Metal processing detection system and method based on computer vision

Abstract

The invention discloses a metal processing detection system and method based on computer vision, and relates to the technical field of defect detection.Rotatable polarization filters are installed in front of a lens of a vision sensor, the polarization filters are arranged at four angles respectively, and four sets of polarization images are collected; calculating the polarization degree of each pixel point on the surface during metal processing by using the gray values of the four groups of polarization images, setting a polarization region interval, and dividing a metal surface reflection region into a specular reflection region and a diffuse reflection region; acquiring images of different wavebands by using a multispectral vision sensor, and constructing a multispectral image set; for the same wave band, the images of different exposure times are synthesized to obtain an HDR high dynamic range image; setting a weight for the HDR image of each wave band, and combining the weighting of the HDR image and the suppression of the polarization degree to carry out multispectral image fusion so as to obtain a final image; and a gray threshold value and an area threshold value are respectively calculated through the average value and the standard deviation, and the metal processing defects are judged by using the two threshold values.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, specifically to a metal processing inspection system and method based on computer vision. Background Technology

[0002] Traditional metal surface defect detection mainly relies on manual visual inspection or stroboscopic inspection. These methods are inefficient, highly subjective, easily affected by the experience and fatigue of the inspectors, and are mostly performed on a sampling basis after the process is completed, making it difficult to adjust process parameters in real time to avoid losses, thus failing to meet the needs of modern high-speed production. Subsequent non-destructive automated inspection methods such as eddy current, infrared, and magnetic flux leakage detection, while improving inspection efficiency, are susceptible to environmental electromagnetic interference, have limited working distances, and, based on their detection principles, can only detect specific types of defects, making it difficult to achieve comprehensive inspection of metal processing. However, with the development of computer technology, computer vision inspection technology, through image processing and pattern recognition, achieves automated detection of metal surface defects, offering advantages such as high real-time performance, high accuracy, and non-contact non-destructive operation. It acquires 2D or 3D image information of the metal surface through optical imaging, and determines the defect situation through image preprocessing, feature extraction, and classification recognition processes. It can adapt to the inspection requirements of complex industrial scenarios such as the complex lighting conditions in hot rolling and the harsh environments of medium and heavy plate production.

[0003] However, computer vision inspection also has its own unique drawbacks. Due to the smoothness of the metal surface and the inconsistency of the light source during processing, including the inconsistency of multiple levels such as wavelength and position type, the image captured by the sensor is unclear due to the reflection of light from the metal. This increases the difficulty of defect detection and produces a large number of misjudgments. Summary of the Invention

[0004] The purpose of this invention is to provide a computer vision-based metal processing inspection system and method to solve the problems raised in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A computer vision-based method for metal processing inspection, the method comprising the following steps: S100. Set up a multispectral light source in the metal processing area, integrate a linear polarizer on the multispectral light source, set up a multispectral vision sensor, install a rotatable polarizing filter in front of the vision sensor lens, set the polarizing filter to four angles respectively, acquire four sets of polarization images, and extract the gray value of each pixel in the four sets of polarization images. Furthermore, the specific steps for acquiring four sets of polarization images are as follows: S101. Let θ denote the angle of the integrated linear polarizer on the multispectral light source. p The angle of the rotatable polarizing filter mounted in front of the vision sensor lens is denoted as θ.a The rotating polarizing filter is set with four angles, namely θ a ={0°, 45°, 90°, 135°}; Acquire metal processing images at four angles, and extract the grayscale value of all pixels in each image as I. θa (x, y), by integrating the gray values ​​of the four angle images, four sets of polarization image gray values ​​are obtained as {I 0° (x, y), I 45° (x, y), I 90° (x, y), I 135° (x, y)}.

[0006] By integrating a linear polarizer into a multispectral light source, installing a rotatable polarizing filter in front of the vision sensor lens, and acquiring polarized images at four angles (0°, 45°, 90°, and 135°), optical information of metal surfaces under different polarization states can be comprehensively captured. The polarized images at different angles contain the differences in the polarization characteristics of the light reflected from the metal surface, providing rich raw data for subsequent calculations of polarization degree and differentiation of reflection types.

[0007] S200. Calculate the degree of polarization of each pixel on the surface during metal processing using the gray values ​​of four sets of polarization images, set the polarization region interval, and divide the metal surface reflection region into specular reflection and diffuse reflection regions. Furthermore, the specific steps for dividing the metal surface reflection into specular reflection and diffuse reflection regions are as follows: S201. Calculate the degree of polarization of each pixel on the surface during metal processing using the grayscale values ​​of each polarization angle from four sets of polarization image grayscale values. The formula is: ; In the formula, DoP(x,y) represents the polarization degree of the pixel (x,y), and I 0° I 45° I 90° I 135° These represent the grayscale values ​​of pixel (x, y) at angles of 0°, 45°, 90°, and 135°, respectively. In the formula, the pixel value is symbolized as I. θa (x, y) is simplified to I θa Repeatedly calculate the polarization degree of each pixel to obtain the polarization degree of all pixels in the complete image during metal processing. S202. Draw a histogram of polarization degree for all pixels in the image. The horizontal axis of the histogram represents the degree of polarization, and the vertical axis represents the number of pixels. Detect the maximum double peak in the histogram, with one peak indicating specular reflection dominance and the other indicating diffuse reflection dominance. Extract the minimum valley value between the maximum double peaks as the polarization degree threshold T. Use the polarization degree threshold T to determine the polarization degree of pixels in the image, specifically: When DoP(x,y)≥T, the pixel (x,y) is determined to be a specular reflection point; when DoP(x,y)<T, the pixel (x,y) is determined to be a diffuse reflection point. If no maximum bimodal peak is detected in the histogram, the preset empirical polarization threshold T is enabled. de The preset empirical polarization degree threshold is used instead of the polarization degree threshold T for judgment. S203. Extract the number of pixels in the minimum area of ​​specular reflection during metal processing in history as the noise threshold. After judging the polarization degree of all pixels using the polarization degree threshold, the pixels are divided into specular reflection points and diffuse reflection points. When pixels of the same type are adjacent, they form a reflection area. Judge all reflection areas composed of specular reflection points. When the number of pixels in the area is less than the noise threshold, the corresponding reflection area is judged as noise and classified as a diffuse reflection area. Finally, all specular reflection areas and diffuse reflection areas are obtained.

[0008] By calculating the degree of polarization of each pixel using the grayscale values ​​of four sets of polarization images, and combining this with a polarization degree threshold (determined by the maximum bimodal valley value of the histogram or a preset empirical value), specular reflection and diffuse reflection areas can be accurately distinguished. This step effectively eliminates the interference of strong specular reflection on metal surfaces on defect detection, because specular reflection easily obscures defect information, while diffuse reflection areas more accurately reflect the state of the metal surface.

[0009] By introducing a noise threshold to filter specular reflection areas, excessively small specular reflection areas are classified as diffuse reflection areas, which reduces the impact of noise on the detection results and improves the accuracy of reflection area division.

[0010] S300: Construct a multispectral array, adjust the bands of the multispectral light source according to the multispectral array, and use a multispectral vision sensor to collect images of different bands to construct a multispectral image set; Furthermore, the specific steps for constructing a multispectral image set are as follows: S301. Construct a multispectral array B, assuming the array contains light source bands of visible light (VIS), near-infrared (NIR), and specific bands λ1 and λ2. These specific bands are actively set by the operator based on different metal surface properties during metal processing. The bands of the multispectral light source are adjusted to represent the four bands in the multispectral array. Metal processing images of the four bands are actively acquired using a multispectral vision sensor, and the grayscale value of the pixel in the b-th band of the metal processing image is extracted as I. b (x, y), b belong to a multispectral array, and the multispectral image gray value set is constructed as {I VIS (x, y), I NIR (x, y), I λ1 (x, y), I λ2 (x, y)}, I VIS(x, y) represents the grayscale value of pixel (x, y) when the wavelength is visible light (VIS). NIR (x, y) represents the grayscale value of pixel (x, y) when the wavelength is near-infrared (NIR). λ1 (x, y) represents the grayscale value of pixel (x, y) when the band is a specific band λ1. λ2 (x, y) represents the gray value of pixel (x, y) when the band is a specific band λ2.

[0011] S400: For the same wavelength band, images are acquired by setting different exposure times, and the images with different exposure times are combined to obtain HDR high dynamic range images; Furthermore, the specific steps for synthesizing images with different exposure times to obtain HDR high dynamic range images are as follows: S401. For the same wavelength band, set different exposure times t and acquire images with different exposure times. Then extract the grayscale values ​​of the pixels in the images. Let the grayscale value of the pixel in the image with the kth exposure time for the b-th wavelength band be I. b，k (x, y) is the image obtained by fusing the grayscale values ​​of image pixels at different exposure times to obtain an HDR high dynamic range image. The formula is: ; In the formula, I b HDR (x, y) represents the grayscale value of the pixel (x, y) after fusion in the b-th band, t k Let w represent the k-th exposure time. k represents the weight of the grayscale value of the image pixel at the k-th exposure time, and n represents the total number of exposure time types; S402. Dynamically calculate the weight of the grayscale value of image pixels for each exposure time, using the following formula: ; In the formula, w k This represents the weight of the grayscale value of the image pixel at the k-th exposure time, where μ represents the ideal grayscale center value. This represents the weight decay coefficient; The grayscale values ​​of all fused pixels are calculated, and the fused pixels are used to construct HDR high dynamic range images for each band.

[0012] By acquiring images with different exposure times for the same wavelength and synthesizing them into HDR images, the problem of uneven image brightness on metal surfaces caused by reflections and shadows can be solved. HDR images can preserve details in both highlight and shadow areas, avoiding the loss of defect information due to underexposure or overexposure, and providing higher-quality image data for subsequent image fusion and defect detection.

[0013] S500: Set weights for HDR images of each band, set polarization suppression coefficients for the polarization degree of each pixel, and combine the weighted HDR images with the suppressed polarization degree to perform multispectral image fusion to obtain the final image; Furthermore, the specific steps for multispectral image fusion, combining HDR image weighting and polarization suppression to obtain the final image, are as follows: S501. Weights are assigned to HDR images for each band, and a polarization suppression coefficient β is pre-set for the polarization degree of each pixel. The rule for setting the polarization suppression coefficient is: β for specular reflection region < β for diffuse reflection region. The weighted HDR image and the suppressed polarization degree are combined for multispectral image fusion, using the following formula: ; In the formula, I fused (x, y) represents the grayscale value of a pixel in the final image, α b This represents the weight of the grayscale value of a pixel in the HDR high dynamic range image of the b-th band. The weight is set by the staff based on prior knowledge. The final image is obtained by using the grayscale values ​​of all the pixels after fusion.

[0014] Weights are assigned to HDR images of each band, and the images are fused using polarization suppression coefficients (smaller coefficients for specular reflection areas and larger coefficients for diffuse reflection areas). This approach integrates the advantages of multi-band images while further suppressing specular reflection interference. Weighted fusion highlights band information sensitive to defect detection, improving the contrast between defective and normal areas in the final image, thus facilitating subsequent defect identification.

[0015] S600. Collect images of defects that existed during historical metal processing, calculate the gray value of the image at the defect location and the number of pixels in the defect area, calculate the gray value threshold and area threshold by the mean and standard deviation respectively, and detect the final image. Use the two thresholds to judge metal processing defects.

[0016] Furthermore, the specific steps for judging metal processing defects using two thresholds are as follows: S601. Collect images of defects that existed during historical metal processing. Calculate the average and standard deviation of the grayscale value and the number of pixels in the defect area in all historical defect images. Subtract the standard deviation from the average to obtain the lower limit of the grayscale threshold. Add the standard deviation to the average to obtain the upper limit of the grayscale threshold. Construct the grayscale threshold interval Iq using the upper and lower limits of the grayscale threshold. Subtract the standard deviation from the average number of pixels in the defect area to obtain the area threshold My. S602. First, perform grayscale judgment, using a grayscale threshold to determine the grayscale value of each pixel in the final image. When I fused When (x, y) ∈ Iq, the corresponding pixel is determined to be a defective pixel. fusedWhen (x, y) ∉ Iq, it is judged as a normal pixel point; adjacent defective pixel points are used to construct a defective area in the final image; Area judgment is performed again. The number of defective pixel points in the defective area in the final image is judged using an area threshold. When the number of defective pixel points in the defective area ≥ My, it is judged as a real defective area. When the number of defective pixel points in the defective area < My, it is judged as a noise area.

[0017] Based on historical defective images, the gray - scale threshold interval and area threshold are calculated, making the threshold setting more in line with the defective features in actual production, and improving the pertinence and accuracy of defective judgment.

[0018] A metal - processing detection system based on computer vision. The metal - processing detection system includes a data - acquisition module, a metal - processing detection device, a reflection - judgment module, a multi - spectral analysis module, a spectral - fusion module, and a defect - detection module; The data - acquisition module is used to acquire metal - processing images with defects in history; The metal - processing detection device is used to respectively set a multi - spectral light source and a multi - spectral vision sensor. A linear polarizer is integrated on the multi - spectral light source, and a rotatable polarization filter is installed in front of the vision - sensor lens. The polarization filter is set at four angles respectively to acquire four groups of polarization images; The reflection - judgment module is used to calculate the polarization degree of each pixel point on the surface during metal processing using the gray - scale values of the four groups of polarization images, set a polarization - region interval, and distinguish the mirror - reflection and diffuse - reflection regions on the metal surface; The multi - spectral analysis module is used to acquire images in different bands. For the same band, images are acquired with different exposure times, and the images with different exposure times are synthesized to obtain an HDR high - dynamic - range image; The spectral - fusion module is used to set weights for each band's HDR image, set a polarization - suppression coefficient for the polarization degree of each pixel point, and combine the weighted HDR image and polarization - degree suppression to perform multi - spectral image fusion to obtain the final image; The defect - detection module is used to calculate the gray - scale threshold and area threshold respectively through the average value and standard deviation, detect the final image, and judge metal - processing defects using the two thresholds.

[0019] The reflection - judgment module includes a polarization - degree calculation unit and a reflection - judgment unit; The polarization - degree calculation unit is used to calculate the polarization degree of each pixel point on the surface during metal processing using the gray - scale values of each - angle polarization image in the four - group polarization - image gray - scale values; The reflection - judgment unit is used to construct a histogram to calculate the polarization - degree threshold, and use the polarization - degree threshold to judge and distinguish the mirror - reflection and diffuse - reflection regions for the polarization degree.

[0020] The multispectral analysis module includes a multispectral image acquisition unit and a high dynamic range image generation unit; The multispectral image acquisition unit is used to construct a multispectral array B, adjust the bands of the multispectral light source to four bands in the multispectral array, and actively acquire metal processing images in the four bands using a multispectral vision sensor. The high dynamic range image generation unit is used to fuse the grayscale values ​​of image pixels at different exposure times to obtain an HDR high dynamic range image.

[0021] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention distinguishes between specular reflection and diffuse reflection regions by polarization degree calculation and suppresses specular reflection interference in image fusion, effectively solving the problem of strong reflection on metal surfaces affecting defect detection.

[0022] 2. By combining multispectral technology and HDR image synthesis, it is possible to use different wavelengths of light to reflect the various characteristics of the metal surface, while also preserving the details of different brightness areas in the image, ensuring that no potential defect information is missed. Attached Figure Description

[0023] Figure 1 This is a schematic diagram illustrating the steps of a computer vision-based metal processing inspection method according to the present invention. Figure 2 This is a module distribution diagram of a computer vision-based metal processing inspection system according to the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Example: Figures 1-2 As shown, the present invention provides a technical solution. A computer vision-based method for metal processing inspection, the method comprising the following steps: S100. Set up a multispectral light source in the metal processing area, integrate a linear polarizer on the multispectral light source, set up a multispectral vision sensor, install a rotatable polarizing filter in front of the vision sensor lens, set the polarizing filter to four angles respectively, acquire four sets of polarization images, and extract the gray value of each pixel in the four sets of polarization images. The specific steps for acquiring four sets of polarization images are as follows: S101. Let θ denote the angle of the integrated linear polarizer on the multispectral light source. p The angle of the rotatable polarizing filter mounted in front of the vision sensor lens is denoted as θ. a The rotating polarizing filter is set with four angles, namely θ a ={0°, 45°, 90°, 135°}; Acquire metal processing images at four angles, and extract the grayscale value of all pixels in each image as I. θa (x, y), by integrating the gray values ​​of the four angle images, four sets of polarization image gray values ​​are obtained as {I 0° (x, y), I 45° (x, y), I 90° (x, y), I 135° (x, y)}.

[0026] By integrating a linear polarizer into a multispectral light source, installing a rotatable polarizing filter in front of the vision sensor lens, and acquiring polarized images at four angles (0°, 45°, 90°, and 135°), optical information of metal surfaces under different polarization states can be comprehensively captured. The polarized images at different angles contain the differences in the polarization characteristics of the light reflected from the metal surface, providing rich raw data for subsequent calculations of polarization degree and differentiation of reflection types.

[0027] S200. Calculate the degree of polarization of each pixel on the surface during metal processing using the gray values ​​of four sets of polarization images, set the polarization region interval, and divide the metal surface reflection region into specular reflection and diffuse reflection regions. The specific steps to divide the reflective region of a metal surface into specular reflection and diffuse reflection regions are as follows: S201. Calculate the degree of polarization of each pixel on the surface during metal processing using the grayscale values ​​of each polarization angle from four sets of polarization image grayscale values. The formula is: ; In the formula, DoP(x,y) represents the polarization degree of the pixel (x,y), and I 0° I 45° I 90° I 135° These represent the grayscale values ​​of pixel (x, y) at angles of 0°, 45°, 90°, and 135°, respectively. In the formula, the pixel value is symbolized as I. θa (x, y) is simplified to I θa Repeatedly calculate the polarization degree of each pixel to obtain the polarization degree of all pixels in the complete image during metal processing. S202. Draw a histogram of polarization degree for all pixels in the image. The horizontal axis of the histogram represents the degree of polarization, and the vertical axis represents the number of pixels. Detect the maximum double peak in the histogram, with one peak indicating specular reflection dominance and the other indicating diffuse reflection dominance. Extract the minimum valley value between the maximum double peaks as the polarization degree threshold T. Use the polarization degree threshold T to determine the polarization degree of pixels in the image, specifically: When DoP(x,y)≥T, the pixel (x,y) is determined to be a specular reflection point; when DoP(x,y)<T, the pixel (x,y) is determined to be a diffuse reflection point. If no maximum bimodal peak is detected in the histogram, the preset empirical polarization threshold T is enabled. de The preset empirical polarization degree threshold is used instead of the polarization degree threshold T for judgment. S203. Extract the number of pixels in the minimum area of ​​specular reflection during metal processing in history as the noise threshold. After judging the polarization degree of all pixels using the polarization degree threshold, the pixels are divided into specular reflection points and diffuse reflection points. When pixels of the same type are adjacent, they form a reflection area. Judge all reflection areas composed of specular reflection points. When the number of pixels in the area is less than the noise threshold, the corresponding reflection area is judged as noise and classified as a diffuse reflection area. Finally, all specular reflection areas and diffuse reflection areas are obtained.

[0028] By calculating the degree of polarization of each pixel using the grayscale values ​​of four sets of polarization images, and combining this with a polarization degree threshold (determined by the maximum bimodal valley value of the histogram or a preset empirical value), specular reflection and diffuse reflection areas can be accurately distinguished. This step effectively eliminates the interference of strong specular reflection on metal surfaces on defect detection, because specular reflection easily obscures defect information, while diffuse reflection areas more accurately reflect the state of the metal surface.

[0029] By introducing a noise threshold to filter specular reflection areas, excessively small specular reflection areas are classified as diffuse reflection areas, which reduces the impact of noise on the detection results and improves the accuracy of reflection area division.

[0030] S300: Construct a multispectral array, adjust the bands of the multispectral light source according to the multispectral array, and use a multispectral vision sensor to collect images of different bands to construct a multispectral image set; The specific steps for constructing a multispectral image set are as follows: S301. Construct a multispectral array B, assuming the array contains light source bands of visible light (VIS), near-infrared (NIR), and specific bands λ1 and λ2. These specific bands are actively set by the operator based on different metal surface properties during metal processing. The bands of the multispectral light source are adjusted to represent the four bands in the multispectral array. Metal processing images of the four bands are actively acquired using a multispectral vision sensor, and the grayscale value of the pixel in the b-th band of the metal processing image is extracted as I. b (x, y), b belong to a multispectral array, and the multispectral image gray value set is constructed as {I VIS (x, y), I NIR (x, y), I λ1 (x, y), I λ2 (x, y)}, I VIS (x, y) represents the grayscale value of pixel (x, y) when the wavelength is visible light (VIS). NIR (x, y) represents the grayscale value of pixel (x, y) when the wavelength is near-infrared (NIR). λ1 (x, y) represents the grayscale value of pixel (x, y) when the band is a specific band λ1. λ2 (x, y) represents the gray value of pixel (x, y) when the band is a specific band λ2.

[0031] S400: For the same wavelength band, images are acquired by setting different exposure times, and the images with different exposure times are combined to obtain HDR high dynamic range images; The specific steps for synthesizing images with different exposure times to obtain HDR high dynamic range images are as follows: S401. For the same wavelength band, set different exposure times t and acquire images with different exposure times. Then extract the grayscale values ​​of the pixels in the images. Let the grayscale value of the pixel in the image with the kth exposure time for the b-th wavelength band be I. b，k (x, y) is the image obtained by fusing the grayscale values ​​of image pixels at different exposure times to obtain an HDR high dynamic range image. The formula is: ; In the formula, I b HDR (x, y) represents the grayscale value of the pixel (x, y) after fusion in the b-th band, t k Let w represent the k-th exposure time. k represents the weight of the grayscale value of the image pixel at the k-th exposure time, and n represents the total number of exposure time types; S402. Dynamically calculate the weight of the grayscale value of image pixels for each exposure time, using the following formula: ; In the formula, w kThis represents the weight of the grayscale value of the image pixel at the k-th exposure time, where μ represents the ideal grayscale center value. This represents the weight decay coefficient; it normalizes the calculated weights. The grayscale values ​​of all fused pixels are calculated, and the fused pixels are used to construct HDR high dynamic range images for each band.

[0032] By acquiring images with different exposure times for the same wavelength and synthesizing them into HDR images, the problem of uneven image brightness on metal surfaces caused by reflections and shadows can be solved. HDR images can preserve details in both highlight and shadow areas, avoiding the loss of defect information due to underexposure or overexposure, and providing higher-quality image data for subsequent image fusion and defect detection.

[0033] S500: Set weights for HDR images of each band, set polarization suppression coefficients for the polarization degree of each pixel, and combine the weighted HDR images with the suppressed polarization degree to perform multispectral image fusion to obtain the final image; The specific steps for multispectral image fusion, which combines weighted HDR image processing with suppressed polarization, to obtain the final image are as follows: S501. Weights are assigned to HDR images for each band, and a polarization suppression coefficient β is pre-set for the polarization degree of each pixel. The rule for setting the polarization suppression coefficient is: β for specular reflection region < β for diffuse reflection region. The weighted HDR image and the suppressed polarization degree are combined for multispectral image fusion, using the following formula: ; In the formula, I fused (x, y) represents the grayscale value of a pixel in the final image, α b This represents the weight of the grayscale value of a pixel in the HDR high dynamic range image of the b-th band. The weight is set by the staff based on prior knowledge. The final image is obtained by using the grayscale values ​​of all the pixels after fusion.

[0034] Weights are assigned to HDR images of each band, and the images are fused using polarization suppression coefficients (smaller coefficients for specular reflection areas and larger coefficients for diffuse reflection areas). This approach integrates the advantages of multi-band images while further suppressing specular reflection interference. Weighted fusion highlights band information sensitive to defect detection, improving the contrast between defective and normal areas in the final image, thus facilitating subsequent defect identification.

[0035] S600. Collect images of defects that existed during historical metal processing, calculate the gray value of the image at the defect location and the number of pixels in the defect area, calculate the gray value threshold and area threshold by the mean and standard deviation respectively, and detect the final image. Use the two thresholds to judge metal processing defects.

[0036] The specific steps for using two thresholds to judge metal processing defects are as follows: S601. Collect images with defects during historical metal processing, calculate the average value and standard deviation of the gray values and the number of regional pixels in the defective areas of all defective images in history. Obtain the lower limit of the gray threshold by subtracting the standard deviation from the average value, and obtain the upper limit of the gray threshold by adding the standard deviation to the average value. Use the upper and lower limits of the gray threshold to construct the gray threshold interval Iq; obtain the area threshold My by subtracting the standard deviation from the average value of the number of pixels in the defective area. S602. First, perform gray judgment. Use the gray threshold to judge the gray value of each pixel in the final image. When I fused (x, y) ∈ Iq, judge the corresponding pixel as a defective pixel. When I fused (x, y) ∉ Iq, judge it as a normal pixel; use adjacent defective pixels to construct a defective area in the final image. Then, perform area judgment. Use the area threshold to judge the number of defective pixels in the defective area of the final image. When the number of defective pixels in the defective area ≥ My, judge it as a real defective area. When the number of defective pixels in the defective area < My, judge it as a noise area.

[0037] Calculating the gray threshold interval and area threshold based on historical defective images makes the threshold setting more in line with the defective characteristics in actual production, improving the pertinence and accuracy of defective judgment.

[0038] A metal processing detection system based on computer vision. The metal processing detection system includes a data acquisition module, a metal processing detection device, a reflection judgment module, a multispectral analysis module, a spectral fusion module, and a defect detection module; The data acquisition module is used to collect metal processing images with defects in history; The metal processing detection device is used to respectively set a multispectral light source and a multispectral vision sensor, integrate a linear polarizer on the multispectral light source, install a rotatable polarization filter in front of the vision sensor lens, and respectively set the polarization filter at four angles to collect four groups of polarization images; The reflection judgment module is used to calculate the polarization degree of each pixel on the surface during metal processing using the gray values of the four groups of polarization images, set the polarization region interval, and distinguish the metal surface reflection area into specular reflection and diffuse reflection areas; The multispectral analysis module is used to collect images in different bands, for the same band, set different exposure times to collect images, and synthesize the images with different exposure times to obtain an HDR high-dynamic range image; The spectral fusion module is used to set weights for HDR images of each band, set polarization suppression coefficients for the polarization degree of each pixel, and combine the weighted HDR image with the suppressed polarization degree to perform multispectral image fusion to obtain the final image. The defect detection module is used to calculate the grayscale threshold and area threshold by the average value and standard deviation, respectively, and to detect the final image. The two thresholds are used to determine metal processing defects.

[0039] The reflection determination module includes a polarization degree calculation unit and a reflection determination unit; The polarization degree calculation unit is used to calculate the polarization degree of each pixel on the surface during metal processing using gray values ​​of each angle polarization image in the four sets of polarization image gray value sets. The reflection judgment unit is used to construct a histogram to calculate the polarization degree threshold, and uses the polarization degree threshold to judge the polarization degree and distinguish between specular reflection and diffuse reflection regions.

[0040] The multispectral analysis module includes a multispectral image acquisition unit and a high dynamic range image generation unit; The multispectral image acquisition unit is used to construct a multispectral array B, adjust the bands of the multispectral light source to four bands in the multispectral array, and actively acquire metal processing images in the four bands using a multispectral vision sensor. The high dynamic range image generation unit is used to fuse the grayscale values ​​of image pixels at different exposure times to obtain an HDR high dynamic range image. Example:

[0041] The surface scratches during the machining of stainless steel gears were detected by acquiring images from four different angles and extracting the grayscale values ​​of the image pixels. Exposure times of 5s, 15s, and 30s were set for the same wavelength band, and images with different exposure times were extracted. The DoP value of the center pixel (1,1) is calculated as sqrt(400+25) / 955 = 0.0216. The DoP values ​​of all pixels were calculated, a histogram was constructed, and the threshold was extracted to be 0.65. Pixel (1,1) was determined to be a diffuse reflection point. Images were acquired using the VIS band at different exposure times, and the weights of the three exposure time images were calculated to be 0.1374, 0.2720, and 0.5906 respectively. The grayscale values ​​of pixel (1,1) in the three exposure time images were assumed to be 255, 200, and 180 respectively. The grayscale value of the HDR image was calculated as: 0.1374 × (255 / 5) + 0.2720 × (200 / 15) + 0.5906 × (180 / 30) = 14.177. The HDR image in the VIR band is obtained by using the HDR grayscale values ​​of all pixels.

[0042] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A computer vision-based metal processing inspection method, characterized in that: The method includes the following steps: S100. Set up a multispectral light source in the metal processing area, integrate a linear polarizer on the multispectral light source, set up a multispectral vision sensor, install a rotatable polarizing filter in front of the vision sensor lens, set the polarizing filter to four angles respectively, acquire four sets of polarization images, and extract the gray value of each pixel in the four sets of polarization images. S200. Calculate the degree of polarization of each pixel on the surface during metal processing using the gray values ​​of four sets of polarization images, set the polarization region interval, and divide the metal surface reflection region into specular reflection and diffuse reflection regions. S300: Construct a multispectral array, adjust the bands of the multispectral light source according to the multispectral array, and use a multispectral vision sensor to collect images of different bands to construct a multispectral image set; S400: For the same wavelength band, images are acquired by setting different exposure times, and the images with different exposure times are combined to obtain HDR high dynamic range images; S500: Set weights for HDR images of each band, set polarization suppression coefficients for the polarization degree of each pixel, and combine the weighted HDR images with the suppressed polarization degree to perform multispectral image fusion to obtain the final image; S600. Collect images of defects that existed during historical metal processing, calculate the gray value of the image at the defect location and the number of pixels in the defect area, calculate the gray value threshold and area threshold by the mean and standard deviation respectively, and detect the final image. Use the two thresholds to judge metal processing defects.

2. The metal processing inspection method based on computer vision according to claim 1, characterized in that: The specific steps for acquiring four sets of polarization images in S100 are as follows: S101. Let θ denote the angle of the integrated linear polarizer on the multispectral light source. p The angle of the rotatable polarizing filter mounted in front of the vision sensor lens is denoted as θ. a The rotating polarizing filter is set with four angles, namely θ a ={0°, 45°, 90°, 135°}; Acquire metal processing images at four angles, and extract the grayscale value of all pixels in each image as I. θa (x, y), by integrating the gray values ​​of the four angle images, four sets of polarization image gray values ​​are obtained as {I 0° (x, y), I 45° (x, y), I 90° (x, y), I 135° (x, y)}.

3. The metal processing inspection method based on computer vision according to claim 2, characterized in that: The specific steps in S200 to divide the metal surface reflection region into specular reflection and diffuse reflection regions are as follows: S201. Calculate the degree of polarization of each pixel on the surface during metal processing using the grayscale values ​​of each polarization angle from four sets of polarization image grayscale values. The formula is: ； In the formula, DoP(x,y) represents the polarization degree of the pixel (x,y), and I 0° I 45° I 90° I 135° These represent the grayscale values ​​of pixel (x, y) at angles of 0°, 45°, 90°, and 135°, respectively. In the formula, the pixel value is symbolized as I. θa (x, y) is simplified to I θa Repeatedly calculate the polarization degree of each pixel to obtain the polarization degree of all pixels in the complete image during metal processing. S202. Draw a histogram of polarization degree for all pixels in the image. The horizontal axis of the histogram represents the degree of polarization, and the vertical axis represents the number of pixels. Detect the maximum double peak in the histogram, with one peak indicating specular reflection dominance and the other indicating diffuse reflection dominance. Extract the minimum valley value between the maximum double peaks as the polarization degree threshold T. Use the polarization degree threshold T to determine the polarization degree of pixels in the image, specifically: When DoP(x,y)≥T, the pixel (x,y) is determined to be a specular reflection point; when DoP(x,y)<T, the pixel (x,y) is determined to be a diffuse reflection point. If no maximum bimodal peak is detected in the histogram, the preset empirical polarization threshold T is enabled. de The preset empirical polarization degree threshold is used instead of the polarization degree threshold T for judgment. S203. Extract the number of pixels in the minimum area of ​​specular reflection during metal processing in history as the noise threshold. After judging the polarization degree of all pixels using the polarization degree threshold, the pixels are divided into specular reflection points and diffuse reflection points. When pixels of the same type are adjacent, they form a reflection area. Judge all reflection areas composed of specular reflection points. When the number of pixels in the area is less than the noise threshold, the corresponding reflection area is judged as noise and classified as a diffuse reflection area. Finally, all specular reflection areas and diffuse reflection areas are obtained.

4. The metal processing inspection method based on computer vision according to claim 3, characterized in that: The specific steps for constructing the multispectral image set in S300 are as follows: S301. Construct a multispectral array B, assuming the array contains light source bands Visible Light (VIS), Near Infrared (NIR), and specific bands λ1 and λ2. These specific bands are actively set by the operator based on different metal surface properties during metal processing. The bands of the multispectral light source are adjusted to represent the four bands in the multispectral array. Metal processing images are actively acquired using a multispectral vision sensor for each of the four bands. The grayscale value of the pixels in the metal processing image for the b-th band is extracted as I. b (x, y), b belong to a multispectral array, and the multispectral image gray value set is constructed as {I VIS (x, y), I NIR (x, y), I λ1 (x, y), I λ2 (x, y)}, I VIS (x, y) represents the grayscale value of pixel (x, y) when the wavelength is visible light (VIS). NIR (x, y) represents the grayscale value of pixel (x, y) when the wavelength is near-infrared (NIR). λ1 (x, y) represents the grayscale value of pixel (x, y) when the band is a specific band λ1. λ2 (x, y) represents the gray value of pixel (x, y) when the band is a specific band λ2.

5. The metal processing inspection method based on computer vision according to claim 4, characterized in that: The specific steps in S400 to synthesize images with different exposure times to obtain HDR high dynamic range images are as follows: S401. For the same wavelength band, set different exposure times t and acquire images with different exposure times. Then extract the grayscale values ​​of the pixels in the images. Let the grayscale value of the pixel in the image with the kth exposure time for the b-th wavelength band be I. b，k (x, y) is the image obtained by fusing the grayscale values ​​of image pixels at different exposure times to obtain an HDR high dynamic range image. The formula is: ； In the formula, I b HDR (x, y) represents the grayscale value of the pixel (x, y) after fusion in the b-th band, t k Let w represent the k-th exposure time. k represents the weight of the grayscale value of the image pixel at the k-th exposure time, and n represents the total number of exposure time types; S402. Dynamically calculate the weights of the gray values of the pixel points in the images with each exposure time. The formula is as follows: ； In the formula, w k This represents the weight of the grayscale value of the image pixel at the k-th exposure time, where μ represents the ideal grayscale center value. This represents the weight decay coefficient; Calculate the gray values of all the fused pixel points, and use the fused pixel points to form the HDR high-dynamic-range images of each band.

6. The metal processing inspection method based on computer vision according to claim 5, characterized in that: The specific steps of combining the weighted sum of the HDR images and the polarization degree suppression in S500 to perform multispectral image fusion to obtain the final image are as follows: S501. Set weights for the HDR images of each band, preset the polarization suppression coefficient β for the polarization degree of each pixel point, and formulate the polarization suppression coefficient setting rule: β in the specular reflection area < β in the diffuse reflection area; Combine the weighted HDR images and the polarization degree suppression to perform multispectral image fusion. The formula is as follows: ； In the formula, I fused (x, y) represents the grayscale value of a pixel in the final image, α b This represents the weight of the grayscale value of a pixel in the HDR high dynamic range image of the b-th band. The weight is set by the staff based on prior knowledge. The final image is obtained by using the grayscale values ​​of all the pixels after fusion.

7. The metal processing inspection method based on computer vision according to claim 6, characterized in that: The specific steps of using two thresholds to judge metal processing defects in S600 are as follows: S601. Collect the images with defects during historical metal processing, calculate the average value and standard deviation of the gray values and the number of regional pixel points in the defect areas of all the images with defects in history. Respectively obtain the lower limit of the gray threshold by subtracting the standard deviation from the average value, and obtain the upper limit of the gray threshold by adding the standard deviation to the average value. Use the upper and lower limits of the gray threshold to construct the gray threshold interval Iq; Obtain the area threshold My by subtracting the standard deviation from the average value of the number of pixel points in the defect area; S602. First, perform grayscale judgment, using a grayscale threshold to determine the grayscale value of each pixel in the final image. When I fused When (x, y) ∈ Iq, the corresponding pixel is determined to be a defective pixel. fused When (x, y) ∉ Iq, it is judged as a normal pixel; the defect region is constructed in the final image using adjacent defective pixels; Perform area judgment again. Use the area threshold to judge the number of defective pixel points in the defect area of the final image. When the number of defective pixel points in the defect area ≥ My, it is judged as a real defect area. When the number of defective pixel points in the defect area < My, it is judged as a noise area.

8. A computer vision-based metal processing inspection system, characterized in that: The metal processing detection system includes a data acquisition module, a metal processing detection device, a reflection judgment module, a multispectral analysis module, a spectral fusion module, and a defect detection module; The data acquisition module is used to acquire the metal processing images with defects in history; The metal processing detection device is used to respectively set a multispectral light source and a multispectral vision sensor, integrate a linear polarizer on the multispectral light source, install a rotatable polarization filter in front of the vision sensor lens, respectively set the polarization filter at four angles, and acquire four groups of polarization images; The reflection judgment module is used to calculate the polarization degree of each pixel point on the surface during metal processing by using the gray values of the four groups of polarization images, set the polarization region interval, and divide the metal surface reflection area into specular reflection and diffuse reflection areas; The multispectral analysis module is used to acquire images of different bands, for the same band, acquire images with different exposure times, and synthesize the images with different exposure times to obtain HDR high-dynamic-range images; The spectral fusion module is used to set weights for the HDR images of each band, set the polarization suppression coefficient for the polarization degree of each pixel point, and combine the weighted sum of the HDR images and the polarization degree suppression to perform multispectral image fusion to obtain the final image; The defect detection module is used to respectively calculate the gray threshold and the area threshold through the average value and the standard deviation, detect the final image, and use the two thresholds to judge metal processing defects.

9. A computer vision-based metal processing inspection system according to claim 8, characterized in that: The reflection judgment module includes a polarization degree calculation unit and a reflection judgment unit; The polarization degree calculation unit is used to calculate the polarization degree of each pixel on the surface during metal processing using gray values ​​of each angle polarization image in the four sets of polarization image gray value sets. The reflection judgment unit is used to construct a histogram to calculate the polarization degree threshold, and uses the polarization degree threshold to judge the polarization degree and distinguish between specular reflection and diffuse reflection regions.

10. A computer vision-based metal processing inspection system according to claim 8, characterized in that: The multispectral analysis module includes a multispectral image acquisition unit and a high dynamic range image generation unit; The multispectral image acquisition unit is used to construct a multispectral array B, adjust the bands of the multispectral light source to four bands in the multispectral array, and actively acquire metal processing images in the four bands using a multispectral vision sensor. The high dynamic range image generation unit is used to fuse the grayscale values ​​of image pixels at different exposure times to obtain an HDR high dynamic range image.

Images