Image enhancement method and system based on virtual polarization filtering

Abstract

The present invention discloses an image enhancement method and system based on virtual polarization filtering. The method comprises: imaging a target area to obtain multiple target polarization images; calculating the Stokes vector of incident light at each pixel on the target image based on the target polarization images to obtain the energy flux density of the incident light; for pixels on the target image with a signal-to-noise ratio less than or equal to a threshold, obtaining an initial grayscale value based on the energy flux density of the incident light; superimposing a virtual polarizer on pixels on the target image with a signal-to-noise ratio greater than the threshold, and obtaining an initial grayscale value based on the energy flux density after passing through the virtual polarizer; and performing overexposure optimization processing based on the initial grayscale value to obtain an enhanced target image. The present invention is applied to the field of image processing. By suppressing polarized light through virtual polarization filtering, the target image is effectively enhanced, making the target area on the image more clearly visible, and significantly reducing the negative impact on target detection under complex lighting conditions.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image enhancement method and system based on virtual polarization filtering. Background Art

[0002] Additive manufacturing (AM), also known as 3D printing, originated in the 1960s. After decades of development, AM has become a vital component in fields such as aerospace, medical, and automotive. During the AM process, a high-energy laser melts layers of powder, forming a high-temperature molten pool. This molten pool then cools and solidifies into the desired shape. Different preparation conditions directly affect the shape of the molten pool. Improper processing parameters can lead to molten pool instability, which in turn can cause defects such as bulges, pores, and cracks. These defects can reduce the mechanical properties and build density of the workpiece, degrading its physical properties and severely impacting its performance and service life. Therefore, effectively detecting defective workpieces and preventing them from entering service is crucial. Traditional non-destructive defect detection methods include radiographic testing, ultrasonic testing, liquid penetrant testing, magnetic particle testing, and eddy current testing. However, these methods are either costly, complex, and slow, or have specific limitations on the objects they can be tested, resulting in a limited range of applications. Given the ever-expanding and dynamic market, these traditional methods may not fully meet production needs in some cases. Therefore, it is necessary to propose a more efficient and low-cost defect detection method.

[0003] In recent years, deep neural network technology has developed rapidly and achieved remarkable success in the field of defect detection. In 1998, LeCun proposed LeNet, which successfully solved the handwritten digit recognition task, marking the true emergence of convolutional neural networks (CNNs) and propelling object detection algorithms into the deep learning stage. In 2013, Ross Girshick et al. proposed R-CNN, applying convolutional neural networks to feature extraction. Leveraging the excellent feature extraction capabilities of CNNs, they improved the detection rate of datasets. In 2015, Kaiming He proposed ResNet, which broke through the limitations of neural network depth and further improved detection accuracy. In the same year, Joseph Redmon et al. proposed YOLO, which uses a unified neural network to simultaneously predict the location and category of objects in an image, achieving fast and efficient object detection. That same year, Shaoqing Ren et al. proposed Faster R-CNN, which increased the speed of training and testing while also improving detection accuracy. In 2016, Gao Huang et al. proposed DenseNet, which established connections between different layers, further alleviating the vanishing gradient problem and improving object detection accuracy. These neural network-based studies have significantly advanced defect detection technology. They have improved model accuracy and processing speed, enabling inspection systems to more effectively identify and classify various surface defects. This research not only enhances the performance and reliability of surface defect detection but also expands its practical applications.

[0004] While numerous excellent methods have been applied to surface defect detection, their performance is often limited by the application scenarios, preventing them from realizing their full potential. For example, in workpiece surface defect detection, the accuracy of these algorithms is significantly affected by complex ambient lighting. However, in actual production processes, lighting conditions are complex and variable, requiring different detection models to adapt to these changing conditions, which increases costs and complexity. Summary of the Invention

[0005] In response to the above-mentioned deficiencies in the prior art, the present invention provides an image enhancement method and system based on virtual polarization filtering, which suppresses polarized light through virtual polarization filtering, thereby effectively enhancing the target image, making the target area on the image more clearly visible, and significantly reducing the negative impact on target detection under complex lighting conditions.

[0006] To achieve the above object, the present invention provides an image enhancement method based on virtual polarization filtering, comprising the following steps:

[0007] Based on N camera systems with different polarization characteristics and imaging simultaneously, the target area is imaged to obtain N target polarization images, where N ≥ 3;

[0008] Construct a target image with the same size as each target polarization image, and calculate the Stokes vector of the incident light at the corresponding pixel point on the target image based on the grayscale value of the pixel point at the same position on the three target polarization images;

[0009] Based on the Stokes vector of the incident light at each pixel on the target image, the energy flux density s of the incident light at each pixel on the target image is calculated p0 ;

[0010] Calculate the signal-to-noise ratio of each pixel on the target image based on the Stokes vector:

[0011] For pixels on the target image whose signal-to-noise ratio is less than or equal to the threshold K, the energy flux density s of the incident light is calculated. p0 Calculate the initial grayscale value of the corresponding pixel;

[0012] For the pixel points on the target image whose signal-to-noise ratio is greater than the threshold K, a block with an angle of After the virtual polarizer is inserted, the energy flux density of the light passing through the virtual polarizer is obtained. Based on energy flux density Calculate the initial grayscale value of the corresponding pixel, where A OP is the polarization angle of the corresponding pixel point on the target image;

[0013] Based on the initial grayscale value, overexposure optimization processing is performed to obtain the final grayscale value of each pixel on the target image, that is, the enhanced target image.

[0014] In one embodiment, when N>3, the calculation of the Stokes vector of the incident light at the pixel point at the corresponding position on the target image based on the grayscale value of the pixel point at the same position on the three target polarization images is specifically:

[0015] The three smallest values ​​are selected from the N grayscale values ​​at corresponding positions on the N target polarization images, and the Stokes vector of the incident light at the pixel point at the corresponding position on the target image is calculated.

[0016] In one embodiment, the Stokes vector of the incident light at each pixel point on the target image is specifically:

[0017]

[0018] Where S(λ) represents the Stokes vector of the incident light at the corresponding pixel point of the target image, I represents the power of the incident light at the corresponding pixel point of the target image, α represents the gain of the camera system, h represents the Planck constant, c represents the speed of light, λ represents the wavelength of the incident light at the corresponding pixel point of the target image, f(λ) represents the quantum efficiency function of the camera system, and ADU1 、A DU2 、A DU3 Represents the grayscale value of the pixel at the same position on the three selected target polarization images;

[0019] W represents a reversible matrix composed of the element values ​​of the Mueller matrix of multiple camera systems, specifically:

[0020]

[0021] Among them, m 111 、m 112 、m 113 Indicates the gray value A DU1 The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image, m 211 、m 212 、m 213 Indicates the gray value A DU2 The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image, m 311 、m 312 、m 313 Indicates the gray value A DU3 The first row of the Mueller matrix of the camera system corresponding to the target polarization image.

[0022] In one embodiment, the energy flux density The energy flux density s p0 The calculation process is:

[0023]

[0024] Among them, S p (λ, Δt) represents the Stokes vector of the incident light flux density at the corresponding pixel point of the target image, and Δt represents the exposure time of the camera system;

[0025]

[0026] The energy flux density s p0 That is the energy flux density Stokes vector S p The first element of (λ, Δt).

[0027] In one embodiment, the initial grayscale value of each pixel on the target image is specifically:

[0028]

[0029] in, represents the initial grayscale value of the corresponding pixel of the target image, Δt′ represents the configured exposure time set on the computer, S NRIt represents the signal-to-noise ratio of the corresponding pixel point of the target image, s0 represents the intensity of the incident light at the corresponding pixel point of the target image, and s1 and s2 represent the linear polarization components of the incident light at the corresponding pixel point of the target image.

[0030] In one embodiment, the final grayscale value of each pixel on the target image is specifically:

[0031]

[0032] in, Indicates the final grayscale value of the corresponding pixel in the target image.

[0033] To achieve the above object, the present invention also provides an image enhancement system based on virtual polarization filtering, which uses the above image enhancement method to perform image enhancement processing.

[0034] In one embodiment, the image enhancement system includes:

[0035] An imaging module includes N camera systems with different polarization characteristics and imaging simultaneously, for imaging a target area to obtain N target polarization images, where N is greater than or equal to 3;

[0036] The incident light calculation module is used to calculate the Stokes vector of the incident light at the pixel point at the corresponding position on the target image based on the grayscale value of the pixel point at the same position on the three target polarization images;

[0037] The energy flux density calculation module is used to calculate the energy flux density s of the incident light at each pixel point on the target image based on the Stokes vector of the incident light at each pixel point on the target image. p0 ;

[0038] A signal-to-noise ratio calculation module is used to calculate the signal-to-noise ratio of each pixel on the target image based on the Stokes vector of the incident light at each pixel;

[0039] The first grayscale value calculation module is used to calculate the energy flux density s of the incident light when the signal-to-noise ratio is less than or equal to the threshold K. p0 Calculate the initial grayscale value of the corresponding pixel on the target image;

[0040] The second gray value calculation module is used to superimpose a gray value with an angle of After the virtual polarizer is inserted, the energy flux density of the light passing through the virtual polarizer is obtained. Then, based on the energy flux density Calculate the initial grayscale value of the corresponding pixel on the target image;

[0041] The overexposure optimization module is used to perform overexposure optimization processing based on the initial grayscale value to obtain the final grayscale value of each pixel on the target image, thereby obtaining an enhanced target image.

[0042] Compared with the prior art, the present invention has the following beneficial technical effects:

[0043] The present invention suppresses polarized light through virtual polarization filtering. At the same time, during the process of virtual polarization filtering, the target pixel points of the virtual polarization filtering are screened based on the signal-to-noise ratio, thereby avoiding the suppression of the information of interest in the image and only suppressing the noise information in the image. At the same time, overexposure optimization processing is performed to avoid calculating erroneous information, thereby ensuring that the enhanced target image is always better than the input target polarization image, achieving effective enhancement of the target image, making the target area on the image more clearly visible, and significantly reducing the negative impact on target detection under complex lighting conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0045] Figure 1 Flowchart of the image enhancement method in Example 1 of the present invention;

[0046] Figure 2 Schematic diagram of the MPIS structure in Example 1 of the present invention;

[0047] Figure 3 This is an example flow chart of image enhancement in Example 1 of the present invention;

[0048] Figure 4 Schematic diagrams of the virtual polarization filter input, output, and grayscale histogram in Example 1 of the present invention, wherein: (A) is a 0° target polarization image and its grayscale histogram, (B) is a 45° target polarization image and its grayscale histogram, (C) is a 90° target polarization image and its grayscale histogram, (D) is a 135° target polarization image and its grayscale histogram, and (E) is an enhanced target image and its grayscale histogram.

[0049] Figure 5 Schematic diagram of the combination of virtual polarization filter input, output and grayscale histogram in Example 1 of the present invention;

[0050] Figure 6 This is a structural block diagram of the image enhancement system in Example 2 of the present invention.

[0051] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0053] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0054] Example 1

[0055] Polarization imaging technology has unique advantages over traditional optical detection methods. It can obtain spectral information, polarization information and spatial information of the detection target, and is currently widely used in industrial imaging, remote sensing, biomedical diagnosis and other fields. Defect detection based on polarization technology is conducive to extracting information such as texture structure, surface material and surface roughness from the polarization information of the detection target, which can effectively improve the accuracy and reliability of detection. To this end, this embodiment discloses an image enhancement method based on virtual polarization filtering, which suppresses polarized light through virtual polarization filtering. At the same time, in the process of virtual polarization filtering, the target pixel points of the virtual polarization filter are screened based on the signal-to-noise ratio, thereby avoiding the suppression of the information of interest in the image and only suppressing the noise information in the image. At the same time, overexposure optimization processing is used to avoid solving erroneous information, thereby ensuring that the enhanced target image is always better than the input target polarization image, achieving effective enhancement of the target image, making the target area on the image more clearly visible, and significantly reducing the negative impact on target detection under complex lighting conditions. Reference Figure 1 In this embodiment, the image enhancement method based on virtual polarization filtering specifically includes the following steps:

[0056] Based on N camera systems with different polarization characteristics and imaging simultaneously, the target area is imaged to obtain N target polarization images, where N ≥ 3;

[0057] Construct a blank target image with the same size as each target polarization image, and calculate the Stokes vector of the incident light at the corresponding pixel point on the target image based on the grayscale value of the pixel point at the same position on the three target polarization images;

[0058] Based on the Stokes vector of the incident light at each pixel on the target image, the energy flux density s of the incident light at each pixel on the target image is calculated p0 ;

[0059] Calculate the signal-to-noise ratio of each pixel on the target image based on the Stokes vector:

[0060] For pixels on the target image whose signal-to-noise ratio is less than or equal to the threshold K, the energy flux density s of the incident light is calculated. p0 Calculate the initial grayscale value of the corresponding pixel;

[0061] For the pixel points on the target image whose signal-to-noise ratio is greater than the threshold K, a block with an angle of After the virtual polarizer is inserted, the energy flux density of the light passing through the virtual polarizer is obtained. Based on energy flux density Calculate the initial grayscale value of the corresponding pixel, where A OP is the polarization angle of the corresponding pixel point on the target image;

[0062] Based on the initial grayscale value, overexposure optimization processing is performed to obtain the final grayscale value of each pixel on the target image, that is, the enhanced target image.

[0063] In nature, light waves are electromagnetic waves, and their electric fields usually propagate in the form of plane sinusoidal waves. Therefore, the electric field E(x, t) of any beam of light propagating along the x-axis can be expressed as:

[0064]

[0065] Where Re[·] represents the real part, E represents the electric field amplitude, j represents the imaginary unit, ω represents the electric field angular frequency, t represents the electric field propagation time, k represents the wave number, x represents the distance vector from the light source to the observation point (imaging distance vector), ψ represents the initial phase of the electric field, and c represents the speed of light. represents the x-axis unit vector, is the unit vector of the electric field vibration direction, that is, the polarization direction of light;

[0066] Taking each pixel in the image as the research object and assuming that each pixel of the image is only affected by a sufficiently small point in the outside world and is not affected by the reflected light from other areas of the outside world, it can be considered that the angular frequency, propagation time, and imaging distance of the imaging light wave of each pixel are the same, so the electric field of the light wave can be expressed as a phasor form for:

[0067]

[0068] The external environment of the electric field is idealized as a vacuum environment. According to the electromagnetic field theory, the electric field and its induced magnetic field are Satisfies the Ampere-Maxwell law, that is:

[0069]

[0070] Where ▽ represents the gradient operator, μ0 represents the vacuum permeability, J represents the current density, and ε0 represents the vacuum dielectric constant;

[0071] Since it is a vacuum environment, J = 0. Taking the curl on both sides of equation (4), we have:

[0072]

[0073] in, Represents the direction of magnetic field vibration, and solving the differential equation yields:

[0074]

[0075] Therefore, the energy flux density S of the electromagnetic field of the light wave is n for:

[0076]

[0077] Then the average power P of the light wave is:

[0078]

[0079] Here, T represents the period of the light wave.

[0080] On the other hand, starting from the morphology of photons, the relationship between the grayscale value of the pixel and the average power of the light wave is analyzed. In the MPIS imaging process, when the image is not overexposed, the grayscale value A of the pixel on the image is DU and the number of photoelectrons N E The functional relationship can be expressed as:

[0081] N E =α·(A DU -A DU0 ) (9)

[0082] Among them, α represents the gain of the camera, A DU0 Indicates the camera bias;

[0083] The quantum efficiency of a camera can effectively convert the number of incident photons N L and the number of photoelectrons N E The relationship is described as:

[0084] N E =f(λ)·N L (10)

[0085] Where f(λ) represents the quantum efficiency function of the camera system, and λ represents the wavelength of the incident light at the corresponding pixel point of the target image;

[0086] Combine equations (9) and (10) and substitute them into the formula available:

[0087]

[0088] Where Δt represents the exposure time of the camera system, h represents Planck's constant, and c represents the speed of light;

[0089] Since the camera bias is often a small value, this embodiment ideally processes it to 0, and we can obtain:

[0090]

[0091] From formula (12), we can see that the grayscale value of the image is a quantity related to the wavelength of light, the average power of light, and the exposure time (there is no boundary condition between these three variables). And from formula (7), we can see that when obtaining the energy flux density of light, the polarization information of the incident light is also obtained. This means that even if the wavelength, exposure time, and individual grayscale information of the incident light are known, the polarization state of the incident light cannot be inferred. Furthermore, regardless of the polarization state of the incident light, under certain conditions, the grayscale value displayed by the image is constant. Therefore, in this embodiment, the polarization state of the virtual incident light is changed by using virtual polarization filtering without affecting the original image information. Image enhancement by suppressing polarized light is theoretically feasible.

[0092] In order to facilitate the description of the polarization state of light, this embodiment uses the Stokes vector S = (s0, s1, s2, s3) T The light is represented in the form of . Among them, component s0 represents the light intensity, components s1 and s2 represent the linear polarization components, component s3 represents the circular polarization component, and T represents the matrix. After the light passes through the optical system, its polarization state will change. The 4×4 Mueller matrix can describe this change. Assume that the Mueller matrix of the i-th camera system is M i , then:

[0093]

[0094] Among them, S i represents the Mueller matrix after the light passes through the i-th camera system, m i11 、m i12 、m i13 、m i14 、m i21 、m i22 、mi23 、m i24 、m i31 、m i32 、m i33 、m i34 、m i41 、m i42 、m i43 、m i44 represents the elements of the Mueller matrix of the i-th camera system;

[0095] Since the circular polarization component s3 of natural light is usually 0, equation (12) can be reduced to:

[0096]

[0097] Taking a certain pixel point of the image obtained after imaging as the research object, we can get:

[0098]

[0099] Among them, s i0 =P i Δt i Indicates the power received by the camera system lens area corresponding to the pixel point, A DUi Represents the gray value of the pixel, λ i Indicates the wavelength of the incident light corresponding to the pixel;

[0100] Therefore, the light intensity I received by three camera systems with different polarization characteristics and simultaneous imaging in the same area can be written in the following matrix form:

[0101]

[0102] Among them, s 10 、s 20 、s 30 Represents the light intensity received by the three camera systems, W represents the reversible matrix composed of the element values ​​of the Mueller matrix of multiple camera systems, m 111 、m 112 、m 113 Indicates the gray value A DU1 The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image, m 211 、m 212 、m 213 Indicates the gray value A DU2 The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image, m 311 、m 312 、m 313 Indicates the gray value A DU3 The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image;

[0103] Since the three camera systems are imaging simultaneously, λ1=λ2=λ3=λ. When the W matrix is ​​reversible, the Stokes vector of the incident light at the corresponding position of each pixel on the target image can be solved by the grayscale values ​​of the three target polarization images, that is:

[0104]

[0105] Among them, S(λ) represents the Stokes vector of the incident light at the corresponding pixel point of the target image, A DU1 、A DU2 、A DU3 Represents the grayscale value of the pixel at the same position on the three selected target polarization images;

[0106] Assuming that the exposure time of the three imaging systems is the same, that is, Δt1=Δt2=Δt3=Δt, this embodiment defines the Stokes vector of the energy flux density of the incident light at the corresponding pixel point of the target image as:

[0107]

[0108] Among them, S p (λ, Δt) represents the Stokes vector of the incident light energy flux density at the corresponding pixel point of the target image;

[0109] According to the first three items in the Stokes vector, the polarization angle A of each pixel in the target image can be obtained. OP Information, expressed as:

[0110]

[0111] For a single pixel, it is obvious that A OP is a quantity that is independent of λ and Δt. At this time, a piece with an angle of The virtual polarizer can effectively suppress polarized light, and the reduced-dimensional Mueller matrix of the virtual polarizer is:

[0112]

[0113] The energy flux density of light after passing through the virtual polarizer is:

[0114]

[0115] Since the wavelength of light does not change after passing through the polarizer, the pixel suppresses the initial grayscale value of the polarized light. for:

[0116]

[0117] Among them, Δt′ represents the exposure time configured by the computer. By changing its value, the exposure of the image can be changed. From formula (22), we can know that the initial grayscale value is a quantity that has nothing to do with λ. Therefore, the image enhancement method in this embodiment does not require information such as the wavelength of the incident light and the exposure time. It can be used according to the input target polarization image and the set exposure time ratio. By performing virtual polarization filtering, the desired target image can be obtained.

[0118] In specific applications, if virtual polarization filtering is performed on each pixel in the target image, the information of interest in the image may be suppressed. In this embodiment, unpolarized light is determined to be information and polarized light is determined to be noise. Therefore, when the signal-to-noise ratio of a pixel on the target image is greater than a threshold value K, it can be considered that the pixel is richer than the noise and does not need virtual polarization filtering. Therefore, in the specific implementation process, only the pixels with a signal-to-noise ratio greater than the threshold value K are subjected to virtual polarization filtering, while other pixels are not subjected to virtual polarization filtering. That is, the initial grayscale value of each pixel on the target image is specifically:

[0119]

[0120]

[0121] Among them, S NR Indicates the signal-to-noise ratio of the corresponding pixel in the target image.

[0122] In the above calculation process, both the calculation of the Stokes vector and the update of the pixel grayscale value rely on formula (12). However, formula (12) is only valid when the image is not overexposed. When the input image is overexposed, erroneous information may be generated, thereby reducing the imaging effect and affecting the accuracy of subsequent target detection. Since it is difficult to avoid image overexposure under complex lighting conditions, this embodiment performs overexposure optimization processing to ensure that the enhanced target image is always better than the input target polarization image.

[0123] Since the circularly polarized light component in nature is usually zero, the light wave can be regarded as a combination of unpolarized light and linearly polarized light. The relationship between its reduced-dimensional Stokes vector can be expressed as:

[0124]

[0125] Among them, S u is the reduced-dimensional Stokes vector of unpolarized light, S p is the reduced-dimensional Stokes vector of polarized light;

[0126] The polarization angle of the linear polarization part is denoted as δ1, which can be obtained using formula (19); the angle of the virtual polarizer is denoted as δ2, and the light intensity s′0 passing through the virtual polarizer is:

[0127]

[0128] If the input image is overexposed, we will discuss two cases below:

[0129] If s u0 >>s p0 ,but At this time, all input images must be overexposed, and the method of this embodiment cannot extract any information from them. The initial grayscale value of the output is 255;

[0130] If s u0 Not much larger than s p0 : At this time, the input image may contain an image that is not overexposed. If so, the initial grayscale value calculated according to the method of this embodiment is Must be greater than the grayscale value A without overexposure DUi The product of the exposure time ratio does not play a filtering role, so we can make To achieve overexposure processing.

[0131] Therefore, the specific process of overexposure optimization processing in this embodiment is as follows:

[0132]

[0133] in, Indicates the final grayscale value of the corresponding pixel in the target image;

[0134] After calculating the final grayscale value of each pixel of the target image, the final target image is obtained, and the image enhancement operation is completed.

[0135] As a preferred embodiment, when the number of camera systems with different polarization characteristics and imaging simultaneously is four or more, that is, when N>3, the Stokes vector of the incident light at the pixel point at the corresponding position on the target image is calculated based on the grayscale value of the pixel point at the same position on the three target polarization images. Specifically, the three smallest values ​​are selected from the N grayscale values ​​at the corresponding positions on the N target polarization images, and the Stokes vector of the incident light at the pixel point at the corresponding position on the target image is calculated to avoid selecting overexposed pixel grayscale values ​​as much as possible.

[0136] The image enhancement method based on virtual polarization filtering in this embodiment is further described below with reference to specific examples.

[0137] In this example, use Figure 2The multi-source polarization imaging system (MPIS) shown here collects image data. The imaging system primarily consists of the workpiece to be measured, an LED light source, a linear polarizer, a motorized rotating frame, a CMOS camera, and a computer. The linear polarizer is a 25.4 mm THORLABS LPVISE100-A, operating in the 400-700 nm wavelength range. The CMOS image sensor has a resolution of 2448 × 2048 pixels, a single pixel size of 3.45 μm, and a lens focal length of 50 mm. The motorized rotating frame is a Standa FPSTA-8MPR16-1, capable of 360° rotation and 0.75 arc-minute step resolution. It is equipped with an 8SMC4-USB controller. The computer used for image acquisition and analysis in the experimental example is a ThinkPad S2 with an i58250U CPU and a maximum frequency of 1.8 GHz. During the experimental example, the workpiece to be measured is illuminated by an LED light source. Light reflected from the workpiece surface passes through the polarizer and is captured by the CMOS image sensor.

[0138] refer to Figure 3 , which is the process of this example, which mainly includes data acquisition, image sampling, input determination, and solving S NR 、Solve The detailed methods of each step are as follows:

[0139] Step 1: MPIS collects images of the defective object and obtains four polarization images of the target at different directions of 0°, 90°, 45°, and 135° under the same viewing angle.

[0140] Step 2: The computer receives four target polarization images as input and samples them using the OpenCV library to obtain the corresponding grayscale value matrix;

[0141] Step 3: Select the pixel points to be processed on the target image and select the three with the smallest values ​​from the four grayscale value matrices to form the input;

[0142] Step 4: Determine the matrix W and solve S based on the three grayscale values ​​input NR ;

[0143] Step 5: Calculate the new final grayscale value of the pixel according to the principle of the method of this embodiment.

[0144] Step 6: If all pixels in the target image have been processed, the enhanced target image is output; otherwise, return to step 3.

[0145] This example evaluates the experimental results by analyzing the grayscale histogram of the image. The grayscale histogram can effectively display the frequency of pixels at different grayscale levels in the image, which helps to analyze the contrast, brightness and dynamic range of the image. In this example, according to the principle of the method of this embodiment, virtual polarization filtering is performed on most pixels in the image in the optimal polarization direction. The expected result is that most of the low and medium grayscale value areas in the grayscale histogram of the ideal output image will show a downward trend. This feature shows that the grayscale value of the low-contrast area is reduced by virtual polarization filtering, thereby suppressing the background expression and improving the visual quality and feature performance of the image.

[0146] refer to Figure 4 、 Figure 5 , are the experimental results and histograms of image enhancement based on virtual polarization filtering in this example. Table 1 shows the distribution of each grayscale value interval of each image.

[0147] Table 1

[0148]

[0149] Depend on Figure 4 、 Figure 5 As shown in Table 1, after image enhancement processing, the number of pixels with grayscale values ​​greater than 204 decreased by two orders of magnitude compared with the images with 0°, 45°, and 90° polarization directions, showing a significant effect in suppressing overexposure. Compared with the image with 135° polarization direction, the number of pixels with grayscale values ​​greater than 152 changed less, only decreasing by about 7%. However, in the low grayscale value area, the number of pixels with grayscale values ​​in the range of [0,50] increased by 33%, while the number of pixels with grayscale values ​​in the range of [51,101] decreased by 25%. This shows that the enhanced image maintains relatively stable features in the high grayscale value area, while the overall grayscale level in the low grayscale value area is reduced, thereby effectively suppressing background noise. This processing method helps to improve the sensitivity and accuracy of the model in defect detection tasks, because the reduction of background noise can highlight the features of the target area, making the identification of defects clearer.

[0150] It is worth noting that although this embodiment Figure 1 The steps in the diagram are shown in the order indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0151] Example 2

[0152] Based on the image enhancement method based on virtual polarization filtering in Example 1, this embodiment discloses an image enhancement system based on virtual polarization filtering, referring to Figure 6 The image enhancement system includes an imaging module, an incident light calculation module, an energy flux density calculation module, a signal-to-noise ratio calculation module, a first grayscale value calculation module, a second grayscale value calculation module, and an overexposure optimization module. Specifically:

[0153] The imaging module includes N camera systems with different polarization characteristics and imaging simultaneously, which are used to image the target area and obtain N target polarization images;

[0154] The incident light calculation module is used to calculate the Stokes vector of the incident light at the pixel point at the corresponding position on the target image based on the grayscale value of the pixel point at the same position on the three target polarization images;

[0155] The energy flux density calculation module is used to calculate the energy flux density s of the incident light at each pixel point on the target image based on the Stokes vector of the incident light at each pixel point on the target image. p0 ;

[0156] The signal-to-noise ratio calculation module is used to calculate the signal-to-noise ratio of each pixel on the target image based on the Stokes vector of the incident light at each pixel;

[0157] The first gray value calculation module is used to calculate the energy flux density s of the incident light when the signal-to-noise ratio is less than or equal to the threshold K. p0 Calculate the initial grayscale value of the corresponding pixel on the target image;

[0158] The second gray value calculation module is used to superimpose a gray value with an angle of 0.01 in the corresponding area when the signal-to-noise ratio is greater than the threshold K. After the virtual polarizer is inserted, the energy flux density of the light passing through the virtual polarizer is obtained. Then, based on the energy flux density Calculate the initial grayscale value of the corresponding pixel on the target image;

[0159] The overexposure optimization module is used to perform overexposure optimization processing based on the initial grayscale value to obtain the final grayscale value of each pixel on the target image, thereby obtaining an enhanced target image.

[0160] In this embodiment, the specific working processes and working principles of the imaging module, incident light calculation module, energy flux density calculation module, signal-to-noise ratio calculation module, first grayscale value calculation module, second grayscale value calculation module, and overexposure optimization module are the same as those in Example 1, and therefore are not further described in this embodiment. Each unit module can be implemented in whole or in part through software, hardware, or a combination thereof. Each unit module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so that the processor can call and execute the corresponding operations of each of the above unit modules.

[0161] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. All equivalent structural transformations made by using the contents of the present invention description and drawings under the inventive concept of the present invention, or direct / indirect application in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. An image enhancement method based on virtual polarization filtering, characterized in that: The steps include: Based on N camera systems with different polarization characteristics and imaging simultaneously, the target area is imaged to obtain N target polarization images, where N ≥ 3; Construct a target image with the same size as each target polarization image, and calculate the Stokes vector of the incident light at the corresponding pixel point on the target image based on the grayscale value of the pixel point at the same position on the three target polarization images; Based on the Stokes vector of the incident light at each pixel on the target image, the energy flux density s of the incident light at each pixel on the target image is calculated p0 ; Calculate the signal-to-noise ratio of each pixel on the target image based on the Stokes vector: For pixels on the target image whose signal-to-noise ratio is less than or equal to the threshold K, the energy flux density s of the incident light is calculated. p0 Calculate the initial grayscale value of the corresponding pixel; For the pixel points on the target image whose signal-to-noise ratio is greater than the threshold K, a block with an angle of After the virtual polarizer is inserted, the energy flux density of the light passing through the virtual polarizer is obtained. Based on energy flux density Calculate the initial grayscale value of the corresponding pixel, where A OP is the polarization angle of the corresponding pixel point on the target image; Based on the initial grayscale value, overexposure optimization processing is performed to obtain the final grayscale value of each pixel on the target image, that is, the enhanced target image.

2. The image enhancement method based on virtual polarization filtering according to claim 1, characterized in that: When N>3, the Stokes vector of the incident light at the pixel point at the corresponding position on the target image is calculated based on the grayscale value of the pixel point at the same position on the three target polarization images, specifically: The three smallest values ​​are selected from the N grayscale values ​​at corresponding positions on the N target polarization images, and the Stokes vector of the incident light at the pixel point at the corresponding position on the target image is calculated.

3. The image enhancement method based on virtual polarization filtering according to claim 1 or 2, characterized in that: The Stokes vector of the incident light at each pixel on the target image is specifically: Where S(λ) represents the Stokes vector of the incident light at the corresponding pixel point of the target image, I represents the power of the incident light at the corresponding pixel point of the target image, α represents the gain of the camera system, h represents the Planck constant, c represents the speed of light, λ represents the wavelength of the incident light at the corresponding pixel point of the target image, f(λ) represents the quantum efficiency function of the camera system, and A DU1 、A DU2 、A DU3 Represents the grayscale value of the pixel at the same position on the three selected target polarization images; W represents a reversible matrix composed of the element values ​​of the Mueller matrix of multiple camera systems, specifically: Among them, m 111 、m 112 、m 113 Indicates the gray value A DU1 The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image, m 211 、m 212 、m 213 Indicates the gray value A DU2 The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image, m 311 、m 312 、m 313 Indicates the gray value A DU3 The first row of the Mueller matrix of the camera system corresponding to the target polarization image.

4. The image enhancement method based on virtual polarization filtering according to claim 3, characterized in that: The energy flux density The energy flux density s p0 The calculation process is: Among them, S p (λ, Δt) represents the Stokes vector of the incident light flux density at the corresponding pixel point of the target image, and Δt represents the exposure time of the camera system; The energy flux density s p0 That is the energy flux density Stokes vector S p The first element of (λ, Δt).

5. The image enhancement method based on virtual polarization filtering according to claim 4, characterized in that: The initial grayscale value of each pixel on the target image is specifically: in, represents the initial grayscale value of the corresponding pixel of the target image, Δt′ represents the configured exposure time set on the computer, S NR It represents the signal-to-noise ratio of the corresponding pixel point of the target image, s0 represents the intensity of the incident light at the corresponding pixel point of the target image, and s1 and s2 represent the linear polarization components of the incident light at the corresponding pixel point of the target image.

6. The image enhancement method based on virtual polarization filtering according to claim 5, characterized in that: The final grayscale value of each pixel on the target image is specifically: in, Indicates the final grayscale value of the corresponding pixel in the target image.

7. An image enhancement system based on virtual polarization filtering, characterized in that: The image enhancement system adopts the image enhancement method according to any one of claims 1 to 6.

8. The image enhancement system based on virtual polarization filtering according to claim 7, characterized in that: The image enhancement system comprises: An imaging module includes N camera systems with different polarization characteristics and imaging simultaneously, for imaging a target area to obtain N target polarization images, where N is greater than or equal to 3; The incident light calculation module is used to calculate the Stokes vector of the incident light at the pixel point at the corresponding position on the target image based on the grayscale value of the pixel point at the same position on the three target polarization images; The energy flux density calculation module is used to calculate the energy flux density s of the incident light at each pixel on the target image based on the Stokes vector of the incident light at each pixel on the target image. p0 ; A signal-to-noise ratio calculation module is used to calculate the signal-to-noise ratio of each pixel on the target image based on the Stokes vector of the incident light at each pixel; The first grayscale value calculation module is used to calculate the energy flux density s of the incident light when the signal-to-noise ratio is less than or equal to the threshold K. p0 Calculate the initial grayscale value of the corresponding pixel on the target image; The second gray value calculation module is used to superimpose a gray value with an angle of After the virtual polarizer is inserted, the energy flux density of the light passing through the virtual polarizer is obtained. Then, based on the energy flux density Calculate the initial grayscale value of the corresponding pixel on the target image; The overexposure optimization module is used to perform overexposure optimization processing based on the initial grayscale value to obtain the final grayscale value of each pixel on the target image, thereby obtaining an enhanced target image.

Images