Image enhancement method and system based on virtual polarization filtering

Abstract

The invention discloses an image enhancement method and system based on virtual polarization filtering, and the method comprises the steps: carrying out the imaging of a target region, and obtaining a plurality of target polarization images; stokes vectors of incident light at all pixel points on the target image are calculated based on the target polarization image, and the energy flux density of the incident light is obtained; for pixel points with signal-to-noise ratios smaller than or equal to a threshold value on the target image, an initial gray value is obtained based on the energy flux density of the incident light; superposing a virtual polaroid on the pixel points with the signal-to-noise ratio greater than a threshold value on the target image, and calculating an initial gray value based on the energy flux density after passing through the virtual polaroid; and performing overexposure optimization processing on the basis of the initial gray value to obtain an enhanced target image. The method is applied to the field of image processing, polarized light is suppressed through virtual polarization filtering, effective enhancement of the target image is achieved, the target area on the image is clearer and visible, and the negative influence on target detection under the complex illumination condition can be remarkably reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image enhancement method and system based on virtual polarization filtering. Background Art

[0002] Additive Manufacturing technology, also known as "3D printing technology", originated in the 1960s. After decades of development, additive manufacturing technology has become an important part of aerospace, medical and automotive fields. In the AM process, a high-energy laser melts the powder layer to form a high-temperature molten pool, which then solidifies into the desired shape after cooling. Different preparation conditions will directly affect the shape of the molten pool. Improper processing parameter settings will lead to unstable molten pools, which in turn induce defects such as convex hulls, pores, and cracks. These defects will reduce the mechanical properties and molding density of the workpiece, causing the physical properties of the workpiece to deteriorate, seriously affecting the actual performance and service life of the workpiece. Therefore, it is crucial to effectively detect defective workpieces and avoid them from being put into use. Traditional non-destructive defect detection methods include radiographic testing, ultrasonic testing, liquid penetrant testing, magnetic particle testing, and eddy current testing. However, these methods are either costly, complex to operate, and slow to detect, or have specific restrictions on the objects to be detected, and have a small scope of application. Given the current expanding and flexible market, these traditional methods may not be able to fully meet production needs in some cases. Therefore, it is necessary to propose a more efficient and low-cost defect detection method.

[0003] In recent years, deep neural network technology has developed rapidly and has achieved remarkable results in the field of defect detection. In 1998, LeCun proposed LeNet, which successfully solved the handwritten digit recognition task, marking the real launch of convolutional neural network (CNN) and promoting the target detection algorithm into the deep learning stage. In 2013, Ross Girshick et al. proposed R-CNN, which applied convolutional neural network to feature extraction and improved the detection rate of data sets through the excellent feature extraction ability of CNN. In 2015, He Kaiming proposed ResNet, which broke through the limitation of neural network depth and further improved the detection accuracy. In the same year, Joseph Redmon et al. proposed YOLO, which uses a unified neural network to simultaneously predict the target position and category in the image, achieving fast and efficient target detection. In the same year, Shaoqing Ren et al. proposed Faster R-CNN, which improved the speed of training and testing stages and also improved the detection accuracy. In 2016, Gao Huang et al. proposed DenseNet, which established the connection relationship between different layers, thereby further alleviating the gradient vanishing problem and improving the accuracy of target detection. These neural network-based studies have greatly promoted the development of defect detection technology. They have improved the accuracy and processing speed of the models, allowing the detection system to more effectively identify and classify various surface defects. These studies have not only improved the performance and reliability of surface defect detection, but also expanded its scope of practical applications.

[0004] Although there are many excellent methods currently applied in the field of surface defect detection, their performance is often limited by different application scenarios when facing specific practical problems, and they fail to fully realize their potential. For example, in the detection of surface defects of workpieces, the detection accuracy of these algorithm models is usually significantly affected by complex environmental lighting. However, in the actual production process, the lighting environment is complex and changeable, which requires different detection models to cope with the changing environment, increasing the cost and difficulty. Summary of the invention

[0005] In view of the deficiencies in the above-mentioned prior art, the present invention provides an image enhancement method and system based on virtual polarization filtering, which suppresses polarized light through virtual polarization filtering, thereby effectively enhancing the target image, making the target area on the image more clearly visible, and significantly reducing the negative impact on target detection under complex lighting conditions.

[0006] To achieve the above object, the present invention provides an image enhancement method based on virtual polarization filtering, comprising the following steps:

[0007] Based on N camera systems with different polarization characteristics and imaging simultaneously, the target area is imaged to obtain N target polarization images, where N ≥ 3;

[0008] Construct a target image with the same size as each target polarization image, and calculate the Stokes vector of the incident light at the pixel point at the corresponding position on the target image based on the grayscale value of the pixel point at the same position on the three target polarization images;

[0009] Based on the Stokes vector of the incident light at each pixel point on the target image, the energy flux density s of the incident light at each pixel point on the target image is calculated. p0 ;

[0010] Calculate the signal-to-noise ratio of each pixel on the target image based on the Stokes vector:

[0011] For pixels on the target image whose signal-to-noise ratio is less than or equal to the threshold K, the energy flux density s of the incident light p0 Calculate and obtain the initial grayscale value of the corresponding pixel;

[0012] For pixels on the target image whose signal-to-noise ratio is greater than the threshold K, a block with an angle of After the virtual polarizer is inserted, the energy flux density of the light passing through the virtual polarizer is obtained. Based on the energy density Calculate the initial grayscale value of the corresponding pixel, where A OP is the polarization angle of the corresponding pixel point on the target image;

[0013] Based on the initial grayscale value, overexposure optimization processing is performed to obtain the final grayscale value of each pixel on the target image, that is, the enhanced target image.

[0014] In one embodiment, when N>3, the Stokes vector of the incident light at the pixel point at the corresponding position on the target image is calculated based on the grayscale value of the pixel point at the same position on the three target polarization images, specifically:

[0015] The three smallest values ​​are selected from the N grayscale values ​​at the corresponding positions on the N target polarization images, and the Stokes vector of the incident light at the pixel point at the corresponding position on the target image is calculated.

[0016] In one embodiment, the Stokes vector of the incident light at each pixel point on the target image is specifically:

[0017]

[0018] Where S(λ) represents the Stokes vector of the incident light at the pixel point corresponding to the target image, I represents the power of the incident light at the pixel point corresponding to the target image, α represents the gain of the camera system, h represents the Planck constant, c represents the speed of light, λ represents the wavelength of the incident light at the pixel point corresponding to the target image, f(λ) represents the quantum efficiency function of the camera system, and ADU1 , A DU2 , A DU3 Represents the grayscale value of the pixel at the same position on the three selected target polarization images;

[0019] W represents a reversible matrix composed of the element values ​​of the Mueller matrix of multiple camera systems, specifically:

[0020]

[0021] Among them, m 111 、m 112 、m 113 Indicates the gray value A DU1 The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image, m 211 、m 212 、m 213 Indicates the gray value A DU2 The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image, m 311 、m 312 、m 313 Indicates the gray value A DU3 The first row of the Mueller matrix of the camera system corresponding to the target polarization image.

[0022] In one embodiment, the energy flux density The energy flux density s p0 The calculation process is:

[0023]

[0024] Among them, S p (λ, Δt) represents the Stokes vector of the energy flux density of the incident light at the corresponding pixel point of the target image, and Δt represents the exposure time of the camera system;

[0025]

[0026] The energy flux density s p0 That is the energy flux density Stokes vector S p The first element of (λ, Δt).

[0027] In one embodiment, the initial grayscale value of each pixel on the target image is specifically:

[0028]

[0029] in, represents the initial grayscale value of the corresponding pixel of the target image, Δt′ represents the configured exposure time set on the computer, and S NRIt represents the signal-to-noise ratio of the pixel point corresponding to the target image, s0 represents the intensity of the incident light at the pixel point corresponding to the target image, and s1 and s2 represent the linear polarization components of the incident light at the pixel point corresponding to the target image.

[0030] In one embodiment, the final grayscale value of each pixel on the target image is specifically:

[0031]

[0032] in, Indicates the final grayscale value of the corresponding pixel in the target image.

[0033] To achieve the above object, the present invention also provides an image enhancement system based on virtual polarization filtering, which uses the above image enhancement method to perform image enhancement processing.

[0034] In one embodiment, the image enhancement system comprises:

[0035] An imaging module, comprising N camera systems with different polarization characteristics and imaging simultaneously, for imaging a target area to obtain N target polarization images, wherein N ≥ 3;

[0036] An incident light calculation module is used to calculate the Stokes vector of the incident light at the pixel point at the corresponding position on the target image according to the grayscale value of the pixel point at the same position on the three target polarization images;

[0037] The energy flux density calculation module is used to calculate the energy flux density s of the incident light at each pixel point on the target image according to the Stokes vector of the incident light at each pixel point on the target image. p0 ;

[0038] A signal-to-noise ratio calculation module, used to calculate the signal-to-noise ratio of each pixel on the target image according to the Stokes vector of the incident light at each pixel;

[0039] The first gray value calculation module is used to calculate the energy flux density s of the incident light when the signal-to-noise ratio is less than or equal to the threshold K. p0 Calculate and obtain the initial grayscale value of the corresponding pixel point on the target image;

[0040] The second gray value calculation module is used to superimpose a gray value with an angle of After the virtual polarizer is inserted, the energy flux density of the light passing through the virtual polarizer is obtained. Then, based on the energy flux density Calculate and obtain the initial grayscale value of the corresponding pixel point on the target image;

[0041] The overexposure optimization module is used to perform overexposure optimization processing based on the initial grayscale value to obtain the final grayscale value of each pixel on the target image, thereby obtaining an enhanced target image.

[0042] Compared with the prior art, the present invention has the following beneficial technical effects:

[0043] The present invention suppresses polarized light through virtual polarization filtering, and at the same time, screens target pixel points of virtual polarization filtering based on the signal-to-noise ratio during the virtual polarization filtering process, thereby avoiding suppression of information of interest in the image and only suppressing noise information in the image. At the same time, overexposure optimization processing is performed to avoid calculating erroneous information, thereby ensuring that the enhanced target image is always better than the input target polarization image, achieving effective enhancement of the target image, making the target area on the image more clearly visible, and significantly reducing the negative impact on target detection under complex lighting conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0045] Figure 1 is a flow chart of the image enhancement method in Embodiment 1 of the present invention;

[0046] Figure 2 This is a schematic diagram of the MPIS structure in Example 1 of the present invention;

[0047] Figure 3 is an example flow chart of image enhancement in Embodiment 1 of the present invention;

[0048] Figure 4 Schematic diagram of the virtual polarization filter input, output and grayscale histogram in Example 1 of the present invention, wherein: (A) is a 0° target polarization image and its grayscale histogram, (B) is a 45° target polarization image and its grayscale histogram, (C) is a 90° target polarization image and its grayscale histogram, (D) is a 135° target polarization image and its grayscale histogram, and (E) is an enhanced target image and its grayscale histogram;

[0049] Figure 5 Schematic diagram of the combination of virtual polarization filter input, output and grayscale histogram in Example 1 of the present invention;

[0050] Figure 6 This is a structural block diagram of the image enhancement system in Example 2 of the present invention.

[0051] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0054] Example 1

[0055] Polarization imaging technology has unique advantages over traditional optical detection methods. It can obtain spectral information, polarization information and spatial information of the detection target. It is currently widely used in industrial imaging, remote sensing, biomedical diagnosis and other fields. Defect detection based on polarization technology is conducive to extracting information such as texture structure, surface material and surface roughness from the polarization information of the detection target, which can effectively improve the accuracy and reliability of detection. To this end, this embodiment discloses an image enhancement method based on virtual polarization filtering, which suppresses polarized light through virtual polarization filtering. At the same time, in the process of virtual polarization filtering, the target pixel points of the virtual polarization filter are screened based on the signal-to-noise ratio, so as to avoid suppressing the information of interest in the image and only suppress the noise information in the image. At the same time, overexposure optimization processing is used to avoid solving erroneous information, so as to ensure that the enhanced target image is always better than the input target polarization image, achieve effective enhancement of the target image, make the target area on the image more clearly visible, and significantly reduce the negative impact on target detection under complex lighting conditions. Reference Figure 1 In this embodiment, the image enhancement method based on virtual polarization filtering specifically includes the following steps:

[0056] Based on N camera systems with different polarization characteristics and imaging simultaneously, the target area is imaged to obtain N target polarization images, where N ≥ 3;

[0057] Construct a blank target image with the same size as each target polarization image, and calculate the Stokes vector of the incident light at the pixel point at the corresponding position on the target image based on the grayscale value of the pixel point at the same position on the three target polarization images;

[0058] Based on the Stokes vector of the incident light at each pixel point on the target image, the energy flux density s of the incident light at each pixel point on the target image is calculated. p0 ;

[0059] Calculate the signal-to-noise ratio of each pixel on the target image based on the Stokes vector:

[0060] For pixels on the target image whose signal-to-noise ratio is less than or equal to the threshold K, the energy flux density s of the incident light p0 Calculate and obtain the initial grayscale value of the corresponding pixel;

[0061] For pixels on the target image whose signal-to-noise ratio is greater than the threshold K, a block with an angle of After the virtual polarizer is inserted, the energy flux density of the light passing through the virtual polarizer is obtained. Based on the energy density Calculate the initial grayscale value of the corresponding pixel, where A OP is the polarization angle of the corresponding pixel point on the target image;

[0062] Based on the initial grayscale value, overexposure optimization processing is performed to obtain the final grayscale value of each pixel on the target image, that is, the enhanced target image.

[0063] In nature, light waves are electromagnetic waves, and their electric fields usually propagate in the form of plane sinusoidal waves. Therefore, the electric field E(x, t) of any beam of light propagating along the x-axis can be expressed as:

[0064]

[0065] Where Re[·] represents the real part, E represents the electric field amplitude, j represents the imaginary unit, ω represents the electric field angular frequency, t represents the electric field propagation time, k represents the wave number, x represents the distance vector from the light source to the observation point (imaging distance vector), ψ represents the initial phase of the electric field, and c represents the speed of light. represents the x-axis unit vector, is the unit vector of the electric field vibration direction, that is, the polarization direction of light;

[0066] Taking each pixel in the image as the research object, and assuming that each pixel of the image is only affected by sufficiently small points in the outside world, and is not affected by reflected light from other areas of the outside world, it can be considered that the angular frequency, propagation time, and imaging distance of the imaging light wave of each pixel are the same, so the electric field of the light wave can be expressed as a phasor form for:

[0067]

[0068] The external environment of the electric field is idealized as a vacuum environment. From the electromagnetic field theory, we know that the electric field and its induced magnetic field Satisfies the Ampere-Maxwell law, that is:

[0069]

[0070] Among them, ▽ represents the gradient operator, μ0 represents the vacuum magnetic permeability, J represents the current density, and ε0 represents the vacuum dielectric constant;

[0071] Since it is a vacuum environment, J = 0, and taking the curl on both sides of equation (4), we have:

[0072]

[0073] in, represents the direction of magnetic field vibration, and solving the differential equation yields:

[0074]

[0075] Therefore, the energy flux density S of the electromagnetic field of the light wave is n for:

[0076]

[0077] Then the average power P of the light wave is:

[0078]

[0079] Where T represents the period of the light wave.

[0080] On the other hand, starting from the morphology of photons, the relationship between the grayscale value of the pixel and the average power of the light wave is analyzed. DU and the number of photoelectrons N E The functional relationship can be expressed as:

[0081] N E =α·(A DU -A DU0 ) (9)

[0082] Among them, α represents the gain of the camera, A DU0 Indicates the camera bias;

[0083] The quantum efficiency of a camera can effectively convert the number of incident photons N L and the number of photoelectrons N E The relationship is described as:

[0084] N E =f(λ)·N L (10)

[0085] Where f(λ) represents the quantum efficiency function of the camera system, and λ represents the wavelength of the incident light at the corresponding pixel point of the target image;

[0086] Combine equation (9) and equation (10) and substitute into the formula available:

[0087]

[0088] Where Δt represents the exposure time of the camera system, h represents Planck's constant, and c represents the speed of light;

[0089] Since the camera bias is often a small value, this embodiment ideally processes it to 0, and the following is obtained:

[0090]

[0091] From formula (12), we can see that the gray value of the image is a quantity related to the wavelength of light, the average power of light, and the exposure time (there is no boundary condition between these three variables). From formula (7), we can see that while obtaining the energy flux density of light, the polarization information of the incident light is also obtained. has been eliminated, which means that even if the wavelength of the incident light, the exposure time, and the single grayscale information are known, the polarization state of the incident light cannot be inferred. Furthermore, regardless of the polarization state of the incident light, under certain conditions, the grayscale value displayed by the image is certain. Therefore, in this embodiment, the polarization state of the virtual incident light is changed by using virtual polarization filtering without affecting the original information of the image. It is theoretically feasible to enhance the image by suppressing polarized light.

[0092] In order to facilitate the description of the polarization state of light, the present embodiment uses the Stokes vector S = (s0, s1, s2, s3) T The light is represented in the form of . Among them, component s0 represents the light intensity, components s1 and s2 represent the linear polarization components, component s3 represents the circular polarization component, and T represents the matrix. After the light passes through the optical system, its polarization state will change. The 4×4 Mueller matrix can describe this change. Assume that the Mueller matrix of the i-th camera system is M i , then:

[0093]

[0094] Among them, S i represents the Mueller matrix after the light passes through the i-th camera system, m i11 、m i12 、m i13 、m i14 、m i21 、m i22 、mi23 、m i24 、m i31 、m i32 、m i33 、m i34 、m i41 、m i42 、m i43 、m i44 represents the elements of the Mueller matrix of the i-th camera system;

[0095] Since the circular polarization component s3 of natural light is usually 0, equation (12) can be reduced to:

[0096]

[0097] Taking a certain pixel point of the image obtained after imaging as the research object, we can get:

[0098]

[0099] Among them, s i0 =P i Δt i Indicates the power received by the camera system lens area corresponding to the pixel point, A DUi Represents the gray value of the pixel, λ i Indicates the wavelength of the incident light corresponding to the pixel;

[0100] Therefore, the light intensity I received by three camera systems with different polarization characteristics and simultaneous imaging in the same area can be written in the following matrix form:

[0101]

[0102] Among them, s 10 、s 20 、s 30 represents the light intensity received by the three camera systems, W represents the reversible matrix composed of the element values ​​of the Mueller matrix of multiple camera systems, m 111 、m 112 、m 113 Indicates the gray value A DU1 The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image, m 211 、m 212 、m 213 Indicates the gray value A DU2 The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image, m 311 、m 312 、m 313 Indicates the gray value A DU3 The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image;

[0103] Since the three camera systems are imaging at the same time, λ1=λ2=λ3=λ. When the W matrix is ​​reversible, the Stokes vector of the incident light at the corresponding position of each pixel on the target image can be solved by the grayscale values ​​of the three target polarization images, that is:

[0104]

[0105] Among them, S(λ) represents the Stokes vector of the incident light at the corresponding pixel point of the target image, A DU1 , A DU2 , A DU3 Represents the grayscale value of the pixel at the same position on the three selected target polarization images;

[0106] Assuming that the exposure time of the three imaging systems is the same, that is, Δt1=Δt2=Δt3=Δt, the energy flux density Stokes vector of the incident light at the corresponding pixel point of the target image is defined as:

[0107]

[0108] Among them, S p (λ, Δt) represents the Stokes vector of the energy flux density of the incident light at the corresponding pixel point of the target image;

[0109] According to the first three items in the Stokes vector, the polarization angle A of each pixel in the target image can be obtained. OP Information, expressed as:

[0110]

[0111] For a single pixel, it is clear that A OP is a quantity that is independent of λ and Δt. At this time, a block with an angle of The virtual polarizer can effectively suppress polarized light, and the reduced-dimensional Mueller matrix of the virtual polarizer is:

[0112]

[0113] The energy flux density of light after passing through the virtual polarizer is:

[0114]

[0115] Since the wavelength of light does not change after passing through the polarizer, the pixel suppresses the initial gray value of the polarized light. for:

[0116]

[0117] Among them, Δt′ represents the exposure time configured by the computer. By changing its value, the exposure of the image can be changed. From formula (22), we can see that the initial gray value is a quantity that is independent of λ. Therefore, the image enhancement method in this embodiment does not require information such as the wavelength of the incident light and the exposure time, and can be based on the input target polarization image and the set exposure time ratio. By performing virtual polarization filtering, the desired target image can be obtained.

[0118] In the specific application process, if virtual polarization filtering is performed on each pixel in the target image, the information of interest in the image may be suppressed. In this embodiment, non-polarized light is determined as information and polarized light is determined as noise. Therefore, when the signal-to-noise ratio of a pixel on the target image is greater than the threshold K, it can be considered that the pixel is richer than the noise and does not need to be virtual polarization filtered. Therefore, in the specific implementation process, only the pixel with a signal-to-noise ratio greater than the threshold K is virtual polarization filtered, and other pixel points are not virtual polarization filtered. That is, the initial grayscale value of each pixel on the target image is specifically:

[0119]

[0120]

[0121] Among them, S NR Indicates the signal-to-noise ratio of the corresponding pixel in the target image.

[0122] In the above calculation process, both in the calculation of the Stokes vector part and in the update of the pixel gray value part, it depends on formula (12). However, formula (12) is only valid when the image is not overexposed. When the input image is overexposed, erroneous information may be generated, thereby reducing the imaging effect and affecting the accuracy of subsequent target detection. Since it is difficult to avoid the overexposure of the image under complex lighting conditions, this embodiment performs overexposure optimization processing to ensure that the enhanced target image is always better than the input target polarization image.

[0123] Since the circularly polarized light component in nature is usually zero, the light wave can be regarded as a combination of unpolarized light and linearly polarized light. The relationship between its reduced-dimensional Stokes vector can be expressed as:

[0124]

[0125] Among them, S u is the reduced-dimensional Stokes vector of unpolarized light, S p is the reduced-dimensional Stokes vector of polarized light;

[0126] The polarization angle of the linear polarization part is denoted as δ1, which can be obtained by formula (19); the angle of the virtual polarizer is denoted as δ2, and the light intensity s′0 passing through the virtual polarizer is:

[0127]

[0128] If the input image is overexposed, there are two cases for discussion below:

[0129] If u0 >>s p0 ,but At this time, all the input images must be overexposed, and the method of this embodiment cannot extract any information from them. The initial grayscale value of the output is 255;

[0130] If u0 Not much larger than s p0 : At this time, the input image may contain an image that is not overexposed. If so, the initial grayscale value calculated by the method of this embodiment is Must be greater than the gray value A without overexposure DUi The product of the exposure time ratio does not play a filtering role, so we can make To achieve overexposure processing.

[0131] Therefore, the process of overexposure optimization processing in this embodiment is specifically as follows:

[0132]

[0133] in, Indicates the final grayscale value of the corresponding pixel of the target image;

[0134] After calculating the final grayscale value of each pixel of the target image, the final target image is obtained, and the image enhancement operation is completed.

[0135] As a preferred implementation, when the number of camera systems with different polarization characteristics and imaging simultaneously is four or more, that is, when N>3, the Stokes vector of the incident light at the pixel point at the corresponding position on the target image is calculated based on the grayscale values ​​of the pixel points at the same position on the three target polarization images. Specifically, the three smallest values ​​are selected from the N grayscale values ​​at the corresponding positions on the N target polarization images, and the Stokes vector of the incident light at the pixel point at the corresponding position on the target image is calculated to avoid selecting overexposed pixel grayscale values ​​as much as possible.

[0136] The image enhancement method based on virtual polarization filtering in this embodiment is further described below with reference to specific examples.

[0137] In this example, use Figure 2The multi-source polarization imaging system (MPIS) shown in the figure collects image data. The imaging system mainly includes the workpiece to be measured, LED light source, linear polarizer, electric rotating frame, CMOS camera and computer. Among them, the linear polarizer uses THORLABS's LPVISE100-A linear polarizer, which has a size of 25.4 mm and an operating wavelength range of 400-700 nm; the resolution of the CMOS image sensor is 2448×2048, the single pixel size is 3.45 μm, and the lens focal length is 50 mm; the electric rotating frame uses Standa's FPSTA-8MPR16-1, which can achieve 360° rotation angle and 0.75 arcmin step resolution polarizer rotation control, and its equipped controller is 8SMC4-USB. The computer model used for image acquisition and analysis in the experimental example is ThinkPad S2, the CPU is i58250U, and its maximum frequency is 1.8 GHz. During the experimental example, the workpiece to be measured is illuminated by an LED light source, and the light is reflected by the workpiece surface and then collected by the CMOS image sensor through the polarizer.

[0138] refer to Figure 3 , which is the process of this example, which mainly includes data acquisition, image sampling, input determination, and solving S NR , Solve And output results in six steps. Specifically, the detailed method of each step is as follows:

[0139] Step 1: MPIS collects images of defective objects and obtains four polarization images of the target at different directions of 0°, 90°, 45° and 135° under the same viewing angle;

[0140] Step 2, the computer receives four target polarization images as input, and uses the opencv library to sample them to obtain the corresponding gray value matrix;

[0141] Step 3, select the pixel points to be processed on the target image, and select the three with the smallest values ​​from the four gray value matrices to form the input;

[0142] Step 4: Determine the matrix W and solve S based on the three grayscale values ​​input NR ;

[0143] Step 5: According to the principle of the method of this embodiment, calculate the new final gray value of the pixel point

[0144] Step 6: If all pixels in the target image have been processed, the enhanced target image is output; otherwise, return to step 3.

[0145] This example evaluates the experimental results by analyzing the grayscale histogram of the image. The grayscale histogram can effectively display the pixel frequency of different grayscale levels in the image, which helps to analyze the contrast, brightness and dynamic range of the image. In this example, according to the principle of the method of this embodiment, most of the pixels in the image are subjected to virtual polarization filtering in the optimal polarization direction. The expected result is that most of the low and medium grayscale value areas in the grayscale histogram of the ideal output image will show a downward trend. This feature shows that the grayscale value of the low contrast area is reduced by virtual polarization filtering, thereby suppressing the background expression and improving the visual quality and feature performance of the image.

[0146] refer to Figure 4 , Figure 5 , are the experimental results and histogram of image enhancement based on virtual polarization filtering in this example, and Table 1 shows the distribution of each gray value interval of each image.

[0147] Table 1

[0148]

[0149] Depend on Figure 4 , Figure 5 As shown in Table 1, after image enhancement, the number of pixels with grayscale values ​​greater than 204 is reduced by two orders of magnitude compared with the images with polarization directions of 0°, 45°, and 90°, showing a significant effect of suppressing overexposure. Compared with the image with polarization direction of 135°, the number of pixels with grayscale values ​​greater than 152 has changed little, only decreasing by about 7%. However, in the low grayscale value area, the number of pixels with grayscale values ​​in the range of [0,50] increased by 33%, while the number of pixels with grayscale values ​​in the range of [51,101] decreased by 25%. This shows that the enhanced image maintains relatively stable features in the high grayscale value area, while the overall grayscale level in the low grayscale value area is reduced, thereby effectively suppressing background noise. This processing method helps to improve the sensitivity and accuracy of the model in defect detection tasks, because the reduction of background noise can highlight the characteristics of the target area, making the identification of defects clearer.

[0150] It is worth noting that although this embodiment Figure 1 The steps in the process are shown in sequence as indicated by the arrows, but the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of the steps, and the steps can be executed in other orders. Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0151] Example 2

[0152] Based on the image enhancement method based on virtual polarization filtering in Example 1, this embodiment discloses an image enhancement system based on virtual polarization filtering. Figure 6 The image enhancement system includes an imaging module, an incident light calculation module, an energy flux density calculation module, a signal-to-noise ratio calculation module, a first gray value calculation module, a second gray value calculation module, and an overexposure optimization module. Specifically:

[0153] The imaging module includes N camera systems with different polarization characteristics and imaging simultaneously, and is used to image the target area to obtain N target polarization images;

[0154] The incident light calculation module is used to calculate the Stokes vector of the incident light at the pixel point at the corresponding position on the target image according to the gray value of the pixel point at the same position on the three target polarization images;

[0155] The energy flux density calculation module is used to calculate the energy flux density s of the incident light at each pixel point on the target image according to the Stokes vector of the incident light at each pixel point on the target image. p0 ;

[0156] The signal-to-noise ratio calculation module is used to calculate the signal-to-noise ratio of each pixel on the target image according to the Stokes vector of the incident light at each pixel;

[0157] The first gray value calculation module is used for calculating the energy flux density s of the incident light when the signal-to-noise ratio is less than or equal to the threshold K. p0 Calculate and obtain the initial grayscale value of the corresponding pixel point on the target image;

[0158] The second gray value calculation module is used to superimpose a gray value with an angle of After the virtual polarizer is inserted, the energy flux density of the light passing through the virtual polarizer is obtained. Then, based on the energy flux density Calculate and obtain the initial grayscale value of the corresponding pixel point on the target image;

[0159] The overexposure optimization module is used to perform overexposure optimization processing based on the initial grayscale value to obtain the final grayscale value of each pixel on the target image, thereby obtaining an enhanced target image.

[0160] In this embodiment, the specific working process and working principle of the imaging module, the incident light calculation module, the energy flux density calculation module, the signal-to-noise ratio calculation module, the first gray value calculation module, the second gray value calculation module, and the overexposure optimization module are the same as those in the method of Embodiment 1, so they are not described in detail in this embodiment. Each unit module can be implemented in whole or in part by software, hardware, or a combination thereof. Each unit module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above unit modules.

[0161] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. An image enhancement method based on virtual polarization filtering, characterized in that: The steps include: Based on N camera systems with different polarization characteristics and imaging simultaneously, the target area is imaged to obtain N target polarization images, where N ≥ 3; Construct a target image with the same size as each target polarization image, and calculate the Stokes vector of the incident light at the pixel point at the corresponding position on the target image based on the grayscale value of the pixel point at the same position on the three target polarization images; Based on the Stokes vector of the incident light at each pixel point on the target image, the energy flux density s of the incident light at each pixel point on the target image is calculated. p0 ; Calculate the signal-to-noise ratio of each pixel on the target image based on the Stokes vector: For pixels on the target image whose signal-to-noise ratio is less than or equal to the threshold K, the energy flux density s of the incident light p0 Calculate and obtain the initial grayscale value of the corresponding pixel; For pixels on the target image whose signal-to-noise ratio is greater than the threshold K, a block with an angle of After the virtual polarizer is inserted, the energy flux density of the light passing through the virtual polarizer is obtained. Based on the energy density Calculate the initial grayscale value of the corresponding pixel, where A OP is the polarization angle of the corresponding pixel point on the target image; Based on the initial grayscale value, overexposure optimization processing is performed to obtain the final grayscale value of each pixel on the target image, that is, the enhanced target image.

2. The image enhancement method based on virtual polarization filtering according to claim 1, characterized in that: When N>3, the Stokes vector of the incident light at the pixel point at the corresponding position on the target image is calculated based on the grayscale value of the pixel point at the same position on the three target polarization images, specifically: The three smallest values ​​are selected from the N grayscale values ​​at the corresponding positions on the N target polarization images, and the Stokes vector of the incident light at the pixel point at the corresponding position on the target image is calculated.

3. The image enhancement method based on virtual polarization filtering according to claim 1 or 2, characterized in that: The Stokes vector of the incident light at each pixel point on the target image is specifically: Where S(λ) represents the Stokes vector of the incident light at the pixel point corresponding to the target image, I represents the power of the incident light at the pixel point corresponding to the target image, α represents the gain of the camera system, h represents the Planck constant, c represents the speed of light, λ represents the wavelength of the incident light at the pixel point corresponding to the target image, f(λ) represents the quantum efficiency function of the camera system, and A DU1 , A DU2 , A DU3 Represents the grayscale value of the pixel at the same position on the three selected target polarization images; W represents a reversible matrix composed of the element values ​​of the Mueller matrix of multiple camera systems, specifically: Among them, m 111 、m 112 、m 113 Indicates the gray value A DU1 The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image, m 211 、m 212 、m 213 Indicates the gray value A DU2 The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image, m 311 、m 312 、m 313 Indicates the gray value A DU3 The first row of the Mueller matrix of the camera system corresponding to the target polarization image.

4. The image enhancement method based on virtual polarization filtering according to claim 3, characterized in that: The energy flux density The energy flux density s p0 The calculation process is: Among them, S p (λ, Δt) represents the Stokes vector of the energy flux density of the incident light at the corresponding pixel point of the target image, and Δt represents the exposure time of the camera system; The energy flux density s p0 That is the energy flux density Stokes vector S p The first element of (λ, Δt).

5. The image enhancement method based on virtual polarization filtering according to claim 4, characterized in that: The initial grayscale value of each pixel on the target image is specifically: in, represents the initial grayscale value of the corresponding pixel of the target image, Δt′ represents the configured exposure time set on the computer, and S NR It represents the signal-to-noise ratio of the pixel point corresponding to the target image, s0 represents the intensity of the incident light at the pixel point corresponding to the target image, and s1 and s2 represent the linear polarization components of the incident light at the pixel point corresponding to the target image.

6. The image enhancement method based on virtual polarization filtering according to claim 5, characterized in that: The final grayscale value of each pixel on the target image is specifically: in, Indicates the final grayscale value of the corresponding pixel in the target image.

7. An image enhancement system based on virtual polarization filtering, characterized in that: The image enhancement system adopts the image enhancement method described in any one of claims 1 to 6.

8. The image enhancement system based on virtual polarization filtering according to claim 7, characterized in that: The image enhancement system comprises: An imaging module, comprising N camera systems with different polarization characteristics and imaging simultaneously, for imaging a target area to obtain N target polarization images, wherein N ≥ 3; An incident light calculation module is used to calculate the Stokes vector of the incident light at the pixel point at the corresponding position on the target image according to the grayscale value of the pixel point at the same position on the three target polarization images; The energy flux density calculation module is used to calculate the energy flux density s of the incident light at each pixel point on the target image according to the Stokes vector of the incident light at each pixel point on the target image. p0 ; A signal-to-noise ratio calculation module, used to calculate the signal-to-noise ratio of each pixel on the target image according to the Stokes vector of the incident light at each pixel; The first gray value calculation module is used to calculate the energy flux density s of the incident light when the signal-to-noise ratio is less than or equal to the threshold K. p0 Calculate and obtain the initial grayscale value of the corresponding pixel point on the target image; The second gray value calculation module is used to superimpose a gray value with an angle of After the virtual polarizer is inserted, the energy flux density of the light passing through the virtual polarizer is obtained. Then, based on the energy flux density Calculate and obtain the initial grayscale value of the corresponding pixel point on the target image; The overexposure optimization module is used to perform overexposure optimization processing based on the initial grayscale value to obtain the final grayscale value of each pixel on the target image, thereby obtaining an enhanced target image.

Images