Method for separating aliasing stripes of transparent element in deflection measurement

Abstract

The invention relates to the technical field of optical measurement, and particularly discloses a method for separating aliasing stripes of a transparent element in deflection measurement, which comprises the following steps: acquiring an initial value of background light intensity; projecting the binary projection circular spot array image in the screen on the transparent element to be detected, and obtaining a binary reflection circular spot array image; storing the relationship between each camera pixel point and the corresponding screen pixel point; acquiring an aliasing stripe image of the transparent element to be detected; calculating a first phase initial value and a first modulation degree initial value of the upper surface reflection fringe image and a second phase initial value and a second modulation degree initial value of the lower surface reflection fringe image; optimizing the background light intensity initial value, the first phase initial value, the second phase initial value, the first modulation degree initial value and the second modulation degree initial value through a neural network; and respectively generating an upper surface reflection fringe image and a lower surface reflection fringe image according to the five optimized initial values. According to the invention, the separated fringe pattern can be rapidly and accurately obtained.

Description

Technical Field

[0001] The present invention relates to the field of optical measurement technology, and more particularly to a method for separating aliased fringes of a transparent element in deflection measurement. Background Art

[0002] Transparent components play an important role in all aspects of social production and life. In the field of optics, transparent components such as lenses, filters and prisms are widely used to manufacture various optical instruments and equipment, such as cameras, telescopes, VR / AR and lasers. In addition, transparent components are also widely used in industries such as automobiles, electronics, aerospace and medical devices. For example, high-precision transparent components such as objective lenses and eyepieces are key components of high-end cameras and projectors. Transparent components used in instruments such as spectrometers and laser interferometers help scientists accurately measure and analyze the characteristics of light. In the field of aerospace, the windshield (windshield cover) of the aircraft cockpit needs to have high strength and excellent transparency to provide pilots with a good field of vision while resisting high pressure, wind speed and temperature changes during flight. Windows on passenger aircraft and spacecraft are required to be transparent, strong and able to withstand extreme environmental conditions, and their surface shape is very important for assembly. In summary, transparent components are an indispensable part of modern industry and scientific research. The surface accuracy of transparent components is crucial to improving the performance and stability of these devices. Therefore, the requirements for the measurement accuracy of the surface shape and quality of transparent components are becoming more and more stringent. Traditional contact measurement methods mainly include three-dimensional coordinate measuring machines and contact point scanning profilometers. Their measurement results are stable and reliable, with high measurement accuracy, and are suitable for measuring various transparent materials. However, contact measurement methods will cause irreversible scratches and damage to the surface of the component, and the volume is so large that it cannot be measured online. Non-contact measurement methods do not require contact with the measurement surface. Common interferometry methods include white light interferometry and laser interferometry, but these methods cannot measure very complex surface shapes. Deflectometry is a non-contact optical measurement technology suitable for measuring the surface shape of transparent components. This method displays the encoded sinusoidal fringe pattern on the screen, uses the surface to be measured to deflect the light emitted from the screen, and the camera captures the deformed fringes after deflection. The surface shape information is inferred based on the pixel correspondence between the deformed fringes and the fringes displayed on the screen. Deflectometry technology has the advantages of high precision, fast speed, and applicability to complex shapes and large-sized components. Therefore, it is widely used in optical manufacturing and optical detection.

[0003] However, when measuring transparent elements by deflectometry, the fringe images reflected from the front and back surfaces are superimposed, so it is difficult to establish pixel correspondence. In addition, the low reflectivity of transparent elements results in a very low signal-to-noise ratio of the captured image, which reduces the measurement accuracy. The original method [D. Sprenger, C. Faber, MC et al. UV-deflectometry: noparasitic reflections, Proc. DGaO, 2010, 111: A19] uses an ultraviolet light source and uses the opaque property of glass to the ultraviolet light source to isolate the lower surface reflection, but the ultraviolet display required by this method is expensive and difficult to obtain. [Z. Hao, Y. Liu, Transparent object shape measurement based on deflectometry, Proceedings, 2018, 2: 548.], Hao et al. used binocular stereo vision to measure the normal direction of transparent objects. The surface shape of the transparent unit to be measured can be calculated using the consistency constraint of the stereo normal vector. However, due to the limitations of the binocular method, its measurement accuracy is low and introduces additional uncertainty. The current mainstream fringe separation method is the multi-frequency phase-shifting deflection measurement method [Ye J, Niu Z, Zhang X, et al. Simultaneous measurement of double surfaces of transparent lenses with phase measuring deflectometry [J]. Optics and Lasers in Engineering, 2021, 137: 106356.], [Tao S, Yue H, Chen H, et al. Elimination of parasitic reflections for objects with high transparency in phase measuring deflectometry [J]. Results in Physics, 2019, 15: 102734.], [Leung YC, Cai L. Untangling parasitic reflection in phase measuring deflectometry by multi-frequency phase-shifting [J]. Applied Optics, 2022, 61 (1): 208-222.], which displays a series of sinusoidal fringe patterns with different frequencies on the screen in sequence, combines the phase shift data and establishes a model, and uses a nonlinear optimization method to solve it.However, this method is very time-consuming and cannot disentangle very thin transparent planes. To this end, Zheng et al. [Zheng W, Li D, Wang R, et al. Parasitic reflection separation deflectometry based on harmonic analysis [J]. Measurement, 2022, 203: 111864.] proposed a parasitic reflection separation deflection method based on harmonic analysis. By analyzing the harmonic components in the power spectrum, the parasitic reflection of thin transparent devices can be disentangled, but only for parallel transparent parts. Due to the low reflectivity of the signal reflected from the lower surface, the intensity of the light entering the camera after three reflections is about one thousandth of the intensity of the light entering the camera after a single reflection. Therefore, the above methods all treat the stripes reflected from the lower surface as ghost images and eliminate them, which also reduces the measurement efficiency of transparent elements. Ye et al. [Ye J, Niu Z, Zhang X, et al. Simultaneous measurement of double surfaces of transparent lenses with phase measuring deflectometry[J]. Optics and Lasers in Engineering, 2021, 137: 106356.] proposed a multi-frequency phase-shifted fringe dealiasing method that can achieve simultaneous measurement of upper and lower surface fringes. However, this method has high requirements on the setting of the initial phase value of the fringes, and phase jumps may occur.

[0004] Neural networks are widely used in learning the phase information of fringe images due to their superior ability to handle nonlinear problems and adaptability [Wang, K., et al. One-step robust deep learning phase unwrapping. Optics Express, 2019, 10: 15100.], [Wang, K., Song, L., Wang, C. et al. On the use of deep learning for phase recovery. Light Sci Appl, 2024, 13: 4.], [Guan J, Li J, Yang X, et al. Error compensation for phase retrieval in deflectometry based on deep learning [J]. Measurement Science and Technology, 2023, 34 (2): 025009.]. The main applications are fringe image detection and recognition, phase information enhancement, phase unwrapping, etc. In surface measurement, neural networks can optimize and enhance some initial parameters through their powerful nonlinear fitting ability. So far, no one has tried to use deep neural networks to optimize initial parameters to achieve transparent element fringe separation and phase decoupling.

[0005] In summary, the following problems remain to be solved in the field of deflection measurement: it is difficult to establish the corresponding relationship between the pixels on the upper and lower surfaces of transparent components, the traditional fringe separation method is slow and has limited application scenarios, and the end-to-end deep learning method has poor reliability. Summary of the invention

[0006] In view of the defects and shortcomings in the prior art, the present invention provides a method for separating aliased fringes of a transparent element in deflection measurement, which can quickly and accurately obtain a separated fringe pattern.

[0007] As a first aspect of the present invention, a method for separating aliased fringes of a transparent element in deflection measurement is provided, comprising the following steps:

[0008] Step S1: construct a deflection measurement system consisting of a camera and a screen, and establish a posture relationship between the camera and the screen;

[0009] Step S2: taking a plurality of black frame images by a camera to obtain an initial value of the background light intensity;

[0010] Step S3: projecting the binary projection circular spot array image on the screen onto the transparent element to be tested, photographing the reflection circular spot array reflected by the upper and lower surfaces of the transparent element to be tested by a camera, and binarizing the photographed reflection circular spot array image to obtain a binary reflection circular spot array image; and calculating the centroid coordinates of each binary reflection circular spot in the binary reflection circular spot array image, and then saving the relationship between the camera pixel point at the centroid of each binary reflection circular spot and the screen pixel point at the centroid of the corresponding binary projection circular spot;

[0011] Step S4: projecting the fringe image on the screen onto the transparent element to be tested, and photographing the aliased fringes reflected by the upper and lower surfaces of the transparent element to be tested by a camera to obtain an aliased fringed image of the transparent element to be tested;

[0012] Step S5: according to the relationship between the camera pixel point at the center of gravity of each binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot, a first phase initial value of the reflection fringe image on the upper surface of the transparent element to be tested and a second phase initial value of the reflection fringe image on the lower surface of the transparent element to be tested are calculated; and the modulation value of each pair of camera pixel points and screen pixel points is optimized to obtain a first modulation initial value of the reflection fringe image on the upper surface of the transparent element to be tested and a second modulation initial value of the reflection fringe image on the lower surface of the transparent element to be tested;

[0013] Step S6: optimizing the background light intensity initial value, the first phase initial value, the second phase initial value, the first modulation degree initial value and the second modulation degree initial value through a neural network to obtain an optimized background light intensity initial value, an optimized first phase initial value, an optimized second phase initial value, an optimized first modulation degree initial value and an optimized second modulation degree initial value;

[0014] Step S7: Generate a reflection fringe image of the upper surface of the transparent element to be tested according to the optimized background light intensity initial value, the optimized first phase initial value and the optimized first modulation initial value, and generate a reflection fringe image of the lower surface of the transparent element to be tested according to the optimized background light intensity initial value, the optimized second phase initial value and the optimized second modulation initial value, so as to achieve separation of the aliased fringe image of the transparent element to be tested.

[0015] Furthermore, the process of taking a plurality of black frame images by a camera to obtain an initial value of the background light intensity further includes:

[0016] The initial value A of the background light intensity init The calculation formula is as follows:

[0017]

[0018] Where n is the black frame image Nds The number of sheets.

[0019] Furthermore, the calculating of the coordinates of the center of gravity of each binary reflection circular spot in the binary reflection circular spot array image and then saving the relationship between the camera pixel point at the center of gravity of each binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot also includes:

[0020] The calculation formula of the centroid coordinates of each binary reflection circular spot is as follows:

[0021]

[0022]

[0023] Among them, C x is the horizontal coordinate of the center of gravity of each binary reflection spot, C y is the ordinate of the centroid of each binary reflection spot, I(x,y) is the pixel point with a pixel value of 1 in the binary reflection spot array image, x is the image abscissa, and y is the image ordinate;

[0024] Determine the correspondence between each binary reflection circular spot and the binary projection circular spot according to the centroid coordinates of each binary reflection circular spot; wherein the binary reflection circular spots in the binary reflection circular spot array image include a first binary reflection circular spot on the upper surface of the transparent element to be tested and a second binary reflection circular spot on the lower surface of the transparent element to be tested;

[0025] The relationship between the camera pixel point at the center of gravity of each first binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot is saved, and the relationship between the camera pixel point at the center of gravity of each second binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot is saved.

[0026] Furthermore, the method of photographing the aliased fringes reflected by the upper and lower surfaces of the transparent element to be tested by a camera to obtain the aliased fringes image of the transparent element to be tested further includes:

[0027] The camera imaging model is defined as follows:

[0028]

[0029] Wherein, I is the pixel value of the aliased fringe image of the transparent element to be tested, A is the background light intensity, B1 is the modulation degree of the reflected fringe image on the upper surface of the transparent element to be tested, B2 is the modulation degree of the reflected fringe image on the lower surface of the transparent element to be tested, is the phase of the reflected fringe image on the upper surface of the transparent element to be tested, It is the phase of the reflected fringe image on the lower surface of the transparent element to be tested.

[0030] Furthermore, the method of calculating the first phase initial value of the reflection fringe image on the upper surface of the transparent element to be tested and the second phase initial value of the reflection fringe image on the lower surface of the transparent element to be tested according to the relationship between the camera pixel point at the center of gravity of each binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot also includes:

[0031] The phase of the camera pixel position in the reflection fringe image on the upper surface of the transparent element to be tested is calculated based on the relationship between the camera pixel at the center of gravity of each first binary reflection circular spot and the screen pixel at the center of gravity of the corresponding binary projection circular spot. The phase of the camera pixel position in the reflection fringe image of the lower surface of the transparent element to be tested is calculated based on the relationship between the camera pixel point at the center of gravity of each second binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot.

[0032] According to the phase of the camera pixel position in the reflection fringe image on the upper surface of the transparent element to be tested

[0033] The first phase initial value of the reflection fringe image on the upper surface of the transparent element to be tested is obtained by fitting According to the phase of the camera pixel position in the reflection fringe image on the lower surface of the transparent element to be tested The second phase initial value of the reflection fringe image of the lower surface of the transparent element to be tested is obtained by fitting

[0034] Furthermore, the optimization of the modulation values ​​of each pair of camera pixels and screen pixels to obtain a first modulation initial value of the reflection fringe image on the upper surface of the transparent element to be tested and a second modulation initial value of the reflection fringe image on the lower surface of the transparent element to be tested also includes:

[0035] The least square method is used to optimize the modulation value of the camera pixel point at the center of gravity of each first binary reflection circular spot and the modulation value of the screen pixel point at the center of gravity of the corresponding binary projection circular spot to obtain the first modulation initial value B of the reflection fringe image on the upper surface of the transparent element to be tested. 1init ; At the same time, the least squares method is used to optimize the modulation value of the camera pixel point at the center of gravity of each second binary reflection circular spot and the modulation value of the screen pixel point at the center of gravity of the corresponding binary projection circular spot to obtain the second modulation initial value B of the reflection fringe image on the lower surface of the transparent element to be tested 2init ; Among them, the formula of least squares method is as follows:

[0036]

[0037] in, is the phase of the camera pixel position in the reflection fringe image on the upper surface of the transparent element to be tested, is the phase of the camera pixel position in the reflected fringe image on the lower surface of the transparent element to be tested, A init is the initial value of the background light intensity.

[0038] Further, the optimization of the background light intensity initial value, the first phase initial value, the second phase initial value, the first modulation degree initial value and the second modulation degree initial value by a neural network to obtain an optimized background light intensity initial value, an optimized first phase initial value, an optimized second phase initial value, an optimized first modulation degree initial value and an optimized second modulation degree initial value also includes:

[0039] The background light intensity initial value, the first modulation degree initial value and the second modulation degree initial value are connected in the channel direction and sent to the target convolutional neural network to obtain an optimized background light intensity initial value, an optimized first modulation degree initial value and an optimized second modulation degree initial value;

[0040] The first phase initial value and the second phase initial value are respectively sent to the target fully connected neural network to obtain an optimized first phase initial value and an optimized second phase initial value.

[0041] Furthermore, the initial value of the background light intensity, the initial value of the first modulation degree, and the initial value of the second modulation degree are connected in the channel direction and sent to the target convolutional neural network to obtain the optimized initial value of the background light intensity, the optimized initial value of the first modulation degree, and the optimized initial value of the second modulation degree, and further includes:

[0042] The initial value A of the background light intensity init , the first modulation initial value B 1init and the second modulation initial value B 2init Connect them in the channel direction to make them into a 3*H*W picture, and then output the optimized 3*H*W picture after nonlinear mapping through the target convolutional neural network, that is, obtain the optimized initial value of the background light intensity Optimized initial value of the first modulation index And the optimized initial value of the second modulation index

[0043] The target convolutional neural network includes a downsampling part and an upsampling part, the downsampling part includes two downsampling blocks, the downsampling block includes a 2×2 maximum pooling layer, each of the downsampling blocks is connected to two 3×3 convolution layers, and the upsampling part includes two upsampling blocks symmetrical to the downsampling blocks; each upsampling block first performs a 2×2 transposed convolution and then performs two 3×3 convolutions; the outputs of the downsampling blocks with the same dimension are connected to the input of each upsampling block.

[0044] Furthermore, the step of sending the first phase initial value and the second phase initial value to the target fully connected neural network respectively to obtain an optimized first phase initial value and an optimized second phase initial value further includes:

[0045] The first phase initial value and the second phase initial value Convert it into a column vector of H*W rows and 1 column, and then input the column vector of H*W rows and 1 column into the target fully connected neural network to output the optimized first phase initial value and the optimized initial value of the second phase The target fully connected neural network includes six fully connected layers with 256 output channels.

[0046] Furthermore, it also includes:

[0047] When training the neural network, the total loss function loss is defined as follows:

[0048] loss = α·l recon +β·l limit +γ·l smooth

[0049] Among them, α, β and γ are hyper parameters, l recon is the reconstruction loss, l limit is the constraint loss, l smooth is the smoothness loss;

[0050] (1) Reconstruction loss l recon The mean square loss is used, which is defined as follows:

[0051]

[0052] Among them, H is the number of pixels in the length direction of the image, and W is the number of pixels in the width direction of the image; represents the pixel value predicted by the neural network, and I(i) represents the pixel value of the aliased fringe image captured by the camera;

[0053] (2) Constraint loss l limit is defined as follows:

[0054]

[0055] in, Represents the phase value of the corresponding camera pixel position predicted by the neural network, Indicates the phase value of the camera pixel position Or the phase value of the camera pixel position N is the number of camera pixels;

[0056] (3) Smoothness loss l smooth is defined as follows:

[0057]

[0058] Among them, u, v represent the row and column coordinates in the image coordinate system, x u,v is the pixel value at u,v.

[0059] The present invention provides a method for separating aliased fringes of a transparent element in deflection measurement, which has the following beneficial effects: using a neural network as a means of parameter calibration, optimizing initial parameters using the powerful nonlinear ability of the neural network, and quickly and accurately obtaining a separated fringe diagram. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present invention, but do not constitute a limitation of the present invention.

[0061] Figure 1 The present invention is a flow chart of a method for separating aliased fringes of a transparent element in deflection measurement.

[0062] Figure 2 It is a structural diagram of the deflection measurement system of the present invention.

[0063] Figure 3 It is the binary projection circular spot array image projected by the screen of the present invention.

[0064] Figure 4 This is the reflection circular spot array image captured by the camera of the present invention.

[0065] Figure 5 It is a structural diagram of the neural network of the present invention.

[0066] FIG6( a ) is an x-direction phase diagram of the reflected fringe image on the upper surface of the transparent element to be tested after optimization by the neural network according to the present invention.

[0067] FIG6( b ) is a y-direction phase diagram of the reflected fringe image on the upper surface of the transparent element to be tested after optimization by the neural network according to the present invention. DETAILED DESCRIPTION

[0068] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation method, structure, characteristics and effects of the separation method of aliased fringes of a transparent element in deflection measurement proposed by the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0069] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so as to describe the embodiments of the present invention described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0070] In this embodiment, a method for separating aliased fringes of a transparent element in deflection measurement is provided, such as Figure 1 As shown, the method for separating aliased fringes of a transparent element in deflection measurement comprises the following steps:

[0071] Step S1: construct a deflection measurement system consisting of a camera and a screen, and establish a posture relationship between the camera and the screen;

[0072] It should be noted that in the deflection measurement system, the camera and the LCD screen are placed on the same side. The camera cannot directly image the LCD screen and needs to rely on a reflector to assist in obtaining a virtual image of the LCD screen. Therefore, a standard plane reflector is placed in the camera's field of view so that the camera can capture the screen pattern reflected by it to establish the positional relationship between the camera and the LCD screen.

[0073] Replace the standard plane reflector with the transparent element to be tested, such as Figure 2 As shown in the figure, the deflectometry system consists of a camera, an LCD screen and a transparent component to be measured. In addition, a computer that can perform neural network training is also required.

[0074] It should be noted that the plano-convex lens described in the present invention is only used as an example to show the transparent element to be tested. The range of transparent elements to be tested that can be measured by the present invention is not limited to the plano-convex lens. As long as the binary reflection circular spots on the front and rear surfaces are not coupled, the double-surface transparent element can be measured.

[0075] Step S2: taking a plurality of black frame images by a camera to obtain an initial value of the background light intensity;

[0076] Preferably, the step of taking a plurality of black frame images by a camera to obtain an initial value of the background light intensity further includes:

[0077] After completing the deflectometry system setup and camera calibration, turn off all light sources including the screen to estimate the camera dark level and leaked background light intensity, and take multiple black frame images N ds , calculate all black frame images N ds The average brightness of the background light intensity is taken as the initial value A init , the initial value of the background light intensity A init The calculation formula is as follows:

[0078]

[0079] Where n is the black frame image N ds The number of sheets. n can be 20.

[0080] Step S3: Project the binary projection circular spot array image on the screen onto the transparent element to be tested. The binary projection circular spot array image is as follows: Figure 3 As shown, the reflection circular spot array reflected by the upper and lower surfaces of the transparent element to be tested is photographed by a camera, and the photographed reflection circular spot array image is as shown in Figure 4 As shown, a threshold value Th is set to perform binarization processing on the captured reflection circular spot array image to obtain a binary reflection circular spot array image; and the centroid coordinates of each binary reflection circular spot in the binary reflection circular spot array image are calculated, and then the relationship between the camera pixel point at the centroid of each binary reflection circular spot and the screen pixel point at the centroid of the corresponding binary projection circular spot is saved;

[0081] It should be noted that the size and spacing of the binary projection circular spot array vary depending on the specific shape of the transparent element to be tested. The binary projection circular spot array is generally arranged in a rectangular shape with n rows and m columns. The camera needs to be able to capture at least 10 reflection circular spots reflected from the upper and lower surfaces, and the reflection circular spots do not overlap. The size of the captured reflection circular spots should be appropriate so that the center of gravity search is reliable. The correspondence between each captured reflection circular spot and the screen projection circular spot is determined by manual search.

[0082] Preferably, the calculating of the coordinates of the center of gravity of each binary reflection circular spot in the binary reflection circular spot array image and then saving the relationship between the camera pixel point at the center of gravity of each binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot also includes:

[0083] The calculation formula of the centroid coordinates of each binary reflection circular spot is as follows:

[0084]

[0085]

[0086] Among them, C x is the horizontal coordinate of the center of gravity of each binary reflection spot, C y is the vertical coordinate of the center of gravity of each binary reflection circular spot, I(x,y) is the pixel point with a pixel value of 1 in the binary reflection circular spot array image, x is the image horizontal coordinate, and y is the image vertical coordinate; then find out the coordinate set C of the center of gravity of the binary reflection circular spot on the upper surface of the transparent element to be tested up ' and the centroid coordinate set C of the binary reflection circular spot on the lower surface of the transparent element to be tested dowm ', and the corresponding binary projection spot centroid coordinate set S'.

[0087] Determine the correspondence between each binary reflection circular spot and the binary projection circular spot according to the centroid coordinates of each binary reflection circular spot; wherein the binary reflection circular spots in the binary reflection circular spot array image include a first binary reflection circular spot on the upper surface of the transparent element to be tested and a second binary reflection circular spot on the lower surface of the transparent element to be tested;

[0088] The relationship between the camera pixel point at the center of gravity of each first binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot is saved, and the relationship between the camera pixel point at the center of gravity of each second binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot is saved.

[0089] Step S4: projecting the fringe image on the screen onto the transparent element to be tested, and photographing the aliased fringes reflected by the upper and lower surfaces of the transparent element to be tested by a camera to obtain an aliased fringed image of the transparent element to be tested;

[0090] It should be noted that sinusoidal fringe images in the horizontal and vertical directions are projected onto the transparent element to be tested.

[0091] Preferably, the step of photographing the aliased fringes reflected from the upper and lower surfaces of the transparent element to be tested by a camera to obtain an aliased fringes image of the transparent element to be tested further includes:

[0092] This deflectometry system works in a dark room, and the camera imaging model is defined as follows:

[0093]

[0094] Wherein, I is the pixel value of the aliased fringe image of the transparent element to be tested, A is the background light intensity, including the camera's dark level, a small amount of ambient light leakage and other constant pixel value responses of the camera; B1 is the modulation degree of the reflected fringe image on the upper surface of the transparent element to be tested, and B2 is the modulation degree of the reflected fringe image on the lower surface of the transparent element to be tested. is the phase of the reflected fringe image on the upper surface of the transparent element to be tested, It is the phase of the reflected fringe image on the lower surface of the transparent element to be tested.

[0095] Step S5: according to the relationship between the camera pixel point at the center of gravity of each binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot, a first phase initial value of the reflection fringe image on the upper surface of the transparent element to be tested and a second phase initial value of the reflection fringe image on the lower surface of the transparent element to be tested are calculated; and the modulation value of each pair of camera pixel points and screen pixel points is optimized to obtain a first modulation initial value of the reflection fringe image on the upper surface of the transparent element to be tested and a second modulation initial value of the reflection fringe image on the lower surface of the transparent element to be tested;

[0096] Preferably, the calculating of the first phase initial value of the reflection fringe image on the upper surface of the transparent element to be tested and the second phase initial value of the reflection fringe image on the lower surface of the transparent element to be tested according to the relationship between the camera pixel point at the center of gravity of each binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot also includes:

[0097] The phase of the camera pixel position in the reflection fringe image on the upper surface of the transparent element to be tested is calculated based on the relationship between the camera pixel at the center of gravity of each first binary reflection circular spot and the screen pixel at the center of gravity of the corresponding binary projection circular spot. The phase of the camera pixel position in the reflection fringe image of the lower surface of the transparent element to be tested is calculated based on the relationship between the camera pixel point at the center of gravity of each second binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot.

[0098] According to the phase of the camera pixel position in the reflection fringe image on the upper surface of the transparent element to be tested The first phase initial value of the reflection fringe image on the upper surface of the transparent element to be tested is obtained by fitting According to the phase of the camera pixel position in the reflection fringe image on the lower surface of the transparent element to be tested The second phase initial value of the reflection fringe image of the lower surface of the transparent element to be tested is obtained by fitting

[0099] Preferably, the optimization of the modulation value of each pair of camera pixels and screen pixels to obtain a first modulation initial value of the reflection fringe image on the upper surface of the transparent element to be tested and a second modulation initial value of the reflection fringe image on the lower surface of the transparent element to be tested also includes:

[0100] The least square method is used to optimize the modulation value of the camera pixel point at the center of gravity of each first binary reflection circular spot and the modulation value of the screen pixel point at the center of gravity of the corresponding binary projection circular spot to obtain the first modulation initial value B of the reflection fringe image on the upper surface of the transparent element to be tested. 1init ; At the same time, the least squares method is used to optimize the modulation value of the camera pixel point at the center of gravity of each second binary reflection circular spot and the modulation value of the screen pixel point at the center of gravity of the corresponding binary projection circular spot to obtain the second modulation initial value B of the reflection fringe image on the lower surface of the transparent element to be tested 2init ; Among them, the formula of least squares method is as follows:

[0101]

[0102] in, is the phase of the camera pixel position in the reflection fringe image on the upper surface of the transparent element to be tested, is the phase of the camera pixel position in the reflected fringe image on the lower surface of the transparent element to be tested, A init is the initial value of the background light intensity. In the initial value estimation stage, the estimated B 1init and B 2init is a constant.

[0103] Step S6: Figure 5 As shown, the background light intensity initial value, the first phase initial value, the second phase initial value, the first modulation degree initial value and the second modulation degree initial value are optimized by a neural network to obtain an optimized background light intensity initial value, an optimized first phase initial value, an optimized second phase initial value, an optimized first modulation degree initial value and an optimized second modulation degree initial value;

[0104] Preferably, the optimization of the background light intensity initial value, the first phase initial value, the second phase initial value, the first modulation degree initial value and the second modulation degree initial value by a neural network to obtain an optimized background light intensity initial value, an optimized first phase initial value, an optimized second phase initial value, an optimized first modulation degree initial value and an optimized second modulation degree initial value also includes:

[0105] The background light intensity initial value, the first modulation degree initial value and the second modulation degree initial value are connected in the channel direction and sent to the target convolutional neural network to obtain an optimized background light intensity initial value, an optimized first modulation degree initial value and an optimized second modulation degree initial value;

[0106] Specifically, the initial value of the background light intensity, the initial value of the first modulation index, and the initial value of the second modulation index are connected in the channel direction and sent to the target convolutional neural network to obtain an optimized initial value of the background light intensity, an optimized initial value of the first modulation index, and an optimized initial value of the second modulation index, and further includes:

[0107] like Figure 5 As shown, the background light intensity initial value A init , the first modulation initial value B 1init and the second modulation initial value B 2init Connect them in the channel direction to make them into a 3*H*W picture, and then output the optimized 3*H*W picture after nonlinear mapping through the target convolutional neural network, that is, obtain the optimized initial value of background light intensity Optimized initial value of the first modulation index And the optimized initial value of the second modulation index

[0108] The target convolutional neural network includes a downsampling part and an upsampling part, the downsampling part includes two downsampling blocks, the downsampling block includes a 2×2 maximum pooling layer, each of the downsampling blocks is connected to two 3×3 convolution layers, and the upsampling part includes two upsampling blocks symmetrical to the downsampling blocks; each upsampling block first performs a 2×2 transposed convolution and then performs two 3×3 convolutions; the outputs of the downsampling blocks with the same dimension are connected to the input of each upsampling block.

[0109] The first phase initial value and the second phase initial value are respectively sent to the target fully connected neural network to obtain an optimized first phase initial value and an optimized second phase initial value.

[0110] Specifically, the step of sending the first phase initial value and the second phase initial value to the target fully connected neural network respectively to obtain an optimized first phase initial value and an optimized second phase initial value further includes:

[0111] like Figure 5 As shown, the first phase initial value and the second phase initial value Convert it into a column vector of H*W rows and 1 column, and then input the column vector of H*W rows and 1 column into the target fully connected neural network to output the optimized first phase initial value and the optimized initial value of the second phase The target fully connected neural network includes six fully connected layers with 256 output channels.

[0112] Preferably, it also includes:

[0113] The neural network is optimized using the Adam optimizer, with the learning rate set to 0.0001, which remains unchanged in the first 60 trainings, and then the learning rate is adjusted following a linear decay strategy; when training the neural network, the weight parameters of the network are updated according to the loss value to achieve gradient descent, and the total loss function loss is defined as follows:

[0114] loss = α·l recon +β·l limit +γ·l smooth

[0115] Among them, α, β and γ are hyperparameters, which are 0.8, 0.1, and 0.1 respectively; l recon is the reconstruction loss, l limit is the constraint loss, l smooth is the smoothness loss;

[0116] (1) Reconstruction loss l recon The mean square error (MSE) loss is used, which is defined as follows:

[0117]

[0118] Where H is the number of pixels in the length direction of the image, W is the number of pixels in the width direction of the image, H and W are 2000 and 2000 respectively; represents the pixel value predicted by the neural network, and I(i) represents the pixel value of the aliased fringe image captured by the camera;

[0119] (2) Constraint loss l limit is defined as follows:

[0120]

[0121] in, Represents the phase value of the corresponding camera pixel position predicted by the neural network, Indicates the phase value of the camera pixel position Or the phase value of the camera pixel position N is the number of camera pixels;

[0122] (3) Smoothness loss l smooth The total variation loss is adopted and is defined as follows:

[0123]

[0124] Among them, u, v represent the row and column coordinates in the image coordinate system, x u,v is the pixel value at u,v;

[0125] It should be noted that the neural network training stops after the loss function converges or the neural network iterates 500 rounds.

[0126] Step S7: Generate a reflection fringe image of the upper surface of the transparent element to be tested according to the optimized background light intensity initial value, the optimized first phase initial value and the optimized first modulation initial value, and generate a reflection fringe image of the lower surface of the transparent element to be tested according to the optimized background light intensity initial value, the optimized second phase initial value and the optimized second modulation initial value, so as to achieve separation of the aliased fringe image of the transparent element to be tested.

[0127] It should be noted that all operations of the present invention must be performed in a dark room.

[0128] like Figure 6(a) to Figure 6(b) As shown in the figure, the simulation experiment shows that the fringe modulation and phase results after neural network optimization are correct, and the neural network also has a certain inhibitory effect on the influence of noise.

[0129] The present invention combines a neural network with a physical imaging model, and can effectively establish a corresponding relationship between a camera, a workpiece, and screen pixels without the need for training data, thereby realizing deflectometry measurement.

[0130] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of the technical solution of the present invention.

Claims

1. A method for separating aliased fringes of a transparent element in deflectometry, characterized in that: The steps include: Step S1: construct a deflection measurement system consisting of a camera and a screen, and establish a posture relationship between the camera and the screen; Step S2: taking a plurality of black frame images by a camera to obtain an initial value of the background light intensity; Step S3: projecting the binary projection circular spot array image on the screen onto the transparent element to be tested, photographing the reflection circular spot array reflected by the upper and lower surfaces of the transparent element to be tested by a camera, and performing a binarization process on the photographed reflection circular spot array image to obtain a binary reflection circular spot array image; and calculating the coordinates of the center of gravity of each binary reflection circular spot in the binary reflection circular spot array image, and then saving the relationship between the camera pixel point at the center of gravity of each binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot; Step S4: projecting the fringe image on the screen onto the transparent element to be tested, and photographing the aliased fringes reflected by the upper and lower surfaces of the transparent element to be tested by a camera to obtain an aliased fringed image of the transparent element to be tested; Step S5: calculating a first phase initial value of the reflection fringe image on the upper surface of the transparent element to be tested and a second phase initial value of the reflection fringe image on the lower surface of the transparent element to be tested according to the relationship between the camera pixel point at the center of gravity of each binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot; and optimizing the modulation values ​​of each pair of camera pixels and screen pixels to obtain a first modulation initial value of the reflection fringe image on the upper surface of the transparent element to be tested and a second modulation initial value of the reflection fringe image on the lower surface of the transparent element to be tested; Step S6: optimizing the background light intensity initial value, the first phase initial value, the second phase initial value, the first modulation degree initial value and the second modulation degree initial value through a neural network to obtain an optimized background light intensity initial value, an optimized first phase initial value, an optimized second phase initial value, an optimized first modulation degree initial value and an optimized second modulation degree initial value; Step S7: Generate a reflection fringe image of the upper surface of the transparent element to be tested according to the optimized background light intensity initial value, the optimized first phase initial value and the optimized first modulation initial value, and generate a reflection fringe image of the lower surface of the transparent element to be tested according to the optimized background light intensity initial value, the optimized second phase initial value and the optimized second modulation initial value, so as to achieve separation of the aliased fringe image of the transparent element to be tested.

2. The method for separating aliased fringes of a transparent element in deflection measurement according to claim 1, characterized in that: The method of taking a plurality of black frame images by a camera to obtain an initial value of the background light intensity also includes: The initial value A of the background light intensity init The calculation formula is as follows: Where n is the black frame image N ds The number of sheets.

3. The method for separating aliased fringes of a transparent element in deflection measurement according to claim 2, characterized in that: The method of calculating the coordinates of the center of gravity of each binary reflection circular spot in the binary reflection circular spot array image and then saving the relationship between the camera pixel point at the center of gravity of each binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot also includes: The calculation formula of the centroid coordinates of each binary reflection circular spot is as follows: Among them, C x is the horizontal coordinate of the center of gravity of each binary reflection spot, C y is the ordinate of the centroid of each binary reflection spot, I(x,y) is the pixel point with a pixel value of 1 in the binary reflection spot array image, x is the image abscissa, and y is the image ordinate; Determine the correspondence between each binary reflection circular spot and the binary projection circular spot according to the centroid coordinates of each binary reflection circular spot; wherein the binary reflection circular spots in the binary reflection circular spot array image include a first binary reflection circular spot on the upper surface of the transparent element to be tested and a second binary reflection circular spot on the lower surface of the transparent element to be tested; The relationship between the camera pixel point at the center of gravity of each first binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot is saved, and the relationship between the camera pixel point at the center of gravity of each second binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot is saved.

4. The method for separating aliased fringes of a transparent element in deflection measurement according to claim 3, characterized in that: The method of photographing the aliased fringes reflected by the upper and lower surfaces of the transparent element to be tested by a camera to obtain an aliased fringes image of the transparent element to be tested also includes: The camera imaging model is defined as follows: Wherein, I is the pixel value of the aliased fringe image of the transparent element to be tested, A is the background light intensity, B1 is the modulation degree of the reflected fringe image on the upper surface of the transparent element to be tested, B2 is the modulation degree of the reflected fringe image on the lower surface of the transparent element to be tested, is the phase of the reflected fringe image on the upper surface of the transparent element to be tested, It is the phase of the reflected fringe image on the lower surface of the transparent element to be tested.

5. The method for separating aliased fringes of a transparent element in deflection measurement according to claim 4, characterized in that: The method further comprises: calculating the first phase initial value of the reflection fringe image on the upper surface of the transparent element to be tested and the second phase initial value of the reflection fringe image on the lower surface of the transparent element to be tested based on the relationship between the camera pixel point at the center of gravity of each binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot, and obtaining the first phase initial value of the reflection fringe image on the upper surface of the transparent element to be tested and the second phase initial value of the reflection fringe image on the lower surface of the transparent element to be tested. The phase of the camera pixel position in the reflection fringe image on the upper surface of the transparent element to be tested is calculated based on the relationship between the camera pixel at the center of gravity of each first binary reflection circular spot and the screen pixel at the center of gravity of the corresponding binary projection circular spot. The phase of the camera pixel position in the reflection fringe image of the lower surface of the transparent element to be tested is calculated based on the relationship between the camera pixel point at the center of gravity of each second binary reflection circular spot and the screen pixel point at the center of gravity of the corresponding binary projection circular spot. According to the phase of the camera pixel position in the reflection fringe image on the upper surface of the transparent element to be tested The first phase initial value of the reflection fringe image on the upper surface of the transparent element to be tested is obtained by fitting According to the phase of the camera pixel position in the reflection fringe image on the lower surface of the transparent element to be tested The second phase initial value of the reflection fringe image of the lower surface of the transparent element to be tested is obtained by fitting 6. The method for separating aliased fringes of a transparent element in deflection measurement according to claim 5, characterized in that: The method of optimizing the modulation value of each pair of camera pixels and screen pixels to obtain a first modulation initial value of the reflection fringe image on the upper surface of the transparent element to be tested and a second modulation initial value of the reflection fringe image on the lower surface of the transparent element to be tested also includes: The least square method is used to optimize the modulation value of the camera pixel point at the center of gravity of each first binary reflection circular spot and the modulation value of the screen pixel point at the center of gravity of the corresponding binary projection circular spot to obtain the first modulation initial value B of the reflection fringe image on the upper surface of the transparent element to be tested. 1init ; At the same time, the least squares method is used to optimize the modulation value of the camera pixel point at the center of gravity of each second binary reflection circular spot and the modulation value of the screen pixel point at the center of gravity of the corresponding binary projection circular spot to obtain the second modulation initial value B of the reflection fringe image on the lower surface of the transparent element to be tested 2init ; Among them, the formula of least squares method is as follows: in, is the phase of the camera pixel position in the reflection fringe image on the upper surface of the transparent element to be tested, is the phase of the camera pixel position in the reflected fringe image on the lower surface of the transparent element to be tested, A init is the initial value of the background light intensity.

7. The method for separating aliased fringes of a transparent element in deflection measurement according to claim 6, characterized in that: The method of optimizing the background light intensity initial value, the first phase initial value, the second phase initial value, the first modulation degree initial value, and the second modulation degree initial value through a neural network to obtain an optimized background light intensity initial value, an optimized first phase initial value, an optimized second phase initial value, an optimized first modulation degree initial value, and an optimized second modulation degree initial value, further includes: The background light intensity initial value, the first modulation degree initial value and the second modulation degree initial value are connected in the channel direction and sent to the target convolutional neural network to obtain an optimized background light intensity initial value, an optimized first modulation degree initial value and an optimized second modulation degree initial value; The first phase initial value and the second phase initial value are respectively sent to the target fully connected neural network to obtain an optimized first phase initial value and an optimized second phase initial value.

8. The method for separating aliased fringes of a transparent element in deflection measurement according to claim 7, characterized in that: The initial value of the background light intensity, the initial value of the first modulation index, and the initial value of the second modulation index are connected in the channel direction and sent to the target convolutional neural network to obtain the optimized initial value of the background light intensity, the optimized initial value of the first modulation index, and the optimized initial value of the second modulation index, and further include: The initial value A of the background light intensity init , the first modulation initial value B 1init and the second modulation initial value B 2init Connect them in the channel direction to make them into a 3*H*W picture, and then output the optimized 3*H*W picture after nonlinear mapping through the target convolutional neural network, that is, obtain the optimized initial value of the background light intensity Optimized initial value of the first modulation index And the optimized initial value of the second modulation index The target convolutional neural network includes a downsampling part and an upsampling part, the downsampling part includes two downsampling blocks, the downsampling block includes a 2×2 maximum pooling layer, each of the downsampling blocks is connected to two 3×3 convolution layers, and the upsampling part includes two upsampling blocks symmetrical to the downsampling blocks; each upsampling block first performs a 2×2 transposed convolution and then performs two 3×3 convolutions; the outputs of the downsampling blocks with the same dimension are connected to the input of each upsampling block.

9. The method for separating aliased fringes of a transparent element in deflection measurement according to claim 7, characterized in that: The step of sending the first phase initial value and the second phase initial value to the target fully connected neural network respectively to obtain the optimized first phase initial value and the optimized second phase initial value further includes: The first phase initial value and the second phase initial value Convert it into a column vector of H*W rows and 1 column, and then input the column vector of H*W rows and 1 column into the target fully connected neural network to output the optimized first phase initial value and the optimized initial value of the second phase The target fully connected neural network includes six fully connected layers with 256 output channels.

10. The method for separating aliased fringes of a transparent element in deflection measurement according to claim 7, characterized in that: Also includes: When training the neural network, the total loss function loss is defined as follows: loss=α·l recon +β·l limit +γ·l smooth Among them, α, β and γ are hyperparameters, l recon is the reconstruction loss, l limit is the constraint loss, l smooth is the smoothness loss; (1) Reconstruction loss l recon The mean square loss is used, which is defined as follows: Among them, H is the number of pixels in the length direction of the image, and W is the number of pixels in the width direction of the image; represents the pixel value predicted by the neural network, and I(i) represents the pixel value of the aliased fringe image captured by the camera; (2) Constraint loss l limit is defined as follows: in, Represents the phase value of the corresponding camera pixel position predicted by the neural network, The phase value representing the camera pixel position Or the phase value of the camera pixel position N is the number of camera pixels; (3) Smoothness loss l smooth is defined as follows: Among them, u, v represent the row and column coordinates in the image coordinate system, x u,v is the pixel value at u,v.

Images