A method for separating overlapping fringes of transparent elements in deflection measurement

Abstract

This invention relates to the field of optical measurement technology, specifically disclosing a method for separating overlapping fringes of a transparent element in deflection measurement. The method includes: obtaining an initial value of background light intensity; projecting a binary projection circular spot array image from a screen onto the transparent element under test to obtain a binary reflection circular spot array image; saving the relationship between each camera pixel and its corresponding screen pixel; obtaining an overlapping fringe image of the transparent element under test; calculating the first initial phase value and the first initial modulation value of the upper surface reflection fringe image, and the second initial phase value and the second initial modulation value of the lower surface reflection fringe image; optimizing the initial values ​​of background light intensity, first initial phase value, second initial phase value, first initial modulation value, and second initial modulation value using a neural network; and generating the upper surface reflection fringe image and the lower surface reflection fringe image based on the optimized five initial values. This invention can quickly and accurately obtain separated fringe patterns.

Description

Technical Field

[0001] This invention relates to the field of optical measurement technology, and more specifically, to a method for separating overlapping fringes of a transparent element in deflection measurement. Background Technology

[0002] Transparent components play a vital role in all aspects of social production and daily life. In the field of optics, transparent components such as lenses, filters, and prisms are widely used in the manufacture of various optical instruments and equipment, such as cameras, telescopes, VR / AR devices, and lasers. Furthermore, transparent components are also widely used in industries such as automotive, electronics, aerospace, and medical devices. For example, high-precision transparent components such as objective lenses and eyepieces are key components of high-end cameras and projectors. Transparent components used in instruments such as spectrometers and laser interferometers help scientists accurately measure and analyze the properties of light. In the aerospace field, the windshields of aircraft cockpits require high strength and excellent transparency to provide pilots with good visibility while resisting high pressure, wind speed, and temperature changes during flight. Windows on passenger planes and spacecraft require transparency, robustness, and the ability to withstand extreme environmental conditions; their surface shape is crucial for assembly. In summary, transparent components are an indispensable part of modern industry and scientific research. The surface accuracy of transparent components is critical to improving the performance and stability of these devices. Therefore, the requirements for the measurement accuracy of the surface shape and quality of transparent components are becoming increasingly stringent. Traditional contact measurement methods mainly include coordinate measuring machines (CMMs) and contact point scanning profilometers. These methods offer stable and reliable results with high accuracy, making them suitable for measuring various transparent materials. However, contact measurement methods can cause irreversible scratches and damage to the component surface, and their large size prevents online measurement. Non-contact measurement methods do not require contact with the measurement surface. Common interferometry methods include white light interferometry and laser interferometry, but these methods cannot measure very complex surface shapes. Deflection measurement is a non-contact optical measurement technique suitable for measuring the surface shape of transparent components. This method displays an encoded sinusoidal fringe pattern on a screen. The surface being measured deflects the light emitted from the screen, and a camera captures the deformed fringes after deflection. The surface shape information is inferred based on the pixel correspondence between the deformed fringes and the fringes displayed on the screen. Deflection measurement technology has advantages such as high accuracy, speed, and applicability to complex shapes and large-sized components. Therefore, it has wide applications in optical manufacturing and optical inspection.

[0003] However, when measuring transparent elements using deflection, the stripe images reflected from the front and back surfaces are superimposed, making it difficult to establish pixel correspondences. Furthermore, the low reflectivity of transparent elements results in a very low signal-to-noise ratio in the captured image, thus reducing measurement accuracy. The initial method [D. Sprenger, C. Faber, MC et al. UV-deflectometry: noparasitic reflections, Proc. DGaO, 2010, 111: A19] uses an ultraviolet light source, utilizing the opacity of glass to block reflections from the lower surface; however, this method requires expensive and difficult-to-obtain ultraviolet displays. In [Z. Hao, Y. Liu, Transparent object shape measurement based on deflectometry, Proceedings, 2018, 2: 548.], Hao et al. used binocular stereo vision to measure the normal direction of transparent objects. The surface shape of the transparent element under test can be calculated using the consistency constraint of the stereo normal vectors. However, due to the limitations of the binocular method, its measurement accuracy is low, and it introduces additional uncertainties. Currently, the mainstream fringe separation method is the multi-frequency phase-shifting deflection measurement method [Ye J, Niu Z, Zhang X, et al. Simultaneous measurement of double surfaces of transparent lenses with phase measuring deflectometry[J]. Optics and Lasersin Engineering, 2021, 137: 106356.], [Tao S, Yue H, Chen H, et al. Elimination of parasitic reflections for objects with high transparency in phase measuring deflectometry[J]. Results in Physics, 2019, 15: 102734.], [Leung YC, Cai L. Untangling parasitic reflection in phase measuring deflectometry by multi-frequency phase-shifting[J]. Applied Optics, 2022, 61(1): 208-222.]. A series of sinusoidal fringe patterns with different frequencies are displayed sequentially on the screen. The phase shift data are combined and a model is established. The nonlinear optimization method is used to solve the problem.However, this method is very time-consuming and cannot untangle very thin transparent planes. To address this, Zheng et al. [Zheng W, Li D, Wang R, et al. Parasitic reflection separation deflectometry based on harmonic analysis[J]. Measurement, 2022, 203: 111864.] proposed a parasitic reflection separation deflection method based on harmonic analysis. By analyzing the harmonic components in the power spectrum, the parasitic reflection of thin transparent devices can be untangled, but this method only applies to parallel transparent devices. Since the reflectivity of the signal reflected from the lower surface is low, the light intensity entering the camera after three reflections is about one-thousandth of the light intensity entering the camera after a single reflection. Therefore, the above methods eliminate the stripes reflected from the lower surface as ghost images, which also leads to a decrease in the measurement efficiency of transparent elements. Ye et al. [Ye J, Niu Z, Zhang X, et al. Simultaneous measurement of double surfaces of transparent lenses with phase measuring deflectometry[J]. Optics and Lasers in Engineering, 2021, 137: 106356.] proposed a multi-frequency phase-shifting fringe dealiasing method, which can simultaneously measure the fringes on the upper and lower surfaces. However, this method has very high requirements for setting the initial phase value of the fringes and can exhibit phase jump phenomena.

[0004] Neural networks, with their superior ability to handle nonlinear problems and their adaptive capabilities, are widely used in learning the phase information of fringe images [Wang, K., et al. One-step robust deep learning phase unwrapping. Optics Express, 2019, 10: 15100.], [Wang, K., Song, L., Wang, C. et al. On the use of deep learning for phase recovery. Light SciAppl, 2024, 13: 4.], [Guan J, Li J, Yang X, et al. Error compensation for phase retrieval in deflectometry based on deep learning[J]. Measurement Science and Technology, 2023, 34(2): 025009.]. Their main applications include fringe image detection and recognition, phase information enhancement, and phase unwrapping. In surface shape measurement, neural networks can optimize and enhance some initial parameters through their powerful nonlinear fitting capabilities. Currently, no one has attempted to use deep neural networks to optimize initial parameters to achieve transparent element fringe separation and phase decoupling.

[0005] In summary, the following problems remain to be solved in the field of deflection measurement: difficulty in establishing the pixel correspondence between the upper and lower surfaces of transparent elements, slow speed and limited application scenarios of traditional stripe separation methods, and poor reliability of end-to-end deep learning methods. Summary of the Invention

[0006] To address the shortcomings and deficiencies in existing technologies, this invention provides a method for separating overlapping fringes of transparent elements in deflection measurements, which can quickly and accurately obtain separated fringe patterns.

[0007] As a first aspect of the present invention, a method for separating overlapping fringes of a transparent element in a deflection measurement is provided, comprising the following steps:

[0008] Step S1: Construct a deflection measurement system consisting of a camera and a screen, and establish the pose relationship between the camera and the screen;

[0009] Step S2: Take multiple black frame images with the camera to obtain the initial value of the background light intensity;

[0010] Step S3: Project the binary projection circular spot array image from the screen onto the transparent element under test. Capture the reflection circular spot array reflected from the upper and lower surfaces of the transparent element under test using a camera. Perform binarization processing on the captured reflection circular spot array image to obtain a binary reflection circular spot array image. Calculate the centroid coordinates of each binary reflection circular spot in the binary reflection circular spot array image, and then save the relationship between the camera pixel at the centroid of each binary reflection circular spot and the screen pixel at the corresponding binary projection circular spot centroid.

[0011] Step S4: Project the stripe image on the screen onto the transparent element under test, and capture the superimposed stripes reflected from the upper and lower surfaces of the transparent element under test with a camera to obtain the superimposed stripe image of the transparent element under test.

[0012] Step S5: Based on the relationship between the camera pixel at the centroid of each binary reflection spot and the screen pixel at the centroid of the corresponding binary projection spot, calculate the first initial phase value of the reflection stripe image on the upper surface of the transparent element under test and the second initial phase value of the reflection stripe image on the lower surface of the transparent element under test; and optimize the modulation values ​​of each pair of camera pixels and screen pixels to obtain the first initial modulation value of the reflection stripe image on the upper surface of the transparent element under test and the second initial modulation value of the reflection stripe image on the lower surface of the transparent element under test.

[0013] Step S6: Optimize the initial values ​​of background light intensity, first phase, second phase, first modulation degree, and second modulation degree using a neural network to obtain optimized initial values ​​of background light intensity, first phase, second phase, first modulation degree, and second modulation degree.

[0014] Step S7: Generate a reflection stripe image of the upper surface of the transparent element under test based on the optimized initial value of background light intensity, the optimized initial value of the first phase, and the optimized initial value of the first modulation. At the same time, generate a reflection stripe image of the lower surface of the transparent element under test based on the optimized initial value of background light intensity, the optimized initial value of the second phase, and the optimized initial value of the second modulation, so as to separate the overlapping stripe images of the transparent element under test.

[0015] Furthermore, the step of capturing multiple black frame images with a camera to obtain the initial value of the background light intensity also includes:

[0016] The initial value of the background light intensity A init The calculation formula is as follows:

[0017]

[0018] Where n is the black frame image Nds The number of sheets.

[0019] Furthermore, the step of calculating the centroid coordinates of each binary reflection circle in the binary reflection circle array image, and then saving the relationship between the camera pixel at the centroid of each binary reflection circle and the screen pixel at the centroid of the corresponding binary projection circle, also includes:

[0020] The formula for calculating the centroid coordinates of each binary reflection spot is as follows:

[0021]

[0022]

[0023] Among them, C x Let C be the x-coordinate of the centroid of each binary reflection spot. y Let I(x,y) be the ordinate of the centroid of each binary reflective circular spot, and let I(x,y) be the pixel with a pixel value of 1 in the binary reflective circular spot array image, where x is the abscissa of the image and y is the ordinate of the image.

[0024] The correspondence between each binary reflection circle and the binary projection circle is determined based on the centroid coordinates of each binary reflection circle; wherein, the binary reflection circles in the binary reflection circle array image include a first binary reflection circle on the upper surface of the transparent element under test and a second binary reflection circle on the lower surface of the transparent element under test;

[0025] Save the relationship between the camera pixel at the centroid of each first binary reflection spot and the screen pixel at the centroid of the corresponding binary projection spot, and simultaneously save the relationship between the camera pixel at the centroid of each second binary reflection spot and the screen pixel at the centroid of the corresponding binary projection spot.

[0026] Furthermore, the step of capturing images of the superimposed fringes reflected from the upper and lower surfaces of the transparent element under test using a camera to obtain an image of the superimposed fringes of the transparent element under test also includes:

[0027] The camera imaging model is defined as follows:

[0028]

[0029] Where I is the pixel value of the aliased fringe image of the transparent element under test, A is the background light intensity, B1 is the modulation degree of the reflected fringe image on the upper surface of the transparent element under test, and B2 is the modulation degree of the reflected fringe image on the lower surface of the transparent element under test. It is the phase of the reflected stripe image on the upper surface of the transparent element under test. It is the phase of the reflected stripe image on the lower surface of the transparent element under test.

[0030] Further, the step of calculating the first initial phase value of the reflective fringe image on the upper surface of the transparent element under test and the second initial phase value of the reflective fringe image on the lower surface of the transparent element under test based on the relationship between the camera pixel at the centroid of each binary reflective spot and the screen pixel at the centroid of the corresponding binary projected spot also includes:

[0031] Based on the relationship between the camera pixel at the centroid of each first binary reflection spot and the screen pixel at the centroid of the corresponding binary projection spot, the phase of the camera pixel position in the reflection stripe image on the upper surface of the transparent element under test is calculated. Based on the relationship between the camera pixel at the centroid of each second binary reflection spot and the screen pixel at the centroid of the corresponding binary projection spot, the phase of the camera pixel position in the reflection stripe image of the lower surface of the transparent element under test is calculated.

[0032] Based on the phase of the camera pixel positions in the reflective stripe image on the upper surface of the transparent element under test.

[0033] The first initial phase value of the image of the reflective stripes on the upper surface of the transparent element under test is obtained by fitting. Based on the phase of the camera pixel positions in the reflective stripe image of the lower surface of the transparent element under test. The second initial phase value of the image of the reflective fringe on the lower surface of the transparent element under test is obtained by fitting.

[0034] Furthermore, the optimization of the modulation values ​​of each pair of camera pixels and screen pixels to obtain the first initial modulation value of the reflective stripe image on the upper surface of the transparent element under test and the second initial modulation value of the reflective stripe image on the lower surface of the transparent element under test further includes:

[0035] The modulation values ​​of the camera pixels at the centroid of each first binary reflection spot and the screen pixels at the centroid of the corresponding binary projection spot are optimized using the least squares method to obtain the first initial modulation value B of the reflection stripe image on the upper surface of the transparent element under test. 1init Simultaneously, the least squares method is used to optimize the modulation values ​​of the camera pixels at the centroid of each second binary reflection spot and the screen pixels at the centroid of the corresponding binary projection spot, so as to obtain the initial second modulation value B of the reflection stripe image on the lower surface of the transparent element under test. 2init The formula for the least squares method is as follows:

[0036]

[0037] in, The phase of the camera pixel position in the image of the reflected stripes on the upper surface of the transparent element under test. Let A be the phase of the camera pixel position in the image of the reflected stripes on the lower surface of the transparent element under test. init This is the initial value of the background light intensity.

[0038] Furthermore, the optimization of the initial values ​​of background light intensity, first phase, second phase, first modulation degree, and second modulation degree using a neural network to obtain optimized initial values ​​of background light intensity, first phase, second phase, first modulation degree, and second modulation degree further includes:

[0039] The initial values ​​of background light intensity, first modulation degree, and second modulation degree are concatenated in the channel direction and then fed into the target convolutional neural network to obtain optimized initial values ​​of background light intensity, first modulation degree, and second modulation degree.

[0040] The first initial phase value and the second initial phase value are respectively fed into the target fully connected neural network to obtain the optimized first initial phase value and the optimized second initial phase value.

[0041] Furthermore, the step of concatenating the initial values ​​of background light intensity, the first modulation degree, and the second modulation degree in the channel direction and then feeding them into the target convolutional neural network to obtain optimized initial values ​​of background light intensity, the first modulation degree, and the second modulation degree further includes:

[0042] The initial value A of the background light intensity init The initial value of the first modulation system, B. 1init Second modulation initial value B 2init The images are concatenated along the channel direction to form a 3*H*W image. Then, after non-linear mapping through the target convolutional neural network, the optimized 3*H*W image is output, thus obtaining the optimized initial value of the background light intensity. Optimized initial values ​​of the first modulation system and the optimized initial value of the second modulation system

[0043] The target convolutional neural network includes a downsampling part and an upsampling part. The downsampling part includes two downsampling blocks, each of which includes a 2×2 max pooling layer. Each downsampling block is followed by two 3×3 convolutional layers. The upsampling part includes two upsampling blocks that are symmetrical to the downsampling blocks. Each upsampling block first performs a 2×2 transposed convolution, and then performs two 3×3 convolutions. The outputs of downsampling blocks with the same dimensions are connected to the inputs of each upsampling block.

[0044] Furthermore, the step of feeding the first initial phase value and the second initial phase value into the target fully connected neural network respectively to obtain the optimized first initial phase value and the optimized second initial phase value also includes:

[0045] The first phase initial value Second phase initial value The vector is converted into an H*W row, 1 column vector, and then input into the target fully connected neural network to output the optimized first phase initial value. and the optimized second phase initial value The target fully connected neural network comprises six fully connected layers with 256 output channels each.

[0046] Furthermore, it also includes:

[0047] When training the neural network, the overall loss function is defined as follows:

[0048] loss = α·l recon +β·l limit +γ·l smooth

[0049] Where α, β, and γ are hyperparameters, l recon To rebuild the losses, l limit To constrain the loss, l smooth For smoothness loss;

[0050] (1) Reconstruction loss recon The mean squared loss is defined as follows:

[0051]

[0052] Where H is the number of pixels in the long direction of the image, and W is the number of pixels in the wide direction of the image; I(i) represents the pixel value predicted by the neural network, and I(i) represents the pixel value of the aliased stripe image captured by the camera.

[0053] (2) Constraint loss l limit The definition is as follows:

[0054]

[0055] in, This represents the phase value of the corresponding camera pixel position predicted by the neural network. Phase value representing the position of a camera pixel Or the phase value at the camera pixel location N is the number of camera pixels;

[0056] (3) Smoothness loss l smooth The definition is as follows:

[0057]

[0058] Where u,v represent the row and column coordinates in the image coordinate system, and x u,v These are the pixel values ​​at points u and v.

[0059] The present invention provides a method for separating overlapping fringes of transparent elements in deflection measurement, which has the following advantages: using a neural network as a means of parameter calibration, the initial parameters are optimized by utilizing the powerful nonlinear ability of the neural network, and the separated fringe pattern is obtained quickly and accurately. Attached Figure Description

[0060] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the following detailed description to explain the invention, but do not constitute a limitation thereof.

[0061] Figure 1 This is a flowchart of the method for separating overlapping stripes of transparent elements in the deflection measurement of the present invention.

[0062] Figure 2 This is a structural diagram of the deflection measurement system of the present invention.

[0063] Figure 3 This is a binary projection circular spot array image of the screen projection of the present invention.

[0064] Figure 4 This is an image of a reflective circular spot array captured by the camera of this invention.

[0065] Figure 5 This is a structural diagram of the neural network of the present invention.

[0066] Figure 6(a) is the x-direction phase map of the reflective stripe image on the upper surface of the transparent element under test after neural network optimization according to the present invention.

[0067] Figure 6(b) is the y-direction phase map of the reflective stripe image on the upper surface of the transparent element under test after neural network optimization according to the present invention. Detailed Implementation

[0068] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a method for separating overlapping fringes of transparent elements in deflection measurement according to the present invention. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0069] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of the invention described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0070] This embodiment provides a method for separating overlapping fringes of a transparent element in deflection measurement, such as... Figure 1 As shown, the method for separating overlapping fringes of transparent elements in the deflection measurement includes the following steps:

[0071] Step S1: Construct a deflection measurement system consisting of a camera and a screen, and establish the pose relationship between the camera and the screen;

[0072] It should be noted that in the deflection measurement system, the camera and the LCD screen are placed on the same side. The camera cannot directly image the LCD screen and needs to rely on a reflector to obtain a virtual image of the LCD screen. Therefore, a standard plane reflector is placed in the camera's field of view so that the camera can capture the screen pattern reflected by it, in order to establish the pose relationship between the camera and the LCD screen.

[0073] Replace the standard plane mirror with the transparent element to be tested, such as Figure 2 As shown, the deflection measurement system consists of a camera, an LCD screen, and a transparent element to be measured. In addition, a computer capable of training neural networks is also required.

[0074] It should be noted that the plano-convex lens described in this invention is only an example to illustrate the transparent element to be tested. The range of transparent elements to be tested that can be measured by this invention is not limited to the plano-convex lens. As long as the binary reflection spots on the front and rear surfaces do not couple, the double-surface transparent element can be measured.

[0075] Step S2: Take multiple black frame images with the camera to obtain the initial value of the background light intensity;

[0076] Preferably, the step of capturing multiple black frame images with a camera to obtain the initial value of the background light intensity further includes:

[0077] After completing the setup of the deflection measurement system and camera calibration, all light sources, including the screen, were turned off to estimate the camera's dark level and the intensity of leaked background light. Multiple black frame images (N) were then captured. ds Calculate all black frame images N ds The average brightness is used as the initial value A of the background light intensity. init The initial value A of the background light intensity init The calculation formula is as follows:

[0078]

[0079] Where n is the black frame image N ds The number of sheets. n can be 20.

[0080] Step S3: Project the binary projection circular array image from the screen onto the transparent element under test. The binary projection circular array image is as follows: Figure 3 As shown, the array of reflected circular spots on the upper and lower surfaces of the transparent element under test is captured by a camera. The captured images of the reflected circular spot array are as follows. Figure 4 As shown, a threshold Th is set to binarize the captured reflection circle array image to obtain a binary reflection circle array image; the centroid coordinates of each binary reflection circle in the binary reflection circle array image are calculated, and then the relationship between the camera pixel at the centroid of each binary reflection circle and the screen pixel at the centroid of the corresponding binary projection circle is saved.

[0081] It should be noted that the size and spacing of the binary projection circular spot array vary depending on the specific shape of the transparent element under test. The binary projection circular spot array is generally arranged in a rectangle of n rows and m columns. The camera must be able to capture at least 10 reflected circular spots from both the upper and lower surfaces, with no overlap between the reflected spots. The size of the captured reflected circular spots should be appropriate to ensure reliable centroid finding. The correspondence between each captured reflected circular spot and the screen projection circular spot is determined manually.

[0082] Preferably, the step of calculating the centroid coordinates of each binary reflection circle in the binary reflection circle array image and then saving the relationship between the camera pixel at the centroid of each binary reflection circle and the screen pixel at the centroid of the corresponding binary projection circle further includes:

[0083] The formula for calculating the centroid coordinates of each binary reflection spot is as follows:

[0084]

[0085]

[0086] Among them, C x Let C be the x-coordinate of the centroid of each binary reflection spot. y Let I(x,y) be the ordinate of the centroid of each binary reflection spot, where I(x,y) is the pixel with a value of 1 in the binary reflection spot array image, x is the abscissa of the image, and y is the ordinate of the image; then, find the set C of coordinates of the centroids of the binary reflection spots on the upper surface of the transparent element to be tested. up 'and the set of centroid coordinates of the binary reflected circular spot on the lower surface of the transparent element under test, C dowm ', and the corresponding set of binary projected circular spot centroid coordinates S'.

[0087] The correspondence between each binary reflection circle and the binary projection circle is determined based on the centroid coordinates of each binary reflection circle; wherein, the binary reflection circles in the binary reflection circle array image include a first binary reflection circle on the upper surface of the transparent element under test and a second binary reflection circle on the lower surface of the transparent element under test;

[0088] Save the relationship between the camera pixel at the centroid of each first binary reflection spot and the screen pixel at the centroid of the corresponding binary projection spot, and simultaneously save the relationship between the camera pixel at the centroid of each second binary reflection spot and the screen pixel at the centroid of the corresponding binary projection spot.

[0089] Step S4: Project the stripe image on the screen onto the transparent element under test, and capture the superimposed stripes reflected from the upper and lower surfaces of the transparent element under test with a camera to obtain the superimposed stripe image of the transparent element under test.

[0090] It should be noted that sinusoidal fringe images in the horizontal and vertical directions are projected onto the transparent element under test.

[0091] Preferably, the step of capturing images of the superimposed fringes reflected from the upper and lower surfaces of the transparent element under test using a camera to obtain an image of the superimposed fringes of the transparent element under test further includes:

[0092] This deflection measurement system operates in a darkroom, and the camera imaging model is defined as follows:

[0093]

[0094] Where I is the pixel value of the aliased fringe image of the transparent element under test, A is the background light intensity, including the camera's dark level, a small amount of ambient light leakage, and other constant pixel value responses of the camera; B1 is the modulation index of the reflected fringe image on the upper surface of the transparent element under test, and B2 is the modulation index of the reflected fringe image on the lower surface of the transparent element under test. It is the phase of the reflected stripe image on the upper surface of the transparent element under test. It is the phase of the reflected stripe image on the lower surface of the transparent element under test.

[0095] Step S5: Based on the relationship between the camera pixel at the centroid of each binary reflection spot and the screen pixel at the centroid of the corresponding binary projection spot, calculate the first initial phase value of the reflection stripe image on the upper surface of the transparent element under test and the second initial phase value of the reflection stripe image on the lower surface of the transparent element under test; and optimize the modulation values ​​of each pair of camera pixels and screen pixels to obtain the first initial modulation value of the reflection stripe image on the upper surface of the transparent element under test and the second initial modulation value of the reflection stripe image on the lower surface of the transparent element under test.

[0096] Preferably, the step of calculating the first initial phase value of the reflective fringe image on the upper surface of the transparent element under test and the second initial phase value of the reflective fringe image on the lower surface of the transparent element under test based on the relationship between the camera pixel at the centroid of each binary reflective spot and the screen pixel at the centroid of the corresponding binary projected spot further includes:

[0097] Based on the relationship between the camera pixel at the centroid of each first binary reflection spot and the screen pixel at the centroid of the corresponding binary projection spot, the phase of the camera pixel position in the reflection stripe image on the upper surface of the transparent element under test is calculated. Based on the relationship between the camera pixel at the centroid of each second binary reflection spot and the screen pixel at the centroid of the corresponding binary projection spot, the phase of the camera pixel position in the reflection stripe image of the lower surface of the transparent element under test is calculated.

[0098] Based on the phase of the camera pixel positions in the reflective stripe image on the upper surface of the transparent element under test. The first initial phase value of the image of the reflective stripes on the upper surface of the transparent element under test is obtained by fitting. Based on the phase of the camera pixel positions in the reflective stripe image of the lower surface of the transparent element under test. The second initial phase value of the image of the reflective fringe on the lower surface of the transparent element under test is obtained by fitting.

[0099] Preferably, optimizing the modulation values ​​of each pair of camera pixels and screen pixels to obtain a first initial modulation value for the reflective stripe image of the upper surface of the transparent element under test and a second initial modulation value for the reflective stripe image of the lower surface of the transparent element under test further includes:

[0100] The modulation values ​​of the camera pixels at the centroid of each first binary reflection spot and the screen pixels at the centroid of the corresponding binary projection spot are optimized using the least squares method to obtain the first initial modulation value B of the reflection stripe image on the upper surface of the transparent element under test. 1init Simultaneously, the least squares method is used to optimize the modulation values ​​of the camera pixels at the centroid of each second binary reflection spot and the screen pixels at the centroid of the corresponding binary projection spot, so as to obtain the initial second modulation value B of the reflection stripe image on the lower surface of the transparent element under test. 2init The formula for the least squares method is as follows:

[0101]

[0102] in, The phase of the camera pixel position in the image of the reflected stripes on the upper surface of the transparent element under test. Let A be the phase of the camera pixel position in the image of the reflected stripes on the lower surface of the transparent element under test. init This is the initial value of the background light intensity. During the initial value estimation stage, the estimated B... 1init and B 2init It is a constant.

[0103] Step S6: As Figure 5 As shown, the initial values ​​of background light intensity, first phase, second phase, first modulation degree, and second modulation degree are optimized using a neural network to obtain optimized initial values ​​of background light intensity, first phase, second phase, first modulation degree, and second modulation degree.

[0104] Preferably, the step of optimizing the initial values ​​of background light intensity, first phase, second phase, first modulation degree, and second modulation degree using a neural network to obtain optimized initial values ​​of background light intensity, first phase, second phase, first modulation degree, and second modulation degree further includes:

[0105] The initial values ​​of background light intensity, first modulation degree, and second modulation degree are concatenated in the channel direction and then fed into the target convolutional neural network to obtain optimized initial values ​​of background light intensity, first modulation degree, and second modulation degree.

[0106] Specifically, the step of concatenating the initial value of the background light intensity, the initial value of the first modulation degree, and the initial value of the second modulation degree in the channel direction and then feeding them into the target convolutional neural network to obtain optimized initial values ​​of the background light intensity, the first modulation degree, and the second modulation degree, further includes:

[0107] like Figure 5 As shown, the initial value A of the background light intensity is... init The initial value of the first modulation system, B. 1init Second modulation initial value B 2init The images are concatenated along the channel direction to form a 3*H*W image. Then, after non-linear mapping through the target convolutional neural network, the optimized 3*H*W image is output, thus obtaining the optimized initial value of the background light intensity. Optimized initial values ​​of the first modulation system and the optimized initial value of the second modulation system

[0108] The target convolutional neural network includes a downsampling part and an upsampling part. The downsampling part includes two downsampling blocks, each of which includes a 2×2 max pooling layer. Each downsampling block is followed by two 3×3 convolutional layers. The upsampling part includes two upsampling blocks that are symmetrical to the downsampling blocks. Each upsampling block first performs a 2×2 transposed convolution, and then performs two 3×3 convolutions. The outputs of downsampling blocks with the same dimensions are connected to the inputs of each upsampling block.

[0109] The first initial phase value and the second initial phase value are respectively fed into the target fully connected neural network to obtain the optimized first initial phase value and the optimized second initial phase value.

[0110] Specifically, the step of feeding the first initial phase value and the second initial phase value into the target fully connected neural network respectively to obtain the optimized first initial phase value and the optimized second initial phase value further includes:

[0111] like Figure 5 As shown, the first phase initial value is... Second phase initial value The vector is converted into an H*W row, 1 column vector, and then input into the target fully connected neural network to output the optimized first phase initial value. and the optimized second phase initial value The target fully connected neural network comprises six fully connected layers with 256 output channels.

[0112] Preferred options also include:

[0113] The neural network is optimized using the Adam optimizer with a learning rate of 0.0001, which remains constant for the first 60 training iterations and then adjusts according to a linear decay strategy. During training, the network's weights are updated based on the loss value to achieve gradient descent. The overall loss function is defined as follows:

[0114] loss = α·l recon +β·l limit +γ·l smooth

[0115] Where α, β, and γ are hyperparameters, taking values ​​of 0.8, 0.1, and 0.1, respectively; recon To rebuild the losses, l limit To constrain the loss, l smooth For smoothness loss;

[0116] (1) Reconstruction loss recon The mean squared loss (MSE) is used, defined as follows:

[0117]

[0118] Where H is the number of pixels in the long direction of the image and W is the number of pixels in the wide direction of the image, with H and W being 2000 and 2000 respectively; I(i) represents the pixel value predicted by the neural network, and I(i) represents the pixel value of the aliased stripe image captured by the camera.

[0119] (2) Constraint loss l limit The definition is as follows:

[0120]

[0121] in, This represents the phase value of the corresponding camera pixel position predicted by the neural network. Phase value representing the position of a camera pixel Or the phase value at the camera pixel location N is the number of camera pixels;

[0122] (3) Smoothness loss l smooth The total variation loss is used and defined as follows:

[0123]

[0124] Where u,v represent the row and column coordinates in the image coordinate system, and x u,v These are the pixel values ​​at points u and v;

[0125] It should be noted that neural network training stops after the loss function converges or after 500 iterations.

[0126] Step S7: Generate a reflection stripe image of the upper surface of the transparent element under test based on the optimized initial value of background light intensity, the optimized initial value of the first phase, and the optimized initial value of the first modulation. At the same time, generate a reflection stripe image of the lower surface of the transparent element under test based on the optimized initial value of background light intensity, the optimized initial value of the second phase, and the optimized initial value of the second modulation, so as to separate the overlapping stripe images of the transparent element under test.

[0127] It should be noted that all operations of this invention must be performed in a dark room.

[0128] like Figures 6(a) to 6(b) As shown in the simulation experiment, the fringe modulation and phase results after neural network optimization are correct, and the neural network also has a certain suppression effect on noise.

[0129] This invention combines neural networks with physical imaging models, enabling the effective establishment of the correspondence between camera, workpiece, and screen pixels without the need for training data, thus achieving deflection measurement.

[0130] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for separating overlapping fringes of a transparent element in deflection measurement, characterized in that, Includes the following steps: Step S1: Construct a deflection measurement system consisting of a camera and a screen, and establish the pose relationship between the camera and the screen; Step S2: Take multiple black frame images with the camera to obtain the initial value of the background light intensity; Step S3: Project the binary projection circular spot array image on the screen onto the transparent element under test, and capture the reflection circular spot array reflected from the upper and lower surfaces of the transparent element under test using a camera. Perform binarization processing on the captured reflection circular spot array image to obtain a binary reflection circular spot array image. The centroid coordinates of each binary reflection circle in the binary reflection circle array image are calculated, and then the relationship between the camera pixel at the centroid of each binary reflection circle and the screen pixel at the centroid of the corresponding binary projection circle is saved. Step S4: Project the stripe image on the screen onto the transparent element under test, and capture the superimposed stripes reflected from the upper and lower surfaces of the transparent element under test with a camera to obtain the superimposed stripe image of the transparent element under test. Step S5: Based on the relationship between the camera pixel at the centroid of each binary reflection spot and the screen pixel at the centroid of the corresponding binary projection spot, calculate the first initial phase value of the reflection stripe image on the upper surface of the transparent element under test and the second initial phase value of the reflection stripe image on the lower surface of the transparent element under test. And optimize the modulation values ​​of each pair of camera pixels and screen pixels to obtain the first initial modulation value of the reflective stripe image on the upper surface of the transparent element under test and the second initial modulation value of the reflective stripe image on the lower surface of the transparent element under test. Step S6: Optimize the initial values ​​of background light intensity, first phase, second phase, first modulation degree, and second modulation degree using a neural network to obtain optimized initial values ​​of background light intensity, first phase, second phase, first modulation degree, and second modulation degree. Step S7: Generate a reflection stripe image of the upper surface of the transparent element under test based on the optimized initial value of background light intensity, the optimized initial value of the first phase, and the optimized initial value of the first modulation. At the same time, generate a reflection stripe image of the lower surface of the transparent element under test based on the optimized initial value of background light intensity, the optimized initial value of the second phase, and the optimized initial value of the second modulation, so as to separate the overlapping stripe images of the transparent element under test.

2. The method for separating overlapping fringes of a transparent element in deflection measurement according to claim 1, characterized in that, The step of capturing multiple black frame images with a camera to obtain the initial value of the background light intensity also includes: The initial value of the background light intensity The calculation formula is as follows: in, is the brightness of the i-th black frame image, and n is the number of black frame images.

3. The method for separating overlapping fringes of a transparent element in deflection measurement according to claim 2, characterized in that, The process of calculating the centroid coordinates of each binary reflection circle in the binary reflection circle array image and then saving the relationship between the camera pixel at the centroid of each binary reflection circle and the screen pixel at the centroid of the corresponding binary projection circle also includes: The formula for calculating the centroid coordinates of each binary reflection spot is as follows: Among them, C x Let C be the x-coordinate of the centroid of each binary reflection spot. y Let be the ordinate of the centroid of each binary reflection spot. Let x be the pixel with a value of 1 in the binary reflectance circular spot array image, and y be the image's horizontal coordinate and vertical coordinate. The correspondence between each binary reflection circle and the binary projection circle is determined based on the centroid coordinates of each binary reflection circle; wherein, the binary reflection circles in the binary reflection circle array image include a first binary reflection circle on the upper surface of the transparent element under test and a second binary reflection circle on the lower surface of the transparent element under test; Save the relationship between the camera pixel at the centroid of each first binary reflection spot and the screen pixel at the centroid of the corresponding binary projection spot, and simultaneously save the relationship between the camera pixel at the centroid of each second binary reflection spot and the screen pixel at the centroid of the corresponding binary projection spot.

4. The method for separating overlapping fringes of a transparent element in deflection measurement according to claim 3, characterized in that, The step of capturing images of the superimposed fringes reflected from the upper and lower surfaces of the transparent element under test using a camera to obtain an image of the superimposed fringes of the transparent element under test also includes: The camera imaging model is defined as follows: Where I is the pixel value of the aliased stripe image of the transparent element under test, A is the background light intensity, B1 is the modulation degree of the reflected stripe image on the upper surface of the transparent element under test, B2 is the modulation degree of the reflected stripe image on the lower surface of the transparent element under test, φ1 is the phase of the reflected stripe image on the upper surface of the transparent element under test, and φ2 is the phase of the reflected stripe image on the lower surface of the transparent element under test.

5. The method for separating overlapping fringes of a transparent element in deflection measurement according to claim 4, characterized in that, The step of calculating the first initial phase value of the reflective fringe image on the upper surface of the transparent element under test and the second initial phase value of the reflective fringe image on the lower surface of the transparent element under test based on the relationship between the camera pixel at the centroid of each binary reflective spot and the screen pixel at the centroid of the corresponding binary projected spot further includes: Based on the relationship between the camera pixel at the centroid of each first binary reflection spot and the screen pixel at the centroid of the corresponding binary projection spot, the phase φ1' of the camera pixel position in the reflection stripe image on the upper surface of the transparent element under test is calculated; based on the relationship between the camera pixel at the centroid of each second binary reflection spot and the screen pixel at the centroid of the corresponding binary projection spot, the phase φ2' of the camera pixel position in the reflection stripe image on the lower surface of the transparent element under test is calculated. Based on the phase φ1' of the camera pixel position in the reflective fringe image of the upper surface of the transparent element under test, the first initial phase value φ of the reflective fringe image of the upper surface of the transparent element under test is fitted to obtain the initial phase value φ. 1init Based on the phase φ2' of the camera pixel position in the reflection stripe image of the lower surface of the transparent element under test, the second initial phase value φ of the reflection stripe image of the lower surface of the transparent element under test is fitted to obtain the second initial phase value φ. 2init .

6. The method for separating overlapping fringes of a transparent element in deflection measurement according to claim 5, characterized in that, The optimization of the modulation values ​​of each pair of camera pixels and screen pixels to obtain the first initial modulation value of the reflective stripe image on the upper surface of the transparent element under test and the second initial modulation value of the reflective stripe image on the lower surface of the transparent element under test further includes: The modulation values ​​of the camera pixels at the centroid of each first binary reflection spot and the screen pixels at the centroid of the corresponding binary projection spot are optimized using the least squares method to obtain the first initial modulation value B of the reflection stripe image on the upper surface of the transparent element under test. 1init Simultaneously, the least squares method is used to optimize the modulation values ​​of the camera pixels at the centroid of each second binary reflection spot and the screen pixels at the centroid of the corresponding binary projection spot, so as to obtain the initial second modulation value B of the reflection stripe image on the lower surface of the transparent element under test. 2init The formula for the least squares method is as follows: Where φ1' is the phase of the camera pixel position in the reflected stripe image of the upper surface of the transparent element under test, φ2' is the phase of the camera pixel position in the reflected stripe image of the lower surface of the transparent element under test, and A init This is the initial value of the background light intensity.

7. The method for separating overlapping fringes of a transparent element in deflection measurement according to claim 6, characterized in that, The optimization of the initial values ​​of background light intensity, first phase, second phase, first modulation degree, and second modulation degree using a neural network to obtain optimized initial values ​​of background light intensity, first phase, second phase, first modulation degree, and second modulation degree further includes: The initial values ​​of background light intensity, first modulation degree, and second modulation degree are concatenated in the channel direction and then fed into the target convolutional neural network to obtain optimized initial values ​​of background light intensity, first modulation degree, and second modulation degree. The first initial phase value and the second initial phase value are respectively fed into the target fully connected neural network to obtain the optimized first initial phase value and the optimized second initial phase value.

8. The method for separating overlapping fringes of a transparent element in deflection measurement according to claim 7, characterized in that, The step of concatenating the initial values ​​of background light intensity, first modulation degree, and second modulation degree along the channel direction and then feeding them into the target convolutional neural network to obtain optimized initial values ​​of background light intensity, first modulation degree, and second modulation degree further includes: The initial value A of the background light intensity init The initial value of the first modulation system, B. 1init Second modulation initial value B 2init The images are concatenated along the channel direction to form a 3*H*W image. Then, after non-linear mapping through the target convolutional neural network, the optimized 3*H*W image is output, thus obtaining the optimized initial value of the background light intensity. Optimized initial value of the first adjustment system and the optimized initial value of the second modulation system Where H is the number of pixels in the long direction of the image, and W is the number of pixels in the wide direction of the image; The target convolutional neural network includes a downsampling part and an upsampling part. The downsampling part includes two downsampling blocks, each of which includes a 2×2 max pooling layer. Each downsampling block is followed by two 3×3 convolutional layers. The upsampling part includes two upsampling blocks that are symmetrical to the downsampling blocks. Each upsampling block first performs a 2×2 transposed convolution, and then performs two 3×3 convolutions. The outputs of downsampling blocks with the same dimensions are connected to the inputs of each upsampling block.

9. The method for separating overlapping fringes of a transparent element in deflection measurement according to claim 7, characterized in that, The step of feeding the first initial phase value and the second initial phase value into the target fully connected neural network respectively to obtain the optimized first initial phase value and the optimized second initial phase value further includes: The first phase initial value φ 1init Second phase initial value φ 2init The vector is converted into an H*W row, 1 column vector, and then input into the target fully connected neural network to output the optimized first phase initial value. and the optimized second phase initial value The target fully connected neural network comprises six fully connected layers with 256 output channels.

10. The method for separating overlapping fringes of a transparent element in deflection measurement according to claim 7, characterized in that, Also includes: When training the neural network, the total loss function The definition is as follows: Where α, β, and γ are hyperparameters. To rebuild the losses, To limit losses, For smoothness loss; (1) Reconstruction losses The mean squared loss is defined as follows: Where H is the number of pixels in the long direction of the image, and W is the number of pixels in the wide direction of the image; This represents the pixel value predicted by the neural network. This represents the pixel value of the aliased stripe image captured by the camera; (2) Constraint loss The definition is as follows: in, This represents the phase value of the corresponding camera pixel position predicted by the neural network. φ1' or φ2' represents the phase value of a camera pixel position; N is the number of camera pixels. (3) Smoothness loss The definition is as follows: Where u,v represent the row and column coordinates in the image coordinate system, and x u,v These are the pixel values ​​at points u and v.

Images