Phase deflection application method and system based on orthogonal imaging model

Abstract

The invention relates to a phase deflection application method and system based on an orthogonal imaging model, a telecentric lens is introduced to replace a traditional imaging lens, carrier frequency phase nonlinear errors caused by light non-parallelism are avoided, and the three-dimensional reconstruction precision is remarkably improved. In order to solve the problem that calibration in the telecentric system is tedious, a special double-layer circular calibration plate (ceramic and photoetching glass) is designed, the use of a high-precision mobile platform is avoided, and the system cost is reduced. Meanwhile, aiming at the problems of unclear images, nonlinear errors and the like in the view field generated by a telecentric system, the image quality and the measurement precision of the system are remarkably improved by combining an image sharpening technology with an image deblurring technology of a Restormer network and using a flat field correction and gamma correction method.

Description

Technical Field

[0001] The present disclosure relates to the field of three-dimensional measurement technology, and in particular to a phase deflection application system based on an orthogonal imaging model, a phase deflection application method based on an orthogonal imaging model, and an electronic device. Background Art

[0002] Three-dimensional imaging technology is a core technology that can obtain the three-dimensional coordinates and features (such as reflectivity) of an object's surface. Its research background covers the needs and development of multiple fields. With the advent of the digital age, three-dimensional imaging technology has received widespread attention and application, especially in industrial manufacturing, optical engineering, medical imaging and other fields. The emergence of this technology stems from the limitations of two-dimensional imaging. Traditional two-dimensional images can only reflect the plane information of an object and cannot accurately describe its three-dimensional shape. By introducing depth information, three-dimensional imaging technology can provide richer and more comprehensive visual perception.

[0003] Phase deflectometry (PMD), as an optical measurement technology based on the reflection principle, has received increasing attention in recent years. Compared with traditional three-dimensional measurement methods, PMD can efficiently and accurately extract the phase information of the object surface by precisely controlling the interaction between light and mirror objects, thereby realizing the three-dimensional morphology reconstruction of mirror objects. This method has significant application prospects in the field of high-precision measurement of mirror objects, especially in scenarios such as optical component manufacturing and automotive surface inspection that have extremely high requirements for surface smoothness. PMD provides a new idea to break through the bottleneck of traditional measurement.

[0004] However, the three-dimensional measurement of highly reflective mirror objects is still a technical challenge. Although the phase deflection technology (PMD) based on the traditional pinhole imaging model can solve the three-dimensional measurement of highly reflective mirror objects, it will also introduce nonlinear carrier frequency errors, which makes it difficult to meet high-precision requirements.

[0005] In addition, traditional PMD systems usually use ordinary lenses that have large near and small far imaging. This lens will cause significant carrier frequency phase nonlinear errors due to the perspective projection effect, especially in complex surfaces or large-angle reflections, resulting in reduced measurement accuracy. At the same time, nonlinear distortion in the lens field of view will further affect the 3D reconstruction results and cannot meet the needs of higher-precision measurements.

[0006] PMD technology is based on the law of optical reflection. By replacing the projection device (projector) in the structured light projection method with a display device (display), phase information is extracted from the "passively" reflected structured light. Then a gradient calculation model is constructed to obtain gradient data, and finally the three-dimensional morphology of the surface is restored based on the gradient data integral. In the current telecentric PMD system, the calibration of the telecentric system is relatively complicated, especially when the positive and negative signs of the calibration parameters are determined, there will be ambiguity, and the mechanical adjustment process of the load is often required. In addition, the use of telecentric lenses will have a small imaging depth of field, resulting in reduced image quality and affecting subsequent three-dimensional reconstruction. Therefore, although the orthogonal imaging model solves the problem of objects being larger when near and smaller when far, the initial calibration process is relatively complicated and will introduce the problem of small imaging depth of field, which will cause defocus blur, that is, unclear feature points and other problems, which will affect the subsequent 3D reconstruction accuracy; the uneven response of the camera sensor and the uneven lighting will affect the image quality and cause inconsistent brightness in the image. The display screen is used to project sinusoidal fringe patterns, and the LCD display screen has a nonlinear brightness response problem, that is, the output grayscale value and the actual brightness are not linearly related. The nonlinear errors introduced by the display screen, the camera itself and the ambient light cause the overall linearity of the system to decrease, affecting the phase extraction and surface gradient calculation, and thus affecting the final 3D reconstruction accuracy. Summary of the invention

[0007] On the one hand, the present application proposes a phase deflection application system based on an orthogonal imaging model, comprising:

[0008] An LCD display for projecting a pre-coded sinusoidal fringe pattern onto the surface of the object being measured;

[0009] A rotating platform, used to provide three-axis tilt drive for the object to be measured and the calibration plate placed thereon; wherein the calibration plate is used to provide calibration reference data of different angles and postures for the CMOS camera;

[0010] A CMOS camera, used for capturing the deformation pattern reflected by the surface of the measured object and the calibration reference data through a telecentric lens thereon, and uploading them to a computer;

[0011] A computer is used to control the LCD display screen, the rotating platform and the CMOS camera; and, receiving and, according to the calibration reference data, performing analytical modulation on the deformation pattern, obtaining the surface shape height information of the object to be measured and constructing an orthogonal imaging model corresponding to the object to be measured based on the three-dimensional reconstruction technology; and, using the image sharpening technology combined with the deblurring technology of the Restormer model and the flat field correction and gamma correction methods, performing deblurring and correction enhancement processing on the orthogonal imaging model, and regenerating and outputting the restored image of the orthogonal imaging model;

[0012] The LCD display screen, the rotating platform and the CMOS camera are respectively connected to the computer for communication.

[0013] As an optional implementation scheme of the present application, optionally, the calibration plate is a double-layer circular calibration plate, which is composed of an upper and lower glass calibration plate and a ceramic calibration plate; the glass calibration plate and the ceramic calibration plate are both provided with a plurality of circular holes arranged in a matrix array and with upper and lower phase deflections, and the center of the circular hole is used to provide the feature points required for calibration.

[0014] As an optional implementation scheme of the present application, optionally, the computer is also used to: pre-generate the sinusoidal fringe pattern and send it to the LCD display screen.

[0015] As an optional implementation scheme of the present application, optionally, the computer is also used to: pre-generate rotation control instructions for the rotating platform, and control the rotating platform to present different angles and postures, so that the calibration plate presents different reference surfaces, for providing calibration reference data of different angles and postures for the CMOS camera.

[0016] As an optional implementation scheme of the present application, optionally, on a computer, the image sharpening technology combined with the deblurring technology of the Restormer model and the flat field correction and gamma correction methods are used to perform deblurring and correction enhancement processing on the orthogonal imaging model, and regenerate and output a restored image of the orthogonal imaging model, including:

[0017] Deblurring the orthogonal imaging model using an image sharpening technique combined with a deblurring technique of a Restormer model;

[0018] Using flat field correction and gamma correction methods, the orthogonal imaging model is subjected to image correction and enhancement processing;

[0019] After the processing is completed, the restored image of the orthogonal imaging model is regenerated and output.

[0020] On the other hand, the present application proposes a phase deflection application method based on an orthogonal imaging model, which is implemented based on the above-mentioned phase deflection application system based on the orthogonal imaging model, and includes the following steps:

[0021] The computer generates a sinusoidal fringe pattern in advance and sends it to the LCD display screen, and simultaneously sends a rotation control instruction;

[0022] The rotating platform responds to the rotation control instruction and presents a corresponding angle and posture, so that the calibration plate on it presents a corresponding reference surface, providing calibration reference data of the corresponding angle and posture for the CMOS camera;

[0023] The LCD display screen projects a pre-coded sinusoidal fringe pattern onto the surface of the object to be measured;

[0024] The computer activates the CMOS camera, captures the deformation pattern reflected by the surface of the measured object and the calibration reference data through the telecentric lens thereon, and uploads them to the computer;

[0025] According to the above steps, several groups of the calibration reference data and the deformation patterns at different angles and postures are collected and uploaded to a computer;

[0026] The computer receives and analyzes and modulates the deformation pattern according to the calibration reference data, obtains the surface shape height information of the object under test and constructs an orthogonal imaging model corresponding to the object under test based on the three-dimensional reconstruction technology; and uses image sharpening technology combined with the deblurring technology of the Restormer model as well as the flat field correction and gamma correction methods to deblur and correct the orthogonal imaging model, and regenerates and outputs the restored image of the orthogonal imaging model.

[0027] In another aspect, the present application further provides an electronic device, comprising:

[0028] processor;

[0029] a memory for storing processor-executable instructions;

[0030] Wherein, the processor is configured to implement the phase deflection application method based on the orthogonal imaging model when executing the executable instructions.

[0031] Technical effects of the present invention:

[0032] This application avoids the carrier frequency phase nonlinear error caused by the non-parallelism of light by introducing a telecentric lens instead of a traditional camera, and significantly improves the accuracy of 3D reconstruction. At the same time, in order to solve the problems of unclear images and nonlinear errors in the field of view generated by the telecentric system, the image sharpening technology combined with the image deblurring technology of the Restormer network and the use of flat field correction and gamma correction methods significantly improves the image quality and measurement accuracy of the system.

[0033] The main advantages of the present invention include:

[0034] (1) A phase deflection system based on the orthogonal imaging model was built, and a flat calibration plate with a known height was designed for the calibration of the orthogonal imaging model. This method solves the problem of ambiguity in the signs of external parameters in traditional calibration, while ensuring the calibration accuracy and saving the high cost of a high-precision mobile platform.

[0035] (2) Since the imaging depth of the orthogonal imaging model is small, it is easy to cause blurred imaging within the same field of view. To solve this problem, this paper uses image sharpening technology combined with the Restormer deep learning network model to deblur the image. The model effectively restores the details of the blurred area, and the image sharpening technology further improves the edge clarity, thereby overcoming the blur problem caused by the small imaging depth of field, significantly improving the accuracy of corner detection and image quality, and reducing PMD system errors.

[0036] (3) In the orthogonal imaging model, the system still has nonlinear errors of the imaging system, namely, the nonlinear response of the sensor and the inconsistent response of the camera image sensor and the difference in illumination. This paper draws on the flat field correction method to complete the correction of the uneven response of the camera sensor in the PMD system, as well as the artifacts and noise caused by the uneven distribution of ambient light, and completes the gamma correction of the display screen. The measurement accuracy and stability of the system have been significantly improved.

[0037] Experimental results show that the proposed method is superior to the traditional PMD system in the 3D reconstruction of standard concave mirrors, showing higher accuracy and robustness, effectively reducing system errors, and opening up a new technical path for high-precision and low-cost 3D measurement of mirror objects. In the future, the adaptability and application potential of this method in complex industrial scenarios will be further studied.

[0038] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0040] Figure 1 It is a schematic diagram of the system composition structure of the present invention;

[0041] Figure 2 It is a schematic diagram of imaging of a theoretical model of a telecentric lens of the present invention;

[0042] Figure 3 It is a schematic diagram of geometric calibration of the PMD system of the present invention;

[0043] Figure 4 A schematic diagram showing comparison of image deblurring and reconstruction results of the present invention;

[0044] Figure 5 Schematic diagram of the circular calibration plate of the present invention in application state: (a) schematic diagram of the calibration plate, (b) actual diagram of two circular calibration plates combined;

[0045] Figure 6It is a schematic diagram of the detection result of the center of the ceramic calibration plate of the present invention;

[0046] Figure 7 Taking images for different lighting conditions of the present invention;

[0047] Figure 8 This is a flat field correction effect diagram of the present invention;

[0048] Fig. 9 The histogram comparison before and after the flat field correction of the present invention is (a) the original image and (b) the flat field corrected image;

[0049] Fig.10 Comparison of grayscale perceived by the human eye: (a) grayscale perceived by the human eye, (b) actual grayscale;

[0050] Fig.11 Grayscale image acquisition for the present invention: (a) grayscale image generated by computer, (b) grayscale image taken by camera;

[0051] Fig.12 It is the grayscale fitting of the response curve of the present invention;

[0052] Fig.13 The grayscale response curves of the display screen of the present invention at different contrasts;

[0053] Fig.14 is a symmetrical compensation curve diagram of the present invention;

[0054] Fig.15 The image after nonlinear compensation correction of the present invention;

[0055] Fig.16 It is a schematic diagram of the system composition structure of the electronic device of the present invention. DETAILED DESCRIPTION

[0056] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0057] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0058] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

[0059] Example 1

[0060] On the one hand, the present application proposes a phase deflection application system based on an orthogonal imaging model, comprising:

[0061] An LCD display for projecting a pre-coded sinusoidal fringe pattern onto the surface of the object being measured;

[0062] A rotating platform, used to provide three-axis tilt drive for the object to be measured and the calibration plate placed thereon; wherein the calibration plate is used to provide calibration reference data of different angles and postures for the CMOS camera;

[0063] A CMOS camera, used for capturing the deformation pattern reflected by the surface of the measured object and the calibration reference data through a telecentric lens thereon, and uploading them to a computer;

[0064] A computer is used to control the LCD display screen, the rotating platform and the CMOS camera; and, receiving and, according to the calibration reference data, performing analytical modulation on the deformation pattern, obtaining the surface shape height information of the object to be measured and constructing an orthogonal imaging model corresponding to the object to be measured based on the three-dimensional reconstruction technology; and, using the image sharpening technology combined with the deblurring technology of the Restormer model and the flat field correction and gamma correction methods, performing deblurring and correction enhancement processing on the orthogonal imaging model, and regenerating and outputting the restored image of the orthogonal imaging model;

[0065] The LCD display screen, the rotating platform and the CMOS camera are respectively connected to the computer for communication.

[0066] like Figure 1 The figure shows the structure of the system. The hardware of the system (PMD system for short) consists of a telecentric lens (set on a CMOS camera), a CMOS camera, a computer, an LCD display, and a rotating platform. The CMOS camera is connected to the 10G Ethernet port of the computer via a network cable for real-time image data acquisition. A 24-inch LCD display (computer side) is used for image display and control, and a 27-inch LCD display displays sinusoidal fringes for measurement, which are connected to the display DP interface of the computer.

[0067] The rotating platform is a three-axis tilting platform, which can be selected by the user, such as the precision manual angular displacement stage of model OMXC114C. The angular displacement stage has a tilt angle range of ±4°, a pitch angle range of ±4°, and a rotation angle range of ±4°. It has three degrees of freedom, the position can be adjusted freely, and small angle rotation adjustment can be performed.

[0068] PMD systems can be divided into various different systems according to the number of cameras used and the type of lenses used. The system constructed in this paper mainly includes an LCD display, a high-precision camera and a plane for reference. The principle of the system is to project the encoded sinusoidal fringe pattern onto the surface of the object under test through an LCD display through computer control. Subsequently, the camera captures the deformation of these stripes on the surface of the object, which reflects the unevenness of the surface of the object. By accurately calibrating the camera and obtaining the intrinsic parameters of the camera, this information helps to analyze the modulated fringe pattern. Then, the system combines the captured fringe pattern with the system geometric structure parameters previously calculated or measured to infer the distribution relationship of the surface gradient.

[0069] After obtaining the gradient information, this system uses mathematical integration technology to process these gradient relationships and finally converts them into accurate height information, thereby achieving detailed reconstruction of the surface shape of the object.

[0070] A fringe pattern with a known period is projected on the LCD display. Any light ray emitted by the pixel of the CMOS camera is reflected from point O on the reference surface to point A on the display. When there is a defect or protrusion on the surface to be tested, which causes an angle θ change between the surface to be tested and the reference surface, the point where the light ray passing through point O is reflected on the LCD display is offset to point A', and the offset angle ∠AOA' is 2θ.

[0071] Let ∠OAA'=α and we can get from the sine theorem of triangle:

[0072]

[0073] The known period of the stripes on the LCD display is T, and the phase change represented by AA' is known to be:

[0074]

[0075] Combining equations (2-1) and (2-2), we can get the phase gradient relationship in the telecentric phase deflection system:

[0076]

[0077] According to formula (2-3), the surface gradient distribution of the test surface in two vertical directions can be obtained respectively:

[0078]

[0079] In the PMD system, it should be pointed out that the height of the object to be measured must be much smaller than |OA| (the distance from the display screen to the reference plane) for each pixel in the formula derivation. In actual use, the height of the object to be measured must be close to the height of the reference plane. When α is approximately close to 90, formula (2-3) can be simplified to

[0080]

[0081] In actual measurement and analysis, using a single-direction sinusoidal fringe pattern often fails to fully capture the details of the surface gradient. Therefore, in actual operation, eight different directions of projected fringes (four-step shift) are often used to more comprehensively obtain the gradient information of the object surface. In this way, by integrating the gradient information in the horizontal and vertical directions, the surface height of the object can be reconstructed more accurately. This method can effectively improve the accuracy and reliability of measurement.

[0082] When using an LCD display to take multiple sinusoidal fringe images with different initial phases through a camera, the number of phase shifts in the phase shift method is □. According to the phase shift method, the light intensity I of each pixel on the sinusoidal fringe image reflected by the measured element is n (x,y) can be expressed by the following formula:

[0083]

[0084] Where A(x,y) is the background light intensity, B(x,y) is the intensity modulation parameter, is the phase encoding value, Indicates the relative position change of the fringe image. By shooting the common intensity image, there are three unknown quantities A(x, y), B(x, y) and So the phase to be demodulated, the N value is at least three, It can be deduced as:

[0085]

[0086] Different phase shifting methods have their own characteristics. The four-step phase shifting method can effectively eliminate the interference of background items and surface reflectivity on the measurement results. Because of the symmetry relationship of the four-step phase shift, it can effectively reduce the symmetric phase error compared to the three-step phase shifting method. The five-step phase shifting method can ensure that the influence of the phase shift error is reduced in terms of accuracy, but it also has higher requirements on the number of sinusoidal fringe images collected and photographed. Generally, with the increase of the number of phase shifting steps, the accuracy of the measurement will also improve, but the more phase shifting steps, the more sinusoidal fringe images are required, which will lead to the cumulative increase of the amount of calculation and the increase of the time required to process the image. Therefore, considering both the calculation speed and the experimental accuracy, the four-step phase shifting method is used to complete the phase extraction of the sinusoidal fringe image.

[0087] Unlike traditional structured light techniques such as phase measurement profilometry, the PMP system obtains height information directly through the unwrapped phase information. The surface height information of high-gloss objects is obtained by integrating the horizontal and vertical gradient data in the phase deflectometry system (PMD).

[0088] Since the height information cannot be directly obtained from the phase information, the three-dimensional morphology can only be obtained by integrating the surface gradient. The relationship between the gradient and the height is expressed as:

[0089]

[0090] where g x (x,y) and g y (x, y) represents the calculated gradient values ​​in the x direction and y direction respectively. Therefore, the final height distribution of the surface of the object to be measured is required, and the gradients in the two perpendicular directions are integrated:

[0091] Z = ∫g x dx+g y dy(0-9)

[0092] The direct integration method as shown in formula (2-14) is one of the simplest and most intuitive reconstruction methods in phase deflection. It reconstructs the original phase or height distribution by integrating the known phase gradient element by element. Although the method is simple and intuitive, and has high computational efficiency, it has some limitations in practical applications, such as sensitivity to noise and dependence on the integration path. Due to its sensitivity to noise, it is very easy to cause accumulated errors. At the same time, the direct section division method is highly dependent on the integration path, and different path integrals may lead to completely different reconstruction results.

[0093] like Figure 2 As shown, this is the imaging diagram of the theoretical model of the telecentric lens.

[0094] Z C The direction is the direction of the camera optical axis, and the center of the focal plane of the camera lens is taken as the center of the camera coordinate system O C Build X C Y C Z C Three-dimensional rectangular coordinate system, along Z C Direction O W Establish a 3D right angle X for the world origin W Y W Z WThe world coordinate system plane ∏1 is used to describe the actual position of any object in the physical world environment. Plane ∏2' is the actual imaging plane of the CMOS camera. For the convenience of calculation and reference, the imaging plane ∏2 is established as the projection of the focus coincidence point in the vertical direction of the CMOS image plane. Plane ∏2 is a virtual imaging plane obtained by symmetric rotation transformation about the camera coordinate system. Plane ∏3 is the plane where the computer image is located. Point P1 (x1, y1) is the actual point in the world coordinate system. Point P 2μ (x 2μ ,y 2μ ) is the ideal image point corresponding to the ideal point P1 on the imaging plane ∏2. The actual image point when considering lens distortion is the point P on the plane ∏2'. 2d (x 2d ,y 2d ).

[0095] According to the theoretical model imaging diagram of the telecentric lens, the ideal imaging is actually the conversion process of three-dimensional coordinates. The relationship between the world coordinate system and the camera coordinate system can be described by the 3×3 rotation matrix R and the translation vector T. The coordinate value of the point in the camera coordinate is recorded as (x c ,y c ,z c ), so the corresponding relationship matrix between the point P1(x1,y1) in the world coordinate system and the camera coordinate system is:

[0096]

[0097] Where r ij is the element of the rotation matrix R, [t x t y t z ] T is the translation vector T.

[0098] The conversion from camera coordinates to image coordinates does not have a projection center due to the nature of orthogonal projection of a double telecentric lens. Therefore, the projection relationship matrix from the projection point to the image coordinates is expressed as:

[0099]

[0100] In formula (3-2), m is the magnification of the lens. The difference between the telecentric imaging model and the pinhole imaging model is that the magnification of the pinhole imaging is the distance from the object to the optical center. The magnification changes with the distance of the object lens, but once the telecentric lens and camera model are determined, the magnification is fixed and does not change with depth.

[0101] The transformation relationship matrix between the image pixel coordinates of a point on the computer screen and the corresponding point on the imaging plane ∏2 is:

[0102]

[0103] In formula (3-3), d x and d y are the actual physical sizes of unit pixels in the x and y directions respectively. The mathematical transformation matrix of an actual physical point from the camera coordinate system to the pixel coordinate system is:

[0104]

[0105] Due to the orthogonal projection property of the telecentric lens, there is no principal point ((u0, v0). And because the physical coordinate system is established on the target plane, z1 = 0. Combining equations (3-1), (3-2) and (3-3), we get the mathematical transformation relationship matrix from world coordinates to pixel coordinates:

[0106]

[0107] The calibration mode used by the system of the present invention is a flat circular calibration plate. The calibration plate at different angles and positions is photographed by a fixed telecentric camera. For each acquired view of the calibration plate, the plane where the 2D circular calibration plate is located is taken as the X W OY W The direction perpendicular to the plane calibration plate is set to Z W The world coordinate system is established by the axis, so we can rewrite equation (3-5) as follows:

[0108]

[0109] In the formula, A and Ks are the internal parameter matrix of the telecentric camera and the external parameter matrix after the stage. The formula is different from the pinhole camera model formed by the ordinary lens. Because of the orthogonal projection property of the telecentric lens on the object side, t z It is a meaningless parameter, the equation is linear, and there is no proportional factor on the left side of equation (3-9).

[0110] From formula (3-9), we can know that:

[0111]

[0112] From H = AK s , can be deduced.

[0113]

[0114] Since the rotation matrix R is an orthogonal matrix, it satisfies the following constraints:

[0115]

[0116] Let T s=(t x t y ) T , C T =(r 31 r 32 ), we can conclude

[0117]

[0118] I 2×2 is the truncated matrix of the upper left corner of the unit matrix, combined with formula (3-12), we get:

[0119]

[0120] The following homography matrix constraint formula describing the camera's internal and external parameters can be derived through equations (3-11) and (3-14):

[0121]

[0122] The unknown number that needs to be determined in the formula is l1=α 2 β 2 , l2=α 2 +γ 2 , l3=β 2 , l4=βγ, so we can list the corresponding mathematical constraint matrix equation to solve it, let The matrix equation is:

[0123]

[0124] There are 4 unknowns to be solved in equation (3-16), and at least N point pairs from 4 images are needed to get the final solution. In order to maximize the accuracy of the solution, we take at least 15 pictures of the calibration plate in different poses to use the least squares method to calculate L = (G T G) -1 Gw can be calculated. The final internal parameter coefficient solution is:

[0125]

[0126] From this (3-17), the internal parameter matrix of the telecentric lens calibration has been obtained, and the truncated external parameter matrix of the i-th image can also be obtained:

[0127]

[0128] For each image, the camera's external parameter matrix is ​​different. From (3-9), we can see that for the i-th image, only the 2×2 R in the upper left corner of the rotation matrix R s The elements in the matrix can be directly obtained from Since R is orthogonal, the remaining matrix elements can be obtained by:

[0129]

[0130] r3=r1×r2(0-27)

[0131] Among them, r1, r2, r3 are all column vectors of the rotation matrix R. In the above formula, only r 13 and r 23 The value of , but the signs of the two cannot be obtained. From a physical point of view, this is a problem in which the telecentric camera cannot confirm whether the deflection is perpendicular to the calibration plate, upward or downward.

[0132] In order to get r 13 and r 23 All the information needs to be introduced into Z W Since the calibration mode is based on a 2D planar target, no markers containing Z-axis information are available. 13 and r 23 The relationship between the signs of is:

[0133]

[0134] Want to restore 13 and r 23 The sign of , only the translation displacement Z on the original image is required d Get a new image with Z d Substitute the coordinates of into the above formula (3-22) to determine r 13 and r 23 The sign of the actual displacement Z d It is not necessarily guaranteed to be perfectly perpendicular to the calibration plate and Z W There will be translation error when the direction is parallel, and the restored r 13 and r 23 It may not completely satisfy the original ideal equations (3-19) and (3-20). However, their positive and negative signs can be determined and satisfy the ideal values. Therefore, the rotation matrix R can be fully restored, solving the key problem of using this calibration method.

[0135] To determine r 13 and r 23 The sign of the translation displacement Z is required dTherefore, the present invention designs a method that is specially used for the telecentric lens calibration of this system and does not require the control of a high-precision mobile platform with high cost. Different from the traditional method, when the calibration plate is tilted too much when shooting with a telecentric camera, the defocus degree of the captured image will also increase. The calibration plate in the form of a matrix queue will affect the accuracy of the corner point detection algorithm. For this reason, a calibration plate with circular dots is selected to find the characteristic circle on the image, and the center of the circle is used as the characteristic point to complete the calibration. A special planar circular calibration plate is designed in combination with an algorithm to achieve r 13 and r 23 The sign restoration of θ also avoids the influence of camera defocus on the calibration accuracy (see the subsequent correction section).

[0136] like Figure 3 The PMD system geometry calibration diagram is shown. The geometry calibration of the system is based on the camera calibration. Each pixel in the system can be calibrated through the geometry calibration to calculate the corresponding α The values ​​of parameters such as |OA| and the integration step required to integrate the gradient information to reconstruct the height information. When calibrating the geometry, it is necessary to establish the precise spatial correspondence between the LCD display and the reference surface. The integration step used in the calculation is based on the image captured by the camera, that is, the actual distance between two adjacent pixels in the image on the reference plane. This setting helps ensure that the data obtained through image analysis accurately reflects the physical size.

[0137] The spatial position calibration of the display screen is not only complex but also the calibration accuracy is easily affected by the imaging quality of the display screen, the flatness of the plane mirror, the defocus degree of the calibration plate when shooting the display screen, etc.

[0138] According to formula (2-3), for each pixel point (u, v) of the camera imaged in this system, α , |OA|, θ are all different, so they can be changed to:

[0139]

[0140] Since θ is generally less than 1°, tan2θ is approximated as a linear correlation function of tanθ, and the corresponding vertical and horizontal gradient relationships can be listed as follows (the two vertical directions are the x and y directions of the camera coordinate axis, respectively):

[0141]

[0142] In the formula, a(u,v), b(u,v) and c(u,v) are all parameter constants that are independent of the gradient. It is the difference between each set of collected phase data and the phase data of the initial reference plane. Therefore, only three sets of corresponding phase and gradient information are needed to solve a, b, and c corresponding to each pixel.

[0143] First, place the mirror-like object to be reconstructed on the reference plane, display the sinusoidal fringe pattern on the LCD screen, and project the sinusoidal fringes onto the surface of the object through the screen. At the same time, the camera shoots the object and the calibration plate placed on the reference plane. The camera calibration has been completed in the previous section to obtain the camera intrinsic parameters. Substituting into formula (3-9) can obtain the truncated extrinsic parameter matrix K of the reference surface relative to the telecentric camera s , we can find the r of the third column of the rotation matrix 13 and r 23 and t in the translation vector x and t y , which gives the normal vector n0=(A0,B0,1) perpendicular to the reference plane. At the same time, the phase of the two perpendicular directions of the surface to be measured is solved by the four-step phase shift method mentioned above. The initial phase used as the initial reference surface.

[0144] By rotating the mirror-like object to be tested, the reference plane, and the entire platform where the calibration plate is located, the mirror-like object to be tested and the initial reference plane are at an angle. Resolve the phase in both directions and phase difference According to the circular calibration plate taken after rotation, the characteristic circle is extracted to calculate the rotation matrix and translation vector to obtain the normal vector n of the new plane after rotation. i =(A i ,B i ,1). After obtaining the normal vector of each rotated plane and the normal vector of the initial reference plane, we can calculate the coordinates of the two planes in the camera coordinate system X after rotation. C and Y C Angle of direction:

[0145]

[0146] By taking the tangent of θ in equations (3-26) and (3-27) respectively, we can get the gradient tanθ. Repeating the process of rotating the plane and solving the angle three times, we can get three sets of phase differences And the gradient value tanθ, the three unknown variables a(u,v), b(u,v), and c(u,v) in equations (3-24) and (3-25) can be solved by at least three constraint equations.

[0147] In actual experiments, there are often calibration errors, and it is impossible to accurately obtain the phase difference using only the data of the three sets of images. Therefore, the present invention selects an adjustable high-precision rotating platform, adjusts a small angle each time, rotates the reference plane 10 times or more, and the angle change between the initial reference plane and the maximum reference plane is about 1°, so that 10 groups or more of gradient values ​​tanθ and phase difference can be obtained. Yes. For each image pixel captured by the camera, the least squares method is used to constrain the fitting of a(u,v), b(u,v), and c(u,v) based on 10 sets of phase gradient data.

[0148] The advantages of telecentric lenses in eliminating perspective distortion, stabilizing magnification, and providing high-precision images make them widely used in the field of high-precision measurement, especially in the field of metrology.

[0149] When camera calibration requires changing the calibration plate posture multiple times, when the calibration plate is placed at a large angle, some edges of the chessboard may exceed the optimal imaging range of the camera, causing some areas of the image to be blurred.

[0150] These problems affect the accuracy of corner extraction during the calibration process, and thus affect the final calibration results. In order to solve these problems, the image deblurring technology based on deep learning is combined with the image sharpening technology, which can effectively improve the spatial resolution of the image and further clarify the blurred area.

[0151] Preferably, the orthogonal imaging model is deblurred and corrected and enhanced by using an image sharpening technique combined with a Restormer model deblurring technique and a flat field correction and gamma correction method, and a restored image of the orthogonal imaging model is regenerated and output, including:

[0152] Deblurring the orthogonal imaging model using a deblurring technique based on a Restormer model;

[0153] Using flat field correction and gamma correction methods, the orthogonal imaging model is subjected to image correction and enhancement processing;

[0154] After the processing is completed, the restored image of the orthogonal imaging model is regenerated and output.

[0155] The sharpening process and correction method will be described below respectively.

[0156] Image sharpening technology enhances the edges and details of the image, so that even when the calibration plate is placed at a large angle, the lost details can be restored. Using image deblurring technology based on deep learning can effectively improve the spatial resolution of the image, making the blurred area clearer and improving the accuracy of corner detection and extraction.

[0157] During the calibration process of the telecentric camera, the image deblurring technology can restore the details lost when the angle is large, making the chessboard corners in the blurred area clearer. At the same time, it not only improves the extraction accuracy of the corners, but also improves the quality of the overall calibration image. The high-quality calibration image enables the entire system to calculate the camera parameters and distortion coefficients more accurately during calibration, thereby improving the final 3D reconstruction and measurement accuracy.

[0158] The phase deflection system of the present invention uses a combination of image sharpening and Restormer model to achieve image deblurring. Image sharpening Restormer is the basic framework of image deblurring and reconstruction. Sharpening can enhance the high-frequency components (such as texture and detail information) in the image, making the blurred area more prominent. Restormer can effectively restore the details and texture information of the image through its multi-scale processing capability and local context perception mechanism.

[0159] The deblurring technology based on the Restormer model can be used to deblur the orthogonal imaging model by referring to the following steps:

[0160] Restormer is an advanced deep learning model designed specifically for image restoration tasks, including deblurring. Typically based on the Transformer architecture, it can efficiently handle blur in images and restore clear details. The following is a high-level overview of the steps to deblur an image based on the Restormer model:

[0161] 1. Data preparation

[0162] Collect blurry images: First, you need a dataset containing blurry images for training and validating the model.

[0163] Prepare clear images: As a reference for supervised learning, corresponding clear images (i.e., the original images before blurring) are also required.

[0164] 2. Model Construction

[0165] Choose or build a Restormer model: You can base your model on an existing Restormer model architecture, or use an open source implementation as a basis and fine-tune it to your needs.

[0166] Define the loss function: Loss functions such as mean squared error (MSE) or structural similarity index (SSIM) are usually used to measure the difference between the model output and the real clear image.

[0167] 3. Model training

[0168] Data preprocessing: Scale the blurred and clear images to appropriate sizes and perform normalization.

[0169] Training process: Train the Restormer model using the preprocessed dataset. This involves feeding the blurred image into the model, calculating the loss, and updating the model weights via backpropagation.

[0170] Validation and Tuning: During training, use the validation set to evaluate the performance of the model and adjust hyperparameters such as learning rate and batch size as needed.

[0171] 4. Model reasoning (defuzzification)

[0172] Load the trained model: In the inference phase, load the trained Restormer model.

[0173] Input blurred image: Input the image to be deblurred into the model.

[0174] Get a clear image: The model will output a deblurred image.

[0175] 5. Post-processing (optional)

[0176] Image enhancement: Further enhancement processing is performed on the deblurred image, such as contrast adjustment and sharpening, to improve the visual effect.

[0177] Quality assessment: Use objective metrics (such as PSNR, SSIM) or subjective evaluation to evaluate the deblurring effect.

[0178] 6. Deployment

[0179] Integration into applications: Integrate the deblurring feature into image editing software, mobile applications, or web services.

[0180] Performance optimization: Consider model compression, quantization, or using a more efficient inference framework to reduce computing resources and latency.

[0181] like Figure 4 The following are the comparison pictures before and after sharpening. It can be seen that: (a) is an "out-of-focus" blurred image, which reflects that part of the calibration plate is blurred due to exceeding the depth of field of the lens, thus affecting the accurate imaging of the calibration plate. This kind of blur phenomenon has a significant impact on the accuracy of corner point extraction. In contrast, (b) is the result after image deblurring. The edge clarity of each dot in the processed image is significantly improved, and the overall image quality is greatly improved.

[0182] Therefore, after being processed by the image deblurring technology of the present invention, the performance of the calibration plate image of the present system in various indicators has been significantly improved, proving the effectiveness of the deblurring technology in improving image quality. First, PSNR evaluates the difference in quality between the image after processing (such as deblurring, compression, etc.) and the original image, indicating that the deblurring process effectively restores the information in the image, making it visually closer to reality. In the image restoration of the experiment of the present invention, the improved image deblurring method is used, which exceeds the traditional deblurring method and the Restormer model method in multiple evaluation indicators, especially in key indicators such as PSNR, SSIM, AG and SF. The PMD system has not only been significantly improved visually, but also greatly improved the accuracy and reliability in the phase deflection measurement and three-dimensional reconstruction process, effectively reducing the system error and effectively improving the accuracy of the subsequent reconstructed surface shape.

[0183] The telecentric phase deflection system used in this paper will have some system error problems, such as defocus blur error and nonlinear distortion error. The defocus blur error is mainly due to the small depth of field of the telecentric lens, which causes blurring in some image areas during the calibration process or when the object surface is not in the focal plane. Especially when the calibration plate is tilted or the object height changes greatly, this blurring will cause the corner point extraction accuracy to decrease, affecting the system calibration results and 3D reconstruction accuracy. At the same time, the nonlinearity of the camera imaging system or the grayscale response of the display will also cause geometric or brightness distortion of the image, affecting the accuracy of phase measurement.

[0184] In order to solve the problem of feature blur caused by the small depth of field of the telecentric system, this paper adopts the Restormer network based on deep learning to achieve fast correction of defocus blur. The Restormer network can effectively restore image details in high-resolution image processing through the self-attention mechanism and deep convolutional feedforward network, thereby improving the clarity of blurred areas, especially the accuracy of corner extraction during the calibration process. By using this network, the system in this paper can significantly improve the image quality and reduce the error caused by defocus blur, thereby improving the overall calibration accuracy of the system.

[0185] In order to solve the nonlinear errors that are easy to occur in the system, this paper intends to use flat field correction and gamma correction technology. Flat field correction corrects the inconsistent response of image sensors and illumination differences, eliminates artifacts and noise, improves the brightness uniformity of the image, and ensures the accuracy of feature extraction. Gamma correction corrects the nonlinear relationship between the grayscale value and the actual brightness of the display screen, ensuring that accurate grayscale information can be obtained in phase measurement, thereby reducing errors caused by nonlinear response.

[0186] The calibration plate used in this embodiment is a double-layer circular calibration plate, which is composed of an upper and lower glass calibration plate and a ceramic calibration plate; the glass calibration plate and the ceramic calibration plate are both provided with a plurality of circular holes arranged in a matrix array and with upper and lower phase deflections, and the center of the circular hole is used to provide the feature points required for calibration.

[0187] like Figure 5 As shown, the design of the circular calibration plate is shown in Figure (a) (the figure shows a superimposed composite state, with the glass calibration plate on the upper layer and the ceramic calibration plate on the lower layer, and (b) is a physical picture of the composite state). Two calibration plates with the same overall size of 38mm×38mm were designed. One is made of an opaque and non-reflective ceramic calibration plate, and the other is a transparent and reflective photolithographic glass calibration plate. The diameter of the black dots in the grid is 1.8mm. The distance between the circular black dots (the center distance between adjacent points) is 6mm, and the number of dots on each calibration plate is the same, 5×5. The circular pattern of the photolithographic glass calibration plate can be regarded as the dot pattern of all the ceramic as a whole offset 45° to the lower right corner. Each calibration plate is 1mm thick, made with high-precision machining technology, and has a low thermal expansion coefficient, so the accuracy can reach ±1μm.

[0188] Because the user only needs to confirm r 13 and r 23 The sign of r 13 and r 23 It is not necessary to completely satisfy the original ideal equations (3-19) and (3-20), so it is only necessary to ensure that the point brought into equation (3-22) is the same as the offset Z of the calibration plate perpendicular to the optical axis plane in the previous figure. d , all points are translated relative to the plane of the previous figure, so the X W and Y W The change will not affect r 13 and r 23 The positive and negative signs of the dots can be judged, so the dot offset of the upper glass plate relative to the ceramic glass plate will not cause calibration errors.

[0189] like Figure 6 The center detection diagram of the ceramic calibration plate is shown in the figure. When it is necessary to change the calibration plate posture for multiple image acquisition and timing, the algorithm detects the 5×5 circular black dot where the ceramic calibration plate is located. The corner point detection algorithm can detect the position of the circular point in the image and mark the center of the circle, and mark the pixel values ​​in the pixel coordinate system in sequence, that is, the u and v values ​​on the left side of the equation (3-9). The figure shows the world coordinates of the dot in the world coordinate system. Take the plane where the 2D calibration target is located as X W OY W The Z direction is perpendicular to the circular calibration plate. WThe world coordinate system is established along the X axis, and the origin of the coordinate system is set to the position where the center of the characteristic circle in the upper left corner of the calibration plate is located. The green upper left corner (0,0) in the figure is the starting point, and the W The axis connects all feature points in a "Z" shape, with the blue feature point in the lower right corner as the end point. The purpose of sorting is to put the corresponding pixel coordinates and world coordinates into equation (3-9) one by one to solve the homography matrix H. After solving H, substitute it into equation (3-16) to solve the internal parameter matrix A of the telecentric camera, and calculate the external parameter matrix of each calibration image through (3-18)

[0190] Through multiple sets of calibration plate images with different postures, multiple sets of corresponding points are obtained to solve and obtain the external parameter matrix After that, the Z that needs to be offset d The displacement is the thickness of the superimposed calibration plate, 1 mm, so here Z d = 1. As shown in the center detection diagram of the glass calibration plate, the sorting logic is the same as that of the ceramic calibration plate. The green upper left corner (0,0) is the starting point, and the X W The axis connects the feature points of all the glass calibration plates in a "Z" shape, with the blue feature point in the lower right corner as the end point.

[0191] Determine r 13 and r 23 The sign of the circular feature point on the ceramic calibration plate is positive or negative, just add the coordinates of the center of the circle to the Z d Combined with the pixel coordinates of the center of the calibration plate detected in the glass calibration plate and substituted into formula (3-22), the solution can be confirmed. 13 and r 23 The positive and negative signs of the telecentric lens ultimately describe the calibration parameters A and K s All can be confirmed.

[0192] This paper adopts the "two-point correction method" for flat field correction, which assumes that the detector pixel responds linearly. First, let the camera expose the dark field and record the offset value of each pixel; then, image the uniform object under uniform illumination to obtain a uniform field image so that all pixels are close to the maximum grayscale; finally, subtract the dark field image from the uniform field image and multiply it by the relative calibration correction gain (Gain).

[0193] like Figure 7 As shown, images are taken with a nearly saturated uniform square and a dark field without light, respectively, to obtain dark field images and bright field images required for flat field correction, and then normal images with noise interference such as ambient light shading are taken under normal conditions to perform the following flat field correction.

[0194] (1) Dark field correction (FPN)

[0195] (a) First, we need to obtain the dark field pixel value of the system (take its average value), in order to determine the dark pixel value generated by the CMOS under specific environmental conditions (such as the same light, temperature, and time). This process can be expressed as:

[0196] G B (x,y)=Ki(x,y)t+G off (0-34)

[0197] G off is the bias of the constant image, K is the conversion gain, and i(x,y) is the dark current of the normal pixel (or dark frame), which is also the signal emitted by the detector when there is no incident light (hence the dark frame).

[0198] (2) Bright field correction (PRNU)

[0199] (b) To perform flat field correction, a reference image G is required. R (x,y). To this end, the sensor is photographed under the illumination of a uniform light field X0, whose light intensity should be as close as possible to the saturation level of the sensor. In addition, the exposure time should be the same as the previously acquired dark field image G B The exposure time of (x, y) is kept consistent. By shooting under high light intensity conditions, the response non-uniformity of each pixel of the sensor can be accurately captured, thus providing accurate data support for subsequent correction. Reference image G R (x,y) can be expressed as:

[0200] G R (x,y)=η(x,y)X0+Ki(x,y)+G off (0-35)

[0201] η(x,y) is the response rate of each pixel. Subtracting equation (4-13) from equation (4-14) yields:

[0202] η(x,y)=[G R (x,y)-G B (x,y)] / X0 (0-36)

[0203] Through this process, users obtain the response rate data of each pixel, which is equivalent to establishing a complete response rate database for all pixels of the image sensor. This helps users to make accurate corrections in the subsequent image processing process and improve image quality.

[0204] The corrected image G(x,y) is expressed as:

[0205] G(x,y)=eta(x,y)X(x,y)+Ki(x,y)+G off (0-37)

[0206] Combining formula (4-13), we can get the corrected relationship:

[0207]

[0208] Because X0 is a uniform square, C0 can be converted to the desired rectified image by multiplying it by a suitable value. The final correction equation is:

[0209]

[0210] m is the reference image G R (x,y) and dark field image G B The average of the images obtained by subtracting (x,y).

[0211] Figure 8 It can be seen that before correction, there is obvious uneven brightness in the image. The pixels in some areas have large brightness deviations due to inconsistent sensor responses, which affects the overall visual quality. The background noise also makes the details in some areas insufficient. After correction, the brightness in the image becomes more uniform, eliminating artifacts and noise caused by differences in brightness and response. The overall image quality is significantly improved, the details are clearer, and the visual effect is more realistic and natural.

[0212] The histograms before and after correction are as follows: Fig. 9 As shown in the figure, after calculating the image histogram before and after correction, it is found that the histogram before correction is more scattered and has large gaps, indicating that the pixel grayscale values ​​in the image are unevenly distributed and there are strong contrast inconsistencies. The corrected histogram is more detailed and dense in comparison, which shows that after flat field correction, the pixel grayscale value distribution in the image becomes more uniform and the brightness difference between pixels decreases. After correction, the distribution of pixel grayscale values ​​is more concentrated and delicate, indicating that the brightness contrast of the image is more natural, the difference between pixels is reduced, and the overall tone transition is smoother.

[0213] Gamma Correction

[0214] The nonlinear characteristics of the display make the brightness look better to the user. Most LCD displays on the market show obvious nonlinear characteristics when displaying grayscale values. This nonlinear performance comes from the fact that the human eye's perception of brightness is not linear. Fig.10 Specifically, the human eye is more sensitive to changes in low brightness, and as brightness increases, the perception of brightness changes gradually decreases. This phenomenon usually follows an exponential or logarithmic relationship.

[0215] Due to this perception characteristic of the human eye, the relationship between the grayscale value of the display and the actual brightness is often designed as a gamma correction curve. The purpose of this gamma correction curve is to make the brightness output of the display more consistent with the perception characteristics of the human eye, so as to provide a more uniform visual experience at various brightness levels. As shown in formula (4-19), in practical applications, this gamma correction can be adjusted by a preset gamma value. For example, the standard gamma value is usually set to 2.2, which indicates that there is a nonlinear relationship between the output brightness of the display and the input signal, which is used to compensate for the human eye's perception characteristics of brightness. In this way, the display can maintain consistency in visual effects over a larger brightness range and avoid overly obvious changes in low-brightness areas.

[0216] Its input gray value I in And the output gray value I out The response relationship can be expressed as

[0217] I out =(I in +ε) γ (0-40)

[0218] Among them, ε is the compensation coefficient (usually 0 by default), and γ is the gamma coefficient set by the LCD display to a positive value.

[0219] During the 3D reconstruction measurement of the telecentric phase deflectometry system, uncorrected gamma will cause the relationship between the grayscale value and the actual brightness on the LCD display to be distorted, while the photosensitive element of the CMOS camera will actually capture the change in light intensity, which will cause inconsistency between the image data received by the camera and the actual projection information, such as the error in the sinusoidality of the captured sinusoidal fringe pattern and the error in the corner point extraction during the calibration of the calibration plate. This inconsistency will directly affect the decoding of the fringe phase by the 3D reconstruction algorithm, and ultimately lead to deviations in the reconstructed 3D surface data. Therefore, ensuring that the gamma correction of the LCD display is in place is an important prerequisite for improving the accuracy of 3D reconstruction measurement.

[0220] like Fig.11 As shown, first, the computer generates an image I with grayscale ranging from 0 to 255 with equal intervals. in1 , I in2 ,…I in255 The reference plane mirror is placed on the plane to be tested, and the grayscale image on the plane mirror is taken with a telecentric camera. To ensure the accuracy of the experiment, each grayscale image is taken 5 times to obtain all the grayscale images. out1 , I out2 ,…I out255 Then, the average gray value of each gray image is obtained. Grayscale response obtained as a telecentric camera.

[0221] The grayscale value generated by the computer is the horizontal axis, and the grayscale value of the captured image is the vertical axis. in and The discrete points of are plotted in the coordinate system. The polynomial is defined and fitted using the least squares method to generate the fitted curve value. The fitting result is given by Fig.12 It can be seen that there is a nonlinear difference between the grayscale value displayed on the LCD display and the grayscale value captured by the camera. In order to ensure the accuracy of the 3D reconstructed surface, it is necessary to complete the nonlinear calibration of the LCD display and then perform correction.

[0222] Since the telecentric phase deflection detection system also has requirements for the contrast response of the LCD display, changing the contrast of the display will affect the display of the captured grayscale image. Increasing the contrast of the LCD display will make the difference between the grayscale levels in the image more obvious, making the darker grayscale look darker and the brighter grayscale look brighter. Reducing the contrast will reduce the difference in grayscale levels and make the grayscale in the image look more uniform. Therefore, it is necessary to select a suitable display contrast to confirm the nonlinear grayscale response of the appropriate display. A computer generates a grayscale image in the contrast range of 0%, 25%, 50%, 75%, and 100%. Use a camera to capture the image of the display in the plane mirror and draw the corresponding grayscale response curve. The results are shown below. Fig.13 As shown:

[0223] Fig.13 In the grayscale response curve of the display screen at different contrasts, the x-axis is the grayscale value of the image generated by the computer on the LCD display screen, and the y-axis is the grayscale value of the image of the plane mirror photographed by the telecentric camera.

[0224] It can be seen that the grayscale response of the display is nonlinear, and correction must be performed to reconstruct the measurement accuracy. When the contrast of the LCD display is low, it can be seen that the projected grayscale value of the display is similar to a curve close to the x-axis, and the maximum grayscale value captured by the CMOS camera is only about 100, which leads to extremely insufficient utilization of the camera response range. If used in a phase deflection system, the grayscale change value of the phase may be too small or missing to obtain the correct gradient information.

[0225] When the grayscale value of the LCD display is in the low-frequency region, it can be seen that the grayscale information collected by addition is close to a straight line without change, and the image quality is significantly reduced, resulting in the inability to eliminate the noise interference in the low-frequency range and a reduction in the image signal-to-noise ratio. When the contrast of the LCD display is 75% and 100%, it can be seen that the response curve has an approximately horizontal curve in the high-frequency range of the projected grayscale above 200 grayscale, which means that the response of the camera has been oversaturated. In the phase deflection system measurement, if the grayscale change is based on more than 200, the phase change information will be lost, and the true gradient of the object to be measured cannot be obtained.

[0226] In summary, we need to select a curve that is approximately linear and whose grayscale response range is close to 0-255. This curve can meet the requirements when the contrast of the LCD display is in the range of 50%. It will not waste the grayscale range of the LCD display and will not cause interference from too little response in the low-frequency range and too much noise.

[0227] Will Fig.13 After the 50% grayscale response curve is normalized and fitted to obtain a new curve, it is necessary to find the curve expression of the correction compensation to compensate the original function. By finding the inverse function of the polynomial fitting curve, that is, finding its symmetric function about y=x, such as Fig.14 As shown in the symmetrical compensation curve, the red line is the fitted compensation curve. The original fitting curve is compensated according to the compensation function fitted by the compensation curve, and the grayscale value of the grayscale image of the LCD display screen projected onto the plane mirror captured by the CMOS camera is compensated to be consistent with the grayscale on the LCD display screen generated by the ideal computer.

[0228] Depend on Fig.15 It can be seen that for each nonlinear grayscale response, the original fitted nonlinear curve can be compensated by the correction function. From the fitting result of the pink-purple correction curve, it can be seen that the corrected result is consistent with the ideal grayscale on the computer LCD display, and the nonlinear error of the system is optimized, which effectively improves the subsequent folded phase extraction quality and the three-dimensional reconstruction accuracy of the entire telecentric phase deflection system.

[0229] Therefore, the present invention combines image sharpening technology with the Restormer deep learning network model to deblur the blurred area in order to solve the out-of-focus blur problem. This effectively restores the image details, allowing the system to accurately extract corner points even at large angles during the calibration process, significantly improving image quality and calibration accuracy.

[0230] Flat field correction and gamma correction methods are further used to address nonlinear distortion errors. Flat field correction effectively reduces artifacts and noise in the image by calibrating the response non-uniformity and illumination differences of the camera sensor, ensuring brightness uniformity and the accuracy of feature extraction. Gamma correction solves the nonlinear problem of the grayscale response of the display screen. By fitting the compensation curve, the grayscale output of the LCD display screen is closer to the ideal state, thereby improving the accuracy of phase measurement. Through these optimization methods, the system has been significantly improved in terms of three-dimensional reconstruction and measurement accuracy, laying a solid foundation for subsequent high-precision phase deflection measurement.

[0231] Obviously, those skilled in the art should understand that the implementation of all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. Those skilled in the art can understand that the implementation of all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated as: HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0232] Example 2

[0233] Based on the implementation principle of Example 1, on the other hand, the present application proposes a phase deflection application method based on an orthogonal imaging model, which is implemented based on the above-mentioned phase deflection application system based on the orthogonal imaging model, and includes the following steps:

[0234] The computer generates a sinusoidal fringe pattern in advance and sends it to the LCD display screen, and simultaneously sends a rotation control instruction;

[0235] The rotating platform responds to the rotation control instruction and presents a corresponding angle and posture, so that the calibration plate on it presents a corresponding reference surface, providing calibration reference data of the corresponding angle and posture for the CMOS camera;

[0236] The LCD display screen projects a pre-coded sinusoidal fringe pattern onto the surface of the object to be measured;

[0237] The computer activates the CMOS camera, captures the deformation pattern reflected by the surface of the measured object and the calibration reference data through the telecentric lens thereon, and uploads them to the computer;

[0238] According to the above steps, several groups of the calibration reference data and the deformation patterns at different angles and postures are collected and uploaded to a computer;

[0239] The computer receives and analyzes and modulates the deformation pattern according to the calibration reference data, obtains the surface shape height information of the object under test and constructs an orthogonal imaging model corresponding to the object under test based on the three-dimensional reconstruction technology; and uses image sharpening technology combined with the image deblurring technology of the Restormer network and the flat field correction and gamma correction methods to deblur and correct the orthogonal imaging model, and regenerates and outputs the restored image of the orthogonal imaging model.

[0240] Please understand this in conjunction with Example 1, and no further details will be given here.

[0241] The modules or steps of the present invention described above can be implemented by a general-purpose computing system, they can be concentrated on a single computing system, or distributed on a network composed of multiple computing systems, and optionally, they can be implemented by a program code executable by a computing system, so that they can be stored in a storage system and executed by the computing system, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0242] Example 3

[0243] like Fig.16 As shown, further, in another aspect, the present application also proposes an electronic device, including:

[0244] processor;

[0245] a memory for storing processor-executable instructions;

[0246] Wherein, the processor is configured to implement the phase deflection application method based on the orthogonal imaging model described in Example 2 when executing the executable instructions.

[0247] The electronic device of the embodiment of the present disclosure includes a processor and a memory for storing instructions executable by the processor, wherein the processor is configured to implement the phase deflection application method based on the orthogonal imaging model described in the above embodiment 2 when executing the executable instructions.

[0248] Here, it should be noted that the number of processors can be one or more. At the same time, the electronic device of the embodiment of the present disclosure may also include an input system and an output system. Among them, the processor, memory, input system and output system may be connected through a bus or in other ways, which are not specifically limited here.

[0249] The memory, as a computer-readable storage medium, can be used to store software programs, computer executable programs and various modules, such as the program or module corresponding to the phase deflection application method based on the orthogonal imaging model in the embodiment of the present disclosure. The processor executes various functional applications and data processing of the electronic device by running the software programs or modules stored in the memory.

[0250] The input system can be used to receive input numbers or signals. The signal can be a key signal related to user settings and function control of the device / terminal / server. The output system can include display devices such as display screens.

[0251] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A phase deflection application system based on an orthogonal imaging model, characterized in that: include: An LCD display for projecting a pre-coded sinusoidal fringe pattern onto the surface of the object being measured; A rotating platform, used to provide three-axis tilt drive for the object to be measured and the calibration plate placed thereon; wherein the calibration plate is used to provide calibration reference data of different angles and postures for the CMOS camera; A CMOS camera, used for capturing the deformation pattern reflected by the surface of the measured object and the calibration reference data through a telecentric lens thereon, and uploading them to a computer; A computer is used to control the LCD display screen, the rotating platform and the CMOS camera; and, receiving and, according to the calibration reference data, performing analytical modulation on the deformation pattern, obtaining the surface shape height information of the object to be measured and constructing an orthogonal imaging model corresponding to the object to be measured based on the three-dimensional reconstruction technology; and, using the image sharpening technology combined with the deblurring technology of the Restormer model and the flat field correction and gamma correction methods, performing deblurring and correction enhancement processing on the orthogonal imaging model, and regenerating and outputting the restored image of the orthogonal imaging model; The LCD display screen, the rotating platform and the CMOS camera are respectively connected to the computer for communication.

2. The phase deflection application system based on the orthogonal imaging model according to claim 1, characterized in that: The calibration plate is a double-layer circular calibration plate, which is composed of a glass calibration plate and a ceramic calibration plate in the upper and lower layers; the glass calibration plate and the ceramic calibration plate are both provided with a plurality of circular holes arranged in a matrix array and with upper and lower phase deflections, and the centers of the circular holes are used to provide the feature points required for calibration.

3. The phase deflection application system based on the orthogonal imaging model according to claim 1, characterized in that: The computer is also used to: pre-generate the sinusoidal fringe pattern and send it to the LCD display screen.

4. The phase deflection application system based on the orthogonal imaging model according to claim 1, characterized in that: The computer is also used to: pre-generate a rotation control instruction for the rotating platform, and control the rotating platform to present different angles and postures, so that the calibration plate presents different reference surfaces, which is used to provide calibration reference data of different angles and postures for the CMOS camera.

5. The phase deflection application system based on the orthogonal imaging model according to claim 1, characterized in that: On a computer, the image sharpening technology is combined with the deblurring technology of the Restormer model and the flat field correction and gamma correction methods to perform deblurring and correction enhancement processing on the orthogonal imaging model, and regenerate and output a restored image of the orthogonal imaging model, including: Deblurring the orthogonal imaging model using an image sharpening technique combined with a deblurring technique of a Restormer model; Using flat field correction and gamma correction methods, the orthogonal imaging model is subjected to image correction and enhancement processing; After the processing is completed, the restored image of the orthogonal imaging model is regenerated and output.

6. A phase deflection application method based on an orthogonal imaging model, implemented based on the phase deflection application system based on an orthogonal imaging model as claimed in any one of claims 1 to 5, characterized in that: The steps include: The computer generates a sinusoidal fringe pattern in advance and sends it to the LCD display screen, and simultaneously sends a rotation control instruction; The rotating platform responds to the rotation control instruction and presents a corresponding angle and posture, so that the calibration plate on it presents a corresponding reference surface, providing calibration reference data of the corresponding angle and posture for the CMOS camera; The LCD display screen projects a pre-coded sinusoidal fringe pattern onto the surface of the object to be measured; The computer activates the CMOS camera, captures the deformation pattern reflected by the surface of the measured object and the calibration reference data through the telecentric lens thereon, and uploads them to the computer; According to the above steps, several groups of the calibration reference data and the deformation patterns at different angles and postures are collected and uploaded to a computer; The computer receives and analyzes and modulates the deformation pattern according to the calibration reference data, obtains the surface shape height information of the object under test and constructs an orthogonal imaging model corresponding to the object under test based on the three-dimensional reconstruction technology; and uses image sharpening technology combined with the deblurring technology of the Restormer model as well as the flat field correction and gamma correction methods to deblur and correct the orthogonal imaging model, and regenerates and outputs the restored image of the orthogonal imaging model.

7. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the phase deflection application method based on the orthogonal imaging model as described in claim 6 when executing the executable instructions.

Images