Object surface shape information determination method, apparatus, device, medium, and product

Abstract

The present invention discloses a method, device, equipment, medium and product for determining the surface shape information of an object. The method includes: obtaining an interference fringe pattern of the object to be measured and determining image description information of the interference fringe pattern; the interference fringe pattern is obtained based on the interference of light beams reflected after at least two light waves propagate to a reference surface and the surface of the object to be measured respectively; the interference fringe pattern, image description information, preset image processing method description information and Zernike polynomial image are input into a pre-trained surface reconstruction model to obtain Zernike polynomial coefficients; based on Zernike polynomial images of different orders and Zernike polynomial coefficients, a surface shape map of the object to be measured is obtained. The technical solution provided by the embodiment of the present invention can achieve the effect of making the determined surface shape map highly compatible with the object to be measured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer processing, and in particular, to an object surface shape information determination method, device, equipment, medium and product. BACKGROUND

[0002] In the manufacturing process of chips, some materials such as wafers are usually used. In order to ensure the accuracy of chip manufacturing, the surface shape of some material objects required for manufacturing needs to be accurately measured and reconstructed.

[0003] The existing surface reconstruction method usually relies on a Fizeau interferometer to reconstruct the surface shape of the object. Some control parameters and strategies in the measurement process of the Fizeau interferometer depend more on the user's engineering experience to determine, and the user's requirements are higher, and there are problems of complicated surface reconstruction process and poor accuracy. SUMMARY

[0004] The present application provides an object surface shape information determination method, device, equipment, medium and product, so as to realize that the determined surface shape map is highly adaptive to the object to be measured, and to achieve the technical effect of improving the convenience of object surface shape information reconstruction.

[0005] According to an aspect of the present application, an object surface shape information determination method is provided, which comprises:

[0006] Obtaining an interference fringe pattern of an object to be measured, and determining image description information of the interference fringe pattern; wherein the interference fringe pattern is obtained based on the coherent interference of light beams reflected after at least two light waves propagate to a reference surface and a surface of the object to be measured, respectively;

[0007] Inputting the interference fringe pattern, the image description information, preset image processing mode description information and a Zernike polynomial image into a pre-trained surface reconstruction model to obtain Zernike polynomial coefficients; wherein the Zernike polynomial image is an image represented based on Zernike polynomials of different orders; and the preset image processing mode description information at least includes phase unwrapping processing mode description information and Zernike coefficient solving mode description information;

[0008] Based on the Zernike polynomial images of different orders and the Zernike polynomial coefficients, a surface shape map of the object to be measured is obtained.

[0009] According to another aspect of the present application, an object surface shape information determination device is provided, which comprises:

[0010] An image description information determination module is configured to acquire an interference fringe pattern of the object to be measured and determine image description information of the interference fringe pattern, wherein the interference fringe pattern is obtained based on interference of light beams reflected after at least two light beams propagate to a reference surface and a surface of the object to be measured, respectively.

[0011] A Zernike polynomial coefficient determination module is configured to input the interference fringe pattern, the image description information, preset image processing mode description information and a Zernike polynomial image into a pre-trained surface shape reconstruction model to obtain Zernike polynomial coefficients, wherein the Zernike polynomial image is an image represented based on Zernike polynomials of different orders, and the preset image processing mode description information at least includes phase unwrapping processing mode description information and Zernike coefficient solving mode description information.

[0012] A surface shape map determination module is configured to obtain a surface shape map of the object to be measured based on the Zernike polynomial images of different orders and the Zernike polynomial coefficients.

[0013] According to another aspect of the present application, an electronic device is provided, which comprises:

[0014] at least one processor; and a memory connected to the at least one processor in communication; wherein

[0015] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the object surface shape information determination method according to any one of the embodiments of the present application.

[0016] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to implement the object surface shape information determination method according to any one of the embodiments of the present application when executed by the processor.

[0017] According to another aspect of the present application, a computer program product is provided, which comprises a computer program for implementing the object surface shape information determination method according to any one of the embodiments of the present application when executed by a processor.

[0018] The technical scheme of the embodiment of the present application is: an interference fringe pattern of a to-be-measured object is obtained, and image description information of the interference fringe pattern is determined; the interference fringe pattern, the image description information, preset image processing mode description information, and a Zernike polynomial image are input into a pre-trained surface shape reconstruction model to obtain Zernike polynomial coefficients; the Zernike polynomial image is an image represented based on Zernike polynomials of different orders; the preset image processing mode description information at least includes phase unwrapping processing mode description information and Zernike coefficient solving mode description information; based on the Zernike polynomial images of different orders and the Zernike polynomial coefficients, a surface shape map of the to-be-measured object is obtained, thereby solving the problem that the prior art depends on a Fizeau interferometer for object surface shape reconstruction, and the surface reconstruction process is complicated and has poor accuracy, and realizing the technical effect of improving the convenience of object surface shape information reconstruction by inputting the interference fringe pattern, the image description information, the preset image processing mode description information, and the Zernike polynomial image into the end-to-end trained surface shape reconstruction model to obtain the Zernike polynomial coefficients, and combining the Zernike polynomial image and the Zernike polynomial coefficients to determine the surface shape map of the to-be-measured object, so that the determined surface shape map is highly adaptive to the to-be-measured object.

[0019] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0021] Figure 1 is a flow chart of an object surface shape information determination method according to the first embodiment of the present application;

[0022] Figure 2 is a schematic diagram of the object surface shape information determination method according to the first embodiment of the present application;

[0023] Figure 3 is a flow chart of an object surface shape information determination method according to the second embodiment of the present application;

[0024] Figure 4 is a schematic diagram of the object surface shape information determination method according to the second embodiment of the present application;

[0025] Figure 5 Figure 1 is a structural schematic diagram of an object surface shape information determination device according to an embodiment of the present application;

[0026] Figure 6 Figure 2 is a structural schematic diagram of an electronic device implementing an object surface shape information determination method according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts should fall within the scope of the present application.

[0028] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to include only those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to the process, method, product, or device.

[0029] Embodiment One

[0030] Figure 1 Figure 1 is a flowchart of an object surface shape information determination method according to an embodiment of the present application. The present embodiment can be applied to the case of reconstructing the three-dimensional surface shape of an object to be measured. The method can be performed by an object surface shape information determination device, which can be implemented in the form of hardware and / or software, and can be configured in a computing device. As shown in Figure 1, the method comprises the following steps. Figure 1

[0031] S110, obtaining an interference fringe pattern of an object to be measured, and determining image description information of the interference fringe pattern.

[0032] ​The object to be measured can be any object whose surface state needs to be reconstructed, such as a wafer (i.e., a silicon wafer) or an object of other materials. The interference fringe pattern is obtained based on the interference of light beams reflected from the reference surface and the surface of the object to be measured after at least two light waves propagate to the reference surface and the surface of the object to be measured, respectively. It can be understood that when laser is emitted by a laser, part of the laser is reflected on the reference surface to form reference light, and the other part of the laser is reflected on the surface of the object to be measured to form measurement light. When the reference light and the measurement light meet, interference occurs, and the interference fringes generated thereby are captured by a camera to form an interference fringe image. The surface shape information of the object to be measured is contained in the interference fringe image. For example, if the surface of the object to be measured is flat, the interference fringes of the reflected light are uniform; if the surface has slight undulations, the interference fringes will be curved or deformed. The image description information is the guiding information for generating the preprocessed interference fringe image. For example, the image description information can be a feature representation of quantitative or qualitative description information of the image content of the interference fringe pattern in different evaluation dimensions. For example, the image description information includes but is not limited to text features of content understanding and noise type, distortion description of the interference fringe image, etc. The content understanding can include expressions of visual features such as color, texture, shape, edge, etc.

[0033] In the embodiment, an interferometer or other optical device can be used to collect the interference fringe pattern of the object to be measured. Further, the interference fringe pattern can be input into a pre-trained visual language model to analyze and process the interference fringe pattern, extract text features of the description information of the interference fringe image in various dimensions, obtain the image description information, and provide guiding information for preprocessing the interference fringe image based on the image description information for the subsequent surface reconstruction model.

[0034] In S120, the interference fringe pattern, the image description information, the preset image processing mode description information, and the Zernike polynomial image are input into a pre-trained surface reconstruction model to obtain Zernike polynomial coefficients.

[0035] Among them, the preset image processing method description information includes at least phase unwrapping processing method description information and Zernike coefficient solution method description information. Phase unwrapping processing method description information refers to information that uses natural language to describe the phase extraction and unwrapping processing method, which is used to guide the surface reconstruction model on how to perform phase extraction and unwrapping, that is, to guide and supervise the model to output a continuous phase unwrapping map. For example, the phase unwrapping processing method description information includes step-by-step instructions on how to extract the phase of pixels in the image and how to restore the continuous phase distribution from the wrapped phase data. Zernike coefficient solution method description information refers to information that uses natural language to describe the calculation method for solving Zernike polynomial coefficients. Natural language includes but is not limited to one or more combinations of images, symbols, videos, and text. Zernike polynomial images are images represented by Zernike polynomials of different orders, that is, images that use Zernike polynomials to describe and simulate wavefront aberrations or wavefront shapes. These images show the changes in the wavefront or surface shape corresponding to each term of the Zernike polynomial. Each Zernike polynomial corresponds to a specific type of aberration, such as spherical aberration, coma, astigmatism, etc. Through the Zernike polynomials, complex wavefront aberrations can be decomposed into a series of basic aberration patterns, which appear as specific ripple patterns on the image, so that they can be intuitively displayed in the image. This image is the Zernike polynomial image. In other words, the Zernike polynomial image includes the wavefront image corresponding to each term in the Zernike polynomial. The Zernike polynomial is a set of continuous function sequences that form a completely orthogonal set on the unit disk, which is used to describe the wavefront error or surface shape. The Zernike polynomial coefficients refer to the coefficients corresponding to each term in the Zernike polynomial, which are used to characterize the aberration amount corresponding to the Zernike polynomial. The surface reconstruction model can be a pre-trained model used to determine the Zernike polynomial coefficients and reconstruct the surface information of the object to be measured based on the Zernike polynomial coefficients.

[0036] It should be noted that the preset image processing method description information can be input dynamically in real time or predetermined. In practical applications, the interference fringe pattern, image description information, preset image processing method description information, and Zernike polynomial image can be used as input to the surface reconstruction model, and the Zernike polynomial coefficients can be output.

[0037] In the embodiment, the surface shape reconstruction model comprises an image preprocessing model, a phase unwrapping model and a wavefront fitting model; the interference fringe image, the image description information, the preset image processing mode description information and the Zernike polynomial image are input into the pre-trained surface shape reconstruction model to obtain the Zernike polynomial coefficients, comprising: based on the image preprocessing model in the surface shape reconstruction model, the interference fringe image is processed according to the image description information to obtain a first interference fringe image; based on the phase unwrapping model in the surface shape reconstruction model, the first interference fringe image is processed according to the phase unwrapping processing mode description information in the preset image processing mode description information to obtain a phase unwrapping image; wherein the preset image processing mode description information further comprises wavefront fitting requirement information and polynomial conversion guide information; based on the wavefront fitting model in the surface shape reconstruction model, the wavefront fitting guide features are determined according to the phase unwrapping image, the Zernike coefficient solving mode description information, the wavefront fitting requirement information, the polynomial conversion guide information and the Zernike polynomial image, and the Zernike polynomial coefficients are determined based on the wavefront fitting guide features.

[0038] The wavefront fitting requirement information refers to a natural language description of how to perform wavefront fitting and the target of wavefront fitting. The polynomial conversion guide information can be description information for guiding how to convert the Zernike polynomial in the discrete unit circle into an orthogonal polynomial, for example, by using an orthogonalization algorithm such as the Gram-Schmit orthogonalization method to perform polynomial conversion.

[0039] Specifically, the image description information and the interference fringe image can be input into the image preprocessing model to preprocess the interference fringe image, such as denoising, image enhancement and the like, to obtain a processed interference fringe image as the first interference fringe image. Further, the first interference fringe image and the phase unwrapping processing mode description information are input into the phase unwrapping model to perform phase extraction and unwrapping processing on the first interference fringe image, to obtain a processed image as the phase unwrapping image. Still further, the phase unwrapping image and the wavefront fitting requirement information, the Zernike coefficient solving mode description information, the Zernike polynomial image and the polynomial conversion guide information are input into the wavefront fitting model to perform feature extraction, to obtain the wavefront fitting guide features, and the Zernike polynomial coefficients are output by analyzing the wavefront fitting guide features.

[0040] In the embodiment, the wavefront fitting guide features are determined according to the phase unwrapping image, the Zernike coefficient solving mode description information, the wavefront fitting requirement information, the polynomial conversion guide information and the Zernike polynomial image, comprising: based on the wavefront fitting requirement information and the phase unwrapping image, a first fusion feature is obtained; based on the Zernike coefficient solving mode description information and the Zernike polynomial image, a second fusion feature is obtained; based on the first fusion feature, the second fusion feature and the polynomial conversion guide information, the wavefront fitting guide features are determined.

[0041] In actual application, the wave surface fitting demand information and the phase unwrapping map can be fused to obtain a first fused feature. The features corresponding to each solving step in the Zernike coefficient solving description feature are sequentially position coded, and each term of the Zernike polynomial in the Zernike polynomial image is sequentially position coded, so that the model can learn the position corresponding to each term of the wave surface image; at the same time, the Zernike coefficient solving method description information is sequentially position coded according to the solving steps, so that the model can understand the calculation process of the Zernike coefficient solving. The position coded Zernike polynomial image and the position coded Zernike coefficient solving method description information are fused to obtain a second fused feature. Further, the second fused feature and the polynomial conversion guide information are fused to obtain a third fused feature, and then the first fused feature and the third fused feature are fused to obtain a wave surface fitting guide feature. It should be noted that if the preset image processing method description information is of a text type, a pre-trained visual language model can be used to process the preset image processing method description information to obtain its feature representation. The wave surface fitting guide feature is determined using the feature representation, the interference fringe pattern, the image description information and the Zernike polynomial image, so as to obtain the Zernike polynomial coefficient by analyzing the polynomial conversion guide feature.

[0042] For example, referring to Figure 2 , the first fused feature 23 is obtained by fusing the wave surface fitting demand information 22 and the phase unwrapping map 21. The second fused feature 26 is obtained by position coding and feature fusion based on the Zernike polynomial image 24 and the Zernike coefficient solving method description information 25. The feature representation 28 is obtained by feature processing of the polynomial conversion guide information 27. The third fused feature 29 is obtained by position coding and feature fusion based on the second fused feature 26 and the feature representation 28. The wave surface fitting guide feature 30 is obtained by feature fusion based on the first fused feature 23 and the third fused feature 29. The Zernike polynomial coefficient 31 is obtained based on the wave surface fitting guide feature 30.

[0043] The benefits of the arrangement in this embodiment are that by decomposing the characteristics of determining the coefficients of Zernike polynomials into multiple parts, one part fuses the data characteristics of the two modalities of phase unwrapping map and wavefront fitting requirements information to generate an understanding feature required for wavefront fitting tasks, i.e., the first fused feature. Another part is to fuse the image representation of each term of the Zernike polynomial as visual data with the description information of the Zernike coefficient solving method to generate a process feature representing the entire process of obtaining the coefficients of the Zernike polynomial by wavefront fitting, i.e., the second fused feature. And when fusing the multi-modal data characteristics, the position coding is used to ensure that the feature order is consistent with the execution order of the fitting process. Further, in order to facilitate the calculation of the coefficients of the Zernike polynomial, the Gram-Schmit orthogonalization method is used to convert the Zernike polynomials in the discrete unit circle into orthogonal polynomials to express the description information, i.e., polynomial conversion guide information. Finally, the three parts of features are fully fused in the order to form the final wavefront fitting guide feature. The generalization ability of the surface shape reconstruction model and the understanding ability of the image and description information are realized, and the surface shape reconstruction model is enabled to output the coefficients of the Zernike polynomial according to the wavefront fitting guide feature.

[0044] S130, based on the Zernike polynomial images of different orders and the coefficients of the Zernike polynomials, obtaining a surface shape map of the object to be measured.

[0045] In this embodiment, the coefficients of the Zernike polynomial can be applied to the Zernike polynomial images of different orders to multiply and accumulate, and a continuous surface shape map is obtained, which is the surface shape map of the object to be measured. The actual surface shape of the object to be measured can be intuitively displayed through the surface shape map.

[0046] The technical scheme provided in the embodiment is characterized in that: an interference fringe pattern of a to-be-measured object is acquired, and image description information of the interference fringe pattern is determined; the interference fringe pattern is obtained based on coherent interference of light beams reflected after at least two light waves propagate to a reference surface and a surface of the to-be-measured object respectively; the interference fringe pattern, the image description information, preset image processing mode description information, and a Zernike polynomial image are input into a pre-trained surface shape reconstruction model to obtain Zernike polynomial coefficients; the Zernike polynomial image is an image represented based on Zernike polynomials of different orders; the preset image processing mode description information at least includes phase unwrapping processing mode description information and Zernike coefficient solving mode description information; based on the Zernike polynomial images of different orders and the Zernike polynomial coefficients, a surface shape map of the to-be-measured object is obtained, thereby solving the problem in the prior art that object surface shape reconstruction is dependent on a Fizeau interferometer, the surface reconstruction process is complicated, and the accuracy is poor, and achieving the technical effects of improving the accuracy of object surface shape information determination and improving the convenience of object surface shape information reconstruction.

[0047] Embodiment Two

[0048] Figure 3 is a flowchart of an object surface shape information determination method according to Embodiment Two of the present application, and the surface shape reconstruction model can be pre-trained on the basis of the foregoing embodiment. The specific implementation can be referred to the technical scheme of the present embodiment. The same or corresponding technical terms as those in the foregoing embodiment are not described herein.

[0049] As shown in Figure 3 , the method specifically includes the following steps:

[0050] S210, a plurality of first samples are acquired; the first samples include an original interference fringe pattern of a measured object, image description information of the original interference fringe pattern, phase unwrapping processing mode description information, Zernike coefficient solving mode description information, wave surface fitting requirement information and polynomial conversion guide information, a Zernike polynomial image, second interference fringe data, phase unwrapping theoretical data, and Zernike polynomial theoretical data.

[0051] The measured object can be an object of the same type as the to-be-measured object or an object of a different type from the to-be-measured object. The second interference fringe data refers to an interference fringe image obtained by preprocessing the original interference fringe data or a feature representation corresponding to the preprocessed interference fringe image. The phase unwrapping theoretical data refers to a phase unwrapping image output by the expected model or a feature representation of the phase unwrapping image. The Zernike polynomial theoretical data refers to Zernike polynomial coefficients output by the expected model or a feature representation of the Zernike polynomial coefficients.

[0052] In this embodiment, to improve the accuracy of the model, as many first samples as possible can be obtained to train the surface shape reconstruction model based on the first samples.

[0053] S220, for each first sample, input the original interference fringe image in the first sample and the image description information into an image preprocessing model in the surface shape reconstruction model to be trained to obtain third interference fringe data.

[0054] In this embodiment, the original interference fringe image in the first sample and the image description information can be input into the image preprocessing model in the surface shape reconstruction model to be trained. The model pre-processes the original interference fringe image and outputs the third interference fringe data. The third interference fringe data includes a third interference fringe image obtained by preprocessing the original interference fringe image and a feature representation of the third interference fringe image.

[0055] For example, referring to Figure 4 The original interference fringe image 41 in the first sample and the image description information 42 can be input into the image preprocessing model 43 to denoise the interference fringe image. The third interference fringe image 44 in the third interference fringe data is obtained and displayed.

[0056] S230, input the third interference fringe data and the phase unwrapping processing mode description information in the first sample into a phase unwrapping model in the surface shape reconstruction model to obtain phase unwrapping prediction data.

[0057] In this embodiment, the third interference fringe data and the phase unwrapping processing mode description information in the first sample can be input into the phase unwrapping model in the surface shape reconstruction model to be trained to perform phase extraction and unwrapping processing on the first interference fringe image, and obtain the phase unwrapping prediction data. The phase unwrapping prediction data includes a phase unwrapping prediction image obtained by phase extraction and unwrapping processing on the third interference fringe image and a feature representation of the phase unwrapping prediction image.

[0058] For example, continuing to refer to Figure 4The third interference fringe data and the phase unwrapping processing mode description information in the first sample are input into the phase unwrapping model 45 to perform image phase extraction and unwrapping, and a phase unwrapping prediction graph in phase unwrapping prediction data is obtained, and is displayed.

[0059] In the embodiment, the phase unwrapping prediction graph and the wave surface fitting requirement information, the Zernike coefficient solving mode description information, the Zernike polynomial graph and the polynomial conversion guide information in the first sample are input into the wave surface fitting model in the surface shape reconstruction model to be trained to obtain first wave surface fitting guide features. Further, the Zernike polynomial prediction data is obtained in combination with the first wave surface fitting guide features. The Zernike polynomial prediction data includes Zernike polynomial prediction coefficients and a feature representation of the Zernike polynomial prediction coefficients. For example, still referring to FIG. 4, the phase unwrapping prediction graph 45 and the wave surface fitting requirement information, the Zernike coefficient solving mode description information, the Zernike polynomial graph and the polynomial conversion guide information in the first sample are input into the wave surface fitting model 47 to perform wafer surface shape reconstruction to obtain the Zernike polynomial prediction data.

[0060] In the embodiment, the phase unwrapping prediction graph and the wave surface fitting requirement information, the Zernike coefficient solving mode description information, the Zernike polynomial graph and the polynomial conversion guide information in the first sample are input into the wave surface fitting model in the surface shape reconstruction model to be trained to obtain first wave surface fitting guide features. Further, the Zernike polynomial prediction data is obtained in combination with the first wave surface fitting guide features. The Zernike polynomial prediction data includes Zernike polynomial prediction coefficients and a feature representation of the Zernike polynomial prediction coefficients. For example, still referring to FIG. 4, the phase unwrapping prediction graph 45 and the wave surface fitting requirement information, the Zernike coefficient solving mode description information, the Zernike polynomial graph and the polynomial conversion guide information in the first sample are input into the wave surface fitting model 47 to perform wafer surface shape reconstruction to obtain the Zernike polynomial prediction data. Figure 4

[0061] It should be noted that the manner of determining the first wave surface fitting guide features is similar to the manner of determining the wave surface fitting guide features according to the phase unwrapping graph, the Zernike coefficient solving mode description information, the wave surface fitting requirement information, the polynomial conversion guide information and the Zernike polynomial graph, and details are not repeated here.

[0062] In order to increase the explainability of the entire end-to-end surface shape reconstruction model, the third interference fringe graph, the phase unwrapping prediction graph and the surface shape graph determined based on the Zernike polynomial prediction data and the Zernike polynomial graph can be displayed to achieve a visualized intermediate result, and the visualized intermediate result is achieved through a visual decoder.

[0063] S250, based on a target loss function, processing the second interference fringe data and the third interference fringe data in the first sample, the phase unwrapping theoretical data and the phase unwrapping prediction data in the first sample, and the Zernike polynomial theoretical data and the Zernike polynomial prediction data in the first sample to obtain a loss value.

[0064] ​In the embodiment, the target loss function can be used to compare the second interference fringe data and the third interference fringe data in the first sample, compare the phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample, and compare the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample, to obtain a final loss value, so as to correct the model parameters of the surface shape reconstruction model to be trained based on the loss value.

[0065] Optionally, the target loss function includes a first loss function, a second loss function and a third loss function. Based on the target loss function, the loss value is obtained by processing the second interference fringe data and the third interference fringe data in the first sample, the phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample, and the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample, including: based on the first loss function, the first error value is obtained by processing the second interference fringe data and the third interference fringe data in the first sample; based on the second loss function, the second error value is obtained by processing the phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample; based on the third loss function, the third error value is obtained by processing the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample; and based on the first error value, the second error value and the third error value, the loss value is obtained.

[0066] In the embodiment, the second interference fringe data and the third interference fringe data in the first sample can be used as the input parameters of the first loss function to obtain the first error value. The phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample can be used as the input parameters of the second loss function to obtain the second error value. The Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample can be used as the input parameters of the third loss function to obtain the third error value. Further, the first error value, the second error value and the third error value are summed to obtain the loss value.

[0067] For example, the target loss function can be represented as:

[0068] L total =L 图像预处理 +L 相位提取解包裹 +L 面形拟合 ; wherein, L total represents the loss value, the result of L 图像预处理 represents the first error value, the result of L 相位提取解包裹 represents the second error value, and the result of L 面形拟合 represents the third error value.

[0069] The technical scheme provided by the embodiment trains the image preprocessing model, the phase unwrapping model and the wave surface fitting model in the surface reconstruction model by using the target loss function, improves the model processing accuracy, realizes end-to-end surface shape reconstruction of the to-be-tested object, and improves the efficiency of surface shape reconstruction.

[0070] Optionally, the first loss function comprises a fourth loss function, a fifth loss function and a sixth loss function; the second interference fringe data comprises a second interference fringe image and a second interference fringe feature, and the second interference fringe feature comprises a second fringe sub-feature in at least one row dimension; the third interference fringe data comprises a third interference fringe image and a third interference fringe feature, and the third interference fringe feature comprises a third fringe sub-feature in at least one row dimension. The second interference fringe feature is a feature representation of the second interference fringe image. The third interference fringe feature is a feature representation of the third interference fringe image. The row dimension can be used to represent a row of a feature matrix of the interference fringe feature.

[0071] Based on this, based on the first loss function, the second interference fringe data and the third interference fringe data in the first sample are processed to obtain a first error value, comprising: for the second interference fringe data and the third interference fringe data in the first sample, based on the fourth loss function, the second fringe sub-feature and the third fringe sub-feature in the same row dimension are processed to obtain a fourth error value; the second fringe sub-feature comprises a first feature component in at least one column dimension, and the third fringe sub-feature comprises a second feature component in at least one column dimension; based on the fifth loss function, the first feature component and the second feature component in the same column dimension are processed to obtain a fifth error value; based on the sixth loss function, the second interference fringe image and the third interference fringe image are processed to obtain a sixth error value; and based on the fourth error value, the fifth error value and the sixth error value, the first error value is obtained. Wherein, the column dimension can be used to represent a column of the feature matrix of the interference fringe feature.

[0072] Specifically, for the second interference fringe data and the third interference fringe data corresponding to the same first sample, the fourth loss function can be used to process the loss of the second fringe sub-feature and the third fringe sub-feature in the same row dimension to obtain the fourth error value. The fifth loss function is used to process the logarithm of the second feature component in each column dimension to obtain the corresponding intermediate value in each column dimension, and then the product of the intermediate value and the first feature component in the same column dimension is processed to obtain the fifth error value. The fourth loss function is used to process the loss of the second interference fringe image and the third interference fringe image corresponding to the same first sample to obtain the sixth error value. Further, the fourth error value, the fifth error value and the sixth error value are summed to obtain the first error value.

[0073] Exemplarily, the first loss function can be represented as: wherein, is a fourth loss function, the result of is a fourth error value; is a fifth loss function, the result of is a fifth error value; is a sixth loss function, the result of is a sixth error value. Wherein,

[0074]

[0075] wherein, N represents the number of row dimensions of the interference fringe feature (that is, the number of rows of the matrix), y i represents the second fringe sub-feature under the i-th row dimension, represents the third fringe sub-feature under the i-th row dimension; n represents the column dimension of the interference fringe feature, represents the first feature component under the i-th column dimension in the second fringe sub-feature under the j-th row dimension, represents the second feature component under the i-th column dimension in the third fringe sub-feature under the j-th row dimension, log() represents a logarithmic function; M represents the number of first samples in a batch during training, x i represents the second interference fringe image in the i-th first sample, represents the third interference fringe image corresponding to the i-th first sample.

[0076] The technical scheme provided by the embodiment of the application uses the fourth loss function to constrain the difference between the visual feature (that is, the second interference fringe feature) of the noise-free interference fringe image (that is, the second interference fringe image) and the visual feature (that is, the third interference fringe feature of the third interference fringe image) obtained by model reasoning. And uses the sixth loss function to constrain the difference between the second interference fringe image and the visual feature decoding image (that is, the second interference fringe image and the third interference fringe image) obtained by model reasoning. And uses the fifth loss function to measure the difference between the visual feature of the second interference fringe image and the statistical distribution of the feature output by the model reasoning. Realize to improve the robustness of the model to abnormal values, narrow the difference between the probability distribution predicted by the model and the theoretical data, improve the model preprocessing precision.

[0077] It should be noted that, based on the first loss function, the second interference fringe data and the third interference fringe data in the first sample are processed to obtain the first error value. The way is similar to the way of processing the phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample based on the second loss function to obtain the second error value, which will not be repeated here.

[0078] In the embodiment, the third loss function comprises a seventh loss function, an eighth loss function, a ninth loss function and a tenth loss function, the Zernike polynomial theoretical data comprises Zernike polynomial theoretical coefficients and theoretical coefficient eigenvectors, and the Zernike polynomial prediction data comprises Zernike polynomial prediction coefficients and prediction coefficient eigenvectors. The theoretical coefficient eigenvectors are characteristic representations of the Zernike polynomial theoretical coefficients. The prediction coefficient eigenvectors are characteristic representations of the Zernike polynomial prediction coefficients.

[0079] Based on this, based on the third loss function, the Zernike polynomial theoretical data and the Zernike polynomial prediction data in the first sample are processed to obtain a third error value, comprising: for the Zernike polynomial theoretical data and the Zernike polynomial prediction data in the first sample, based on the seventh loss function, the Zernike polynomial coefficients of the same term in the Zernike polynomial theoretical coefficients and the Zernike polynomial prediction coefficients are processed to obtain a seventh error value; based on the eighth loss function, the Zernike polynomial coefficients of each term in the Zernike polynomial prediction coefficients are logarithmically processed to obtain a first coefficient corresponding to each Zernike polynomial coefficient, and based on the first coefficient and the Zernike polynomial coefficient of the same term in the Zernike polynomial theoretical coefficients, an eighth error value is determined; based on the ninth loss function, the Zernike polynomial features in the same row dimension of the prediction coefficient eigenvectors and the theoretical coefficient eigenvectors are processed to obtain a ninth error value; wherein the prediction coefficient eigenvectors comprise at least one third characteristic component in a column dimension, and the theoretical coefficient eigenvectors comprise at least one fourth characteristic component in a column dimension; based on the tenth loss function, the third characteristic component and the fourth characteristic component in the same column dimension are processed to obtain a tenth error value; based on the seventh error value, the eighth error value, the ninth error value and the tenth error value, the third error value is determined.

[0080] Specifically, for the Zernike polynomial theoretical data and the Zernike polynomial predicted data corresponding to the first sample, the Zernike polynomial coefficients of the same term in the Zernike polynomial theoretical coefficients and the Zernike polynomial predicted coefficients can be loss processed by the seventh loss function to obtain a seventh error value. The Zernike polynomial coefficients of each term in the Zernike polynomial predicted coefficients are logarithmically processed by the eighth loss function to obtain a first coefficient corresponding to each Zernike polynomial coefficient. Further, the first coefficient and the Zernike polynomial coefficients of the same term in the Zernike polynomial theoretical coefficients are multiplied to obtain an eighth error value. According to the ninth loss function, the Zernike polynomial features in the same row dimension of the predicted coefficient feature vector and the theoretical coefficient feature vector are loss processed to obtain a ninth error value. The third feature components in each column dimension are logarithmically processed by the tenth loss function to obtain intermediate data corresponding to each third feature component in each column dimension, and then the intermediate data and the fourth feature components in the same column dimension are multiplied to obtain a tenth error value. Further, the seventh error value, the eighth error value, the ninth error value and the tenth error value are summed to obtain a third error value.

[0081] For example, the third loss function can be expressed as:

[0082] wherein, the seventh loss function is the result of is the seventh error value; the eighth loss function is the result of is the eighth error value; the ninth loss function is the result of is the ninth error value; the tenth loss function is the result of is the tenth error value. Wherein,

[0083]

[0084] In the formula, n represents the number of Zernike polynomial coefficients, a i represents the i-th Zernike polynomial theoretical coefficient, represents the i-th Zernike polynomial predicted coefficient, N represents the number of row dimensions of the Zernike polynomial coefficient feature (i.e. the theoretical coefficient feature vector and the predicted coefficient feature vector), M represents the column dimension of the Zernike polynomial coefficient feature, y i represents the i-th row dimension of the theoretical coefficient feature vector, represents the i-th row dimension of the predicted coefficient feature vector, represents the fourth feature component of the i-th column dimension in the j-th row dimension of the theoretical coefficient feature vector, represents the third feature component in the i-th column dimension of the predicted coefficient feature vector in the j-th row dimension.

[0085] S260, based on the loss value, correcting the model parameter in the surface shape reconstruction model to be trained.

[0086] In this embodiment, based on the loss value, the model parameter in the surface shape reconstruction model to be trained is continuously adjusted to make the model optimal, so as to obtain the trained surface shape reconstruction model.

[0087] S270, taking the target loss function convergence as the training target to obtain the trained surface shape reconstruction model.

[0088] The training target refers to training to achieve the target loss function convergence as the target.

[0089] In this embodiment, the training error of the target loss function, i.e. the loss parameter, can be used as a condition for detecting whether the target loss function currently reaches convergence, such as whether the training error is less than a preset error or the error change trend is stable, or whether the current iteration number is equal to a preset number. If it is detected that the convergence condition is reached, such as the training error of the target loss function is less than the preset error or the error change trend is stable, it indicates that the surface shape reconstruction model to be trained is trained, and at this time the iteration training can be stopped. If it is detected that the current convergence condition is not reached, the first sample can be further used to train the surface shape reconstruction model to be trained until the training error of the target loss function is within a preset range. When the training error of the target loss function reaches convergence, the trained surface shape reconstruction model to be trained is used as the surface shape reconstruction model, so that the subsequent interference fringe pattern, image description information, phase unwrapping processing mode description information, wave surface fitting requirement information, Zernike coefficient solving mode description information, Zernike polynomial image and polynomial conversion guidance information are input into the pre-trained surface shape reconstruction model to obtain the Zernike polynomial coefficient, and then based on the Zernike polynomial image and the Zernike polynomial coefficient of different orders, the surface shape of the object to be measured is obtained.

[0090] The technical scheme of the embodiment is used to realize end-to-end integrated training of a surface shape reconstruction model from an interference fringe image to surface shape reconstruction, without separately training each model, thereby improving the accuracy of model surface shape reconstruction as a whole. Meanwhile, the output features of the interference fringe image denoising link, the output features of the phase extraction unwrapping link, and the input features of the wave surface fitting surface shape reconstruction link are visualized through the decoder to realize intermediate results, thereby increasing the explainability of the surface shape reconstruction model as a whole. Further, in the wave surface fitting link, the Zernike coefficient solving mode is usually used to describe information and a Zernike polynomial image to guide the model to realize wave surface fitting, thereby solving the problem of low data processing efficiency in Zernike polynomial solving using data formulas, and improving the Zernike polynomial coefficient determination efficiency while ensuring the accuracy of the Zernike polynomial coefficients.

[0091] Embodiment three

[0092] Figure 5 is a structural schematic diagram of an object surface shape information determination device according to embodiment three of the present application. As shown in the figure, Figure 5 the device includes an image description information determination module 310, a Zernike polynomial coefficient determination module 320, and a surface shape map determination module 330.

[0093] The image description information determination module 310 is configured to acquire an interference fringe image of a to-be-measured object and determine image description information of the interference fringe image. The interference fringe image is obtained based on coherent interference of reflected light beams after at least two light waves propagate to a reference surface and a surface of the to-be-measured object, respectively. The Zernike polynomial coefficient determination module 320 is configured to input the interference fringe image, the image description information, preset image processing mode description information, and a Zernike polynomial image into a pre-trained surface shape reconstruction model to obtain Zernike polynomial coefficients. The Zernike polynomial image is an image represented based on different orders of Zernike polynomials. The preset image processing mode description information at least includes phase unwrapping processing mode description information and Zernike coefficient solving mode description information. The surface shape map determination module 330 is configured to obtain a surface shape map of the to-be-measured object based on the different orders of Zernike polynomial images and the Zernike polynomial coefficients.

[0094] The technical scheme of the embodiment comprises the following steps: obtaining an interference fringe pattern of a to-be-measured object, and determining image description information of the interference fringe pattern; the interference fringe pattern is obtained based on coherent interference of light beams reflected after at least two light beams propagate to a reference surface and a surface of the to-be-measured object; the interference fringe pattern, the image description information, preset image processing mode description information, and a Zernike polynomial image are input into a pre-trained surface shape reconstruction model to obtain Zernike polynomial coefficients; the Zernike polynomial image is an image represented based on Zernike polynomials of different orders; the preset image processing mode description information at least comprises phase unwrapping processing mode description information and Zernike coefficient solving mode description information; based on the Zernike polynomial image of different orders and the Zernike polynomial coefficients, a surface shape map of the to-be-measured object is obtained, thereby solving the problem that the prior art relies on a Fizeau interferometer for object surface shape reconstruction, and the surface reconstruction process is complicated and has poor accuracy, and achieving the technical effects of improving the accuracy of object surface shape information determination and improving the convenience of object surface shape information reconstruction.

[0095] On the basis of the above device, optionally, the surface shape reconstruction model comprises an image preprocessing model, a phase unwrapping model, and a wave surface fitting model; the Zernike polynomial coefficient determination module 320 comprises:

[0096] A first interference fringe pattern determination unit is configured to process the interference fringe pattern according to the image description information based on the image preprocessing model in the surface shape reconstruction model, to obtain a first interference fringe pattern.

[0097] A phase unwrapping pattern determination unit is configured to process the first interference fringe pattern according to phase unwrapping processing mode description information in the preset image processing mode description information based on the phase unwrapping model in the surface shape reconstruction model, to obtain a phase unwrapping pattern; the preset image processing mode description information further comprises wave surface fitting requirement information and polynomial conversion guide information.

[0098] A Zernike polynomial coefficient determination unit is configured to determine a wave surface fitting guide feature based on the phase unwrapping pattern, the Zernike coefficient solving mode description information, the wave surface fitting requirement information, the polynomial conversion guide information, and a Zernike polynomial image based on the wave surface fitting model in the surface shape reconstruction model, and determine Zernike polynomial coefficients based on the wave surface fitting guide feature.

[0099] On the basis of the above device, optionally, the Zernike polynomial coefficient determination unit comprises:

[0100] The first fusion feature determination unit is configured to obtain first fusion features based on the wave front fitting requirement information and the phase unwrapping map.

[0101] The second fusion feature determination unit is configured to obtain second fusion features based on the Zernike coefficient solving mode description information and the Zernike polynomial image.

[0102] The wave front fitting guide feature determination unit is configured to determine wave front fitting guide features based on the first fusion features, the second fusion features and the polynomial conversion guide information.

[0103] On the basis of the above device, optionally, the device further comprises a surface shape reconstruction model determination module configured to train the surface shape reconstruction model; wherein the surface shape reconstruction model determination module comprises:

[0104] The first sample determination unit is configured to obtain a plurality of first samples; the first samples comprise original interference fringe patterns of measured objects, image description information of the original interference fringe patterns, phase unwrapping processing mode description information, Zernike coefficient solving mode description information, wave front fitting requirement information and polynomial conversion guide information, Zernike polynomial images, second interference fringe data, phase unwrapping theoretical data and Zernike polynomial theoretical data.

[0105] The third interference fringe data determination unit is configured to, for each of the first samples, input the original interference fringe pattern and the image description information in the first sample into an image preprocessing model in the surface shape reconstruction model to be trained to obtain third interference fringe data.

[0106] The phase unwrapping prediction data determination unit is configured to input the third interference fringe data and the phase unwrapping processing mode description information in the first sample into a phase unwrapping model in the surface shape reconstruction model to obtain phase unwrapping prediction data.

[0107] The Zernike polynomial prediction data determination unit is configured to input the phase unwrapping prediction data and the wave front fitting requirement information, the Zernike coefficient solving mode description information, the Zernike polynomial image and the polynomial conversion guide information in the first sample into a wave front fitting model in the surface shape reconstruction model to obtain Zernike polynomial prediction data.

[0108] a loss value determination unit configured to determine a loss value based on a target loss function and processing the second interferometric fringe data and the third interferometric fringe data in the first sample, the phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample, and the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample;

[0109] a parameter correction unit configured to correct a model parameter in the surface profile reconstruction model based on the loss value;

[0110] a surface profile reconstruction model determination unit configured to take the convergence of the target loss function as a training target to obtain a trained surface profile reconstruction model.

[0111] In the above device, optionally, the target loss function comprises a first loss function, a second loss function and a third loss function, and the loss value determination unit comprises:

[0112] a first error value determination unit configured to determine a first error value based on the first loss function and processing the second interferometric fringe data and the third interferometric fringe data in the first sample;

[0113] a second error value determination unit configured to determine a second error value based on the second loss function and processing the phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample;

[0114] a third error value determination unit configured to determine a third error value based on the third loss function and processing the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample;

[0115] a loss value determination sub-unit configured to determine a loss value based on the first error value, the second error value and the third error value.

[0116] In the above device, optionally, the first loss function comprises a fourth loss function, a fifth loss function and a sixth loss function; the second interferometric fringe data comprises a second interferometric fringe pattern and a second interferometric fringe feature, and the second interferometric fringe feature comprises a second fringe sub-feature in at least one row dimension; the third interferometric fringe data comprises a third interferometric fringe pattern and a third interferometric fringe feature, and the third interferometric fringe feature comprises a third fringe sub-feature in at least one row dimension; and the first error value determination unit comprises:

[0117] a fourth error value determination unit configured to, for the second interference fringe data and the third interference fringe data in the first sample, determine a fourth error value based on a fourth loss function for processing a second fringe sub-feature and a third fringe sub-feature in a same row dimension, wherein the second fringe sub-feature comprises a first feature component in at least one column dimension, and the third fringe sub-feature comprises a second feature component in the at least one column dimension;

[0118] a fifth error value determination unit configured to, based on a fifth loss function, determine a fifth error value for processing the first feature component and the second feature component in a same column dimension;

[0119] a sixth error value determination unit configured to, based on a sixth loss function, determine a sixth error value for processing the second interference fringe pattern and the third interference fringe pattern;

[0120] a first error value determination sub-unit configured to, based on the fourth error value, the fifth error value, and the sixth error value, determine a first error value.

[0121] In the above device, optionally, the third loss function comprises a seventh loss function, an eighth loss function, a ninth loss function, and a tenth loss function, the Zernike polynomial theoretical data comprises Zernike polynomial theoretical coefficients and a theoretical coefficient feature vector, and the Zernike polynomial prediction data comprises Zernike polynomial prediction coefficients and a prediction coefficient feature vector; and the third error value determination unit comprises:

[0122] a seventh error value determination unit configured to, for the Zernike polynomial theoretical data and the Zernike polynomial prediction data in the first sample, determine a seventh error value based on a seventh loss function for processing Zernike polynomial coefficients in a same term of the Zernike polynomial theoretical coefficients and the Zernike polynomial prediction coefficients;

[0123] an eighth error value determination unit configured to, based on an eighth loss function, respectively perform logarithmic processing on Zernike polynomial coefficients in each term of the Zernike polynomial prediction coefficients to obtain a first coefficient corresponding to each of the Zernike polynomial coefficients, and determine an eighth error value based on the first coefficient and a Zernike polynomial coefficient in a same term of the first coefficient in the Zernike polynomial theoretical coefficients;

[0124] a ninth error value determination unit configured to, based on a ninth loss function, determine a ninth error value for processing Zernike polynomial features in a same row dimension of the prediction coefficient feature vector and the theoretical coefficient feature vector; wherein the prediction coefficient feature vector comprises a third feature component in at least one column dimension, and the theoretical coefficient feature vector comprises a fourth feature component in the at least one column dimension;

[0125] a tenth error value determination unit configured to determine a tenth error value based on a tenth loss function and the third feature component and the fourth feature component in the same column dimension;

[0126] a third error value determination sub-unit configured to determine a third error value based on the seventh error value, the eighth error value, the ninth error value and the tenth error value.

[0127] The object surface shape information determination apparatus provided by the embodiments of the present application can perform the object surface shape information determination method provided by any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the performing method.

[0128] Embodiment Four

[0129] Figure 6 is a structural schematic diagram of an electronic device for implementing the object surface shape information determination method of the embodiments of the present application. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0130] As shown in Figure 6 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is in communication with the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0131] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0132] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the object surface shape information determination method.

[0133] In some embodiments, the object surface shape information determination method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the object surface shape information determination method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the object surface shape information determination method by any other appropriate means, such as by means of firmware.

[0134] The various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0135] Computer programs for implementing the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer program, when executed, can cause instructions defined in the flow charts and / or block diagrams to be implemented. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, and partially on a remote machine or entirely on a remote machine or server.

[0136] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0137] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0138] The systems and techniques described here can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here, or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0139] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0140] The embodiment of the present application further provides a computer program product, comprising a computer program which, when executed by a processor, implements the object surface shape information determination method provided in any embodiment of the present application.

[0141] The computer program code implementing the application can be written in one or more programming languages or combinations of languages including object oriented languages such as Java, Smalltalk, Python, C++, C# or conventional procedural programming languages such as "C" or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0142] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps recited in the present application can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which are not limited herein.

[0143] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining object face shape information, characterized in that: include: Obtaining an interference fringe pattern of the object to be measured and determining image description information of the interference fringe pattern; wherein the interference fringe pattern is obtained based on the interference of light beams reflected after at least two light waves propagate to a reference surface and the surface of the object to be measured; Inputting the interference fringe pattern, the image description information, the preset image processing method description information, and the Zernike polynomial image into a pre-trained surface reconstruction model to obtain Zernike polynomial coefficients; wherein the Zernike polynomial image is an image represented by Zernike polynomials of different orders; and the preset image processing method description information includes at least phase unwrapping processing method description information and Zernike coefficient solution method description information; Obtaining a surface shape map of the object to be measured based on the Zernike polynomial images of different orders and the Zernike polynomial coefficients; The surface reconstruction model includes an image preprocessing model, a phase unwrapping model, and a wavefront fitting model; the interference fringe pattern, the image description information, the preset image processing method description information, and the Zernike polynomial image are input into the pre-trained surface reconstruction model to obtain the Zernike polynomial coefficients, including: Based on the image preprocessing model in the surface reconstruction model, the interference fringe pattern is processed according to the image description information to obtain a first interference fringe pattern; Based on the phase unwrapping model in the surface reconstruction model, the first interference fringe pattern is processed according to the phase unwrapping processing method description information in the preset image processing method description information to obtain a phase unwrapping pattern; wherein the preset image processing method description information also includes wavefront fitting requirement information and polynomial conversion guidance information; Based on the wavefront fitting model in the surface reconstruction model, the wavefront fitting guiding features are determined according to the phase unwrapping diagram, the Zernike coefficient solution method description information, the wavefront fitting requirement information, the polynomial conversion guidance information and the Zernike polynomial image, and the Zernike polynomial coefficients are determined based on the wavefront fitting guiding features.

2. The method according to claim 1, characterized in that The determining of the wavefront fitting guidance feature according to the phase unwrapping diagram, the Zernike coefficient solution method description information, the wavefront fitting requirement information, the polynomial conversion guidance information, and the Zernike polynomial image includes: Obtaining a first fusion feature based on the wavefront fitting requirement information and the phase unwrapping map; Obtaining a second fusion feature based on the Zernike coefficient solution method description information and the Zernike polynomial image; A wavefront fitting guiding feature is determined based on the first fusion feature, the second fusion feature and the polynomial conversion guiding information.

3. The method according to claim 1, characterized in that The method further comprises: The face reconstruction model is obtained by training; wherein, The training to obtain the face reconstruction model includes: Acquire multiple first samples; the first samples include an original interference fringe pattern of a measured object, image description information of the original interference fringe pattern, description information of a phase unwrapping processing method, description information of a Zernike coefficient solution method, wavefront fitting requirement information and polynomial conversion guidance information, the Zernike polynomial image, second interference fringe data, phase unwrapping theoretical data, and Zernike polynomial theoretical data; the second interference fringe data is an interference fringe image after preprocessing the original interference fringe data as output by a desired model, or a feature representation corresponding to the preprocessed interference fringe image; For each of the first samples, inputting the original interference fringe pattern and image description information in the first sample into an image preprocessing model in the surface reconstruction model to be trained to obtain third interference fringe data; Inputting the third interference fringe data and the phase unwrapping processing mode description information in the first sample into the phase unwrapping model in the surface reconstruction model to obtain phase unwrapping prediction data; Inputting the phase unwrapping prediction data and the wavefront fitting requirement information, Zernike coefficient solution method description information, Zernike polynomial image, and polynomial conversion guidance information in the first sample into the wavefront fitting model in the surface reconstruction model to obtain Zernike polynomial prediction data; Based on the target loss function, processing the second interference fringe data and the third interference fringe data in the first sample, the phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample, and the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample to obtain a loss value; Based on the loss value, modifying the model parameters in the face reconstruction model; The convergence of the target loss function is used as a training goal to obtain a trained face reconstruction model.

4. The method according to claim 3, characterized in that The target loss function includes a first loss function, a second loss function, and a third loss function. The process of processing the second interference fringe data and the third interference fringe data in the first sample, the phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample, and the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample based on the target loss function to obtain a loss value includes: processing the second interference fringe data and the third interference fringe data in the first sample based on the first loss function to obtain a first error value; processing the phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample based on the second loss function to obtain a second error value; Processing the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample based on the third loss function to obtain a third error value; A loss value is obtained based on the first error value, the second error value, and the third error value.

5. The method according to claim 4, characterized in that The first loss function includes a fourth loss function, a fifth loss function, and a sixth loss function; the second interference fringe data includes a second interference fringe pattern and a second interference fringe feature, and the second interference fringe feature includes a second fringe sub-feature in at least one row dimension; the third interference fringe data includes a third interference fringe pattern and a third interference fringe feature, and the third interference fringe feature includes a third fringe sub-feature in at least one row dimension; The processing of the second interference fringe data and the third interference fringe data in the first sample based on the first loss function to obtain a first error value includes: For the second interference fringe data and the third interference fringe data in the first sample, processing the second fringe sub-feature and the third fringe sub-feature in the same row dimension based on a fourth loss function to obtain a fourth error value; wherein the second fringe sub-feature includes a first characteristic component in at least one column dimension, and the third fringe sub-feature includes a second characteristic component in at least one column dimension; Processing the first feature component and the second feature component in the same column dimension based on a fifth loss function to obtain a fifth error value; processing the second interference fringe pattern and the third interference fringe pattern based on a sixth loss function to obtain a sixth error value; A first error value is obtained based on the fourth error value, the fifth error value, and the sixth error value.

6. The method according to claim 4, characterized in that The third loss function includes a seventh loss function, an eighth loss function, a ninth loss function, and a tenth loss function; the Zernike polynomial theoretical data includes Zernike polynomial theoretical coefficients and theoretical coefficient feature vectors; the Zernike polynomial predicted data includes Zernike polynomial predicted coefficients and predicted coefficient feature vectors; and the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample are processed based on the third loss function to obtain a third error value, including: For the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample, processing the Zernike polynomial coefficients of the same item in the Zernike polynomial theoretical coefficients and the Zernike polynomial predicted coefficients based on a seventh loss function to obtain a seventh error value; performing logarithmic processing on each Zernike polynomial coefficient of the Zernike polynomial prediction coefficients based on an eighth loss function to obtain a first coefficient corresponding to each Zernike polynomial coefficient, and determining an eighth error value based on the first coefficient and the Zernike polynomial coefficient of the same term as the first coefficient in the Zernike polynomial theoretical coefficients; Based on a ninth loss function, processing the Zernike polynomial features in the same row dimension in the prediction coefficient eigenvector and the theoretical coefficient eigenvector to obtain a ninth error value; wherein the prediction coefficient eigenvector includes a third eigencomponent in at least one column dimension, and the theoretical coefficient eigenvector includes a fourth eigencomponent in at least one column dimension; Processing the third characteristic component and the fourth characteristic component in the same column dimension based on a tenth loss function to obtain a tenth error value; A third error value is determined based on the seventh error value, the eighth error value, the ninth error value, and the tenth error value.

7. A device for determining object face shape information, characterized in that: include: An image description information determination module is configured to obtain an interference fringe pattern of the object to be measured and determine image description information of the interference fringe pattern; wherein the interference fringe pattern is obtained based on the interference of light beams reflected after at least two light waves propagate to a reference surface and the surface of the object to be measured, respectively; a Zernike polynomial coefficient determination module, configured to input the interference fringe pattern, the image description information, the preset image processing method description information, and the Zernike polynomial image into a pre-trained surface reconstruction model to obtain Zernike polynomial coefficients; wherein the Zernike polynomial image is an image represented by Zernike polynomials of different orders; and the preset image processing method description information includes at least phase unwrapping processing method description information and Zernike coefficient solution method description information; a surface shape map determining module, configured to obtain a surface shape map of the object to be measured based on the Zernike polynomial images of different orders and the Zernike polynomial coefficients; The surface reconstruction model includes an image preprocessing model, a phase unwrapping model, and a wavefront fitting model; the Zernike polynomial coefficient determination module includes: a first interference fringe pattern determining unit, configured to process the interference fringe pattern according to the image description information based on an image preprocessing model in a surface reconstruction model to obtain a first interference fringe pattern; a phase unwrapping map determining unit, configured to process the first interference fringe pattern based on the phase unwrapping model in the surface reconstruction model and according to phase unwrapping processing mode description information in preset image processing mode description information to obtain a phase unwrapping map; wherein the preset image processing mode description information further includes wavefront fitting requirement information and polynomial conversion guidance information; A Zernike polynomial coefficient determination unit is configured to determine, based on a wavefront fitting model in the surface reconstruction model, wavefront fitting guidance features according to the phase unwrapping diagram, the Zernike coefficient solution method description information, the wavefront fitting requirement information, the polynomial conversion guidance information, and the Zernike polynomial image, and determine Zernike polynomial coefficients based on the wavefront fitting guidance features.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the object face shape information determination method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the object face shape information determination method according to any one of claims 1 to 6 when executed.

Images