Object surface shape information determination method and device, equipment, medium and product

Abstract

The invention discloses an object surface shape information determination method and device, equipment, a medium and a product. The method comprises the following steps: acquiring an interference fringe pattern of a to-be-detected object, and determining image description information of the interference fringe pattern; the interference fringe pattern is obtained based on interference of light beams reflected after at least two light waves are transmitted to a reference plane and the surface of the to-be-measured object respectively; inputting the interference fringe pattern, the image description information, the preset image processing mode description information and the Zernike polynomial image into a pre-trained surface shape reconstruction model to obtain a Zernike polynomial coefficient; and based on the Zernike polynomial images of different orders and the Zernike polynomial coefficients, obtaining a surface shape graph of the to-be-measured object. According to the technical scheme provided by the embodiment of the invention, the effect that the determined surface shape graph is highly matched with the to-be-detected object can be realized.

Description

Technical Field

[0001] The present invention relates to the field of computer processing technology, and in particular to a method, device, equipment, medium and product for determining object face shape information. Background Art

[0002] In the process of chip manufacturing, some materials, such as wafers, are usually used. In order to ensure the accuracy of chip manufacturing, it is usually necessary to accurately measure and reconstruct the surface shapes of some material objects required for manufacturing.

[0003] Existing surface reconstruction methods usually rely on Fizeau interferometers to reconstruct the surface structure of an object. Some control parameters and strategies of Fizeau interferometers during the measurement process rely heavily on the user's engineering experience to determine, which places high demands on the user, and there are problems such as cumbersome surface reconstruction process and poor accuracy. Summary of the invention

[0004] The present invention provides a method, device, equipment, medium and product for determining object face shape information, so as to achieve the technical effect of making the determined surface shape map highly compatible with the object to be measured, thereby improving the convenience of reconstructing the object face shape information.

[0005] According to one aspect of the present invention, a method for determining face shape information of an object is provided, the method comprising:

[0006] Obtaining an interference fringe pattern of the object to be measured, and determining image description information of the interference fringe pattern; wherein the interference fringe pattern is obtained based on the interference of light beams reflected by at least two light waves propagating to a reference surface and a surface of the object to be measured respectively;

[0007] Inputting the interference fringe pattern, the image description information, the preset image processing method description information and the Zernike polynomial image into a pre-trained surface reconstruction model to obtain Zernike polynomial coefficients; wherein the Zernike polynomial image is an image represented by Zernike polynomials of different orders; the preset image processing method description information at least includes phase unwrapping processing method description information and Zernike coefficient solution method description information;

[0008] A surface shape map of the object to be measured is obtained based on the Zernike polynomial images of different orders and the Zernike polynomial coefficients.

[0009] According to another aspect of the present invention, there is provided a device for determining face shape information of an object, the device comprising:

[0010] An image description information determination module is used to obtain an interference fringe pattern of the object to be measured and determine the image description information of the interference fringe pattern; wherein the interference fringe pattern is obtained based on the interference of light beams reflected by at least two light waves propagating to a reference surface and a surface of the object to be measured respectively;

[0011] A Zernike polynomial coefficient determination module, used for inputting the interference fringe pattern, the image description information, the preset image processing method description information and the Zernike polynomial image into a pre-trained surface reconstruction model to obtain Zernike polynomial coefficients; wherein the Zernike polynomial image is an image represented by Zernike polynomials of different orders; the preset image processing method description information at least includes phase unwrapping processing method description information and Zernike coefficient solution method description information;

[0012] The surface shape map determining module is used to obtain the surface shape map of the object to be measured based on the Zernike polynomial images of different orders and the Zernike polynomial coefficients.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and a memory in communication with the at least one processor; wherein,

[0015] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the object face shape information determination method described in any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the object face shape information determination method described in any embodiment of the present invention when executed.

[0017] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the method for determining the face shape information of an object as described in any embodiment of the present invention.

[0018] The technical solution of the embodiment of the present invention is to obtain the interference fringe pattern of the object to be measured and determine the image description information of the interference fringe pattern; wherein the interference fringe pattern is obtained after the interference of the light beams reflected by at least two light waves propagating to the reference surface and the surface of the object to be measured respectively; the interference fringe pattern, the image description information, the preset image processing method description information and the Zernike polynomial image are input into a pre-trained surface reconstruction model to obtain the Zernike polynomial coefficients; wherein the Zernike polynomial image is an image represented by Zernike polynomials of different orders; the preset image processing method description information at least includes phase unwrapping processing method description information and Zernike coefficient solution method description information; based on The Zernike polynomial images and Zernike polynomial coefficients of different orders are used to obtain the surface shape map of the object to be measured, which solves the problem that the existing technology relies on Fizeau interferometer to reconstruct the surface shape of the object, and the surface reconstruction process is cumbersome and the accuracy is poor. The interference fringe pattern, image description information, preset image processing method description information and Zernike polynomial image are input into the end-to-end trained surface reconstruction model to obtain Zernike polynomial coefficients, and the surface shape map of the object to be measured is determined by combining the Zernike polynomial image and the Zernike polynomial coefficients, so that the determined surface shape map is highly compatible with the object to be measured, and the technical effect of improving the convenience of reconstructing the surface shape information of the object is achieved.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flow chart of a method for determining object face shape information provided according to Embodiment 1 of the present invention;

[0022] Figure 2 is a schematic diagram of a method for determining object face shape information provided in accordance with a first embodiment of the present invention;

[0023] Figure 3 is a flow chart of a method for determining object face shape information provided according to Embodiment 2 of the present invention;

[0024] Figure 4 is a schematic diagram of a method for determining object face shape information provided in accordance with a second embodiment of the present invention;

[0025] Figure 5 is a schematic diagram of the structure of a device for determining object face shape information provided according to Embodiment 3 of the present invention;

[0026] Figure 6 It is a schematic diagram of the structure of an electronic device for implementing the method for determining the object face shape information according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] Embodiment 1

[0030] Figure 1 This is a flow chart of a method for determining object surface shape information according to the first embodiment of the present invention. This embodiment is applicable to the case of reconstructing the three-dimensional surface shape of the object to be measured. The method can be executed by an object surface shape information determination device. The object surface shape information determination device can be implemented in the form of hardware and / or software. The object surface shape information determination device can be configured in a computing device. Figure 1 As shown, the method includes:

[0031] S110, obtaining an interference fringe pattern of the object to be measured, and determining image description information of the interference fringe pattern.

[0032] The object to be measured can be any object whose surface state needs to be reconstructed, such as a wafer (i.e., a silicon wafer) or an object of other materials. The interference fringe pattern is obtained based on the interference of the reflected light beams after at least two light waves propagate to the reference surface and the surface of the object to be measured. It can be understood that when the laser is emitted by the laser, part of the laser is reflected on the reference surface to form the reference light, and the other part of the laser is reflected on the surface of the object to be measured to form the measurement light. When the reference light and the measurement light meet, interference occurs, and the interference fringes generated are captured by the camera to form an interference fringe image. The interference fringe image contains the surface shape information of the object to be measured. For example, if the surface of the object to be measured is flat, the interference fringes of the reflected light are uniform; if the surface has slight undulations, the interference fringes will be curved or deformed. The image description information is the guiding information for generating the preprocessed interference fringe image. For example, the image description information can be the feature representation of the quantitative or qualitative description information of the image content of the interference fringe image under different evaluation dimensions. For example, the image description information includes but is not limited to the content understanding of the interference fringe image and the text features of the noise type and distortion description. Content understanding can include the expression of visual features such as color, texture, shape, and edges.

[0033] In this embodiment, an interferometer or other optical devices can be used to collect the interference fringe pattern of the object to be measured. Then, the interference fringe pattern can be input into the pre-trained visual language model, the interference fringe pattern can be analyzed and processed, and the text features of the description information of the interference fringe image in various dimensions can be extracted to obtain the image description information, so as to provide guidance information for pre-processing the interference fringe image for the subsequent surface reconstruction model based on the image description information.

[0034] S120, inputting the interference fringe pattern, image description information, preset image processing method description information and Zernike polynomial image into a pre-trained surface reconstruction model to obtain Zernike polynomial coefficients.

[0035] Among them, the preset image processing method description information at least includes phase unwrapping processing method description information and Zernike coefficient solution method description information. Phase unwrapping processing method description information refers to information that describes the phase extraction and unwrapping processing method in natural language, which is used to guide the surface reconstruction model on how to perform phase extraction and unwrapping, that is, to guide and supervise the model to output a continuous phase unwrapping map. For example, the phase unwrapping processing method description information includes step instructions on how to extract the phase of the pixel point in the image and how to restore the continuous phase distribution from the wrapped phase data. Zernike coefficient solution method description information refers to information that describes the calculation method for solving Zernike polynomial coefficients in natural language. Natural language includes but is not limited to one or more combinations of images, symbols, videos, and text. Zernike polynomial images are images represented by Zernike polynomials of different orders, that is, images that use Zernike polynomials to describe and simulate wavefront aberrations or wavefront shapes. These images show the changes in the wavefront or surface shape corresponding to each term of the Zernike polynomial. Each Zernike polynomial corresponds to a specific type of aberration, such as spherical aberration, coma, astigmatism, etc. Through Zernike polynomials, complex wavefront aberrations can be decomposed into a series of basic aberration patterns, which appear as specific ripple patterns on the image, so that they can be intuitively displayed in the image. The image is the Zernike polynomial image. In other words, the Zernike polynomial image includes the wavefront image corresponding to each term in the Zernike polynomial. The Zernike polynomial is a set of continuous function sequences that form a completely orthogonal set on the unit disk, which is used to describe the wavefront error or surface shape. The Zernike polynomial coefficients refer to the coefficients corresponding to each term in the Zernike polynomial, which are used to characterize the aberration amount corresponding to the Zernike polynomial. The surface reconstruction model can be a pre-trained model used to determine the Zernike polynomial coefficients and reconstruct the surface information of the object to be measured based on the Zernike polynomial coefficients.

[0036] It should be noted that the preset image processing method description information can be input in real time or can be predetermined. In practical applications, the interference fringe pattern, image description information, preset image processing method description information and Zernike polynomial image can be used as inputs of the surface reconstruction model to output Zernike polynomial coefficients.

[0037] In this embodiment, the surface reconstruction model includes an image preprocessing model, a phase unwrapping model and a wavefront fitting model; the interference fringe pattern, image description information, preset image processing method description information and Zernike polynomial image are input into the pre-trained surface reconstruction model to obtain Zernike polynomial coefficients, including: based on the image preprocessing model in the surface reconstruction model, the interference fringe pattern is processed according to the image description information to obtain a first interference fringe pattern; based on the phase unwrapping model in the surface reconstruction model, the first interference fringe pattern is processed according to the phase unwrapping processing method description information in the preset image processing method description information to obtain a phase unwrapping pattern; wherein the preset image processing method description information also includes wavefront fitting requirement information and polynomial conversion guidance information; based on the wavefront fitting model in the surface reconstruction model, the wavefront fitting guiding features are determined according to the phase unwrapping pattern, the Zernike coefficient solution method description information, the wavefront fitting requirement information, the polynomial conversion guidance information and the Zernike polynomial image, and the Zernike polynomial coefficients are determined based on the wavefront fitting guiding features.

[0038] The wave surface fitting requirement information refers to the description of how to perform wave surface fitting and the target of wave surface fitting in natural language. The polynomial conversion guidance information can be the description information for guiding how to convert the Zernike polynomials in the discrete unit circle into orthogonal polynomials, for example, by performing polynomial conversion through an orthogonalization algorithm (such as the Gram-Schmit orthogonalization method).

[0039] Specifically, the image description information and the interference fringe pattern can be input into the image preprocessing model, and the interference fringe pattern is preprocessed, such as denoising, image enhancement, etc., to obtain the processed interference fringe pattern as the first interference fringe pattern. Further, the first interference fringe pattern and the description information of the phase unwrapping processing method are input into the phase unwrapping model, and the first interference fringe pattern is subjected to phase extraction and unwrapping processing to obtain the processed image as the phase unwrapping image. Further, the phase unwrapping image and the wavefront fitting requirement information, the description information of the Zernike coefficient solution method, the Zernike polynomial image, and the polynomial conversion guidance information are input into the wavefront fitting model, and feature extraction is performed to obtain the wavefront fitting guiding features, and the Zernike polynomial coefficients are output by analyzing the wavefront fitting guiding features.

[0040] In this embodiment, the wavefront fitting guidance features are determined according to the phase unwrapping diagram, the description information of the Zernike coefficient solution method, the wavefront fitting requirement information, the polynomial conversion guidance information and the Zernike polynomial image, including: obtaining the first fusion feature based on the wavefront fitting requirement information and the phase unwrapping diagram; obtaining the second fusion feature based on the description information of the Zernike coefficient solution method and the Zernike polynomial image; determining the wavefront fitting guidance feature based on the first fusion feature, the second fusion feature and the polynomial conversion guidance information.

[0041] In practical applications, the wavefront fitting requirement information and the phase unwrapping map can be feature fused to obtain the first fused feature. The features corresponding to each solution step in the Zernike coefficient solution description feature are position-encoded in sequence, and the wavefront image corresponding to each item of the Zernike polynomial in the Zernike polynomial image can be position-encoded in sequence, so that the model can know the position corresponding to the wavefront image corresponding to each item; at the same time, the Zernike coefficient solution method description information can be position-encoded in the order of the solution steps, so that the model can understand the calculation process of the Zernike coefficient solution. The position-encoded Zernike polynomial image and the position-encoded Zernike coefficient solution method description information are feature-fused to obtain the second fused feature. Further, the second fused feature and the polynomial conversion guidance information are feature-fused to obtain the third fused feature, and then the first fused feature and the third fused feature are feature-fused to obtain the wavefront fitting guidance feature. It should be noted that if the preset image processing method description information is a text type, the pre-trained visual language model can be used to perform feature processing on the preset image processing method description information to obtain its feature representation. The wavefront fitting guiding features are determined using its feature representation and interference fringe pattern, image description information and Zernike polynomial image, so as to obtain Zernike polynomial coefficients by analyzing the guiding features of polynomial conversion.

[0042] For example, see Figure 2 , based on the feature fusion of the wavefront fitting requirement information 21 and the phase unwrapping image 22, a first fused feature 23 is obtained. Position encoding and feature fusion are performed based on the Zernike polynomial image 24 and the Zernike coefficient solution method description information 25 to obtain a second fused feature 26. Feature processing is performed on the polynomial conversion guidance information 27 to obtain a feature representation 28. Position encoding and feature fusion are performed based on the second fused feature 26 and the feature representation 28 to obtain a third fused feature 29. Based on the feature fusion of the first fused feature 23 and the third fused feature 29, a wavefront fitting guide feature 30 is obtained. Based on the wavefront fitting guide feature 30, the Zernike polynomial coefficient 31 is obtained.

[0043] The advantage of this setting is that by decomposing the features for determining the Zernike polynomial coefficients into multiple parts, one part fuses the data features of the two modes of phase unwrapping diagram and wavefront fitting requirement information to generate the understanding features of the wavefront fitting task requirements, that is, the first fusion feature. The other part is to fuse the image performance of each item of the Zernike polynomial as visual data with the description information of the Zernike coefficient solution method to generate the process features that characterize the entire wavefront fitting to obtain the Zernike polynomial coefficients, that is, the second fusion feature. And when the multimodal data features are fused, the position encoding is used to ensure that the feature order is consistent with the execution order of the fitting process. Further, in order to facilitate the calculation of the Zernike polynomial coefficients, the Gram-Schmit orthogonalization method will be used to convert the Zernike polynomials in the discrete unit circle into orthogonal polynomials and express them with description information, that is, polynomial conversion guidance information. Finally, these three parts of the features are fully integrated in order to form the final wavefront fitting guidance features. Through the generalization ability of the surface reconstruction model and the ability to understand images and description information, according to the wavefront fitting guidance features, the surface reconstruction model realizes the output of the Zernike polynomial coefficients.

[0044] S130 , obtaining a surface shape diagram of the object to be measured based on Zernike polynomial images of different orders and Zernike polynomial coefficients.

[0045] In this embodiment, the Zernike polynomial coefficients can be applied to Zernike polynomial images of different orders, multiplied and accumulated to obtain a continuous surface shape map, which is the surface shape map of the object to be measured. The surface shape map can intuitively display the actual surface shape of the object to be measured.

[0046] The technical solution provided in this embodiment obtains the interference fringe pattern of the object to be measured and determines the image description information of the interference fringe pattern; wherein the interference fringe pattern is obtained after the reflected light beams after at least two light waves propagate to the reference surface and the surface of the object to be measured interfere with each other; the interference fringe pattern, the image description information, the preset image processing method description information and the Zernike polynomial image are input into a pre-trained surface reconstruction model to obtain the Zernike polynomial coefficients; wherein the Zernike polynomial image is an image represented by Zernike polynomials of different orders; the preset image processing method description information at least includes phase unwrapping processing method description information and Zernike coefficient solution method description information; based on Zernike polynomials of different orders The invention relates to a polynomial image and Zernike polynomial coefficients, and obtains a surface shape map of the object to be measured, which solves the problem that the prior art relies on Fizeau interferometer to reconstruct the surface shape of the object, and has the problems of cumbersome surface reconstruction process and poor accuracy. The invention realizes that the interference fringe pattern, image description information, preset image processing method description information and Zernike polynomial image are input into an end-to-end trained surface reconstruction model to obtain Zernike polynomial coefficients, and the surface shape map of the object to be measured is determined by combining the Zernike polynomial image and the Zernike polynomial coefficients, so that the determined surface shape map is highly compatible with the object to be measured, and achieves the technical effect of improving the accuracy of determining the surface shape information of the object and improving the convenience of reconstructing the surface shape information of the object.

[0047] Embodiment 2

[0048] Figure 3 This is a flow chart of a method for determining face shape information of an object provided according to the second embodiment of the present invention. On the basis of the above embodiment, a face shape reconstruction model can also be obtained by pre-training. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same as or corresponding to the above embodiment are not repeated here.

[0049] like Figure 3 As shown, the method specifically comprises the following steps:

[0050] S210, obtaining multiple first samples; the first samples include the original interference fringe pattern of the measured object, image description information of the original interference fringe pattern, phase unwrapping processing method description information, Zernike coefficient solution method description information, wavefront fitting requirement information and polynomial conversion guidance information, Zernike polynomial image, second interference fringe data, phase unwrapping theoretical data and Zernike polynomial theoretical data.

[0051] The measured object may be an object of the same type as the object to be measured, or an object of a different type from the object to be measured. The second interference fringe data refers to the interference fringe image after preprocessing the original interference fringe data output by the expected model, or the feature representation corresponding to the preprocessed interference fringe image. The phase unwrapping theoretical data refers to the phase unwrapping diagram output by the expected model or the feature representation of the phase unwrapping diagram. The Zernike polynomial theoretical data refers to the Zernike polynomial coefficients output by the expected model or the feature representation of the Zernike polynomial coefficients.

[0052] In this embodiment, in order to improve the accuracy of the model, as many and rich first samples as possible may be obtained to obtain a face reconstruction model based on the first sample training.

[0053] S220 . For each first sample, input the original interference fringe pattern and image description information in the first sample into an image preprocessing model in the surface reconstruction model to be trained to obtain third interference fringe data.

[0054] In this embodiment, the original interference fringe pattern and image description information in the first sample can be input into the image preprocessing model in the surface reconstruction model to be trained, and the model preprocesses the original interference fringe pattern and outputs the third interference fringe data. The third interference fringe data includes the third interference fringe pattern after the original interference fringe pattern is preprocessed and the feature representation of the third interference fringe pattern.

[0055] For example, see Figure 4 The original interference fringe image 41 and the image description information 42 in the first sample can be input into the image preprocessing model 43 to denoise the interference fringe image. The third interference fringe image 44 in the third interference fringe data is obtained and displayed.

[0056] S230, inputting the third interference fringe data and the phase unwrapping processing method description information in the first sample into the phase unwrapping model in the surface reconstruction model to obtain phase unwrapping prediction data.

[0057] In this embodiment, the third interference fringe data and the phase unwrapping processing method description information in the first sample can be input into the phase unwrapping model in the surface reconstruction model to be trained, and the first interference fringe pattern is subjected to phase extraction and unwrapping processing to obtain phase unwrapping prediction data. The phase unwrapping prediction data includes a phase unwrapping prediction graph after the phase extraction and unwrapping processing of the third interference fringe pattern and a feature representation of the phase unwrapping prediction graph.

[0058] For example, see Figure 4, the phase unwrapping processing method description information in the third interference fringe data and the first sample is input into the phase unwrapping model 45, image phase extraction and unwrapping are performed, and the phase unwrapping prediction graph 45 in the phase unwrapping prediction data is obtained and displayed.

[0059] S240, input the phase unwrapping prediction data and the wavefront fitting requirement information in the first sample, the Zernike coefficient solution method description information, the Zernike polynomial image and the polynomial conversion guidance information into the wavefront fitting model in the surface reconstruction model to obtain the Zernike polynomial prediction data.

[0060] In this embodiment, the phase unwrapping prediction map and the wavefront fitting requirement information, the Zernike coefficient solution method description information, the Zernike polynomial image and the polynomial conversion guidance information in the first sample can be input into the wavefront fitting model in the surface reconstruction model to be trained to obtain the first wavefront fitting guidance feature. Then, the Zernike polynomial prediction data is obtained in combination with the first wavefront fitting guidance feature. The Zernike polynomial prediction data includes the Zernike polynomial prediction coefficients and the feature representation of the Zernike polynomial prediction coefficients. For example, continue to refer to Figure 4 The phase unwrapping prediction data and the wavefront fitting requirement information in the first sample, the description information of the Zernike coefficient solution method, the Zernike polynomial image and the polynomial conversion guidance information are input into the wavefront fitting model 47 to reconstruct the wafer surface shape and obtain the Zernike polynomial prediction data.

[0061] It should be noted that the method of determining the first wavefront fitting guiding features is similar to the method of determining the wavefront fitting guiding features based on the phase unwrapping diagram, the Zernike coefficient solution method description information, the wavefront fitting requirement information, the polynomial conversion guidance information and the Zernike polynomial image, and will not be repeated here.

[0062] In order to increase the interpretability of the entire end-to-end surface reconstruction model, the third interference fringe pattern, the phase unwrapping prediction pattern, and the surface shape pattern determined based on the Zernike polynomial prediction data and the Zernike polynomial image can be displayed to achieve the effect of visualizing the intermediate results. This visualization effect is achieved through a visual decoder.

[0063] S250. Based on the target loss function, the second interference fringe data and the third interference fringe data in the first sample, the phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample, and the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample are processed to obtain a loss value.

[0064] In this embodiment, the target loss function can be used to compare the second interference fringe data and the third interference fringe data in the first sample, to compare the phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample, and to compare the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample to obtain the final loss value, so as to correct the model parameters of the surface reconstruction model to be trained based on the loss value.

[0065] Optionally, the target loss function includes a first loss function, a second loss function and a third loss function. Based on the target loss function, the second interference fringe data and the third interference fringe data in the first sample, the phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample, and the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample are processed to obtain a loss value, including: based on the first loss function, the second interference fringe data and the third interference fringe data in the first sample are processed to obtain a first error value; based on the second loss function, the phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample are processed to obtain a second error value; based on the third loss function, the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample are processed to obtain a third error value; based on the first error value, the second error value and the third error value, the loss value is obtained.

[0066] In this embodiment, the second interference fringe data and the third interference fringe data in the first sample can be used as input parameters of the first loss function to obtain a first error value. The phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample are used as input parameters of the second loss function to obtain a second error value. The Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample are used as input parameters of the third loss function to obtain a third error value. Further, the first error value, the second error value, and the third error value are summed to obtain a loss value.

[0067] For example, the objective loss function can be expressed as:

[0068] L total =L 图像预处理 +L 相位提取解包裹 +L 面形拟合 Among them, L total Represents the loss value, L 图像预处理 The result represents the first error value, L 相位提取解包裹 The result represents the second error value, L 面形拟合 The result of represents the third error value.

[0069] The technical solution provided in this embodiment uses a target loss function to perform integrated training on the surface reconstruction model including the image preprocessing model, the phase unwrapping model and the wavefront fitting model, thereby improving the model processing accuracy and realizing end-to-end surface shape reconstruction of the object to be measured, thereby improving the efficiency of surface shape reconstruction.

[0070] Optionally, the first loss function includes a fourth loss function, a fifth loss function, and a sixth loss function; the second interference fringe data includes a second interference fringe pattern and a second interference fringe feature, and the second interference fringe feature includes a second fringe sub-feature in at least one row dimension; the third interference fringe data includes a third interference fringe pattern and a third interference fringe feature, and the third interference fringe feature includes a third fringe sub-feature in at least one row dimension. The second interference fringe feature is a feature representation of the second interference fringe pattern. The third interference fringe feature is a feature representation of the third interference fringe pattern. The row dimension can be used to characterize the rows of the feature matrix of the interference fringe feature.

[0071] Based on this, based on the first loss function, the second interference fringe data and the third interference fringe data in the first sample are processed to obtain a first error value, including: for the second interference fringe data and the third interference fringe data in the first sample, based on the fourth loss function, the second fringe sub-feature and the third fringe sub-feature in the same row dimension are processed to obtain a fourth error value; the second fringe sub-feature includes a first characteristic component in at least one column dimension, and the third fringe sub-feature includes a second characteristic component in at least one column dimension; based on the fifth loss function, the first characteristic component and the second characteristic component in the same column dimension are processed to obtain a fifth error value; based on the sixth loss function, the second interference fringe pattern and the third interference fringe pattern are processed to obtain a sixth error value; based on the fourth error value, the fifth error value and the sixth error value, the first error value is obtained. Wherein, the column dimension can be used to characterize the column of the characteristic matrix of the interference fringe feature.

[0072] Specifically, for the second interference fringe data and the third interference fringe data corresponding to the same first sample, the second fringe sub-features and the third fringe sub-features in the same row dimension can be processed by the fourth loss function to obtain a fourth error value. The second characteristic component in each column dimension is processed by the fifth loss function to obtain the corresponding intermediate value in each column dimension, and then the intermediate value and the first characteristic component in the same column dimension are multiplied to obtain the fifth error value. The second interference fringe pattern and the third interference fringe pattern corresponding to the same first sample are processed by the fourth loss function to obtain a sixth error value. Further, the fourth error value, the fifth error value and the sixth error value are summed to obtain the first error value.

[0073] Exemplarily, the first loss function can be expressed as: The result is the fourth error value; is the fifth loss function, The result is the fifth error value; is the sixth loss function, The result is the sixth error value.

[0074]

[0075] Where N represents the number of row dimensions of the interference fringe feature (i.e., the number of rows in the matrix), y i represents the second stripe sub-feature under the i-th row dimension, represents the third fringe feature in the i-th row dimension; n represents the column dimension of the interference fringe feature, represents the first feature component of the second stripe sub-feature in the jth row dimension under the i-th column dimension, represents the second feature component in the ith column dimension of the third stripe sub-feature in the jth row dimension, log() represents the logarithmic function; M represents the number of first samples in a batch during training, x i represents the two interference fringe patterns in the i-th first sample, It represents the third interference fringe pattern corresponding to the i-th first sample.

[0076] The technical solution provided by the embodiment of the present invention uses the fourth loss function to constrain the difference between the visual features (i.e., the second interference fringe features) of the noise-free interference fringe image (i.e., the second interference fringe image) and the visual features (i.e., the third interference fringe features of the third interference fringe image) obtained by model reasoning. And the sixth loss function is used to constrain the difference between the second interference fringe image and the visual feature decoded image (i.e., the second interference fringe image and the third interference fringe image) obtained by model reasoning. And the fifth loss function is used to measure the difference between the visual features of the second interference fringe image and the feature statistical distribution output by model reasoning. The robustness of the model to outliers is improved, the difference between the probability distribution predicted by the model and the theoretical data is reduced, and the accuracy of model preprocessing is improved.

[0077] It should be noted that the way of processing the second interference fringe data and the third interference fringe data in the first sample based on the first loss function to obtain the first error value is similar to the way of processing the phase unwrapping theoretical data and the phase unwrapping prediction data in the first sample based on the second loss function to obtain the second error value, which will not be elaborated here.

[0078] In this embodiment, the third loss function includes the seventh loss function, the eighth loss function, the ninth loss function and the tenth loss function, the Zernike polynomial theoretical data includes the Zernike polynomial theoretical coefficients and the theoretical coefficient feature vector, and the Zernike polynomial prediction data includes the Zernike polynomial prediction coefficients and the prediction coefficient feature vector. The theoretical coefficient feature vector is the feature representation of the Zernike polynomial theoretical coefficients. The prediction coefficient feature vector is the feature representation of the Zernike polynomial prediction coefficients.

[0079] Based on this, based on the third loss function, the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample are processed to obtain a third error value, including: for the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample, based on the seventh loss function, the Zernike polynomial coefficients of the same item in the Zernike polynomial theoretical coefficients and the Zernike polynomial predicted coefficients are processed to obtain a seventh error value; based on the eighth loss function, the Zernike polynomial coefficients of each item in the Zernike polynomial predicted coefficients are logarithmically processed to obtain the first coefficient corresponding to each Zernike polynomial coefficient, and based on the first coefficient and the Zernike polynomial The eighth error value is determined by using the Zernike polynomial coefficient of the same term as the first coefficient in the theoretical coefficient of the formula; based on the ninth loss function, the Zernike polynomial features in the same row dimension in the prediction coefficient eigenvector and the theoretical coefficient eigenvector are processed to obtain a ninth error value; wherein the prediction coefficient eigenvector includes a third eigencomponent in at least one column dimension, and the theoretical coefficient eigenvector includes a fourth eigencomponent in at least one column dimension; based on the tenth loss function, the third eigencomponent and the fourth eigencomponent in the same column dimension are processed to obtain the tenth error value; based on the seventh error value, the eighth error value, the ninth error value and the tenth error value, the third error value is determined.

[0080] Specifically, for the Zernike polynomial theoretical data and Zernike polynomial prediction data corresponding to the first sample, the seventh loss function can be used to perform loss processing on the Zernike polynomial coefficients of the same item in the Zernike polynomial theoretical coefficients and the Zernike polynomial prediction coefficients to obtain the seventh error value. Through the eighth loss function, the Zernike polynomial coefficients of each item in the Zernike polynomial prediction coefficients are logarithmically processed to obtain the first coefficient corresponding to each Zernike polynomial coefficient. Then, the first coefficient and the Zernike polynomial coefficients of the same item as the first coefficient in the Zernike polynomial theoretical coefficients are multiplied to obtain the eighth error value. According to the ninth loss function, the Zernike polynomial features in the same row dimension in the prediction coefficient feature vector and the theoretical coefficient feature vector are subjected to loss processing to obtain the ninth error value. Through the tenth loss function, the third characteristic component in each column dimension is logarithmically processed to obtain the intermediate data corresponding to the third characteristic component in each column dimension, and then, the intermediate data in the same column dimension and the fourth characteristic component are multiplied to obtain the tenth error value. Further, the seventh error value, the eighth error value, the ninth error value and the tenth error value are summed to obtain a third error value.

[0081] Exemplarily, the third loss function can be expressed as:

[0082] in, is the seventh loss function, The result is the seventh error value; The eighth loss function, The result is the eighth error value; is the ninth loss function, The result is the ninth error value; is the tenth loss function, The result is the tenth error value.

[0083]

[0084] In the formula, n represents the number of Zernike polynomial coefficients, a i represents the theoretical coefficient of the i-th Zernike polynomial, represents the prediction coefficient of the i-th Zernike polynomial, N represents the number of row dimensions of the Zernike polynomial coefficient feature (i.e., the theoretical coefficient feature vector and the prediction coefficient feature vector), M represents the column dimension of the Zernike polynomial coefficient feature, y i represents the theoretical coefficient eigenvector under the i-th row dimension, represents the prediction coefficient feature vector under the i-th row dimension, represents the fourth eigencomponent of the i-th column dimension in the theoretical coefficient eigenvector under the j-th row dimension, Represents the third feature component of the i-th column dimension in the prediction coefficient feature vector under the j-th row dimension.

[0085] S260: Based on the loss value, modify the model parameters in the face reconstruction model to be trained.

[0086] In this embodiment, the model parameters in the face reconstruction model to be trained are adjusted continuously based on the loss value to optimize the model, so as to obtain a trained face reconstruction model.

[0087] S270, taking the convergence of the target loss function as the training goal to obtain a trained face reconstruction model.

[0088] Among them, the training goal refers to the training to achieve the convergence of the target loss function.

[0089] In this embodiment, the training error of the target loss function, that is, the loss parameter, can be used as a condition for detecting whether the target loss function has reached convergence, such as whether the training error is less than a preset error or whether the error change trend tends to be stable, or whether the current number of iterations is equal to the preset number. If the convergence condition is detected, such as the training error of the target loss function is less than the preset error or the error change tends to be stable, it indicates that the training of the face reconstruction model to be trained is completed, and the iterative training can be stopped at this time. If it is detected that the convergence condition is not reached at present, the first sample can be further obtained to train the face reconstruction model to be trained until the training error of the target loss function is within the preset range. When the training error of the target loss function reaches convergence, the trained surface reconstruction model to be trained is used as the surface reconstruction model, so that the interference fringe pattern, image description information, phase unwrapping processing method description information, wavefront fitting requirement information, Zernike coefficient solution method description information, Zernike polynomial image and polynomial conversion guidance information are subsequently input into the pre-trained surface reconstruction model to obtain the Zernike polynomial coefficients, and then based on the Zernike polynomial images and Zernike polynomial coefficients of different orders, the surface shape map of the object to be measured is obtained.

[0090] The technical solution of this embodiment uses the first sample to realize an end-to-end integrated training surface reconstruction model from interference fringe image to surface reconstruction, without the need to train each model separately, which improves the accuracy of model surface reconstruction as a whole. At the same time, the output features of the interference fringe image denoising link, the output features of the phase extraction unwrapping link, and the input features of the wavefront fitting surface reconstruction link are visualized through the decoder to realize the visualization of the intermediate results, which increases the interpretability of the surface reconstruction model as a whole. Furthermore, in the wavefront fitting link, the Zernike coefficient solution method is usually used to describe information and Zernike polynomial images to guide the model to realize wavefront fitting, which solves the problem of low data processing efficiency when using data formulas to solve Zernike polynomials, and improves the efficiency of determining Zernike polynomial coefficients while ensuring the accuracy of Zernike polynomial coefficients.

[0091] Embodiment 3

[0092] Figure 5 FIG. 1 is a schematic diagram of a device for determining face shape information of an object according to Embodiment 3 of the present invention. Figure 5 As shown, the device includes: an image description information determination module 310, a Zernike polynomial coefficient determination module 320 and a surface shape map determination module 330.

[0093] Among them, the image description information determination module 310 is used to obtain the interference fringe pattern of the object to be measured and determine the image description information of the interference fringe pattern; wherein the interference fringe pattern is obtained after the light beams reflected after at least two light waves propagate to the reference surface and the surface of the object to be measured interfere with each other; the Zernike polynomial coefficient determination module 320 is used to input the interference fringe pattern, the image description information, the preset image processing method description information and the Zernike polynomial image into a pre-trained surface reconstruction model to obtain the Zernike polynomial coefficients; wherein the Zernike polynomial image is an image represented by Zernike polynomials of different orders; the preset image processing method description information at least includes phase unwrapping processing method description information and Zernike coefficient solution method description information; the surface shape map determination module 330 is used to obtain the surface shape map of the object to be measured based on the Zernike polynomial images of different orders and the Zernike polynomial coefficients.

[0094] The technical solution of this embodiment is to obtain the interference fringe pattern of the object to be measured and determine the image description information of the interference fringe pattern; wherein the interference fringe pattern is obtained after the interference of the light beams reflected by at least two light waves propagating to the reference surface and the surface of the object to be measured respectively; the interference fringe pattern, the image description information, the preset image processing method description information and the Zernike polynomial image are input into a pre-trained surface reconstruction model to obtain the Zernike polynomial coefficients; wherein the Zernike polynomial image is an image represented by Zernike polynomials of different orders; the preset image processing method description information at least includes phase unwrapping processing method description information and Zernike coefficient solution method description information; based on Zernike polynomials of different orders The invention relates to a surface shape map of the object to be measured, and a surface shape map of the object to be measured is obtained by combining the interference fringe pattern, image description information, preset image processing method description information and Zernike polynomial image into an end-to-end trained surface reconstruction model to obtain Zernike polynomial coefficients. The surface shape map of the object to be measured is determined by combining the Zernike polynomial image and the Zernike polynomial coefficients, so that the determined surface shape map is highly compatible with the object to be measured, thereby achieving the technical effect of improving the accuracy of determining the surface shape information of the object and improving the convenience of reconstructing the surface shape information of the object.

[0095] On the basis of the above device, optionally, the surface reconstruction model includes an image preprocessing model, a phase unwrapping model and a wavefront fitting model; the Zernike polynomial coefficient determination module 320 includes:

[0096] A first interference fringe pattern determining unit is used to process the interference fringe pattern according to the image description information based on the image preprocessing model in the surface reconstruction model to obtain a first interference fringe pattern;

[0097] A phase unwrapping map determining unit, configured to process the first interference fringe map based on the phase unwrapping model in the surface reconstruction model and according to the phase unwrapping processing mode description information in the preset image processing mode description information to obtain a phase unwrapping map; wherein the preset image processing mode description information also includes wavefront fitting requirement information and polynomial conversion guidance information;

[0098] A Zernike polynomial coefficient determination unit is used to determine the wavefront fitting guidance features based on the wavefront fitting model in the surface reconstruction model, according to the phase unwrapping diagram, the Zernike coefficient solution method description information, the wavefront fitting requirement information, the polynomial conversion guidance information and the Zernike polynomial image, and determine the Zernike polynomial coefficients based on the wavefront fitting guidance features.

[0099] On the basis of the above device, optionally, the Zernike polynomial coefficient determination unit includes:

[0100] A first fusion feature determination unit, configured to obtain a first fusion feature based on the wavefront fitting requirement information and the phase unwrapping diagram;

[0101] A second fusion feature determination unit, configured to obtain a second fusion feature based on the Zernike coefficient solution method description information and the Zernike polynomial image;

[0102] A wavefront fitting guiding feature determining unit is used to determine a wavefront fitting guiding feature based on the first fusion feature, the second fusion feature and the polynomial conversion guidance information.

[0103] On the basis of the above device, optionally, the device further comprises: a face reconstruction model determination module, used for training to obtain the face reconstruction model; wherein the face reconstruction model determination module comprises:

[0104] A first sample determination unit is used to obtain a plurality of first samples; the first samples include an original interference fringe pattern of a measured object, image description information of the original interference fringe pattern, description information of a phase unwrapping processing method, description information of a Zernike coefficient solution method, wavefront fitting requirement information and polynomial conversion guidance information, the Zernike polynomial image, second interference fringe data, phase unwrapping theoretical data, and Zernike polynomial theoretical data;

[0105] a third interference fringe data determining unit, configured to input, for each of the first samples, an original interference fringe image and image description information in the first sample into an image preprocessing model in a surface reconstruction model to be trained, to obtain third interference fringe data;

[0106] a phase unwrapping prediction data determining unit, configured to input the third interference fringe data and the phase unwrapping processing mode description information in the first sample into a phase unwrapping model in the surface reconstruction model to obtain phase unwrapping prediction data;

[0107] a Zernike polynomial prediction data determination unit, configured to input the phase unwrapping prediction data and the wavefront fitting requirement information, the Zernike coefficient solution method description information, the Zernike polynomial image, and the polynomial conversion guidance information in the first sample into the wavefront fitting model in the surface reconstruction model to obtain the Zernike polynomial prediction data;

[0108] a loss value determining unit, configured to process the second interference fringe data and the third interference fringe data in the first sample, the phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample, and the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample based on a target loss function to obtain a loss value;

[0109] A parameter correction unit, used for correcting the model parameters in the face reconstruction model based on the loss value;

[0110] The face reconstruction model determination unit is used to take the convergence of the target loss function as a training target to obtain a trained face reconstruction model.

[0111] Based on the above device, optionally, the target loss function includes a first loss function, a second loss function and a third loss function, and the loss value determination unit includes:

[0112] A first error value determining unit, configured to process the second interference fringe data and the third interference fringe data in the first sample based on the first loss function to obtain a first error value;

[0113] A second error value determining unit, configured to process the phase unwrapping theoretical data and the phase unwrapping prediction data in the first sample based on the second loss function to obtain a second error value;

[0114] A third error value determining unit is used to process the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample based on the third loss function to obtain a third error value;

[0115] The loss value determining subunit is used to obtain a loss value based on the first error value, the second error value and the third error value.

[0116] On the basis of the above device, optionally, the first loss function includes a fourth loss function, a fifth loss function and a sixth loss function; the second interference fringe data includes a second interference fringe pattern and a second interference fringe feature, and the second interference fringe feature includes a second fringe sub-feature in at least one row dimension; the third interference fringe data includes a third interference fringe pattern and a third interference fringe feature, and the third interference fringe feature includes a third fringe sub-feature in at least one row dimension; the first error value determination unit includes:

[0117] a fourth error value determining unit, configured to process the second fringe sub-feature and the third fringe sub-feature in the same row dimension based on a fourth loss function for the second interference fringe data and the third interference fringe data in the first sample, to obtain a fourth error value; wherein the second fringe sub-feature includes a first characteristic component in at least one column dimension, and the third fringe sub-feature includes a second characteristic component in at least one column dimension;

[0118] a fifth error value determining unit, configured to process the first feature component and the second feature component in the same column dimension based on a fifth loss function to obtain a fifth error value;

[0119] a sixth error value determining unit, configured to process the second interference fringe pattern and the third interference fringe pattern based on a sixth loss function to obtain a sixth error value;

[0120] The first error value determining subunit is configured to obtain a first error value based on the fourth error value, the fifth error value and the sixth error value.

[0121] On the basis of the above device, optionally, the third loss function includes a seventh loss function, an eighth loss function, a ninth loss function and a tenth loss function, the Zernike polynomial theoretical data includes Zernike polynomial theoretical coefficients and theoretical coefficient feature vectors, and the Zernike polynomial prediction data includes Zernike polynomial prediction coefficients and prediction coefficient feature vectors; the third error value determination unit includes:

[0122] a seventh error value determining unit, configured to process, based on a seventh loss function, the Zernike polynomial coefficients of the same item in the Zernike polynomial theoretical coefficients and the Zernike polynomial predicted coefficients in the first sample, to obtain a seventh error value;

[0123] an eighth error value determining unit, configured to perform logarithmic processing on each Zernike polynomial coefficient of the Zernike polynomial prediction coefficients based on an eighth loss function to obtain a first coefficient corresponding to each Zernike polynomial coefficient, and determine an eighth error value based on the first coefficient and the Zernike polynomial coefficient of the same item as the first coefficient in the Zernike polynomial theoretical coefficients;

[0124] a ninth error value determining unit, configured to process the Zernike polynomial features in the same row dimension in the prediction coefficient feature vector and the theoretical coefficient feature vector based on a ninth loss function, to obtain a ninth error value; wherein the prediction coefficient feature vector includes a third feature component in at least one column dimension, and the theoretical coefficient feature vector includes a fourth feature component in at least one column dimension;

[0125] a tenth error value determining unit, configured to process the third characteristic component and the fourth characteristic component in the same column dimension based on a tenth loss function to obtain a tenth error value;

[0126] The third error value determining subunit is configured to determine a third error value based on the seventh error value, the eighth error value, the ninth error value and the tenth error value.

[0127] The object face shape information determination device provided in the embodiment of the present invention can execute the object face shape information determination method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0128] Embodiment 4

[0129] Figure 6 1 is a schematic diagram of the structure of an electronic device that implements the object face information determination method of an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0130] like Figure 6 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0131] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0132] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the object face shape information determination method.

[0133] In some embodiments, the object face shape information determination method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the object face shape information determination method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the object face shape information determination method in any other appropriate manner (e.g., by means of firmware).

[0134] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific logic products (ASSPs), systems on chips (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0135] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0136] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0137] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0138] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0139] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0140] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the method for determining the face shape information of an object provided in any embodiment of the present invention.

[0141] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, Python, C++, C#, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0142] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0143] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for determining object face information, characterized in that: include: Obtaining an interference fringe pattern of the object to be measured, and determining image description information of the interference fringe pattern; wherein the interference fringe pattern is obtained based on the interference of light beams reflected by at least two light waves propagating to a reference surface and a surface of the object to be measured respectively; Inputting the interference fringe pattern, the image description information, the preset image processing method description information and the Zernike polynomial image into a pre-trained surface reconstruction model to obtain Zernike polynomial coefficients; wherein the Zernike polynomial image is an image represented by Zernike polynomials of different orders; the preset image processing method description information at least includes phase unwrapping processing method description information and Zernike coefficient solution method description information; A surface shape map of the object to be measured is obtained based on the Zernike polynomial images of different orders and the Zernike polynomial coefficients.

2. The method according to claim 1, characterized in that The surface reconstruction model includes an image preprocessing model, a phase unwrapping model and a wavefront fitting model; the interference fringe pattern, the image description information, the preset image processing method description information and the Zernike polynomial image are input into the pre-trained surface reconstruction model to obtain the Zernike polynomial coefficients, including: Based on the image preprocessing model in the surface reconstruction model, the interference fringe pattern is processed according to the image description information to obtain a first interference fringe pattern; Based on the phase unwrapping model in the surface reconstruction model, the first interference fringe pattern is processed according to the phase unwrapping processing method description information in the preset image processing method description information to obtain a phase unwrapping pattern; wherein the preset image processing method description information also includes wavefront fitting requirement information and polynomial conversion guidance information; Based on the wavefront fitting model in the surface reconstruction model, the wavefront fitting guiding features are determined according to the phase unwrapping diagram, the description information of the Zernike coefficient solution method, the wavefront fitting requirement information, the polynomial conversion guidance information and the Zernike polynomial image, and based on the wavefront fitting guiding features, the Zernike polynomial coefficients are determined.

3. The method according to claim 2, characterized in that The step of determining the wavefront fitting guidance feature according to the phase unwrapping diagram, the Zernike coefficient solution method description information, the wavefront fitting requirement information, the polynomial conversion guidance information and the Zernike polynomial image includes: Based on the wavefront fitting requirement information and the phase unwrapping diagram, obtaining a first fusion feature; Based on the Zernike coefficient solution method description information and the Zernike polynomial image, a second fusion feature is obtained; A wavefront fitting guiding feature is determined based on the first fusion feature, the second fusion feature and the polynomial conversion guiding information.

4. The method according to claim 1, characterized in that: The method further comprises: The face reconstruction model is obtained by training; wherein, The training to obtain the face reconstruction model includes: Acquire a plurality of first samples; the first samples include an original interference fringe pattern of a measured object, image description information of the original interference fringe pattern, description information of a phase unwrapping processing method, description information of a Zernike coefficient solution method, wavefront fitting requirement information and polynomial conversion guidance information, the Zernike polynomial image, second interference fringe data, phase unwrapping theoretical data, and Zernike polynomial theoretical data; For each of the first samples, the original interference fringe pattern and image description information in the first sample are input into an image preprocessing model in the surface reconstruction model to be trained to obtain third interference fringe data; Inputting the third interference fringe data and the phase unwrapping processing method description information in the first sample into the phase unwrapping model in the surface reconstruction model to obtain phase unwrapping prediction data; Inputting the phase unwrapping prediction data and the wavefront fitting requirement information, Zernike coefficient solution method description information, Zernike polynomial image and polynomial conversion guidance information in the first sample into the wavefront fitting model in the surface reconstruction model to obtain Zernike polynomial prediction data; Based on the target loss function, the second interference fringe data and the third interference fringe data in the first sample, the phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample, and the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample are processed to obtain a loss value; Based on the loss value, modifying the model parameters in the face reconstruction model; The convergence of the target loss function is used as a training goal to obtain a trained face reconstruction model.

5. The method according to claim 4, characterized in that The target loss function includes a first loss function, a second loss function and a third loss function. Based on the target loss function, the second interference fringe data and the third interference fringe data in the first sample, the phase unwrapping theoretical data and the phase unwrapping predicted data in the first sample, and the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample are processed to obtain a loss value, including: Based on the first loss function, processing the second interference fringe data and the third interference fringe data in the first sample to obtain a first error value; Based on the second loss function, the phase unwrapping theoretical data and the phase unwrapping prediction data in the first sample are processed to obtain a second error value; Based on the third loss function, the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample are processed to obtain a third error value; A loss value is obtained based on the first error value, the second error value, and the third error value.

6. The method according to claim 5, characterized in that The first loss function includes a fourth loss function, a fifth loss function and a sixth loss function; the second interference fringe data includes a second interference fringe pattern and a second interference fringe feature, and the second interference fringe feature includes a second fringe sub-feature in at least one row dimension; the third interference fringe data includes a third interference fringe pattern and a third interference fringe feature, and the third interference fringe feature includes a third fringe sub-feature in at least one row dimension; The step of processing the second interference fringe data and the third interference fringe data in the first sample based on the first loss function to obtain a first error value includes: For the second interference fringe data and the third interference fringe data in the first sample, based on a fourth loss function, the second fringe sub-feature and the third fringe sub-feature in the same row dimension are processed to obtain a fourth error value; wherein the second fringe sub-feature includes a first characteristic component in at least one column dimension, and the third fringe sub-feature includes a second characteristic component in at least one column dimension; Based on a fifth loss function, processing the first feature component and the second feature component in the same column dimension to obtain a fifth error value; Based on a sixth loss function, processing the second interference fringe pattern and the third interference fringe pattern to obtain a sixth error value; A first error value is obtained based on the fourth error value, the fifth error value, and the sixth error value.

7. The method according to claim 5, characterized in that The third loss function includes a seventh loss function, an eighth loss function, a ninth loss function and a tenth loss function, the Zernike polynomial theoretical data includes Zernike polynomial theoretical coefficients and theoretical coefficient feature vectors, and the Zernike polynomial predicted data includes Zernike polynomial predicted coefficients and predicted coefficient feature vectors; based on the third loss function, the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample are processed to obtain a third error value, including: For the Zernike polynomial theoretical data and the Zernike polynomial predicted data in the first sample, based on a seventh loss function, processing the Zernike polynomial coefficients of the same item in the Zernike polynomial theoretical coefficients and the Zernike polynomial predicted coefficients to obtain a seventh error value; Based on an eighth loss function, logarithmically processing the Zernike polynomial coefficient of each item in the Zernike polynomial prediction coefficients is performed respectively to obtain a first coefficient corresponding to each item of the Zernike polynomial coefficient, and determining an eighth error value based on the first coefficient and the Zernike polynomial coefficient of the same item as the first coefficient in the Zernike polynomial theoretical coefficients; Based on a ninth loss function, the Zernike polynomial features in the same row dimension in the prediction coefficient feature vector and the theoretical coefficient feature vector are processed to obtain a ninth error value; wherein the prediction coefficient feature vector includes a third feature component in at least one column dimension, and the theoretical coefficient feature vector includes a fourth feature component in at least one column dimension; Based on the tenth loss function, the third characteristic component and the fourth characteristic component in the same column dimension are processed to obtain a tenth error value; Based on the seventh error value, the eighth error value, the ninth error value, and the tenth error value, a third error value is determined.

8. A device for determining object face shape information, characterized in that: include: An image description information determination module is used to obtain an interference fringe pattern of the object to be measured and determine the image description information of the interference fringe pattern; wherein the interference fringe pattern is obtained based on the interference of light beams reflected by at least two light waves propagating to a reference surface and a surface of the object to be measured respectively; A Zernike polynomial coefficient determination module, used for inputting the interference fringe pattern, the image description information, the preset image processing method description information and the Zernike polynomial image into a pre-trained surface reconstruction model to obtain Zernike polynomial coefficients; wherein the Zernike polynomial image is an image represented by Zernike polynomials of different orders; the preset image processing method description information at least includes phase unwrapping processing method description information and Zernike coefficient solution method description information; The surface shape map determining module is used to obtain the surface shape map of the object to be measured based on the Zernike polynomial images of different orders and the Zernike polynomial coefficients.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory in communication with the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the object face shape information determination method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the object face shape information determination method according to any one of claims 1 to 7 when executed.

Images