Training device, prediction device, training method and equipment of three-dimensional optical imaging prediction model

Abstract

This invention discloses a training device, prediction device, training method, and equipment for a three-dimensional optical imaging prediction model, comprising: a first acquisition module for acquiring two-dimensional mask design drawings and etching process parameters of multiple etched workpieces, wherein the two-dimensional mask design drawings include two-dimensional mask area information of the etching mask; a second acquisition module for acquiring target three-dimensional optical imaging information of each etched workpiece; a first prediction module for inputting the two-dimensional mask area information and etching process parameters of each etched workpiece into the three-dimensional optical imaging prediction model to be trained, and predicting the predicted three-dimensional optical imaging information of each etched workpiece; a loss calculation module for calculating the model inference loss based on the predicted three-dimensional optical imaging information of each etched workpiece and the target three-dimensional optical imaging information; and a training module for training the three-dimensional optical imaging prediction model based on the model inference loss, thereby obtaining the trained three-dimensional optical imaging prediction model.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional shape detection technology, specifically to a training device, prediction device, training method, and equipment for a three-dimensional optical imaging prediction model. Background Technology

[0002] Etching of materials such as glass is a processing technique that uses chemical, physical, or laser methods to create uniform roughness or specific patterns on the surface. Chemical etching, in particular, involves numerous steps, including 2D pattern design, pretreatment, photoresist coating, pattern transfer, development, chemical etching, photoresist removal, and post-processing. Evaluating the 3D optical imaging effect of the etched pattern is crucial for improving and optimizing the 2D pattern design.

[0003] In related technologies, after the 2D drawing design is completed, it is usually necessary to make a physical photomask and actually carry out etching tests to verify the three-dimensional (3D) optical imaging effect of the etched product. That is, the three-dimensional (3D) optical imaging effect cannot be predicted in advance, and it is impossible to quickly and accurately verify whether the 2D drawing design meets the product design requirements, resulting in a longer overall design cycle and serious material waste. Summary of the Invention

[0004] This invention provides a training device, prediction device, training method, and equipment for a three-dimensional optical imaging prediction model, in order to solve the problem in related technologies of how to evaluate the three-dimensional optical imaging effect of a product after etching, so as to quickly and accurately verify whether the 2D drawing design meets the product design requirements.

[0005] In a first aspect, the present invention provides a training device for a three-dimensional optical imaging prediction model, the training device comprising: The first acquisition module is used to acquire two-dimensional mask design drawings and etching process parameters of multiple etched parts. The two-dimensional mask design drawings include two-dimensional mask area information of the etching mask corresponding to the etched parts. The second acquisition module is used to acquire the target three-dimensional optical imaging information of each of the etched workpieces; The first prediction module is used to input the two-dimensional mask area information and etching process parameters of each etched workpiece into the three-dimensional optical imaging prediction model to be trained, and to predict the predicted three-dimensional optical imaging information of each etched workpiece. The loss calculation module is used to calculate the model inference loss based on the predicted three-dimensional optical imaging information of each of the etched parts and the target three-dimensional optical imaging information. The training module is used to train the three-dimensional optical imaging prediction model based on the model inference loss until the model converges, thus obtaining the trained three-dimensional optical imaging prediction model.

[0006] In some embodiments, the two-dimensional mask region information includes the two-dimensional coordinate information of each vertex in each first mask region, and the two-dimensional coordinate information includes two-dimensional horizontal coordinates; The etching process parameters include at least one of the following parameters: coating material, coating thickness, photoresist thickness, exposure energy, solution concentration, development time, and etching duration.

[0007] In some embodiments, the two-dimensional mask region information includes the two-dimensional coordinate information of each vertex in each first mask region, and the second acquisition module includes: The detection submodule is used to detect the measured three-dimensional optical imaging information of each etched workpiece. The measured three-dimensional optical imaging information includes the three-dimensional coordinate information corresponding to each spatial position point of the etched workpiece. The planarization processing submodule is used to perform planarization correction processing on the measured three-dimensional optical imaging information of the etched workpiece; The matching calibration submodule is used to determine the target three-dimensional optical imaging information of the etched workpiece based on the two-dimensional mask area information and the measured three-dimensional optical imaging information after flattening and correction. The target three-dimensional optical imaging information includes the three-dimensional coordinate information of the spatial position points that match each vertex in each first mask area.

[0008] In some embodiments, the matching calibration submodule is configured to: obtain first centroid coordinate information of each first mask region based on the two-dimensional coordinate information of each vertex in each first mask region; generate a corresponding two-dimensional feature mask image based on the measured three-dimensional optical imaging information after planarization and correction processing, wherein the two-dimensional feature mask image includes the two-dimensional coordinate information of each spatial location point in each second mask region; obtain second centroid coordinate information of each second mask region based on the two-dimensional coordinate information of each spatial location point in each second mask region; and perform matching calibration between each vertex in each first mask region and each spatial location point in each second mask region based on the first centroid coordinate information of each first mask region and the second centroid coordinate information of each second mask region, thereby determining the three-dimensional coordinate information of the matching spatial location point corresponding to each vertex in each first mask region.

[0009] In some embodiments, the planarization submodule is used to: perform a reference plane fitting on the three-dimensional coordinate information of each spatial location point in the measured three-dimensional optical imaging information using the least squares method to obtain the fitting reference plane; and perform a difference calculation between the measured height coordinates in the three-dimensional coordinate information of each spatial location point and the theoretical tilt reference height of each spatial location point on the fitting reference plane to obtain the measured three-dimensional optical imaging information after planarization correction.

[0010] In some embodiments, the matching calibration submodule is configured to: The measured three-dimensional optical imaging information after flattening and correction is projected into the horizontal coordinate space to obtain the corresponding two-dimensional spatial height distribution map; the two-dimensional spatial height distribution map is then segmented into grayscale binarization according to a preset height discrimination threshold to obtain the corresponding two-dimensional feature mask image.

[0011] In some embodiments, the matching calibration submodule is used for: A first triangulation topology is constructed based on the first centroid coordinate information of each first mask region, and a second triangulation topology is constructed based on the second centroid coordinate information of each second mask region; Traverse and search for currently similar reference triangles between the first triangulation topology and the second triangulation topology. The currently similar reference triangles include the first reference triangle in the first triangulation topology and the second reference triangle in the second triangulation topology. The current affine transformation matrix is ​​determined based on the first centroid coordinate information corresponding to each vertex of the first reference triangle and the first centroid coordinate information corresponding to each vertex of the second reference triangle. For the first centroid coordinate information of each first mask region, perform an affine transformation based on the current affine transformation matrix to obtain the centroid mapping coordinate information of the first mask region; Search for the second centroid coordinate information of the second mask region that matches the centroid mapping coordinate information of the first mask region in the second centroid coordinate information of each second mask region; The matching result with the most matches between the centroid mapping coordinates of the first mask region and the second centroid coordinates of the second mask region is taken as the final matching result from the multiple matching results obtained through multiple traversals. Based on the final matching results, the mapping relationship between each vertex in each first mask region and each spatial location point in each second mask region is determined, and the three-dimensional coordinate information of the matching spatial location point corresponding to each vertex in each first mask region is obtained.

[0012] In some embodiments, the matching calibration submodule is used to establish the current affine transformation equation based on the first centroid coordinate information corresponding to each vertex in the first reference triangle and the second centroid coordinate information corresponding to each vertex in the second reference triangle; and to solve the current affine transformation equation to obtain the current affine transformation matrix.

[0013] In some embodiments, the predicted three-dimensional optical imaging information includes the predicted height coordinates of each vertex of each first mask region in the two-dimensional mask region information, and the target three-dimensional optical imaging information includes the target height coordinates that correspond to and match each vertex of each first mask region in the two-dimensional mask region information; The loss calculation module is used to: for each etched part, obtain the basic loss corresponding to the etched part based on the average absolute error between the predicted height coordinates and the target height coordinates of each vertex of each first mask region; and obtain the current model inference loss based on the mean of the basic losses corresponding to each etched part.

[0014] In a second aspect, the present invention provides a three-dimensional optical imaging prediction device, the three-dimensional optical imaging prediction device comprising: The third acquisition module is used to acquire the two-dimensional mask design drawing and etching process parameters of the workpiece to be etched. The two-dimensional mask design drawing includes the two-dimensional mask area information of the etching mask corresponding to the workpiece to be etched. The second prediction module is used to input the two-dimensional mask area information and the etching process parameters into a pre-trained three-dimensional optical imaging prediction model to predict the three-dimensional optical imaging information of the workpiece after etching.

[0015] Thirdly, the present invention provides a training method for a three-dimensional optical imaging prediction model, the training method comprising: Obtain two-dimensional mask design drawings and etching process parameters for multiple etched parts. The two-dimensional mask design drawings include two-dimensional mask area information of the etching mask corresponding to the etched parts. Obtain the target three-dimensional optical imaging information of each of the etched workpieces; The two-dimensional mask area information and etching process parameters of each etched part are input into the three-dimensional optical imaging prediction model to be trained, and the predicted three-dimensional optical imaging information of each etched part is obtained. The model inference loss is calculated based on the predicted three-dimensional optical imaging information of each etched workpiece and the target three-dimensional optical imaging information. The three-dimensional optical imaging prediction model is trained based on the model inference loss until the model converges, resulting in a fully trained three-dimensional optical imaging prediction model.

[0016] Fourthly, the present invention provides an electronic device, the electronic device comprising: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes these computer instructions to perform the training method for the three-dimensional optical imaging prediction model described in the first aspect.

[0017] According to the training device, prediction device, training method, and equipment for a three-dimensional optical imaging prediction model provided by the present invention, the training device uses two-dimensional mask design drawings and etching process parameters of multiple etched workpieces as training samples for the prediction model, and uses the detected target three-dimensional optical imaging information of the etched workpieces as training sample labels to train the prediction model. The trained three-dimensional optical imaging prediction model can be used to predict the three-dimensional optical imaging information of the workpiece to be etched. Before the actual etching process is performed on the workpiece, the two-dimensional mask area information and etching process parameters are input through the trained three-dimensional optical imaging prediction model to directly predict the three-dimensional optical imaging information of the workpiece after etching. The three-dimensional optical imaging information of the product generated after the process enables the simulation of the three-dimensional optical imaging effect of the product after etching, from the two-dimensional mask design drawing of the workpiece to be etched. The prediction process integrates the etching process parameters of the workpiece to be etched, which effectively reduces the deviation between the simulation results and the actual etching effect. Thus, the predicted three-dimensional optical imaging information can quickly and accurately evaluate whether the two-dimensional mask design drawing of the workpiece to be etched meets the product design requirements without the need for verification and evaluation after actual etching. It can quickly and accurately improve and optimize the two-dimensional mask design drawing of the workpiece to be etched, which is conducive to improving the production cycle of the two-dimensional mask design drawing and reducing the material waste caused by product etching effect test. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 A block diagram illustrating the composition of a training device for a three-dimensional optical imaging prediction model provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a two-dimensional mask design drawing according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the composition structure of a second acquisition module in an embodiment of the present invention; Figure 4 This is a visualization diagram of measured three-dimensional optical imaging information in an embodiment of the present invention; Figure 5 This is a schematic diagram of the spatial projection of measured three-dimensional optical imaging information onto the xy plane of a three-dimensional spatial coordinate system in an embodiment of the present invention. Figure 6 This is a schematic diagram of the data distribution of measured three-dimensional optical imaging information in the XZ plane under a fixed Y-axis coordinate value in an embodiment of the present invention; Figure 7 This is a schematic diagram of the data distribution of measured three-dimensional optical imaging information in the XZ plane under a fixed Y-axis coordinate value after planarization in an embodiment of the present invention; Figure 8 This is a schematic diagram of the spatial projection of the measured three-dimensional optical imaging information after planarization processing onto the xy plane of the three-dimensional spatial coordinate system in an embodiment of the present invention. Figure 9 This is a schematic diagram of the centroid distribution of each first mask region in an embodiment of the present invention; Figure 10 This is a schematic diagram of a two-dimensional feature mask image obtained after binarizing a two-dimensional spatial height distribution map in an embodiment of the present invention. Figure 11 This is a schematic diagram of the centroid distribution of each second mask region in an embodiment of the present invention; Figure 12 This is a schematic diagram of the first and second triangulation topologies in an embodiment of the present invention; Figure 13 This is a schematic diagram illustrating the matching between each first mask region and each second mask region in an embodiment of the present invention; Figure 14 This is a schematic diagram of the training architecture of the three-dimensional optical imaging prediction model in an embodiment of the present invention; Figure 15 A flowchart illustrating a training method for a three-dimensional optical imaging prediction model provided in an embodiment of the present invention; Figure 16 A block diagram of a three-dimensional optical imaging prediction device provided in an embodiment of the present invention; Figure 17 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0022] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0023] In related technologies, after the 2D drawing design is completed, it is usually necessary to make a physical photomask and actually carry out etching tests to verify the three-dimensional (3D) optical imaging effect of the etched product. That is, the three-dimensional (3D) optical imaging effect cannot be predicted in advance, and it is impossible to quickly and accurately verify whether the 2D drawing design meets the product design requirements, resulting in a longer overall design cycle and serious material waste.

[0024] In actual processing and 3D inspection, chemical etching processes have high precision, down to the micrometer level. However, the instruments used in the inspection process cannot achieve absolute flatness, and interference from dust particles is not completely isolated. This results in uneven 3D coordinates obtained during the 3D inspection process. Furthermore, due to equipment limitations, only a very small area can usually be inspected, making it impossible to accurately locate the coordinate information of the 3D inspected product. Meanwhile, 2D drawings are often tens or even hundreds of millimeters in size, ultimately making it impossible to accurately match the 3D inspection area with the 2D drawing design. Consequently, it is impossible to effectively model the 3D optical imaging effect from the 2D drawing design to the product at the microscopic level.

[0025] Therefore, embodiments of the present invention provide a training device and prediction device, training method and electronic device for a three-dimensional optical imaging prediction model, which aims to effectively solve at least one technical problem existing in the above-mentioned related technologies.

[0026] This invention provides a training apparatus for a three-dimensional optical imaging prediction model, which executes a training method for the three-dimensional optical imaging prediction model. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0027] Figure 1 This is a block diagram of a training device for a three-dimensional optical imaging prediction model provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the training device for the three-dimensional optical imaging prediction model includes: The first acquisition module 101 is used to acquire two-dimensional mask design drawings and etching process parameters of multiple etched parts. The two-dimensional mask design drawings include two-dimensional mask area information of the etching mask corresponding to the etched parts. The second acquisition module 102 is used to acquire the target three-dimensional optical imaging information of each etched workpiece; The first prediction module 103 is used to input the two-dimensional mask area information and etching process parameters of each etched workpiece into the three-dimensional optical imaging prediction model to be trained, and to predict the predicted three-dimensional optical imaging information of each etched workpiece. The loss calculation module 104 is used to calculate the model inference loss based on the predicted three-dimensional optical imaging information of each etched part and the target three-dimensional optical imaging information. Training module 105 is used to train a 3D optical imaging prediction model based on the model inference loss until the model converges, thus obtaining a trained 3D optical imaging prediction model.

[0028] In this embodiment of the invention, the processed part can be an industrial product such as glass that can be etched with patterns. The etched processed part refers to a product formed by making an etching mask corresponding to the pattern of the two-dimensional mask design drawing according to a preset two-dimensional mask design drawing, using set etching process parameters, and etching the processed part to be etched through the etching mask. The etched processed part has a pattern corresponding to the two-dimensional mask design drawing formed on it.

[0029] Two-dimensional mask design drawings are used in the etching process of workpieces to design the mask design for etching the workpiece. The drawings contain information about the two-dimensional mask areas on the etching mask. The two-dimensional mask area information is a pre-designed planar geometric pattern in the two-dimensional plane of the workpiece, consisting of closed contours, lines, hollow areas, array structures, trench paths, micro-hole arrangements, etc. It is composed of a series of closed mask patterns, and each closed mask pattern corresponds to a first mask area, which is the key basis for pattern transfer in the etching process.

[0030] Three-dimensional optical imaging information refers to the stereoscopic visual effect obtained by measuring the etched surface of a workpiece under a specific light source after etching, using the human eye or a high-precision optical surface measuring instrument. This ultimately forms a series of optical imaging information that includes three-dimensional spatial coordinate information, as well as depth perception, texture layers, and changes in light and shadow.

[0031] In this embodiment of the invention, the first acquisition module 101 acquires the two-dimensional mask design drawings and etching process parameters used in the respective etching processes of each etched part, and the second acquisition module 102 acquires the target three-dimensional optical imaging information of each etched part formed by the etching process; the first prediction module 103 uses the two-dimensional mask region information and etching process parameters of the etched part as input features of the prediction model and inputs them into the three-dimensional optical imaging prediction model to be trained, thereby predicting the predicted three-dimensional optical imaging information of each etched part; the loss calculation module 104 uses the target three-dimensional optical imaging information of the etched part as the output label of the prediction model, calculates the model inference loss between the predicted three-dimensional optical imaging information and the target three-dimensional optical imaging information, and trains the model through the training module 105, updating the model parameters through backpropagation based on the model inference loss until the model converges to obtain the trained three-dimensional optical imaging prediction model.

[0032] According to an embodiment of the present invention, a training device for a three-dimensional optical imaging prediction model is provided. This device uses two-dimensional mask design drawings and etching process parameters of multiple etched workpieces as training samples for the prediction model, and uses the detected target three-dimensional optical imaging information of the etched workpieces as training sample labels to train the prediction model. The trained three-dimensional optical imaging prediction model can be used to predict the three-dimensional optical imaging information of the workpiece to be etched. Before the actual etching process is performed on the workpiece, the two-dimensional mask area information and etching process parameters are input into the trained three-dimensional optical imaging prediction model to directly predict the product formed after the etching process. The system uses 3D optical imaging information to simulate the 3D optical imaging effect of the product after etching, from the 2D mask design drawing of the workpiece to the etched product. The prediction process incorporates the etching process parameters of the workpiece, effectively reducing the deviation between the simulation results and the actual etching effect. Thus, the predicted 3D optical imaging information can quickly and accurately evaluate whether the 2D mask design drawing of the workpiece meets the product design requirements without the need for verification and evaluation after actual etching. It can quickly and accurately improve and optimize the 2D mask design drawing of the workpiece, which is beneficial to improving the production cycle of the 2D mask design drawing and reducing the material waste caused by product etching effect testing.

[0033] Figure 2 This is a schematic diagram of a two-dimensional mask design drawing according to an embodiment of the present invention, such as... Figure 2 As shown, the two-dimensional mask design drawing of the workpiece includes multiple closed mask patterns A. The mask pattern A is usually a closed polygonal shape. The polygonal shape can be a regular or irregular polygonal shape. The shape, distribution and number of the specific mask patterns A in the two-dimensional mask design drawing can be determined according to the actual drawing design needs. The embodiments of the present invention do not impose special restrictions on this.

[0034] Each mask pattern A corresponds to a first mask region, and each first mask region consists of a series of vertices. In plane geometry, a vertex is the point where two sides of a polygon intersect, or the common endpoint of the two sides of an angle.

[0035] In some embodiments, the two-dimensional mask region information includes the two-dimensional coordinate information of each vertex in each first mask region, and the two-dimensional coordinate information includes two-dimensional lateral coordinates. The two-dimensional lateral coordinates of each vertex in the first mask region are the two-dimensional plane coordinates of each vertex in the two-dimensional plane (XY) coordinate system of the two-dimensional mask design drawing, including the X-axis coordinate and the Y-axis coordinate.

[0036] The format of two-dimensional mask region information can be represented as [polygon1, polygon2, polygon3, ...], where polygon... n =[(x n1 ,y n1 ), (x n2 ,y n2 ), (x n3 ,y n3 ...], polygon n Represents the nth first mask region, (x nm ,y nm ) represents the two-dimensional horizontal coordinate of the m-th vertex in the n-th first mask region.

[0037] In some embodiments, the etching process parameters include, but are not limited to, at least one of the following parameters: coating material, coating thickness, photoresist thickness, exposure energy, development time, solution concentration, and etching duration.

[0038] Figure 3 This is a schematic diagram of the composition structure of a second acquisition module in an embodiment of the present invention. In some embodiments, such as... Figure 3 As shown, the two-dimensional mask region information includes the two-dimensional coordinate information of each vertex in each first mask region, and the second acquisition module 102 includes: The detection submodule 301 is used to detect the measured three-dimensional optical imaging information of each etched workpiece. The measured three-dimensional optical imaging information includes the three-dimensional coordinate information corresponding to each spatial position point of the etched workpiece. The planarization processing submodule 302 is used to perform planarization correction processing on the measured three-dimensional optical imaging information of the etched workpiece; The matching calibration submodule 303 is used to determine the target three-dimensional optical imaging information of the etched workpiece based on the two-dimensional mask area information of the etched workpiece and the measured three-dimensional optical imaging information after flattening and correction. The target three-dimensional optical imaging information includes the three-dimensional coordinate information of the spatial position points that match each vertex in each first mask area.

[0039] In some embodiments, the detection submodule 301 detects the measured three-dimensional optical imaging information of the etched workpiece formed by the etching process using an optical metrology instrument (such as a Zygo device). The measured three-dimensional optical imaging information includes the three-dimensional coordinate information corresponding to each spatial position point of the etched workpiece. The three-dimensional coordinate information is the three-dimensional spatial coordinate of the spatial position point of the etched workpiece in the three-dimensional space (XYZ) coordinate system, including the X-axis coordinate, Y-axis coordinate and Z-axis coordinate. The three-dimensional coordinate information includes the two-dimensional coordinate information of the spatial position point on the reference plane and the measured height coordinate relative to the reference plane. The two-dimensional coordinate information includes the two-dimensional lateral coordinate, which includes the X-axis coordinate and Y-axis coordinate of the spatial position point. The measured height coordinate is the Z-axis coordinate of the spatial position point.

[0040] The measured three-dimensional optical imaging information can be represented in the format of [(x1,y1,z1), (x2,y2,z2), (x3,y3,z3), ...]. The xy plane in the three-dimensional (xyz) coordinate system is meshed. n ∈[0,1,2,3,...],y n ∈[0,1,2,3, ...],z n ∈R (real number field), (x n ,y n ,z n Let x represent the three-dimensional spatial coordinates of the nth spatial location point, where x n y n It is the two-dimensional horizontal coordinate of the nth spatial location point.

[0041] Figure 4 This is a visualization diagram of measured three-dimensional optical imaging information in an embodiment of the present invention. Figure 5 This is a schematic diagram of the spatial projection of measured three-dimensional optical imaging information onto the xy plane of a three-dimensional spatial coordinate system in an embodiment of the present invention. Figure 4 The measured three-dimensional optical imaging information shown is projected onto the xy-plane to obtain the following: Figure 5 The diagram shows a spatial planar representation of the measured three-dimensional optical imaging information.

[0042] from Figure 5It can be seen that the three-dimensional spatial coordinates of the obtained measured three-dimensional optical imaging information are not completely flat. The reason is that the optical metrology instrument has high detection accuracy (micrometer level). Any slight morphological undulations and roughness fluctuations on the workpiece surface will be sensitively captured by the equipment, resulting in a corresponding height deviation in the original detection data. At the same time, it is difficult to achieve an absolutely ideal flatness of the measurement benchmark during the detection process. In addition, the interference factors such as dust particles and small vibrations in the environment cannot be completely isolated. All of these factors together lead to the unevenness of the three-dimensional spatial coordinates of the finally obtained measured three-dimensional optical imaging information.

[0043] Figure 6 This is a schematic diagram of the data distribution of measured three-dimensional optical imaging information in the XZ plane under a fixed Y-axis coordinate value in an embodiment of the present invention. Further, as shown... Figure 6 As shown, by setting a fixed vertical coordinate value y= The measured three-dimensional optical imaging information is longitudinally truncated to obtain the corresponding XZ plane cross-sectional distribution data, which is then visualized. The XZ plane distribution can intuitively identify that the measured three-dimensional optical imaging information has an overall linear tilt characteristic. This tilt characteristic will introduce a system offset and measurement error in the height dimension Z value. Therefore, the measured three-dimensional optical imaging information needs to be flattened and corrected to eliminate the global tilt deviation and restore the true shape and height information.

[0044] In some embodiments, the planarization processing submodule 302 is used to: perform reference plane fitting on the three-dimensional coordinate information of each spatial position point in the measured three-dimensional optical imaging information using the least squares method to obtain the fitting reference plane; and perform difference calculation between the measured height coordinates in the three-dimensional coordinate information of each spatial position point and the theoretical tilt reference height of each spatial position point on the fitting reference plane to obtain the measured three-dimensional optical imaging information after planarization correction.

[0045] Specifically, assume that the three-dimensional coordinates of each spatial location point in the measured three-dimensional optical imaging information are [(x1,y1,z1), (x2,y2,z2), (x3,y3,z3), ..., (x n ,y n ,z n In the process of fitting the reference plane, firstly, the overdetermined matrix equation is constructed: Ak1=z Wherein, A is constructed as an n×3 matrix composed of the three-dimensional coordinate information of each spatial position point in the measured three-dimensional optical imaging information with the measured height coordinate fixed as a constant coefficient of 1, A=[x1,y1,1;x2,y2,1;x3,y3,1;....;x n ,y n [1]; z is constructed as an n×1 matrix composed of the measured height coordinates of each spatial location point in the measured three-dimensional optical imaging information, z=[zx1,y1 ;z x2,y2 ;z x3,y3 ;....;z xn,yn ].

[0046] Then, solve for the plane parameter vector k1 of the fitted reference plane, k1 = (A T A) -1 (A) T z) = [a; b; d], where k1 represents the least squares mapping matrix.

[0047] Finally, based on the plane parameter vector k1, the x-direction tilt coefficient a, the y-direction tilt coefficient b, and the reference height intercept d are obtained, thereby constructing a fitting reference plane, which can be expressed as: z = ax + by + d Where a and b represent the tilt coefficients, and d is the height intercept.

[0048] The leveling submodule 302 calculates the theoretical tilt reference height of each spatial location point on the fitted reference plane by fitting the reference plane, and performs a difference calculation between the measured height coordinates in the three-dimensional coordinate information of each spatial location point and the theoretical tilt reference height to obtain the measured three-dimensional optical imaging information after leveling correction.

[0049] Figure 7 This is a schematic diagram of the data distribution of measured three-dimensional optical imaging information in the XZ plane under a fixed Y-axis coordinate value after planarization in an embodiment of the present invention. Figure 8 This is a schematic diagram of the spatial projection of the measured three-dimensional optical imaging information, after planarization processing, onto the xy plane of the three-dimensional spatial coordinate system in an embodiment of the present invention, for comparison. Figure 6 and Figure 7 It can be seen that the planarization submodule 302 effectively eliminates the overall linear tilt caused by workpiece clamping, detection field of view offset, and imaging system in the measured 3D optical imaging information by least-squares fitting of the global reference plane, corrects the tilt characteristics of the XZ section, avoids the systematic error of Z-axis height introduced by tilt, removes the macroscopic plane tilt component, retains the real microscopic concave and convex features of the workpiece surface, and ensures that the height values ​​of each spatial position point are not affected by the overall attitude offset. Figure 5 and Figure 8 It can be seen that the three-dimensional spatial coordinates of the measured three-dimensional optical imaging information tend to be flat, and the three-dimensional morphology data is more consistent with the actual surface state of the object.

[0050] In some embodiments, since the actual three-dimensional optical imaging information detection can only detect a part of the object, and since the accuracy of the XY space coordinates is limited, it is impossible to accurately determine which specific first mask region of the two-dimensional mask design drawing corresponds to the detection area, that is, it is impossible to establish a mapping relationship. Therefore, it is necessary to perform region matching calibration by using the morphological characteristics of the actual three-dimensional optical imaging information and the two-dimensional mask region information, so as to prepare for subsequent model training.

[0051] In some embodiments, the matching calibration submodule 303 is configured to: obtain first centroid coordinate information of each first mask region based on the two-dimensional coordinate information of each vertex in each first mask region; generate a corresponding two-dimensional feature mask image based on the measured three-dimensional optical imaging information after planarization and correction processing, wherein the two-dimensional feature mask image includes the two-dimensional coordinate information of each spatial location point in each second mask region; obtain second centroid coordinate information of each second mask region based on the two-dimensional coordinate information of each spatial location point in each second mask region; and perform matching calibration between each vertex in each first mask region and each spatial location point in each second mask region based on the first centroid coordinate information of each first mask region and the second centroid coordinate information of each second mask region, thereby determining the three-dimensional coordinate information of the matching spatial location point corresponding to each vertex in each first mask region.

[0052] Figure 9 This is a schematic diagram of the centroid distribution of each first mask region in an embodiment of the present invention, such as... Figure 9 As shown, the matching calibration submodule 303 extracts the centroid of each first mask region (polygonal shape) in the two-dimensional mask region information to obtain the centroid distribution of each first mask region in the two-dimensional mask region information.

[0053] Specifically, for each first mask region (polygonal shape), the two-dimensional coordinate information of each vertex in the first mask region m is represented in sequence as P1(x1, y1), P2(x2, y2), ..., P n (x) n y n Then, the area of ​​the first mask region m can be expressed as:

[0054] Then the first centroid coordinate information of the first mask region m It can be represented as:

[0055] Where d is a constant coefficient in the formula for calculating the centroid of the polygonal shape (first mask region), and n is the number of vertices in the first mask region m. Since the first mask region m is a closed shape region, i+1=1 when i=n.

[0056] In some embodiments, the matching calibration submodule 303 is used to: project the measured three-dimensional optical imaging information after flattening and correction into the lateral coordinate space to obtain the corresponding two-dimensional spatial height distribution map; and perform grayscale binarization segmentation on the two-dimensional spatial height distribution map according to a preset height discrimination threshold to obtain the corresponding two-dimensional feature mask image.

[0057] Specifically, in combination Figure 8 As shown, the measured three-dimensional optical imaging information after flattening and correction is spatially projected onto the xy plane to obtain the corresponding two-dimensional spatial height distribution map.

[0058] Figure 10 This is a schematic diagram of a two-dimensional feature mask image obtained after binarizing a two-dimensional spatial height distribution map in an embodiment of the present invention, as shown below. Figure 10 As shown, in the grayscale binarization segmentation of the two-dimensional spatial height distribution map, for the two-dimensional coordinate information (x, y) of each spatial location point in the two-dimensional spatial height distribution map, if the measured height coordinate corresponding to the spatial location point is greater than the preset height discrimination threshold, the grayscale of the spatial location point is set to 1; if the measured height coordinate corresponding to the spatial location point is less than or equal to the preset height discrimination threshold, the grayscale of the spatial location point is set to 0, thereby separating the background matrix region in the two-dimensional spatial height distribution map. Figure 10 The black area in the middle) and the abnormal protruding defect area ( Figure 10 The white area in the image is used to obtain the corresponding two-dimensional feature mask image. The two-dimensional feature mask image includes the two-dimensional coordinate information of each spatial location point in each second mask region. Figure 10 The white, isolated area represents the second mask area.

[0059] Figure 11 This is a schematic diagram of the centroid distribution of each second mask region in an embodiment of the present invention, such as... Figure 11 As shown, the matching calibration submodule 303 extracts the centroid of each second mask region in the measured three-dimensional optical imaging information to obtain the centroid distribution of each second mask region in the measured three-dimensional optical imaging information. Specifically, for each second mask region, the second centroid coordinate information of the second mask region L is calculated based on the two-dimensional coordinate information (x, y) of each spatial location point in the second mask region L. The second centroid coordinate information of the second mask region L can be represented as:

[0060] Combination Figure 10As shown, the two-dimensional feature mask image corresponding to the measured three-dimensional optical imaging information is a binary image. The region marked as an effective pixel (i.e., gray level 1) in the binary image constitutes the second mask region. The second mask region includes the set of effective pixels within the region. The centroid of the second mask region is essentially the first-order moment centroid (geometric centroid), which is a weighted average position description of the distribution of effective pixels within the second mask region.

[0061] In the formula for calculating the second centroid coordinates of the second mask region L mentioned above... It is the first moment of the second mask region L in the x-direction, representing the x-coordinates of all spatial points within the region, weighted. Weighted summation; It is the first moment of the second mask region L in the y direction, representing the y-coordinate of all spatial points within the region, according to... Weighted summation; Represents the area of ​​a region (in a binary image, it is equal to the total number of valid pixels within the region); This is a weighting function, such as density or grayscale value.

[0062] In some embodiments, the matching calibration submodule 303 is used to match and calibrate each vertex in each first mask region with each spatial location point in each second mask region in the following manner: steps 1 to 7.

[0063] Step 1: Construct a first triangulation topology based on the first centroid coordinate information of each first mask region, and construct a second triangulation topology based on the second centroid coordinate information of each second mask region.

[0064] Figure 12 This is a schematic diagram of the first and second triangulation topologies in an embodiment of the present invention, as shown below. Figure 12 As shown, Delaunay triangulation is performed on the first centroid coordinate information of each first mask region in the two-dimensional mask region information and the second centroid coordinate information of each second mask region in the measured three-dimensional optical imaging information to obtain the first triangulation topology and the second triangulation topology.

[0065] Step 2: Traverse and search for the current similar reference triangles between the first triangulation topology and the second triangulation topology. The current similar reference triangles include the first reference triangle in the first triangulation topology and the second reference triangle in the second triangulation topology.

[0066] From the first triangulation topology and the second triangulation topology, search for the currently similar reference triangle between the first triangulation topology and the second triangulation topology. The currently similar reference triangle includes the first reference triangle in the first triangulation topology and the second reference triangle in the second triangulation topology.

[0067] For example, geometric features of each reference triangle are extracted from the first and second triangulation topologies, respectively. These geometric features include at least one of the following: interior angles, side lengths, side length ratios, area, and perimeter. The morphological similarity between the geometric features of each set of reference triangles is calculated iteratively. Each set of reference triangles includes one reference triangle from the first triangulation topology and one reference triangle from the second triangulation topology. When the morphological similarity between the currently calculated set of reference triangles reaches a similarity threshold, this set of reference triangles is considered the most similar reference triangle between the first and second triangulation topologies. The algorithm for calculating morphological similarity may include, but is not limited to, similarity algorithms such as cosine distance and Euclidean distance.

[0068] Step 3: Determine the current affine transformation matrix based on the first centroid coordinate information corresponding to each vertex of the first reference triangle and the first centroid coordinate information corresponding to each vertex of the second reference triangle.

[0069] In some embodiments, the matching calibration submodule 303 is used to establish the current affine transformation equation based on the first centroid coordinate information corresponding to each vertex in the first reference triangle and the second centroid coordinate information corresponding to each vertex in the second reference triangle; and to solve the current affine transformation equation to obtain the current affine transformation matrix.

[0070] Specifically, the current affine transformation equation is expressed as: Bk2=P Where B is a 3×3 matrix constructed based on the first centroid coordinate information corresponding to each vertex in the first reference triangle. Let P be the first centroid coordinate information corresponding to the three vertices of the first reference triangle; P is a 3×2 matrix constructed based on the second centroid coordinate information corresponding to each vertex of the second reference triangle. , The coordinates of the second centroid corresponding to the three vertices of the second reference triangle are given.

[0071] Solving the above affine transformation equations yields the affine transformation matrix k2, where k2 = B. -1 P, k2 is a 3x2 matrix.

[0072] By establishing the current affine transformation equation between the similar first reference triangle and the second reference triangle in the current traversal, and solving for the current affine transformation matrix, the current affine transformation matrix is ​​used to obtain the corresponding centroid mapping coordinate information from the first centroid coordinate information of the first mask region through affine transformation, so as to align the first centroid coordinate information of the first mask region with the second centroid coordinate information of the second mask region.

[0073] Step 4: For the first centroid coordinate information of each first mask region, perform an affine transformation based on the current affine transformation matrix to obtain the centroid mapping coordinate information of the first mask region.

[0074] The first centroid coordinate information of each first mask region is transformed using the current affine transformation matrix to obtain the corresponding centroid mapping coordinate information. This is used to align the first centroid coordinate information of each first mask region with the second centroid coordinate information of the second mask region, thereby facilitating the search for matching first and second mask regions.

[0075] Step 5: Search for the second centroid coordinate information of the second mask region that matches the centroid mapping coordinate information of the first mask region in the second centroid coordinate information of each second mask region.

[0076] In step 5, matching means that the distance between two centroid coordinates is less than a preset distance threshold, or the error between the centroid x-axis coordinate of the centroid mapping coordinate information of the first mask region and the centroid x-axis coordinate of the second centroid coordinate information of the second mask region is less than a first threshold, and the error between the centroid y-axis coordinate of the centroid mapping coordinate information of the first mask region and the centroid y-axis coordinate of the second centroid coordinate information of the second mask region is less than a second threshold. Then, the two centroid coordinates are determined to be the centroid coordinate points of the matching mapping, that is, the first mask region and the second mask region are matched.

[0077] Step 6: Among the multiple matching results obtained from multiple traversals, the matching result with the most matches between the centroid mapping coordinates of the first mask region and the second centroid coordinates of the second mask region is taken as the final matching result.

[0078] After each traversal and affine transformation, a global centroid coordinate matching of the first and second mask regions is performed. The matching result with the most matches between the centroid mapping coordinates of the first mask region and the second centroid coordinates of the second mask region is taken as the final matching result.

[0079] Figure 13 This is a schematic diagram illustrating the matching between each first mask region and each second mask region in an embodiment of the present invention, as shown below. Figure 13 As shown, Figure 13 The final matching result is shown by matching and calibrating each first mask region in the two-dimensional mask region information with each second mask region in the two-dimensional feature mask image corresponding to the measured three-dimensional optical imaging information. The second mask regions in the two-dimensional feature mask image corresponding to the measured three-dimensional optical imaging information are basically matched and mapped one-to-one with each first mask region in the two-dimensional mask region information.

[0080] Step 7: Determine the mapping relationship between each vertex in each first mask region and each spatial position point in each second mask region based on the final matching result, and obtain the three-dimensional coordinate information of the spatial position point corresponding to each vertex in each first mask region.

[0081] Based on the matching mapping relationship between each first mask region and each second mask region in the final matching result, the mapping relationship between each vertex in each first mask region and each spatial position point in each second mask region is determined, and the three-dimensional coordinate information of the matching spatial position point corresponding to each vertex in each first mask region is obtained. This enables the matching calibration between each vertex in each first mask region and each spatial position point in each second mask region, thereby determining the target height coordinates corresponding to each vertex in each first mask region.

[0082] In some embodiments, the three-dimensional optical imaging prediction model adopts a UNet network structure obtained by modifying the ResNet Backbone based on the residual neural network ResNet.

[0083] Figure 14 This is a schematic diagram of the training architecture of the three-dimensional optical imaging prediction model in an embodiment of the present invention. In some embodiments, the predicted three-dimensional optical imaging information includes the predicted height coordinates of each vertex of each first mask region in the two-dimensional mask region information, and the target three-dimensional optical imaging information includes the target height coordinates that correspond to and match each vertex of each first mask region in the two-dimensional mask region information; such as Figure 14 As shown, the first prediction module 103 uses the two-dimensional mask region information and etching process parameters of the etched workpiece as input features of the prediction model. After embedding and vectorization processing, it is input into the three-dimensional optical imaging prediction model to be trained. The final output task of the model is adjusted to numerical regression mode. The predicted height coordinates of each vertex of each first mask region in the two-dimensional mask region information are obtained directly through numerical regression output, thereby obtaining the predicted three-dimensional optical imaging information. The loss calculation module 104 calculates the model inference loss according to the predicted three-dimensional optical imaging information of each etched workpiece and the target three-dimensional optical imaging information through a preset loss function. The training module 105 trains the three-dimensional optical imaging prediction model according to the model inference loss. The model parameters are updated and modified through backpropagation. When the model inference loss value is reduced to a sufficiently small value and gradually stabilizes with the change of the training cycle, it is considered that the model has converged and reached the optimal effect. At this time, it means that the model has the ability to perform two-dimensional to three-dimensional simulation prediction at the millisecond level, and finally the trained three-dimensional optical imaging prediction model is obtained.

[0084] In some embodiments, the loss function is calculated using the Mean Absolute Error (MAE) with a neglected value (-1).

[0085] In some embodiments, the loss calculation module 104 is configured to: for each etched part, obtain the base loss corresponding to the etched part based on the average absolute error between the predicted height coordinates and the target height coordinates of each vertex of each first mask region; and obtain the current model inference loss based on the average of the base losses corresponding to each etched part.

[0086] Based on the training device for the aforementioned three-dimensional optical imaging prediction model, this embodiment of the invention provides a training method for a three-dimensional optical imaging prediction model. Figure 15 This is a flowchart illustrating a training method for a three-dimensional optical imaging prediction model provided in an embodiment of the present invention, as shown below. Figure 15 As shown, the training methods for the three-dimensional optical imaging prediction model include: Step S1501: Obtain two-dimensional mask design drawings and etching process parameters for multiple etched parts. The two-dimensional mask design drawings include two-dimensional mask area information of the etching mask corresponding to the etched parts.

[0087] The two-dimensional mask region information includes the two-dimensional coordinate information of each vertex in each first mask region of the etching mask, and the two-dimensional coordinate information includes two-dimensional horizontal coordinates; the etching process parameters include at least one of the following parameters: coating material, coating thickness, photoresist thickness, exposure energy, solution concentration, development time, and etching duration.

[0088] Step S1502: Obtain the target three-dimensional optical imaging information of each etched workpiece.

[0089] In some embodiments, step S1502, acquiring the target three-dimensional optical imaging information of each etched workpiece, includes: for each etched workpiece, detecting the measured three-dimensional optical imaging information of the etched workpiece, the measured three-dimensional optical imaging information including the three-dimensional coordinate information corresponding to each spatial position point of the etched workpiece; performing flattening correction processing on the measured three-dimensional optical imaging information of the etched workpiece; determining the target three-dimensional optical imaging information of the etched workpiece based on the two-dimensional mask area information of the etched workpiece and the measured three-dimensional optical imaging information after flattening correction processing, the target three-dimensional optical imaging information including the three-dimensional coordinate information of the spatial position point corresponding to each vertex in each first mask area.

[0090] In some embodiments, determining the target three-dimensional optical imaging information of the etched workpiece based on the two-dimensional mask region information of the etched workpiece and the measured three-dimensional optical imaging information after planarization and correction includes: obtaining the first centroid coordinate information of each first mask region based on the two-dimensional coordinate information of each vertex in each first mask region; generating a corresponding two-dimensional feature mask image based on the measured three-dimensional optical imaging information after planarization and correction, wherein the two-dimensional feature mask image includes the two-dimensional coordinate information of each spatial location point in each second mask region; obtaining the second centroid coordinate information of each second mask region based on the two-dimensional coordinate information of each spatial location point in each second mask region; and matching and calibrating each vertex in each first mask region with each spatial location point in each second mask region based on the first centroid coordinate information of each first mask region and the second centroid coordinate information of each second mask region to determine the three-dimensional coordinate information of the spatial location point corresponding to each vertex in each first mask region.

[0091] In some embodiments, generating a corresponding two-dimensional feature mask image based on the measured three-dimensional optical imaging information after planarization correction includes: projecting the measured three-dimensional optical imaging information after planarization correction into a horizontal coordinate space to obtain a corresponding two-dimensional spatial height distribution map; and performing grayscale binarization segmentation on the two-dimensional spatial height distribution map according to a preset height discrimination threshold to obtain a corresponding two-dimensional feature mask image.

[0092] In some embodiments, based on the first centroid coordinate information of each first mask region and the second centroid coordinate information of each second mask region, each vertex in each first mask region is matched and calibrated with each spatial position point in each second mask region to determine the three-dimensional coordinate information of the matching spatial position points corresponding to each vertex in each first mask region. This includes: constructing a first triangulation topology based on the first centroid coordinate information of each first mask region, and constructing a second triangulation topology based on the second centroid coordinate information of each second mask region; traversing and searching for currently similar reference triangles between the first triangulation topology and the second triangulation topology, wherein the currently similar reference triangles include the first reference triangle in the first triangulation topology and the second reference triangle in the second triangulation topology; and matching the first centroid coordinate information of each vertex of the first reference triangle with the second reference triangle. The first centroid coordinates of each vertex of the quasi-triangle are used to determine the current affine transformation matrix. For the first centroid coordinates of each first mask region, an affine transformation is performed according to the current affine transformation matrix to obtain the centroid mapping coordinates of the first mask region. The second centroid coordinates of each second mask region are searched for to match the centroid mapping coordinates of the first mask region. The matching result with the most matches between the centroid mapping coordinates of the first mask region and the second centroid coordinates of the second mask region is taken as the final matching result. The mapping relationship between each vertex in each first mask region and each spatial position point in each second mask region is determined based on the final matching result, thus obtaining the three-dimensional coordinates of the matching spatial position points corresponding to each vertex in each first mask region.

[0093] In some embodiments, determining the current affine transformation matrix based on the first centroid coordinate information corresponding to each vertex of the first reference triangle and the first centroid coordinate information corresponding to each vertex of the second reference triangle includes: establishing the current affine transformation equation based on the first centroid coordinate information corresponding to each vertex of the first reference triangle and the second centroid coordinate information corresponding to each vertex of the second reference triangle; and solving the current affine transformation equation to obtain the current affine transformation matrix.

[0094] In some embodiments, the planarization correction process for the measured three-dimensional optical imaging information of the etched workpiece includes: using the least squares method to fit the three-dimensional coordinate information of each spatial position point in the measured three-dimensional optical imaging information to a reference plane to obtain the fitting reference plane; and performing a difference calculation between the measured height coordinates in the three-dimensional coordinate information of each spatial position point and the theoretical tilt reference height of each spatial position point on the fitting reference plane to obtain the planarized correction process for the measured three-dimensional optical imaging information.

[0095] Step S1503: Input the two-dimensional mask area information and etching process parameters of each etched part into the three-dimensional optical imaging prediction model to be trained, and predict the three-dimensional optical imaging information of each etched part.

[0096] Step S1504: Calculate the model inference loss based on the predicted three-dimensional optical imaging information of each etched part and the target three-dimensional optical imaging information.

[0097] The predicted three-dimensional optical imaging information includes the predicted height coordinates of each vertex of each first mask region in the two-dimensional mask region information, and the target three-dimensional optical imaging information includes the target height coordinates that correspond to and match each vertex of each first mask region in the two-dimensional mask region information.

[0098] In some embodiments, in step S1504, the model inference loss is calculated based on the predicted three-dimensional optical imaging information and the target three-dimensional optical imaging information of each etched part, including: for each etched part, the base loss corresponding to the etched part is obtained based on the average absolute error between the predicted height coordinates and the target height coordinates of each vertex of each first mask region; and the current model inference loss is obtained based on the mean of the base loss corresponding to each etched part.

[0099] Step S1505: Train the three-dimensional optical imaging prediction model based on the model inference loss until the model converges, and obtain the trained three-dimensional optical imaging prediction model.

[0100] In this embodiment of the invention, the training method of the three-dimensional optical imaging prediction model is implemented by the training device of the three-dimensional optical imaging prediction model in any of the above embodiments. For a detailed description of the training method of the three-dimensional optical imaging prediction model, please refer to the relevant description in the training device of the three-dimensional optical imaging prediction model, which will not be repeated here.

[0101] This invention provides a three-dimensional optical imaging prediction apparatus for executing a three-dimensional optical imaging prediction method flow. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0102] Figure 16 A block diagram of a three-dimensional optical imaging prediction device provided in an embodiment of the present invention is shown below. Figure 16 As shown, the three-dimensional optical imaging prediction device includes: The third acquisition module 1601 is used to acquire the two-dimensional mask design drawing and etching process parameters of the workpiece to be etched. The two-dimensional mask design drawing includes the two-dimensional mask area information of the etching mask corresponding to the workpiece to be etched. The second prediction module 1602 is used to input the two-dimensional mask area information and etching process parameters into a pre-trained three-dimensional optical imaging prediction model to predict the three-dimensional optical imaging information of the workpiece after etching.

[0103] The pre-trained three-dimensional optical imaging prediction model is obtained by training the three-dimensional optical imaging prediction model using the training device described in the above embodiment using the training method described above.

[0104] Based on the above-mentioned three-dimensional optical imaging prediction device, this embodiment of the invention also provides a three-dimensional optical imaging prediction method, including: acquiring a two-dimensional mask design drawing and etching process parameters of the workpiece to be etched, wherein the two-dimensional mask design drawing includes two-dimensional mask area information of the etching mask corresponding to the workpiece to be etched; inputting the two-dimensional mask area information and etching process parameters into a pre-trained three-dimensional optical imaging prediction model to predict the three-dimensional optical imaging information of the workpiece after etching.

[0105] Figure 17 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0106] The following is a detailed reference. Figure 17 This diagram illustrates a suitable structural design for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 1701, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 1702 or a program loaded from memory 1708 into random access memory (RAM) 1703. RAM 1703 also stores various programs and data required for the operation of the electronic device. The processor 1701, ROM 1702, and RAM 1703 are interconnected via bus 1704. Input / output (I / O) interface 1705 is also connected to bus 1704.

[0107] Typically, the following devices can be connected to I / O interface 1705: input devices 1706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 1707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 1708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1709. Communication device 1709 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 17 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0108] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 1709, or installed from a memory 1708, or installed from a ROM 1702. When the computer program is executed by the processor 1701, it performs the functions defined in the training or prediction methods of the embodiments of the present invention.

[0109] Figure 17 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0110] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the training or prediction methods shown in the above embodiments are implemented.

[0111] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0112] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A device for training a three-dimensional optical imaging prediction model, characterized in that, The device includes: The first acquisition module is used to acquire two-dimensional mask design drawings and etching process parameters of multiple etched parts. The two-dimensional mask design drawings include two-dimensional mask area information of the etching mask corresponding to the etched parts. The second acquisition module is used to acquire the target three-dimensional optical imaging information of each of the etched workpieces; The first prediction module is used to input the two-dimensional mask area information and etching process parameters of each etched workpiece into the three-dimensional optical imaging prediction model to be trained, and to predict the predicted three-dimensional optical imaging information of each etched workpiece. The loss calculation module is used to calculate the model inference loss based on the predicted three-dimensional optical imaging information of each of the etched parts and the target three-dimensional optical imaging information. The training module is used to train the three-dimensional optical imaging prediction model based on the model inference loss until the model converges, thus obtaining the trained three-dimensional optical imaging prediction model.

2. The apparatus according to claim 1, characterized in that, The two-dimensional mask region information includes the two-dimensional coordinate information of each vertex in each first mask region, and the two-dimensional coordinate information includes two-dimensional horizontal coordinates; The etching process parameters include at least one of the following parameters: coating material, coating thickness, photoresist thickness, exposure energy, solution concentration, development time, and etching duration.

3. The apparatus of claim 1, wherein, The two-dimensional mask region information includes the two-dimensional coordinate information of each vertex in each first mask region, and the second acquisition module includes: The detection submodule is used to detect the measured three-dimensional optical imaging information of each etched workpiece. The measured three-dimensional optical imaging information includes the three-dimensional coordinate information corresponding to each spatial position point of the etched workpiece. The planarization processing submodule is used to perform planarization correction processing on the measured three-dimensional optical imaging information of the etched workpiece; The matching calibration submodule is used to determine the target three-dimensional optical imaging information of the etched workpiece based on the two-dimensional mask area information and the measured three-dimensional optical imaging information after flattening and correction. The target three-dimensional optical imaging information includes the three-dimensional coordinate information of the spatial position points that match each vertex in each first mask area.

4. The apparatus of claim 3, wherein, The matching calibration submodule is used to: obtain the first centroid coordinate information of each first mask region based on the two-dimensional coordinate information of each vertex in each first mask region; and generate a corresponding two-dimensional feature mask image based on the measured three-dimensional optical imaging information after planarization and correction processing, wherein the two-dimensional feature mask image includes the two-dimensional coordinate information of each spatial location point in each second mask region. Based on the two-dimensional coordinate information of each spatial location point in each second mask region, obtain the second centroid coordinate information of each second mask region; Based on the first centroid coordinate information of each first mask region and the second centroid coordinate information of each second mask region, each vertex in each first mask region is matched and calibrated with each spatial position point in each second mask region to determine the three-dimensional coordinate information of the matching spatial position point corresponding to each vertex in each first mask region.

5. The apparatus of claim 3, wherein, The planarization submodule is used to: use the least squares method to fit the three-dimensional coordinate information of each spatial location point in the measured three-dimensional optical imaging information to a reference plane to obtain the fitting reference plane; and perform a difference calculation between the measured height coordinates in the three-dimensional coordinate information of each spatial location point and the theoretical tilt reference height of each spatial location point on the fitting reference plane to obtain the measured three-dimensional optical imaging information after planarization correction.

6. The apparatus of claim 4, wherein, The matching calibration submodule is used for: The measured three-dimensional optical imaging information after flattening and correction is projected into the horizontal coordinate space to obtain the corresponding two-dimensional spatial height distribution map; the two-dimensional spatial height distribution map is then segmented into grayscale binarization according to a preset height discrimination threshold to obtain the corresponding two-dimensional feature mask image. And / or, the matching calibration submodule is used for: A first triangulation topology is constructed based on the first centroid coordinate information of each first mask region, and a second triangulation topology is constructed based on the second centroid coordinate information of each second mask region; Traverse and search for currently similar reference triangles between the first triangulation topology and the second triangulation topology. The currently similar reference triangles include the first reference triangle in the first triangulation topology and the second reference triangle in the second triangulation topology. The current affine transformation matrix is ​​determined based on the first centroid coordinate information corresponding to each vertex of the first reference triangle and the first centroid coordinate information corresponding to each vertex of the second reference triangle. For the first centroid coordinate information of each first mask region, perform an affine transformation based on the current affine transformation matrix to obtain the centroid mapping coordinate information of the first mask region; Search for the second centroid coordinate information of the second mask region that matches the centroid mapping coordinate information of the first mask region in the second centroid coordinate information of each second mask region; The matching result with the most matches between the centroid mapping coordinates of the first mask region and the second centroid coordinates of the second mask region is taken as the final matching result from the multiple matching results obtained through multiple traversals. Based on the final matching results, the mapping relationship between each vertex in each first mask region and each spatial location point in each second mask region is determined, and the three-dimensional coordinate information of the matching spatial location point corresponding to each vertex in each first mask region is obtained.

7. The apparatus of claim 1, wherein, The predicted three-dimensional optical imaging information includes the predicted height coordinates of each vertex of each first mask region in the two-dimensional mask region information, and the target three-dimensional optical imaging information includes the target height coordinates that correspond to and match each vertex of each first mask region in the two-dimensional mask region information. The loss calculation module is used to: for each etched part, obtain the basic loss corresponding to the etched part based on the average absolute error between the predicted height coordinates and the target height coordinates of each vertex of each first mask region; and obtain the current model inference loss based on the mean of the basic losses corresponding to each etched part.

8. A three-dimensional optical imaging prediction device, characterized by, The device includes: The third acquisition module is used to acquire the two-dimensional mask design drawing and etching process parameters of the workpiece to be etched. The two-dimensional mask design drawing includes the two-dimensional mask area information of the etching mask corresponding to the workpiece to be etched. The second prediction module is used to input the two-dimensional mask area information and the etching process parameters into a pre-trained three-dimensional optical imaging prediction model to predict the three-dimensional optical imaging information of the workpiece after etching. 9.A method for training a three-dimensional optical imaging prediction model, characterized in that, The method includes: Obtain two-dimensional mask design drawings and etching process parameters for multiple etched parts. The two-dimensional mask design drawings include two-dimensional mask area information of the etching mask corresponding to the etched parts. Obtain the target three-dimensional optical imaging information of each of the etched workpieces; The two-dimensional mask area information and etching process parameters of each etched part are input into the three-dimensional optical imaging prediction model to be trained, and the predicted three-dimensional optical imaging information of each etched part is obtained. The model inference loss is calculated based on the predicted three-dimensional optical imaging information of each etched workpiece and the target three-dimensional optical imaging information. The three-dimensional optical imaging prediction model is trained based on the model inference loss until the model converges, resulting in a fully trained three-dimensional optical imaging prediction model.

10. An electronic device, comprising: include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes these computer instructions to perform the training method for the three-dimensional optical imaging prediction model as described in claim 9.

Images