LDI exposure machine light path deformation monitoring method based on multi-mode deformation field analysis

Abstract

The invention relates to the technical field of alignment precision of LDI exposure machines, and particularly discloses an LDI exposure machine light path deformation monitoring method based on multi-mode deformation field analysis, which comprises the following steps: acquiring multi-mode monitoring data of a substrate, the multi-modal monitoring data of the substrate comprises substrate datum point array data, a composite calibration pattern, a near-infrared image, grating temperature field distribution data and vibration spectrum data; obtaining deformation characteristic data of a plurality of substrate areas according to the near-infrared image, the substrate reference point array data and the composite calibration pattern; based on each piece of deformation characteristic data, the grating temperature field distribution data and the vibration spectrum data, obtaining fused deformation characteristic data; and generating a light path continuous deformation field corresponding to each substrate area according to the fused deformation feature data. According to the invention, by acquiring the multi-modal monitoring data of the substrate, such as the reference point array data of the substrate, the composite calibration pattern, the near-infrared image and the like, the state of the substrate is comprehensively reflected from multiple dimensions, and the defects of traditional single data monitoring are overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of LDI exposure machine alignment accuracy, and in particular to a method for monitoring optical path deformation of an LDI exposure machine based on multimodal deformation field analysis. Background Art

[0002] LDI (laser direct imaging) exposure is an essential process in the PCB manufacturing process. It plays a key role in the resolution of fine circuits, smart factories and environmental protection. It can also improve labor efficiency and increase production capacity.

[0003] Existing LDI exposure processes typically utilize traditional LDI exposure machines, parallel exposure machines, and other tools, relying on a single data source. For example, observing substrate fiducial displacement solely through an optical microscope or collecting equipment vibration data solely through vibration sensors. This single-dimensional monitoring method fails to fully reflect the true state of optical path changes. For example, thermal expansion of the substrate during exposure due to localized temperature fluctuations (temperature differences ≥ 3°C) can result in deformations of the order of 0.3-0.5μm. However, optical detection methods struggle to distinguish between thermal deformation and displacement caused by mechanical vibration. While vibration sensors can capture the equipment's resonant frequency, they cannot directly link the vibration signal to deformation in specific areas of the optical path. Summary of the Invention

[0004] The object of the present invention is to provide a method for monitoring optical path deformation of an LDI exposure machine based on multimodal deformation field analysis, so as to solve the technical problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions: A method for monitoring optical path deformation of an LDI exposure machine based on multimodal deformation field analysis, applied to the LDI exposure machine, includes: Acquiring substrate multimodal monitoring data, wherein the substrate multimodal monitoring data includes substrate reference point array data, composite calibration pattern, near-infrared image, grating temperature field distribution data, and vibration spectrum data; acquiring deformation feature data of a plurality of substrate regions according to the near-infrared image, substrate reference point array data, and a composite calibration pattern; Acquire fused deformation feature data based on each of the deformation feature data, the grating temperature field distribution data, and the vibration spectrum data; generating a light path continuous deformation field corresponding to each substrate region according to the fused deformation feature data; Obtaining a micro-deformation of the optical path according to the continuous deformation field of the optical path, and determining whether the micro-deformation of the optical path is greater than a preset value; If it is greater than, the optical path micro-deformation is an abnormal deformation amount, and the corresponding substrate area is judged to be deformed, and deformation error information is generated according to the optical path micro-deformation; If it is not greater than, the optical path micro-deformation is a safe deformation amount, and it is determined that the corresponding substrate area has not been deformed.

[0006] Preferably, the step of acquiring deformation feature data of a plurality of substrate regions according to the near-infrared image, substrate reference point array data and composite calibration pattern comprises: Performing image fusion and registration based on the near-infrared image and the composite calibration pattern to generate a multispectral image; Acquire corner point features of the reference point array based on the multispectral image and the substrate reference point array data, and generate an initial feature point set, wherein the initial feature point set includes measured coordinates and preset coordinates; Fitting the measured coordinates and the preset coordinates through affine transformation, and performing interior point screening to obtain a matching coordinate offset matrix; generating a global deformation distribution map of the substrate according to the matching coordinate offset matrix; Performing optical path region division on the substrate global deformation distribution map to obtain a plurality of optical path division regions, and dividing the substrate into a plurality of substrate optical path regions according to the plurality of optical path division regions; Deformation characteristic data is acquired according to each substrate optical path region, wherein the deformation characteristic data includes regional average displacement, regional maximum strain and regional deformation variance.

[0007] Preferably, the step of obtaining fused deformation feature data based on each of the deformation feature data, grating temperature field distribution data and vibration spectrum data comprises: establishing a spatial reference grid based on the substrate reference point array data; Performing time synchronization calibration on the deformation feature data, grating temperature field distribution data, and vibration spectrum data, and mapping the calibrated deformation feature data, grating temperature field distribution data, and vibration spectrum data to the spatial reference grid to form spatiotemporally aligned multimodal data; Acquiring deformation-vibration correlation feature data and optical path-vibration correlation feature data according to the multimodal data; Acquire a first key characteristic component according to the deformation-vibration correlation characteristic data; Acquire a second key characteristic component according to the optical path-vibration correlation characteristic data; Performing time alignment and normalization on the first key feature component and the second key feature component and then fusing them to construct a fused feature component; The deformation field distribution is predicted based on the fused feature components through a long short-term memory network, and fused deformation feature data is generated according to the light path area division.

[0008] Preferably, the step of generating a light path continuous deformation field corresponding to each substrate region according to the fused deformation feature data includes: Acquire a deformation sensitive area according to the fused deformation feature data, and map the deformation sensitive area to a corresponding optical path number; extracting a deformed optical path region image from the near-infrared image based on the optical path number, and performing adaptive histogram equalization and Gaussian filtering preprocessing on the deformed optical path region image to obtain a preprocessed deformed optical path region image; Acquire a deformation light path region image sequence, near-infrared temperature gradient field data, and vibration spectrum data within a preset sampling period according to each preprocessed deformation light path region image, and generate a multimodal data set; fusing temperature gradient and vibration amplitude features of the multimodal data set to obtain a local position-related deformation variable; Generate a continuous deformation field according to the local position-correlated deformation amount of each pre-processed deformed optical path area image; A preset light path region division mask is obtained, and the continuous deformation field is divided according to the preset light path region division mask to generate a light path continuous deformation field.

[0009] Preferably, the step of obtaining the light path micro-deformation amount according to the light path continuous deformation field includes: Get the working area matrix and light path stitching mask matrix; Obtaining a gradient vector field for the optical path continuous deformation field; Obtaining a magnitude matrix according to the gradient vector field; dividing the amplitude matrix into gradient change amplitude matrices of each light path according to the light path splicing mask matrix; Divide the working area into grid-shaped integration areas according to a preset resolution, and obtain the boundary coordinates and pixel set contained in each grid; Calculate the average value of the gradient amplitude in each grid, aggregate the results by optical path number, and generate the preliminary optical path micro-deformation vector; Obtaining high-precision reference points in each optical path area, and obtaining relative errors based on the preliminary optical path micro-deformation and the measured values ​​of the reference points; The preliminary optical path micro-deformation is corrected by relative error, and the correction result is used as the final optical path micro-deformation.

[0010] Preferably, the step of generating deformation error information according to the optical path micro-deformation comprises: Obtaining abnormal deformation amount and regional parameters according to the optical path micro-deformation amount; Obtaining a timestamp of the occurrence of the abnormal deformation according to the abnormal deformation amount, and associating it with the corresponding LDI exposure machine optical path number and substrate area preset coordinates to obtain deformation integration information; Acquire the grating temperature field distribution data and vibration spectrum data at the time of abnormal deformation according to the regional parameters; generating inducement integration information according to the grating temperature field distribution data and the vibration spectrum data; Deformation error information is obtained according to the deformation integration information and the inducement integration information.

[0011] The present invention also provides an LDI exposure machine optical path deformation monitoring system based on multi-modal deformation field analysis, comprising: A monitoring data acquisition module is used to obtain substrate multimodal monitoring data, wherein the substrate multimodal monitoring data includes substrate reference point array data, composite calibration pattern, near-infrared image, grating temperature field distribution data and vibration spectrum data; a deformation feature extraction module, configured to obtain deformation feature data of a plurality of substrate regions based on the near-infrared image, substrate reference point array data, and a composite calibration pattern; A multimodal fusion module, configured to obtain fused deformation feature data based on each of the deformation feature data, grating temperature field distribution data, and vibration spectrum data; A micro-deformation analysis module is used to generate a continuous optical path deformation field corresponding to each substrate area according to the fused deformation feature data; a judgment module, configured to obtain a micro-deformation of the optical path according to the continuous deformation field of the optical path, and to judge whether the micro-deformation of the optical path is greater than a preset value; If it is greater than, the optical path micro-deformation is an abnormal deformation amount, and the corresponding substrate area is judged to be deformed, and deformation error information is generated according to the optical path micro-deformation; If it is not greater than, the optical path micro-deformation is a safe deformation amount, and it is determined that the corresponding substrate area has not been deformed.

[0012] Preferably, the deformation feature extraction module includes: An image fusion and registration unit, configured to perform image fusion and registration based on the near-infrared image and the composite calibration pattern to generate a multispectral image; a feature point set generating unit, configured to obtain corner point features of a reference point array based on the multispectral image and substrate reference point array data, and generate an initial feature point set, wherein the initial feature point set includes measured coordinates and preset coordinates; A coordinate offset calculation unit, configured to fit the measured coordinates and the preset coordinates through affine transformation and perform interior point screening to obtain a matching coordinate offset matrix; A global deformation map generating unit, configured to generate a global deformation distribution map of the substrate according to the matching coordinate offset matrix; an optical path region division unit, configured to divide the substrate global deformation distribution map into optical path regions to obtain a plurality of optical path division regions, and divide the substrate into a plurality of substrate optical path regions according to the plurality of optical path division regions; The deformation characteristic calculation unit is used to obtain deformation characteristic data according to each substrate optical path area, wherein the deformation characteristic data includes regional average displacement, regional maximum strain and regional deformation variance.

[0013] Preferably, the multimodal fusion module includes: A spatial grid construction unit, configured to establish a spatial reference grid based on the substrate reference point array data; a time-space synchronization calibration unit, configured to perform time-synchronization calibration on the deformation feature data, grating temperature field distribution data, and vibration spectrum data, and map the calibrated deformation feature data, grating temperature field distribution data, and vibration spectrum data to the spatial reference grid to form time-space aligned multimodal data; a correlation feature calculation unit, configured to obtain deformation-vibration correlation feature data and optical path-vibration correlation feature data based on the multimodal data; A first key feature extraction unit, configured to obtain a first key feature component according to the deformation-vibration correlation feature data; A second key feature extraction unit, configured to obtain a second key feature component according to the optical path-vibration correlation feature data; a feature fusion unit, configured to perform time alignment and normalization processing on the first key feature component and the second key feature component and then fuse them to construct a fused feature component; The deformation field prediction unit is used to predict the deformation field distribution based on the fused feature components through a long short-term memory network, and generate fused deformation feature data according to the optical path area division.

[0014] Preferably, the multimodal fusion module includes: a sensitive area identification unit, configured to obtain a deformation sensitive area according to the fused deformation feature data, and map the deformation sensitive area to a corresponding optical path number; an image preprocessing unit, configured to extract a deformed optical path region image from the near-infrared image based on the optical path number, and perform adaptive histogram equalization and Gaussian filtering preprocessing on the deformed optical path region image to obtain a preprocessed deformed optical path region image; A multimodal data set construction unit is used to obtain a deformation light path area image sequence, near-infrared temperature gradient field data and vibration spectrum data within a preset sampling period according to each preprocessed deformation light path area image, and generate a multimodal data set; A local deformation calculation unit is used to fuse the temperature gradient and vibration amplitude features of the multimodal data set to obtain a local position-related deformation; A continuous deformation field generating unit, configured to generate a continuous deformation field according to a local position-correlated deformation amount of each pre-processed deformed optical path region image; The optical path deformation field segmentation unit is used to obtain a preset optical path area division mask, and segment the continuous deformation field according to the preset optical path area division mask to generate an optical path continuous deformation field.

[0015] The beneficial effects of the present application are as follows: the present invention obtains multi-modal monitoring data of the substrate, such as substrate reference point array data, composite calibration patterns, near-infrared images, etc., to comprehensively reflect the substrate status from multiple dimensions and make up for the shortcomings of traditional single data monitoring. These data are used to accurately calculate the deformation characteristic data of the substrate area, avoiding the problems of low precision and inaccurate positioning of traditional methods, and providing a reliable basis for subsequent analysis. Based on each of the deformation characteristic data, grating temperature field distribution data and vibration spectrum data, fused deformation characteristic data is obtained, and multiple data are effectively integrated to deeply analyze the comprehensive impact of different factors on optical path deformation, breaking through the limitation that traditional technology is difficult to associate multiple factors. According to the model, a continuous deformation field of the optical path is generated, and the micro-deformation of the optical path is obtained, which can accurately determine whether the substrate area is deformed and timely discover potential accuracy problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a schematic diagram of a method flow chart according to an embodiment of the present application.

[0017] Figure 2 This is a schematic diagram of the system structure of an embodiment of the present application.

[0018] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0019] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0020] like Figure 1 As shown, the present application provides a method for monitoring optical path deformation of an LDI exposure machine based on multimodal deformation field analysis, which is applied to an LDI exposure machine and includes: S1. Acquire substrate multimodal monitoring data, wherein the substrate multimodal monitoring data includes substrate reference point array data, composite calibration pattern, near-infrared image, grating temperature field distribution data, and vibration spectrum data; S2. Acquire deformation feature data of multiple substrate regions based on the near-infrared image, substrate reference point array data, and composite calibration pattern; S3, acquiring fused deformation feature data based on each of the deformation feature data, the grating temperature field distribution data, and the vibration spectrum data; S4, generating a light path continuous deformation field corresponding to each substrate region according to the fused deformation feature data; S5. Obtaining a micro-deformation of the optical path according to the continuous deformation field of the optical path, and determining whether the micro-deformation of the optical path is greater than a preset value; If it is greater than, the optical path micro-deformation is an abnormal deformation amount, and the corresponding substrate area is judged to be deformed, and deformation error information is generated according to the optical path micro-deformation; If it is not greater than, the optical path micro-deformation is a safe deformation amount, and it is determined that the corresponding substrate area has not been deformed.

[0021] As described in the above steps S1-S5, the process of obtaining multimodal monitoring data of the substrate of the present invention is to design a PCB substrate with a thickness of 1.6 mm and a size of 24.8×28.8 inches, and set 7 groups of test units in the substrate. Each group of test units includes a horizontally designed reference point array composed of 2μm diameter copper-nickel alloy metal dots with an array spacing of 20 mm, covering all LDI exposure machine optical path splicing areas, and vertically designing 3 groups of analytical circuit graphics and triangle coordinates below. The substrate reference point array data is obtained through the reference point array composed of metal dots, and the composite calibration pattern is obtained by analyzing the circuit graphics and triangle coordinates; a 12K high-resolution optical scanner is used to capture near-infrared images, and grating temperature field distribution data is obtained based on the image analysis; the vibration data of the preset time period during the monitoring process is collected by a vibration sensor, and vibration spectrum data is established. These data comprehensively reflect the substrate state from the spatial reference, optical characteristics, temperature distribution and mechanical vibration dimensions, providing multi-dimensional raw data support for subsequent deformation analysis.

[0022] When acquiring deformation feature data based on the substrate reference point array data and the composite calibration pattern, the near-infrared image and the composite calibration pattern are first fused to generate a multispectral image to suppress reflections. The reference point corner features are then extracted through FAST corner detection combined with non-maximum suppression. Outliers are eliminated to obtain a coordinate offset matrix, and a bicubic spline interpolation model is constructed to generate a global deformation distribution map of the substrate. Finally, the deformation feature data of each region is extracted according to the optical path area. This process enhances feature extraction through multispectral images, thereby optimizing and improving matching accuracy. The interpolation model generates a continuous deformation distribution, realizing the conversion from image features to quantitative deformation parameters, and accurately positioning the deformation of each optical path region, avoiding the low precision and inaccurate positioning problems of traditional methods.

[0023] When obtaining fused deformation feature data based on deformation feature data, grating temperature field distribution data and vibration spectrum data, a three-dimensional Cartesian coordinate system is first established and the timestamps of multi-source data are synchronized to form spatiotemporally aligned multimodal data. Then, correlation feature data is obtained through cross-correlation analysis and Pearson correlation coefficient. Key feature components are extracted through PCA and random forest. After fusion, the deformation field distribution is predicted through the LSTM network. This process unifies the spatiotemporal coordinate system to solve the data asynchrony problem, and dimensionality reduction technology simplifies data complexity. By capturing the temporal correlation of multiple factors, multimodal data is effectively fused, and the combined impact of temperature, vibration and deformation is deeply analyzed.

[0024] When generating a continuous deformation field of the optical path based on the fused deformation feature data, the deformation-sensitive areas are screened according to the preset standard deviation and associated with the optical path number, the near-infrared image ROI is extracted and preprocessed, and a spatiotemporal sequence data set is constructed. The displacement field is calculated by the optical flow method and the temperature and vibration characteristics are fused to obtain the local position-related deformation variable. The continuous deformation field is generated through thin plate spline interpolation and Kalman filtering. This process locates the optical path problem through sensitive areas, captures the real-time displacement through the optical flow method, corrects the deformation variable through multi-source data, and improves the data continuity and stability through interpolation and filtering. High-precision reconstruction from discrete points to the global continuous deformation field is achieved, providing dynamic continuous data support for deformation analysis.

[0025] Finally, when obtaining the micro-deformation of the optical path and generating deformation error information based on the continuous deformation field of the optical path, the working area and the optical path splicing range are first obtained, the spatial gradient of the deformation field is calculated by the central difference method, the gradient amplitude is extracted according to the optical path division, and the preliminary micro-deformation is generated by grid integration. After correction by reference point calibration, it is compared with the preset threshold. In the event of an abnormality, error information containing the inducement is generated by combining temperature and vibration data. This process locates the area with severe deformation through gradient analysis, quantifies the deformation energy through grid integration, and ensures accuracy through reference point calibration. It realizes accurate calculation of micro-deformation and abnormality judgment, and timely generates error reports containing multi-dimensional information to ensure the positioning accuracy of the LDI exposure machine and avoid production errors caused by optical path deformation.

[0026] In one embodiment, the step of acquiring deformation feature data of multiple substrate regions based on the near-infrared image, substrate reference point array data, and composite calibration pattern includes: S201, performing image fusion and registration based on the near-infrared image and the composite calibration pattern to generate a multispectral image; S202, acquiring corner point features of a reference point array based on the multispectral image and substrate reference point array data, and generating an initial feature point set, wherein the initial feature point set includes measured coordinates and preset coordinates; S203, fitting the measured coordinates and the preset coordinates through affine transformation, and performing interior point screening to obtain a matching coordinate offset matrix; S204, generating a global deformation distribution map of the substrate according to the matching coordinate offset matrix; S205, performing optical path region division on the substrate global deformation distribution map to obtain a plurality of optical path division regions, and dividing the substrate into a plurality of substrate optical path regions according to the plurality of optical path division regions; S206 , acquiring deformation characteristic data according to each substrate optical path region, wherein the deformation characteristic data includes regional average displacement, regional maximum strain, and regional deformation variance.

[0027] As described in steps S201-S206 above, when the present invention generates a multispectral image by image fusion and registration based on the near-infrared image and the composite calibration pattern, the near-infrared image of the substrate is first collected by a 12K optical scanner, and the near-infrared band's suppression characteristics on metal reflection are utilized (the 700-1100nm band reduces the reflection intensity of the copper-nickel alloy reference point by more than 60%). At the same time, the composite calibration pattern is obtained based on the analytical circuit graphics and triangle coordinates of the substrate design, and its 3mil / 3mil line width visible light characteristics provide high-contrast structural information; SIFT is used to obtain the composite calibration pattern. The feature matching algorithm performs 3×3 median filtering on the near-infrared image to reduce noise and histogram equalization on the composite calibration pattern to enhance contrast. Then, feature points of the two types of images are extracted and mismatched points are eliminated using the RANSAC algorithm (matching error ≤ 0.5 pixel). The affine transformation matrix is ​​calculated to achieve spatial alignment. The near-infrared single channel and the RGB three channels of the composite calibration pattern are superimposed to generate a four-channel multispectral image with a pixel coordinate accuracy of 5μm / pixel. This image not only retains temperature distribution information but also enhances line edge features, increasing the accuracy of subsequent corner point detection to over 95%.

[0028] Then, when obtaining the corner point features of the fiducial point array and generating an initial feature point set based on the multispectral image and substrate fiducial point array data, a 50×50 pixel ROI area is delineated for each design coordinate of the substrate fiducial point array (such as a grid point with a lateral spacing of 20 mm) in the multispectral image. The FAST corner detection algorithm is applied (the response threshold is set to 30, and a grayscale change exceeding 30 is considered a corner point) to detect corner points within the ROI. Non-maximum suppression (NMS) is then used to retain the corner point with the strongest response in each ROI. An initial feature point set containing the design coordinates (x, y) and the measured coordinates (x, y) is generated. The design coordinates come from the substrate design file, and the measured coordinates are obtained through corner detection. The error between the two is controlled within 2 pixels, providing reliable coordinate pair data for subsequent deformation calculations.

[0029] Afterwards, the measured coordinates and the preset coordinates are fitted through affine transformation and inlier screening is performed to obtain the matching coordinate offset matrix. The RANSAC algorithm is used to randomly select 4 pairs of feature points from the initial feature point set as samples, and substitute them into the affine transformation model (x'=a1x+a2y+a3, y'=a4x+a5y+a6) to calculate the transformation parameters a1-a6. The projection error (Euclidean distance d) is calculated for all feature points, and the inlier threshold d=2 pixels is set. Points with an error ≤ the threshold are considered inliers. After 1000 iterations, the model with the most inliers is retained (inlier rate ≥90%). The affine parameters are recalculated based on the inliers to generate the matching coordinate offset matrix Δxy=[x,y]. This matrix records the measured and designed coordinate offsets of each reference point. The error is controlled within 1μm, effectively eliminating outliers caused by abnormal factors such as substrate warping.

[0030] Then, when generating the global deformation distribution map of the substrate based on the matching coordinate offset matrix, the inner point coordinates (x, y, Δx i ,Δy i )Substitute into the bicubic spline interpolation model ΔI(x, y)=∑∑α ij * x i * y i , solve the coefficient α through singular value decomposition (SVD) ij , the least square method is used to make the root mean square error (RMSE) between the predicted offset and the measured value ≤ 0.1μm, the substrate is divided into 0.1mm×0.1mm grids, and the model is applied to calculate the Δx of each grid point i and Δy i The value is generated into a color-coded (red ≥ 5μm, blue ≤ 1μm) global deformation distribution map, which clearly presents the deformation amount of each point on the substrate with a resolution of 0.1mm point, making it easy to analyze local and global deformation trends.

[0031] Afterwards, when the global deformation distribution map of the substrate is divided into optical path areas to obtain multiple substrate optical path areas, the geometric parameters of the 12 optical paths (such as center coordinates, rotation angles, coverage range, and a single optical path covering a 2×2-inch area) are obtained from the LDI device parameters, and a polygonal mask is generated. The mask is applied to the global deformation distribution map, and the image segmentation algorithm is used to crop according to the coordinate range to divide the substrate into 12 rectangular areas consistent with the optical path illumination range. Each area corresponds to an optical path number, and the boundary error between the area division and the optical path splicing is ≤0.5mm, achieving accurate mapping of the global deformation data to each optical path unit.

[0032] Finally, when the deformation characteristic data of each substrate optical path area is obtained, the arithmetic mean values ​​μΔx and μΔy of Δx and Δy are calculated for all grid points in each optical path area as the regional average displacement to reflect the overall deformation trend; 、 Construct the strain matrix, solve the eigenvalues ​​λ1 and λ2, and calculate the principal strains As the maximum strain of the region, identify the severity of local deformation; calculate the standard deviation σΔx and σΔy of Δx and Δy as the regional deformation variance to evaluate the deformation uniformity. For example, the average displacement of a certain optical path region is μΔx = 0.8μm, μΔy = 0.6μm, and the maximum strain ε max =0.01%, with a variance of σΔx²=0.1μm². Combined with the temperature gradient of ΔT=4°C in this area, we can conclude that the deformation is uniform due to thermal expansion. Larger variances may indicate mechanical vibration. These parameters enable quantitative assessment of deformation in each optical path region, providing a basis for optical path calibration.

[0033] In one embodiment, the step of obtaining fused deformation feature data based on each of the deformation feature data, grating temperature field distribution data, and vibration spectrum data includes: S301, establishing a spatial reference grid according to the substrate reference point array data; S302, performing time synchronization calibration on the deformation feature data, grating temperature field distribution data, and vibration spectrum data, and mapping the calibrated deformation feature data, grating temperature field distribution data, and vibration spectrum data to the spatial reference grid to form spatiotemporally aligned multimodal data; S303, acquiring deformation-vibration correlation feature data and optical path-vibration correlation feature data according to the multimodal data; S304, obtaining a first key characteristic component according to the deformation-vibration correlation characteristic data; S305, obtaining a second key characteristic component according to the optical path-vibration correlation characteristic data; S306: performing time alignment and normalization processing on the first key feature component and the second key feature component, and then fusing them to construct a fused feature component; S307 , predicting deformation field distribution through a long short-term memory network based on the fused feature components, and generating fused deformation feature data according to the light path area division.

[0034] As described in the above steps S301-S307, when the present invention establishes a spatial reference grid based on the substrate reference point array data, it first uses the copper-nickel alloy reference point array distributed at a spacing of 20 mm on the substrate, and its design coordinates form a regular grid. A three-dimensional Cartesian coordinate system is established with the lower left corner of the substrate as the origin, with the X axis along the horizontal direction of the substrate, the Y axis along the vertical direction, and the Z axis perpendicular to the substrate. The reference point design coordinates are mapped to grid nodes to form a spatial reference grid with a spacing of 20 mm. The measured coordinates of the reference points are then extracted by a high-precision image acquisition device, and the coordinate deviation is calculated by comparing them with the design coordinates. The design coordinates of the reference points are used as the coordinates of the reference points. Marked nodes are used to construct a grid covering the entire substrate, forming a reference grid matrix. This provides a unified spatial coordinate framework for multimodal data, ensures that deformation, temperature, and vibration data can be aligned and compared in space, and eliminates coordinate deviations caused by factors such as thermal expansion of the substrate material. For example, the design coordinates of the substrate center reference point are (124mm, 144mm), and the measured coordinates are (124.001mm, 144.002mm). This is used as the central node of the spatial reference grid, and the surrounding nodes are expanded at 20mm intervals to form a 12×14 grid matrix covering a 24.8×28.8-inch substrate.

[0035] Then, when the deformation feature data, grating temperature field distribution data, and vibration spectrum data are time-synchronized and mapped to a spatial reference grid to form spatiotemporally aligned multimodal data, the timestamps of the deformation, temperature, and vibration data are first unified using a GPS clock or a high-precision timer. Linear interpolation is applied to the temperature field and vibration data to unify the sampling frequency to 1 Hz to synchronize with the deformation data. The deformation, temperature, and vibration data at each time point are then mapped to the corresponding grid node to form a spatiotemporal data cube. This solves the problem of inconsistent sampling frequencies of multi-source data and ensures one-to-one correspondence of the data in the spatiotemporal dimensions. For example, at t = 10 s, the deformation data of a grid point (x = 100 mm, y = 100 mm) is Δx = 0.2 μm, the temperature field data is 26°C, and the main frequency of the vibration spectrum is 40 Hz. After synchronization, these are stored as four-dimensional data (100, 100, 10, 0.2, 26, 40), facilitating subsequent analysis of the multi-factor association at this spatiotemporal point.

[0036] Next, when obtaining deformation-vibration correlation feature data and optical path-vibration correlation feature data based on multimodal data, the cross-correlation function between the deformation feature and the vibration spectrum in the range of τ = -5s to +5s is calculated for each grid point. The τ corresponding to the peak is taken as the time lag (τ represents the time lag between the deformation variable and the vibration signal). The Pearson correlation coefficient between the vibration main frequency amplitude and the optical path power is calculated for each optical path area (the Pearson correlation coefficient is represented by r). Features with |r| > the preset value are screened, and the correlation between deformation and vibration, and between optical path and vibration is quantified, and key influencing factors are identified. For example, the vibration main frequency of a certain optical path area is 60Hz, and the correlation coefficient with the exposure power of the optical path is r = 0.72, indicating that vibration is intensified during high-power exposure. At the same time, the cross-correlation peak between deformation and vibration occurs at τ = 1s, indicating that significant deformation occurs 1s after vibration, forming the correlation feature data (60Hz, 0.72, 1s).

[0037] Then, when obtaining the first key characteristic component based on the deformation-vibration correlation feature data, the deformation-vibration correlation data is first standardized into a matrix with a mean of 0 and a variance of 1, the covariance matrix is ​​calculated, and the eigenvalues ​​and eigenvectors are solved. The principal components with a cumulative contribution rate of 90% are selected to extract the most representative features from the high-dimensional correlation data to reduce the computational complexity. For example, the deformation-vibration correlation data contains three dimensions: deformation variable, vibration amplitude, and time lag. After PCA, the variance contribution rate of PC1 is 75%, and the weights are w1=0.6, w2=0.5, and w3=-0.3, indicating that the deformation variable and vibration amplitude are the main influencing factors, and the influence of time lag is relatively small.

[0038] Afterwards, when obtaining the second key feature component based on the optical path-vibration correlation feature data, a random forest classifier was constructed, the optical path-vibration correlation feature data was input, the feature importance score was output, and features with an importance score > 0.2 were screened to identify the vibration features that have the greatest impact on the optical path deformation, and redundant features were excluded. For example: after analyzing the vibration data of 12 optical paths, the random forest showed that the importance scores of the vibration main frequency and the number of optical path 7 were 0.5 and 0.3, respectively, indicating that optical path 7 was particularly sensitive to 60Hz vibration. This feature was extracted as the second key component.

[0039] Then, when the first key feature component and the second key feature component are time-aligned and normalized and then fused to construct the fused feature component, the first and second key feature components are first aligned by timestamp to ensure that the features at the same time point correspond. Then, the two types of features are Min-Max normalized separately, and the eigenvalues ​​are scaled to the [0,1] interval to eliminate the difference in feature dimensions and ensure the balance of the fused feature vector. For example, at a certain moment, the first key component PC1=1.5 is normalized to 0.8, and the second key component vibration main frequency 60Hz is normalized to 0.6. The optical path number 7 is one-hot encoded, and after fusion, a one-dimensional vector containing normalized features and one-hot encoding is formed.

[0040] Finally, when predicting the deformation field distribution through the long short-term memory network based on the fused feature components and generating fused deformation feature data by optical path region, the fused feature vectors of the first 100 time steps are used to train the LSTM model, setting the hidden layer to 128 units and a learning rate of 0.001. The current feature vector is input, and the deformation field distribution at the next moment is predicted. The features of each region are extracted through the optical path mask, and the future deformation trend is predicted based on historical multimodal data. Accurate fused deformation features are provided according to the optical path region. For example, after the trained LSTM model is input with the fused feature vector, it is predicted that Δx = 0.92 μm of optical path 7 at time t + 1, an increase of 15% compared to the current value. Combined with the temperature field data, it is judged that the thermal-vibration coupling causes the deformation to intensify, and the fused deformation feature data of the optical path is generated for generating continuous deformation fields and error analysis.

[0041] In one embodiment, the step of generating a continuous optical path deformation field corresponding to each substrate region according to the fused deformation feature data includes: S401, acquiring a deformation sensitive area according to the fused deformation feature data, and mapping the deformation sensitive area to a corresponding optical path number; S402, extracting a deformed optical path region image from the near-infrared image based on the optical path number, and performing adaptive histogram equalization and Gaussian filtering preprocessing on the deformed optical path region image to obtain a preprocessed deformed optical path region image; S403, acquiring a deformation light path region image sequence, near-infrared temperature gradient field data, and vibration spectrum data within a preset sampling period according to each pre-processed deformation light path region image, and generating a multimodal data set; S404, fusing the temperature gradient and vibration amplitude features of the multimodal data set to obtain a local position-related deformation variable; S405, generating a continuous deformation field according to the local position-correlated deformation amount of each pre-processed deformed optical path region image; S406: Obtain a preset light path region division mask, and segment the continuous deformation field according to the preset light path region division mask to generate a light path continuous deformation field.

[0042] As described in the above steps S401-S406, when the present invention obtains the deformation sensitive area based on the fused deformation feature data and maps it to the corresponding optical path number, the standard deviation of each area is first calculated for the fused deformation feature data, and the area with a standard deviation greater than the preset threshold is screened out as the deformation sensitive area, and then the sensitive area is matched to the optical path number through spatial coordinate overlapping calculation, and a mapping table is generated based on the substrate optical path layout design to locate the local area of ​​deformation abnormality to provide a target for single optical path analysis.

[0043] Then, when extracting the deformed light path area image from the near-infrared image based on the light path number and performing preprocessing, the corresponding area is cropped from the near-infrared image according to the light path number, and adaptive histogram equalization is applied to the cropped area to enhance image contrast and expand the grayscale value distribution to the range of 0-255. The image is then denoised using a 3×3 Gaussian kernel to reduce noise, thereby improving the quality of the deformed light path area image and facilitating subsequent feature point detection and displacement calculation. For example, in the original near-infrared image of light path 7, the grayscale values ​​of the reference point area are concentrated in the range of 150-180 due to reflection. After adaptive histogram equalization, the grayscale values ​​are expanded to 50-230. After Gaussian filtering, the noise is reduced from a grayscale value of 20 to 5, clearly showing the edge of the reference point.

[0044] Afterwards, when obtaining relevant data within a preset sampling period based on each preprocessed deformed optical path area image and generating a multimodal dataset, an image of the deformed optical path area is collected every 100ms within the preset sampling period, and near-infrared temperature gradient field data and vibration spectrum data at the corresponding moment are synchronously obtained. A four-dimensional dataset containing image sequences, temperature gradient fields, and vibration spectra is constructed, providing multi-source data in the temporal and spatial dimensions for analyzing the dynamic evolution of deformation and environmental inducements. For example, 50 frames of images are collected within 5s for the ROI of optical path 7, and the temperature gradient increases from 1.5°C to 2.8°C and the vibration main frequency increases from 45Hz to 60Hz are synchronously recorded, forming a multimodal dataset containing visual, thermal, and vibration data.

[0045] Next, when the temperature gradient and vibration amplitude features in the multimodal dataset are fused to obtain the local position-related deformation variable, the Lucas-Kanade optical flow method is applied to the image sequence to calculate the displacement field of each pixel. The displacement component is concatenated with the temperature gradient and the main vibration frequency to form a feature vector. Principal component analysis is performed on the feature vector to extract the principal component with a cumulative contribution rate greater than or equal to a preset value as the local position-related deformation variable. Features directly related to the deformation are extracted from the multimodal data to eliminate noise interference: for example, the horizontal displacement component of a certain pixel point calculated by the optical flow method is 0.1μm / frame and the vertical displacement component is 0.05μm / frame, corresponding to a temperature gradient of 2.5℃ and a main vibration frequency of 55Hz. After concatenating the feature vectors, principal component analysis is performed to obtain a local position-related deformation variable of 0.12μm, reflecting the comprehensive deformation effect.

[0046] Afterwards, when generating a continuous deformation field based on the local position-related deformation variables of each preprocessed deformed optical path area image, the local position variables of the sensitive area are used as control points, and thin plate spline interpolation is applied to generate the global initial deformation field. The time series deformation value of each spatial point is smoothed by Kalman filtering, and the smoothed continuous deformation field is output. The discrete local position variables are expanded into a continuous field to eliminate sampling noise and improve data continuity. For example: the sensitive area of ​​optical path 7 has 10 control points. After thin plate spline interpolation, an initial deformation field with a resolution of 0.1mm is generated. After Kalman filtering, the noise amplitude (standard deviation) is reduced from ±0.2μm to ±0.05μm, clearly showing a trend of deformation decreasing from the edge to the center.

[0047] Finally, when obtaining the preset light path area division mask and dividing the continuous deformation field based on it to generate the light path continuous deformation field, the preset light path area division mask file is read. This file comes from the light path layout file of the LDI device, and the continuous deformation field is multiplied pixel by pixel with the mask matrix to extract the deformation data of each light path area. The deformation field file of each light path is output, and the global deformation field is disassembled into independent data of each light path to facilitate single light path analysis and calibration. For example: after applying the mask of light path 7, the deformation field of the light path area is extracted, showing that its maximum deformation at the edge is 1.2μm and the center is 0.5μm. A deformation field file with the light path number is generated for subsequent micro-deformation calculation and error analysis.

[0048] In one embodiment, the step of obtaining the micro-deformation of the optical path according to the continuous deformation field of the optical path includes: S501, obtaining a working area matrix and an optical path splicing mask matrix; S502, obtaining a gradient vector field for the optical path continuous deformation field; S503, obtaining an amplitude matrix according to the gradient vector field; S504, dividing the amplitude matrix into gradient change amplitude matrices of each light path according to the light path splicing mask matrix; S505, dividing the working area into grid-shaped integration areas according to a preset resolution, and obtaining the boundary coordinates and the pixel set contained in each grid; S506, calculating the average value of the gradient amplitude in each grid, aggregating the results by optical path number, and generating a preliminary optical path micro-deformation vector; S507, obtaining a high-precision reference point in each optical path area, and obtaining a relative error based on the preliminary optical path micro-deformation and the measured value of the reference point; S508 , correcting the preliminary optical path micro-deformation by using the relative error, and using the correction result as the final optical path micro-deformation.

[0049] As described in steps S501-S508 above, when obtaining the working area matrix and the optical path stitching mask matrix, the present invention reads the physical dimensions of the working area from the LDI exposure machine equipment parameters and converts them into pixel coordinates to generate the working area matrix. A polygonal mask matrix is ​​generated for each optical path based on the optical path geometric parameters to clarify the spatial range of deformation analysis and distinguish between the optical path stitching area and the inactive area. Then, when obtaining the gradient vector field from the continuous optical path deformation field, the central difference method is applied pixel by pixel to the continuous optical path deformation field to calculate the rate of change of the deformation field in the x and y directions. A gradient component matrix is ​​generated and merged into a gradient vector field. The rate of change of the deformation field in each direction is quantified, and regions of severe deformation are identified, providing a basis for gradient amplitude calculation. For example, at a pixel on the edge of optical path 7, the gradient component in the x-direction is 0.05 μm / pixel and the gradient component in the y-direction is 0.03 μm / pixel, indicating that the lateral deformation at that point is severe. The gradient vector direction is (59°, 0.058 μm / pixel).

[0050] Afterwards, when obtaining the amplitude matrix based on the gradient vector field, the modulus of the gradient vector field is calculated pixel by pixel to generate the amplitude matrix and normalize it. The directional information of the gradient vector is converted into a scalar value to facilitate subsequent integral calculations and regional comparisons, highlighting areas with significant deformation. For example, the amplitude matrix value in the central area of ​​optical path 7 is 0.02 μm / pixel, and it reaches 0.08 μm / pixel in the edge area, forming a gradient distribution that increases from the center to the edge, indicating that the edge deformation due to thermal expansion is more obvious.

[0051] Next, when the amplitude matrix is ​​divided into the gradient change amplitude matrix of each optical path according to the optical path splicing mask matrix, a matrix point multiplication operation is performed on each optical path, and the global amplitude matrix is ​​multiplied element by element with the mask matrix of the corresponding optical path. The deformation data of each optical path is isolated to avoid interference between adjacent optical paths, which facilitates independent analysis and error tracing of single optical paths. For example: after multiplying the global amplitude matrix with the mask matrix of optical path 7, the amplitude of the non-covered area becomes 0, and only the data of the upper right corner area is retained, forming the gradient change amplitude matrix of optical path 7, which clearly shows the gradient distribution inside the optical path.

[0052] Then, when the working area is divided into grid-shaped integration areas according to the preset resolution and relevant information is obtained, the working area matrix is ​​divided into equally spaced grids according to the preset resolution, and the boundary coordinates and the pixel set contained in each grid are recorded. The continuous deformation field is discretized to facilitate local integral calculation and statistical analysis, thereby improving calculation efficiency and accuracy. For example: the substrate is divided into 248×288 grids, each grid corresponds to a 0.1mm×0.1mm physical area, and optical path 7 covers approximately 40×40 grids, and the edge grids contain more high-amplitude pixels.

[0053] Afterwards, the average value of the gradient amplitude within each grid is calculated. When the results are aggregated by the light path number to generate the preliminary light path micro-deformation vector, the average gradient amplitude is calculated for each grid through its pixel set. The integral values ​​of all grids belonging to the light path are arranged in sequence according to the light path number to generate a vector. The local gradient amplitude is converted into a global deformation energy index through integral calculation, reflecting the cumulative deformation degree of each light path. For example, a grid of light path 7 contains 400 pixels, the average amplitude is 0.06μm, the integral value is 0.06μm, and the value of the corresponding position after aggregation is 0.06, reflecting the contribution of this grid to the deformation of light path 7.

[0054] Next, high-precision reference points are obtained for each optical path area, and the relative error is obtained based on the preliminary optical path micro-deformation and the measured value of the reference point. The reference point is selected in the optical path area to obtain the measured micro-deformation, and the absolute error and relative error between the preliminary result and the measured value are calculated. The accuracy of the preliminary calculation result is evaluated, and the deviation caused by noise or model error is identified to provide a basis for correction. For example, the measured micro-deformation of the reference point of optical path 7 is 1.2μm, and the preliminary calculation result is 1.3μm, with a relative error of 8.3%, which exceeds the preset threshold of 5% and needs to be corrected.

[0055] Finally, when the preliminary optical path micro-deformation is corrected by relative error and the correction result is used as the final optical path micro-deformation, the correction factor is calculated according to the relative error, and the preliminary vector is corrected element by element. The calibration is repeated until the relative errors of all optical paths meet the accuracy requirements, eliminating systematic errors and random noise, improving the accuracy of micro-deformation calculation, and ensuring the reliability of the results.

[0056] In one embodiment, the step of generating deformation error information according to the optical path micro-deformation includes: S509, obtaining abnormal deformation amount and regional parameters according to the optical path micro-deformation amount; S510, obtaining a timestamp of occurrence of the abnormal deformation according to the abnormal deformation amount, and associating it with the corresponding LDI exposure machine optical path number and substrate area preset coordinates to obtain deformation integration information; S511, acquiring grating temperature field distribution data and vibration spectrum data at the time when abnormal deformation occurs according to the regional parameters; S512, generating incentive integration information according to the grating temperature field distribution data and the vibration spectrum data; S513: Obtain deformation error information according to the deformation integration information and the inducement integration information.

[0057] As described in steps S509-S513 above, when the present invention obtains abnormal deformation quantities and regional parameters based on the optical path micro-deformation, it first calculates the global mean and standard deviation of the optical path micro-deformation vector, and uses the 3σ criterion to screen out pixels that exceed the global mean plus 3 times the standard deviation as abnormal deformation quantities. Then, an abnormal region mask is generated through 8-neighborhood connected domain analysis, and regional parameters such as the area, center of mass coordinates, and bounding box of each abnormal region are calculated. The deformation region outside the normal range is located, and the degree of abnormality is quantified to provide a target for subsequent time correlation and inducement analysis.

[0058] Afterwards, the timestamp of the abnormal deformation is obtained according to the abnormal deformation variable and associated with the corresponding information to obtain the deformation integration information. The timestamp is extracted from the abnormal area mask, and the pixel coordinates are converted into physical coordinates through the coordinate mapping function. The optical path number is associated to generate a triple record containing the timestamp, optical path number and physical coordinates. The abnormal deformation is associated with the time and space coordinates to form a complete event record, which supports error tracing and equipment calibration.

[0059] Next, when obtaining the grating temperature field distribution data and vibration spectrum data at the time when the abnormal deformation occurs according to the regional parameters, the thermal image at the corresponding time is extracted from the temperature field database according to the timestamp, and the time domain signal 50ms before and after the timestamp is intercepted from the vibration data buffer. The spectrum is obtained by FFT transformation, and the temperature value and main frequency component corresponding to the coordinates of the abnormal area are extracted to obtain the environmental parameters at the time of the abnormality, providing multi-dimensional data support for analyzing the cause of the deformation.

[0060] Then, when generating the integrated cause information based on the grating temperature field distribution data and vibration spectrum data, the Sobel operator is applied to the temperature field to calculate the temperature gradient vector, and the Pearson correlation coefficient between the temperature and the main vibration frequency within 10 seconds before the anomaly occurs is calculated. The temperature gradient vector, correlation coefficient, and main vibration frequency are integrated into the cause vector to quantify the correlation between environmental factors and deformation, identify the main cause, and provide a basis for compensation strategy.

[0061] Finally, when obtaining deformation error information based on deformation integration information and inducement integration information, the deformation integration information containing timestamp, optical path number, physical coordinates, and deformation amount is merged with the inducement integration information containing temperature gradient, correlation coefficient, and vibration main frequency. The weights of inducements such as temperature gradient and vibration are assigned through the hierarchical analysis method, and deformation error information containing spatiotemporal coordinates, abnormality degree, inducement, and weight is generated. This forms a complete error report, which supports real-time adjustment of LDI equipment and process optimization.

[0062] like Figure 2 As shown, the present invention also provides an LDI exposure machine optical path deformation monitoring system based on multi-modal deformation field analysis, comprising: Monitoring data acquisition module 1, used to obtain substrate multimodal monitoring data, wherein the substrate multimodal monitoring data includes substrate reference point array data, composite calibration pattern, near-infrared image, grating temperature field distribution data and vibration spectrum data; a deformation feature extraction module 2, configured to obtain deformation feature data of a plurality of substrate regions based on the near-infrared image, substrate reference point array data, and a composite calibration pattern; A multimodal fusion module 3, configured to obtain fused deformation feature data based on each of the deformation feature data, the grating temperature field distribution data, and the vibration spectrum data; A micro-deformation analysis module 4 is configured to generate a continuous optical path deformation field corresponding to each substrate region based on the fused deformation feature data; A judgment module 5 is used to obtain a micro deformation of the optical path according to the continuous deformation field of the optical path, and to judge whether the micro deformation of the optical path is greater than a preset value; If it is greater than, the optical path micro-deformation is an abnormal deformation amount, and the corresponding substrate area is judged to be deformed, and deformation error information is generated according to the optical path micro-deformation; If it is not greater than, the optical path micro-deformation is a safe deformation amount, and it is determined that the corresponding substrate area has not been deformed.

[0063] In one embodiment, the deformation feature extraction module 2 includes: An image fusion and registration unit, configured to perform image fusion and registration based on the near-infrared image and the composite calibration pattern to generate a multispectral image; a feature point set generating unit, configured to obtain corner point features of a reference point array based on the multispectral image and substrate reference point array data, and generate an initial feature point set, wherein the initial feature point set includes measured coordinates and preset coordinates; A coordinate offset calculation unit, configured to fit the measured coordinates and the preset coordinates through affine transformation and perform interior point screening to obtain a matching coordinate offset matrix; A global deformation map generating unit, configured to generate a global deformation distribution map of the substrate according to the matching coordinate offset matrix; an optical path region division unit, configured to divide the substrate global deformation distribution map into optical path regions to obtain a plurality of optical path division regions, and divide the substrate into a plurality of substrate optical path regions according to the plurality of optical path division regions; The deformation characteristic calculation unit is used to obtain deformation characteristic data according to each substrate optical path area, wherein the deformation characteristic data includes regional average displacement, regional maximum strain and regional deformation variance.

[0064] In one embodiment, the multimodal fusion module 3 includes: A spatial grid construction unit, configured to establish a spatial reference grid based on the substrate reference point array data; a time-space synchronization calibration unit, configured to perform time-synchronization calibration on the deformation feature data, grating temperature field distribution data, and vibration spectrum data, and map the calibrated deformation feature data, grating temperature field distribution data, and vibration spectrum data to the spatial reference grid to form time-space aligned multimodal data; a correlation feature calculation unit, configured to obtain deformation-vibration correlation feature data and optical path-vibration correlation feature data based on the multimodal data; A first key feature extraction unit, configured to obtain a first key feature component according to the deformation-vibration correlation feature data; A second key feature extraction unit, configured to obtain a second key feature component according to the optical path-vibration correlation feature data; a feature fusion unit, configured to perform time alignment and normalization processing on the first key feature component and the second key feature component and then fuse them to construct a fused feature component; The deformation field prediction unit is used to predict the deformation field distribution based on the fused feature components through a long short-term memory network, and generate fused deformation feature data according to the optical path area division.

[0065] In one embodiment, the multimodal fusion module 4 includes: a sensitive area identification unit, configured to obtain a deformation sensitive area according to the fused deformation feature data, and map the deformation sensitive area to a corresponding optical path number; an image preprocessing unit, configured to extract a deformed optical path region image from the near-infrared image based on the optical path number, and perform adaptive histogram equalization and Gaussian filtering preprocessing on the deformed optical path region image to obtain a preprocessed deformed optical path region image; A multimodal data set construction unit is used to obtain a deformation light path area image sequence, near-infrared temperature gradient field data and vibration spectrum data within a preset sampling period according to each preprocessed deformation light path area image, and generate a multimodal data set; A local deformation calculation unit is used to fuse the temperature gradient and vibration amplitude features of the multimodal data set to obtain a local position-related deformation; A continuous deformation field generating unit, configured to generate a continuous deformation field according to a local position-correlated deformation amount of each pre-processed deformed optical path region image; The optical path deformation field segmentation unit is used to obtain a preset optical path area division mask, and segment the continuous deformation field according to the preset optical path area division mask to generate an optical path continuous deformation field.

[0066] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, value library or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and RAM bus dynamic RAM (RDRAM).

[0067] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, apparatus, article, or method comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, apparatus, article, or method. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, apparatus, article, or method comprising the element.

[0068] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent results or equivalent process transformations made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for monitoring optical path deformation of an LDI exposure machine based on multimodal deformation field analysis, applied to an LDI exposure machine, characterized in that: include: Acquiring substrate multimodal monitoring data, wherein the substrate multimodal monitoring data includes substrate reference point array data, composite calibration pattern, near-infrared image, grating temperature field distribution data, and vibration spectrum data; acquiring deformation feature data of a plurality of substrate regions according to the near-infrared image, substrate reference point array data, and a composite calibration pattern; Acquire fused deformation feature data based on each of the deformation feature data, the grating temperature field distribution data, and the vibration spectrum data; generating a light path continuous deformation field corresponding to each substrate region according to the fused deformation feature data; Obtaining a micro-deformation of the optical path according to the continuous deformation field of the optical path, and determining whether the micro-deformation of the optical path is greater than a preset value; If it is greater than, the optical path micro-deformation is an abnormal deformation amount, and the corresponding substrate area is judged to be deformed, and deformation error information is generated according to the optical path micro-deformation; If it is not greater than, the optical path micro-deformation is a safe deformation amount, and it is determined that the corresponding substrate area has not been deformed.

2. The method for monitoring optical path deformation of an LDI exposure machine based on multimodal deformation field analysis according to claim 1, characterized in that: The step of acquiring deformation feature data of multiple substrate regions based on the near-infrared image, substrate reference point array data, and the composite calibration pattern comprises: Performing image fusion and registration based on the near-infrared image and the composite calibration pattern to generate a multispectral image; Acquire corner point features of the reference point array based on the multispectral image and the substrate reference point array data, and generate an initial feature point set, wherein the initial feature point set includes measured coordinates and preset coordinates; Fitting the measured coordinates and the preset coordinates through affine transformation, and performing interior point screening to obtain a matching coordinate offset matrix; generating a global deformation distribution map of the substrate according to the matching coordinate offset matrix; Performing optical path region division on the substrate global deformation distribution map to obtain a plurality of optical path division regions, and dividing the substrate into a plurality of substrate optical path regions according to the plurality of optical path division regions; Deformation characteristic data is acquired according to each substrate optical path region, wherein the deformation characteristic data includes regional average displacement, regional maximum strain and regional deformation variance.

3. The method for monitoring optical path deformation of an LDI exposure machine based on multimodal deformation field analysis according to claim 1, characterized in that: The step of obtaining fused deformation feature data based on each of the deformation feature data, grating temperature field distribution data, and vibration spectrum data comprises: establishing a spatial reference grid based on the substrate reference point array data; Performing time synchronization calibration on the deformation feature data, grating temperature field distribution data, and vibration spectrum data, and mapping the calibrated deformation feature data, grating temperature field distribution data, and vibration spectrum data to the spatial reference grid to form spatiotemporally aligned multimodal data; Acquiring deformation-vibration correlation feature data and optical path-vibration correlation feature data according to the multimodal data; Acquire a first key characteristic component according to the deformation-vibration correlation characteristic data; Acquire a second key characteristic component according to the optical path-vibration correlation characteristic data; Performing time alignment and normalization on the first key feature component and the second key feature component and then fusing them to construct a fused feature component; The deformation field distribution is predicted based on the fused feature components through a long short-term memory network, and fused deformation feature data is generated according to the light path area division.

4. The method for monitoring optical path deformation of an LDI exposure machine based on multimodal deformation field analysis according to claim 1, characterized in that: The step of generating a light path continuous deformation field corresponding to each substrate area according to the fused deformation feature data includes: Acquire a deformation sensitive area according to the fused deformation feature data, and map the deformation sensitive area to a corresponding optical path number; extracting a deformed optical path region image from the near-infrared image based on the optical path number, and performing adaptive histogram equalization and Gaussian filtering preprocessing on the deformed optical path region image to obtain a preprocessed deformed optical path region image; Acquire a deformation light path region image sequence, near-infrared temperature gradient field data, and vibration spectrum data within a preset sampling period according to each preprocessed deformation light path region image, and generate a multimodal data set; fusing temperature gradient and vibration amplitude features of the multimodal data set to obtain a local position-related deformation variable; Generate a continuous deformation field according to the local position-correlated deformation amount of each pre-processed deformed optical path area image; A preset light path region division mask is obtained, and the continuous deformation field is divided according to the preset light path region division mask to generate a light path continuous deformation field.

5. The method for monitoring optical path deformation of an LDI exposure machine based on multimodal deformation field analysis according to claim 1, characterized in that: The step of obtaining the light path micro-deformation amount according to the light path continuous deformation field includes: Get the working area matrix and light path stitching mask matrix; Obtaining a gradient vector field for the optical path continuous deformation field; Obtaining a magnitude matrix according to the gradient vector field; dividing the amplitude matrix into gradient change amplitude matrices of each light path according to the light path splicing mask matrix; Divide the working area into grid-shaped integration areas according to a preset resolution, and obtain the boundary coordinates and pixel set contained in each grid; Calculate the average value of the gradient amplitude in each grid, aggregate the results by optical path number, and generate the preliminary optical path micro-deformation vector; Obtaining high-precision reference points in each optical path area, and obtaining relative errors based on the preliminary optical path micro-deformation and the measured values ​​of the reference points; The preliminary optical path micro-deformation is corrected by relative error, and the correction result is used as the final optical path micro-deformation.

6. The method for monitoring optical path deformation of an LDI exposure machine based on multimodal deformation field analysis according to claim 1, characterized in that: The step of generating deformation error information according to the optical path micro-deformation comprises: Obtaining abnormal deformation amount and regional parameters according to the optical path micro-deformation amount; Obtaining a timestamp of the occurrence of the abnormal deformation according to the abnormal deformation amount, and associating it with the corresponding LDI exposure machine optical path number and substrate area preset coordinates to obtain deformation integration information; Acquire the grating temperature field distribution data and vibration spectrum data at the time of abnormal deformation according to the regional parameters; generating inducement integration information according to the grating temperature field distribution data and the vibration spectrum data; Deformation error information is obtained according to the deformation integration information and the inducement integration information.

7. An LDI exposure machine optical path deformation monitoring system based on multi-modal deformation field analysis, characterized in that: include: A monitoring data acquisition module is used to obtain substrate multimodal monitoring data, wherein the substrate multimodal monitoring data includes substrate reference point array data, composite calibration pattern, near-infrared image, grating temperature field distribution data and vibration spectrum data; a deformation feature extraction module, configured to obtain deformation feature data of a plurality of substrate regions based on the near-infrared image, substrate reference point array data, and a composite calibration pattern; A multimodal fusion module, configured to obtain fused deformation feature data based on each of the deformation feature data, grating temperature field distribution data, and vibration spectrum data; A micro-deformation analysis module is used to generate a continuous optical path deformation field corresponding to each substrate area according to the fused deformation feature data; a judgment module, configured to obtain a micro-deformation of the optical path according to the continuous deformation field of the optical path, and to judge whether the micro-deformation of the optical path is greater than a preset value; If it is greater than, the optical path micro-deformation is an abnormal deformation amount, and the corresponding substrate area is judged to be deformed, and deformation error information is generated according to the optical path micro-deformation; If it is not greater than, the optical path micro-deformation is a safe deformation amount, and it is determined that the corresponding substrate area has not been deformed.

8. The LDI exposure machine optical path deformation monitoring system based on multi-modal deformation field analysis according to claim 7, characterized in that: The deformation feature extraction module includes: An image fusion and registration unit, configured to perform image fusion and registration based on the near-infrared image and the composite calibration pattern to generate a multispectral image; a feature point set generating unit, configured to obtain corner point features of a reference point array based on the multispectral image and substrate reference point array data, and generate an initial feature point set, wherein the initial feature point set includes measured coordinates and preset coordinates; A coordinate offset calculation unit, configured to fit the measured coordinates and the preset coordinates through affine transformation and perform interior point screening to obtain a matching coordinate offset matrix; A global deformation map generating unit, configured to generate a global deformation distribution map of the substrate according to the matching coordinate offset matrix; an optical path region division unit, configured to divide the substrate global deformation distribution map into optical path regions to obtain a plurality of optical path division regions, and divide the substrate into a plurality of substrate optical path regions according to the plurality of optical path division regions; The deformation characteristic calculation unit is used to obtain deformation characteristic data according to each substrate optical path area, wherein the deformation characteristic data includes regional average displacement, regional maximum strain and regional deformation variance.

9. The LDI exposure machine optical path deformation monitoring system based on multi-modal deformation field analysis according to claim 7, characterized in that: The multimodal fusion module includes: A spatial grid construction unit, configured to establish a spatial reference grid based on the substrate reference point array data; a time-space synchronization calibration unit, configured to perform time-synchronization calibration on the deformation feature data, grating temperature field distribution data, and vibration spectrum data, and map the calibrated deformation feature data, grating temperature field distribution data, and vibration spectrum data to the spatial reference grid to form time-space aligned multimodal data; a correlation feature calculation unit, configured to obtain deformation-vibration correlation feature data and optical path-vibration correlation feature data based on the multimodal data; A first key feature extraction unit, configured to obtain a first key feature component according to the deformation-vibration correlation feature data; A second key feature extraction unit, configured to obtain a second key feature component according to the optical path-vibration correlation feature data; a feature fusion unit, configured to perform time alignment and normalization processing on the first key feature component and the second key feature component and then fuse them to construct a fused feature component; The deformation field prediction unit is used to predict the deformation field distribution based on the fused feature components through a long short-term memory network, and generate fused deformation feature data according to the optical path area division.

10. The LDI exposure machine optical path deformation monitoring system based on multi-modal deformation field analysis according to claim 7, characterized in that: The multimodal fusion module includes: a sensitive area identification unit, configured to obtain a deformation sensitive area according to the fused deformation feature data, and map the deformation sensitive area to a corresponding optical path number; an image preprocessing unit, configured to extract a deformed optical path region image from the near-infrared image based on the optical path number, and perform adaptive histogram equalization and Gaussian filtering preprocessing on the deformed optical path region image to obtain a preprocessed deformed optical path region image; A multimodal data set construction unit is used to obtain a deformation light path area image sequence, near-infrared temperature gradient field data and vibration spectrum data within a preset sampling period according to each preprocessed deformation light path area image, and generate a multimodal data set; A local deformation calculation unit is used to fuse the temperature gradient and vibration amplitude features of the multimodal data set to obtain a local position-related deformation; A continuous deformation field generating unit, configured to generate a continuous deformation field according to a local position-correlated deformation amount of each pre-processed deformed optical path region image; The optical path deformation field segmentation unit is used to obtain a preset optical path area division mask, and segment the continuous deformation field according to the preset optical path area division mask to generate an optical path continuous deformation field.

Images