Method, system and device for three-dimensional topography measurement of a semiconductor based on displacement imaging

Abstract

The application relates to the technical field of optical measurement, in particular to a semiconductor three-dimensional topography measurement method, system and device based on displacement imaging, wherein the method comprises the following steps: projecting a coaxial grating pattern with a projection imaging focus plane coinciding with a telecentric camera focus plane; collecting a multi-focus image sequence along an optical axis direction; calculating the focusing degree at different focus positions according to a computable texture or a natural image texture formed by the grating pattern; inputting the multi-focus image sequence into a pre-trained height prediction model to obtain a discrete height map; extracting corresponding focus function values based on the discrete height map to form a focus curve, and determining a final height value through barycentric calculation to output a three-dimensional topography measurement result. The application can improve the accuracy, continuity and production line applicability of three-dimensional topography measurement of displacement imaging under the condition of a complex semiconductor surface.

Description

Technical Field

[0001] This application relates to the field of optical measurement technology, and in particular to a method, system and device for measuring the three-dimensional topography of semiconductors based on displacement imaging. Background Technology

[0002] With the continuous advancement of semiconductor manufacturing processes, chip packaging density is increasing daily, and the dimensions of interconnect and support structures such as gold wire bonding, solder ball welding, metal pads, and chip edges are constantly shrinking, placing higher demands on semiconductor packaging inspection technology. Three-dimensional topography measurement can acquire the height distribution and spatial contour of the semiconductor surface under test, which can be used to determine whether there are height anomalies or topographic defects in structures such as gold wires, solder balls, pads, and chip surfaces. Therefore, it has significant application value in the field of semiconductor inspection.

[0003] In existing technologies, displacement imaging is a commonly used non-contact method for measuring three-dimensional topography. This method typically controls a camera or imaging component to move along the optical axis, acquiring multiple frames of images at different focal positions to form a multi-focus image sequence. For the same pixel location or local region in the image, the degree of focus at different focal positions is calculated based on methods such as neighborhood convolution, information entropy, Laplacian operator, or variance function. The optimal focal position is then determined based on the response result of the focus degree changing with the focal position. Since different focal positions correspond to the amount of camera displacement along the optical axis, the optimal focal position can be converted into the height value of the corresponding location on the surface under test. The height values ​​of multiple pixel locations together form a height map of the semiconductor surface under test, thus completing the three-dimensional topography reconstruction. For areas with rich surface texture and significant grayscale variations, the above method can obtain a relatively stable focusing response, thus achieving good measurement results.

[0004] However, semiconductor package surfaces typically contain mirrored metal regions, weakly textured regions, rough substrate regions, and elongated structures such as gold wires. The reflectivity and texture intensity of these different regions vary significantly. For weakly textured regions and mirrored, textureless regions, the lack of sufficiently stable grayscale variation information in the image leads to unclear focus response, making it easy to misjudge the optimal focus position. This results in deviations in the height values ​​of the corresponding regions and reduces the accuracy of 3D topography measurement. For elongated structures like gold wires, their surface reflection is easily affected by lighting conditions. When gold wires cross areas with significant background variations, such as chip edges, focus calculations are easily interfered with by background differences, causing errors or omissions in the height points corresponding to the gold wires, resulting in broken lines in the 3D reconstruction results. Meanwhile, while deep learning models can improve the reconstruction quality of complex regions, training samples typically require extensive manual annotation, leading to high sample construction costs. Furthermore, the model inference speed can also affect high-speed inspection in semiconductor production lines. Therefore, improving the accuracy, continuity, and production line applicability of displacement imaging 3D topography measurement under complex semiconductor surface conditions is a pressing issue that needs to be addressed in this field. Summary of the Invention

[0005] This application provides a method, system, and device for measuring the three-dimensional topography of semiconductors based on displacement imaging, which can improve the accuracy, continuity, and production line applicability of displacement imaging three-dimensional topography measurement under complex semiconductor surface conditions. This application provides the following technical solution: In a first aspect, this application provides a method for measuring the three-dimensional topography of a semiconductor based on displacement imaging, the method comprising: Control the coaxial grating to project a grating pattern onto the surface of the semiconductor under test, and make the imaging focal plane of the coaxial grating coincide with the focal plane of the telecentric camera; The telecentric camera is controlled to move along the optical axis to acquire a multi-focus image sequence of the semiconductor surface under test at different focusing positions; The degree of focus of the semiconductor surface under test at different focus positions is calculated based on the multi-focus image sequence. For the mirror surface and metal surface of the semiconductor surface under test, the degree of focus is calculated based on the computable texture formed by the grating pattern. For the rough surface and gold line area of ​​the semiconductor surface under test, the degree of focus is calculated based on the natural image texture. The multi-focus image sequence is input into a pre-trained height prediction model to obtain a discrete height map of the semiconductor surface under test. Based on the height value indicated by the discrete height map, the focus function value corresponding to the height value is extracted from the focus degree and a focus curve is formed. Based on the focus function value corresponding to each height position in the focus curve, the centroid is calculated to determine the final height value of the semiconductor surface under test. Based on the final height value, three-dimensional reconstruction is performed, and the three-dimensional topography measurement result of the semiconductor surface under test is output.

[0006] In one specific implementation, the method is applied to a semiconductor inspection system, which includes a controller, a telecentric camera, a multilayer RGB light source, a coaxial grating, and a grating ruler. The controller is communicatively connected to the telecentric camera, the multilayer RGB light source, the coaxial grating, and the grating ruler, respectively. The coaxial grating is disposed in the coaxial illumination optical path of the telecentric camera. The coaxial grating is a transmissive grating or a reflective grating, and the coaxial grating is composed of periodically arranged bright and dark stripes. The multi-layer RGB light source includes a high-angle light source, a medium-angle light source, a low-angle light source, and an RGB point coaxial light source. The high-angle light source is used to illuminate the top area of ​​the gold wire, the medium-angle light source is used to illuminate the side wall area of ​​the gold wire, the low-angle light source is used to illuminate the surface area of ​​the substrate, and the RGB point coaxial light source is disposed in the optical path of the coaxial grating and is used to cooperate with the coaxial grating to provide coaxial illumination for the mirror area.

[0007] In one specific implementation, controlling the telecentric camera to move along the optical axis and acquiring a multi-focus image sequence of the semiconductor surface under test at different focusing positions includes: The telecentric camera is controlled to move continuously along the optical axis, and the displacement of the telecentric camera is measured in real time by the grating ruler. The pulse signal output by the grating ruler is connected to the counting module of the controller, and the counting module counts the number of pulses of the pulse signal. When the number of pulses reaches a preset trigger threshold, the controller sends a trigger signal to the telecentric camera, causing the telecentric camera to acquire an image frame at the current position. The displacement interval of the telecentric camera when acquiring images is adjusted by adjusting the trigger threshold, so as to acquire multiple frames of images at different focusing positions and form the multi-focus image sequence.

[0008] In one specific implementation, calculating the degree of focus of the semiconductor surface under test at different focusing positions based on the multi-focus image sequence includes: The images in the multi-focus image sequence are smoothed and denoised to obtain denoised images; High-frequency components are extracted from the denoised image based on the Laplacian operator or variance function to obtain the degree of focus of the semiconductor surface under test at different focus positions. The degree of focus is represented by the focus function value at the corresponding focus position. For the mirror surface and the metal surface in the semiconductor surface under test, the computable texture formed by the grating pattern in the image is extracted, and the contrast or gradient magnitude of the computable texture is calculated. Based on the contrast or gradient magnitude, the focus function values ​​of the mirror surface and the metal surface at different focus positions are determined. For the rough surface and gold wire region in the semiconductor surface under test, the natural image texture or structural edge in the image is extracted, and the focus function value of the rough surface and the gold wire region at different focus positions is determined based on the natural image texture or the structural edge. At the same pixel location, the image frames corresponding to different focal positions in the multi-focus image sequence are traversed to obtain the degree of focus of the focal function value of the pixel location as the focal position changes.

[0009] In one specific implementation, the pre-trained height prediction model is a height prediction model built based on a variational autoencoder and an attention mechanism U-Net network. The step of inputting the multi-focus image sequence into the pre-trained height prediction model to obtain a discrete height map of the semiconductor surface under test includes: The multi-focus image sequence is input into the encoding part of the variational autoencoder, and the multi-focus image sequence is compressed and encoded by the encoding part to obtain latent space features; The latent space features are input into the U-Net attention mechanism network, and the U-Net attention mechanism network is used to extract and reconstruct the latent space features. The U-Net attention mechanism network includes an encoding path, a decoding path, and an attention gating module set between the encoding path and the decoding path. The attention gating module receives the feature map output from the encoding path and the upsampled feature map from the decoding path, and generates a spatial attention weight map based on the feature map output from the encoding path and the upsampled feature map from the decoding path. The feature map output by the encoding path is spatially attention-weighted according to the spatial attention weight map to enhance the focus on relevant regions and suppress irrelevant background regions; The feature map output by the U-Net network with the attention mechanism is input into the decoding part of the variational autoencoder, and the decoding part performs decoding and reconstruction, and outputs the discrete height map through the activation function.

[0010] In one specific implementation, the pre-trained height prediction model is trained through the following steps: Multiple sets of original multifocus image sequences are acquired, and the initial focus degree of each set of original multifocus image sequences is calculated using the Laplacian algorithm to obtain the initial height map; The pyramid algorithm is used to perform multi-scale analysis on the initial height map, and the consistency of the results is evaluated based on the degree of focus at different resolution levels to identify suspicious areas in the initial height map. The original multi-focus image sequence, the initial height map, and the marking information of the suspicious areas are displayed using a manual annotation tool. Based on the changes in sharpness of the original multi-focus image sequence at each focal position, the erroneous points in the suspicious areas are corrected to obtain the corrected height map. The original multi-focus image sequence and the corresponding corrected height map are cropped to establish an initial training sample set; The initial training sample set is rotated and mirrored to expand it, resulting in an expanded sample set. The high-precision prediction model, constructed using a U-Net network based on a variational autoencoder and attention mechanism, is trained based on the expanded sample set.

[0011] In one specific implementation, the step of extracting the focus function values ​​corresponding to the height values ​​near the height values ​​indicated by the discrete height map and forming a focus curve based on the focus degree, and performing centroid calculation based on the focus function values ​​corresponding to each height position in the focus curve to determine the final height value of the semiconductor surface under test, includes: The discrete height level corresponding to each pixel position is determined based on the discrete height map, and the corresponding height value is determined based on the discrete height level. Using the height value as the center, determine the height search range among multiple focal positions corresponding to the multi-focus image sequence; The focus function value is extracted from the focus degree at each focus position corresponding to the height search range, and the focus curve is formed according to the height order of the focus position; Based on the focus function values ​​corresponding to each height position in the focus curve, the centroid is calculated, and the calculated centroid position is determined as the final height value of the corresponding pixel position on the surface of the semiconductor under test. The height distribution of the semiconductor surface under test is generated based on the final height value of each pixel position, and three-dimensional reconstruction is performed based on the height distribution to output the three-dimensional topography measurement results.

[0012] Secondly, this application provides a semiconductor three-dimensional topography measurement system based on displacement imaging, which adopts the following technical solution: A semiconductor three-dimensional topography measurement system based on displacement imaging includes: A grating projection module is used to control a coaxial grating to project a grating pattern onto the surface of a semiconductor under test, and to make the imaging focal plane of the coaxial grating coincide with the focal plane of a telecentric camera. The image acquisition module is used to control the telecentric camera to move along the optical axis and acquire multi-focus image sequences of the semiconductor surface under test at different focusing positions; The focusing calculation module is used to calculate the degree of focusing of the semiconductor surface under test at different focusing positions based on the multi-focus image sequence. Specifically, for the mirror surface and metal surface of the semiconductor surface under test, the degree of focusing is calculated based on the computable texture formed by the grating pattern; for the rough surface and gold line region of the semiconductor surface under test, the degree of focusing is calculated based on the natural image texture. The height generation module is used to input the multi-focus image sequence into a pre-trained height prediction model to obtain a discrete height map of the semiconductor surface under test. The 3D reconstruction module is used to extract the focus function value corresponding to the height value near the height value from the focus degree based on the height value indicated by the discrete height map and form a focus curve; to perform centroid calculation based on the focus function value corresponding to each height position in the focus curve to determine the final height value of the semiconductor surface under test; to perform 3D reconstruction based on the final height value and output the 3D topography measurement result of the semiconductor surface under test.

[0013] Thirdly, this application provides an electronic device, the device including a processor and a memory; the memory stores a program, the program being loaded and executed by the processor to implement a semiconductor three-dimensional topography measurement method based on displacement imaging as described in the first aspect.

[0014] Fourthly, this application provides a computer-readable storage medium storing a program that, when executed by a processor, is used to implement a semiconductor three-dimensional topography measurement method based on displacement imaging as described in the first aspect.

[0015] First, a coaxial grating is projected onto the surface of the semiconductor under test, and the imaging focal plane of the coaxial grating is aligned with the focal plane of the telecentric camera. Then, the telecentric camera is moved along the optical axis to acquire a multi-focus image sequence of the semiconductor surface at different focal positions. Subsequently, the degree of focus of the semiconductor surface at different focal positions is calculated based on the multi-focus image sequence. The degree of focus for mirror surfaces and metal surfaces is calculated based on the computable texture formed by the grating pattern, while the degree of focus for rough surfaces and gold line regions is calculated based on the natural image texture. The multi-focus image sequence is then input into a pre-trained height prediction model to obtain a discrete height map of the semiconductor surface. Finally, based on the height value indicated by the discrete height map, the focus function value corresponding to the height value is extracted from the degree of focus and a focus curve is formed. The centroid is calculated based on the focus function value corresponding to each height position in the focus curve to determine the final height value of the semiconductor surface. Based on the final height value, three-dimensional reconstruction is performed, and the three-dimensional topography measurement results are output. Therefore, this application does not simply rely on existing textures in the image for focus determination. Instead, it achieves this by aligning the imaging focusing plane of the coaxial grating with the focusing plane of the telecentric camera. This allows the grating pattern to generate a calculable sharpness response as the focus position changes, thus enabling stable focus levels for both mirror and metallic surfaces. Simultaneously, the focus level for rough surfaces and gold-lined areas can still be calculated using natural image textures, allowing different surface types to be included in the same height measurement process. Furthermore, the discrete height map first provides the height range, and the focus curve then performs centroid calculation near this height value. This ensures that the final height value depends not only on a single focus peak but is determined by multiple focus function values ​​near the height, thereby improving the accuracy, stability, and continuity of the three-dimensional morphology measurement results of the semiconductor surface under test.

[0016] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0017] Figure 1 This is a schematic flowchart of the semiconductor three-dimensional topography measurement method based on displacement imaging in the embodiments of this application.

[0018] Figure 2 This is a schematic diagram of the overall process of the semiconductor three-dimensional topography measurement method based on displacement imaging in the embodiments of this application.

[0019] Figure 3 This is a structural block diagram of the semiconductor three-dimensional topography measurement system based on displacement imaging in the embodiments of this application.

[0020] Figure 4 This is a block diagram of an electronic device for semiconductor three-dimensional topography measurement based on displacement imaging, as described in this application. Detailed Implementation

[0021] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0022] Optionally, this application uses the semiconductor three-dimensional topography measurement method based on displacement imaging provided in various embodiments as an example for description in electronic devices. The electronic device is a terminal or server. The terminal can be a computer, tablet computer, etc. This embodiment does not limit the type of electronic device.

[0023] This application provides a semiconductor three-dimensional topography measurement method based on displacement imaging, applied to a semiconductor inspection system. The semiconductor inspection system includes a controller, a telecentric camera, a multilayer RGB light source, a coaxial grating, and a grating ruler. The controller is communicatively connected to the telecentric camera, the multilayer RGB light source, the coaxial grating, and the grating ruler. The controller controls the coaxial grating to project a grating pattern onto the surface of the semiconductor under test, controls the multilayer RGB light source to illuminate the surface according to a preset timing sequence, controls the telecentric camera to move along the optical axis and acquire images, and also receives the displacement measurement signal output by the grating ruler. The telecentric camera is used to acquire images of the semiconductor surface under test at different focal positions. The telecentric camera adopts a telecentric optical design, which can reduce imaging distortion and maintain stable magnification, making it suitable for three-dimensional topography measurement of semiconductor surfaces.

[0024] Specifically, a coaxial grating is placed in the coaxial illumination path of the telecentric camera. The coaxial grating is either a transmissive or reflective grating, and it consists of periodically arranged bright and dark fringes. The imaging focal plane of the coaxial grating coincides with the focal plane of the telecentric camera. When the telecentric camera focuses on a certain height position on the surface of the semiconductor under test, the grating pattern projected by the coaxial grating onto the semiconductor surface is clearly imaged at that height position. When the telecentric camera moves along the optical axis and deviates from this height position, the grating pattern gradually blurs. Therefore, the clarity of the grating pattern corresponds to the focusing state of the telecentric camera, allowing the grating pattern to participate in the focus calculation as a computable texture in the image.

[0025] Furthermore, the projected light from the coaxial grating exhibits different reflection characteristics on different surface materials. For mirrored and metallic surfaces within the semiconductor under test, the projected light from the coaxial grating reflects more strongly, resulting in a clearer grating pattern in the image acquired by the telecentric camera. This grating pattern forms a computable texture, allowing the focus level to be calculated even on mirrored and metallic surfaces lacking natural texture. For rough surfaces and gold-line regions within the semiconductor under test, the projected light from the coaxial grating reflects less strongly, and the grating pattern may be invisible or have low contrast in the image. However, rough surfaces and gold-line regions typically possess natural image textures or structural edges, thus allowing the focus level to be calculated based on these natural image textures or structural edges.

[0026] Meanwhile, the multi-layer RGB light source includes a high-angle light source, a medium-angle light source, a low-angle light source, and an RGB point coaxial light source. The high-angle light source illuminates the top area of ​​the gold wire, the medium-angle light source illuminates the sidewall area of ​​the gold wire, and the low-angle light source illuminates the surface area of ​​the substrate. The RGB point coaxial light source is placed in the optical path of the coaxial grating and works with the coaxial grating to provide coaxial illumination for the mirror area. The controller controls the high-angle light source, medium-angle light source, low-angle light source, and RGB point coaxial light source to work in a time-division manner according to a preset timing sequence, so that the top area of ​​the gold wire, the sidewall area of ​​the gold wire, the surface area of ​​the substrate, and the mirror area obtain appropriate lighting conditions.

[0027] Through the above setup, the semiconductor testing system establishes a collaborative relationship between the controller, telecentric camera, multi-layer RGB light source, coaxial grating, and grating ruler at the hardware level; at the imaging level, the imaging focal plane of the coaxial grating coincides with the focal plane of the telecentric camera, so that the clarity of the grating pattern can reflect the focusing state; at the illumination level, the multi-layer RGB light source illuminates the top area of ​​the gold wire, the sidewall area of ​​the gold wire, the surface area of ​​the substrate, and the mirror area respectively, so that the mirror surface, metal surface, rough surface, and gold wire area of ​​the semiconductor under test can all obtain image information for displacement imaging three-dimensional topography measurement.

[0028] Reference Figure 1 This is a flowchart illustrating a semiconductor three-dimensional topography measurement method based on displacement imaging according to an embodiment of this application. The method includes at least the following steps: Step S101: Control the coaxial grating to project the grating pattern onto the surface of the semiconductor under test, and make the imaging focal plane of the coaxial grating coincide with the focal plane of the telecentric camera.

[0029] In step S101, the controller controls the coaxial grating to project a grating pattern onto the surface of the semiconductor under test, and configures the optical path so that the imaging focal plane of the coaxial grating coincides with the focal plane of the telecentric camera. When the semiconductor surface under test has different positions along the height direction, the telecentric camera focuses on one of the height positions, and the grating pattern projected by the coaxial grating forms a clear pattern at that height position; when the telecentric camera deviates from this height position, the grating pattern gradually becomes blurred as the degree of defocus increases. Thus, the clarity of the grating pattern corresponds to the focusing state of the telecentric camera on the semiconductor surface under test.

[0030] Specifically, the imaging focal plane of the coaxial grating coincides with the focal plane of the telecentric camera, so that the grating pattern does not simply exist as an illumination background, but can present high contrast and obvious edge changes at the focal position, and low contrast and weak edge changes at the defocus position. For the mirrored and metallic surfaces of the semiconductor under test, which have little natural image texture, the grating pattern projected by the coaxial grating can form a computable texture in the image, enabling the mirrored and metallic surfaces to obtain an image texture response that varies with the focal position.

[0031] Furthermore, for rough surfaces and gold line regions, the reflection of coaxial grating projection light is relatively weak in these areas, making the grating pattern invisible or having low contrast. However, rough surfaces and gold line regions typically possess natural image textures or structural edges. Step S101 controls the coaxial grating projection and defines the overlap relationship of the imaging focal plane, enabling the mirror surface and metal surface to obtain a computable texture formed by the grating pattern, while preserving the natural image textures or structural edge information of the rough surfaces and gold line regions themselves. This ensures that the semiconductor surface under test has image content that can be used to characterize the focusing state in different material regions.

[0032] Step S102: Control the telecentric camera to move along the optical axis and acquire multi-focus image sequences of the semiconductor surface under test at different focusing positions.

[0033] In step S102, the controller controls the telecentric camera to move along the optical axis, enabling the telecentric camera to acquire images of the semiconductor surface under test at multiple different focusing positions. The positional changes of the telecentric camera along the optical axis correspond to different height focusing positions on the semiconductor surface under test. Therefore, the multiple frames acquired at different focusing positions can form a multi-focus image sequence of the semiconductor surface under test, with each frame in the multi-focus image sequence corresponding to a specific focusing position.

[0034] Specifically, the telecentric camera is controlled to move continuously along the optical axis, and the displacement of the telecentric camera is measured in real time using a grating ruler. The grating ruler, as a displacement measuring element, outputs pulse signals corresponding to the displacement changes during the telecentric camera's movement. The pulse signals output by the grating ruler are connected to the counting module of the controller, which counts the number of pulses, enabling the controller to determine the current position change of the telecentric camera along the optical axis based on the number of pulses.

[0035] Furthermore, when the number of pulses reaches a preset trigger threshold, the controller sends a trigger signal to the telecentric camera, causing the telecentric camera to acquire an image frame at the current position. The telecentric camera completes image acquisition during continuous movement, forming a rapid-fire image acquisition method. By triggering image acquisition with grating ruler pulses, each image frame corresponds to a specific displacement position, reducing focus position errors caused by manually setting the acquisition time or fluctuations in movement speed.

[0036] Simultaneously, the displacement interval of the telecentric camera during image acquisition can be adjusted by changing the trigger threshold, allowing for the acquisition of multiple frames at different focal positions and the formation of a multi-focus image sequence. For semiconductor products under test with gradual thickness changes, a larger trigger threshold can be set, resulting in a larger displacement interval between adjacent image frames, thereby reducing the number of acquisition frames and increasing the detection speed. For semiconductor products under test with drastic thickness changes, a smaller trigger threshold can be set, resulting in a smaller displacement interval between adjacent image frames, thereby increasing the number of acquisition frames and improving the sampling resolution in the vertical direction. The trigger threshold can be configured via software according to the detection resolution requirements without replacing the hardware.

[0037] In the above manner, step S102 can achieve displacement-triggered image acquisition by using grating ruler pulse counting during the continuous movement of the telecentric camera along the optical axis, so that the semiconductor surface under test can form a multi-focus image sequence with clear position and adjustable interval at different focusing positions.

[0038] Step S103: Calculate the degree of focus of the semiconductor surface under test at different focus positions based on the multi-focus image sequence. For the mirror surface and metal surface of the semiconductor surface under test, the degree of focus is calculated based on the computable texture formed by the grating pattern. For the rough surface and gold line area of ​​the semiconductor surface under test, the degree of focus is calculated based on the natural image texture.

[0039] In step S103, the degree of focus of the semiconductor surface under test at different focusing positions is calculated based on the multi-focus image sequence. The images in the multi-focus image sequence are smoothed and denoised to obtain denoised images; high-frequency components are extracted from the denoised images based on the Laplacian operator or variance function to obtain the degree of focus of the semiconductor surface under test at different focusing positions, and the degree of focus is represented by the focusing function value at the corresponding focusing position.

[0040] Specifically, for the mirror surface and metal surface of the semiconductor under test, the computable texture formed by the grating pattern in the image is extracted, and the contrast or gradient magnitude of the computable texture is calculated. Based on the contrast or gradient magnitude, the focus function values ​​of the mirror surface and metal surface at different focus positions are determined. For the rough surface and gold line region of the semiconductor under test, the natural image texture or structural edge in the image is extracted, and the focus function values ​​of the rough surface and gold line region at different focus positions are determined based on the natural image texture or structural edge. At the same pixel position, the image frames corresponding to different focus positions in the multi-focus image sequence are traversed to obtain the degree of focus of the focus function value at that pixel position as the focus position changes.

[0041] Specifically, the images in the multi-focus image sequence are smoothed and denoised to obtain the denoised image. The smoothing and denoising process can employ a Gaussian filter to remove thermal noise and shot noise introduced by the camera sensor. The two-dimensional Gaussian kernel function of the Gaussian filter is: ; in, It is a two-dimensional Gaussian kernel function. This represents the coordinate offset of a pixel within the neighborhood. The standard deviation of the Gaussian filter. Used to control the smoothing level. The Gaussian filter is convolved with the image to obtain the denoised grayscale value: ; in, For the original image at pixel location grayscale value at that location This represents the convolution operation. This is the grayscale value after noise reduction.

[0042] Furthermore, the Laplacian operator or variance function is used to calculate the focus function value of the denoised image. The Laplacian operator is used to extract high-frequency components in the image; edges, textures, and raster fringes in the image are all high-frequency contents that can produce gray-level changes. The variance function is used to characterize the dispersion of gray-level values ​​within a local neighborhood. When the telecentric camera is in focus, the edges, textures, or raster patterns in the image are clearer, the local gray-level changes are stronger, and the corresponding focus function value is larger. When the telecentric camera is off-focus, the image is blurred, the local gray-level changes are weaker, and the corresponding focus function value is smaller. Therefore, by calculating the focus function value of the image frame corresponding to different focus positions at the same pixel location or the same local neighborhood, the focus degree of that location at different focus positions can be obtained.

[0043] In one embodiment, the discrete form of the Laplace operator is: ; in, pixel position Laplace response at the location, The grayscale value after denoising. , , and Pixel positions The positions of adjacent pixels in the horizontal and vertical directions.

[0044] In actual calculations, the target Laplacian convolution kernel can be determined from multiple Laplacian convolution kernels of preset sizes based on the texture scale, structural complexity, and noise level of the region under test. Optionally, the target Laplacian convolution kernel can include a 3×3 four-neighbor Laplacian convolution kernel, a 3×3 eight-neighbor Laplacian convolution kernel, or a 5×5 Laplacian convolution kernel. The 3×3 four-neighbor Laplacian convolution kernel is as follows: ; The 3×3 eight-neighbor Laplace convolution kernel is: ; The 5×5 Laplace convolution kernel is: ; in, It is a 3×3 four-neighbor Laplace convolution kernel. It is a 3×3 eight-neighbor Laplace convolution kernel. A 5×5 Laplacian convolution kernel is used. A 3×3 four-neighbor Laplacian convolution kernel primarily responds to second-order changes in the horizontal and vertical directions; a 3×3 eight-neighbor Laplacian convolution kernel increases the response to diagonal gray-level changes; and a 5×5 Laplacian convolution kernel has a larger receptive field, enhancing the response to larger-scale texture and edge changes and reducing the impact of isolated noise points on the calculation results. When the measured region has rich texture or a relatively simple gold line structure, a 3×3 four-neighbor Laplacian convolution kernel or a 3×3 eight-neighbor Laplacian convolution kernel can be used to improve computational efficiency; when the measured region has a complex structure, and the gold line passes through shadow areas or gradient abrupt regions, a 5×5 Laplacian convolution kernel can be used to enhance the response to larger-scale changes and noise resistance.

[0045] The denoised image is convolved with the target Laplacian kernel to obtain the Laplacian response map. The formula for calculating the Laplacian response map is as follows: ; in, pixel position The Laplace response value at that point, The grayscale value after denoising. For the target Laplacian convolution kernel, This represents the convolution operation. This indicates taking the absolute value. The Laplacian response value can be used as the focus function value for the corresponding pixel position; if the local neighborhood is used as the calculation object, the Laplacian response values ​​of multiple pixels in the local neighborhood can be summed, averaged, or the variance taken to obtain the focus function value of that local neighborhood at the current focus position.

[0046] Furthermore, for mirrored and metallic surfaces, natural grayscale textures are scarce, making it difficult to obtain stable focus function values ​​when directly relying on the surface's own grayscale changes. Since the grating pattern in coaxial grating projection reflects strongly on mirrored and metallic surfaces, a computable texture composed of bright and dark stripes appears in the image. Therefore, when calculating the focus degree of such regions, grating stripes within the same pixel location or the same local neighborhood are used as the calculation object. If the Laplacian operator is used, the Laplacian response value is calculated for the local neighborhood containing the grating stripes, and this response value is used as the focus function value at that focus location. If the variance function is used, the grayscale dispersion formed by the bright and dark stripes within that local neighborhood is calculated, and the grayscale dispersion is used as the focus function value at that focus location. When the grating pattern is clear, the boundaries between bright and dark stripes are clear, and the Laplacian response value, gradient magnitude, contrast, or variance function output is large; when the grating pattern is blurred, the boundaries between bright and dark stripes become weaker, and the corresponding output decreases. Thus, the focus function values ​​for mirrored and metallic surfaces at different focus locations can be obtained.

[0047] Furthermore, for rough surfaces and gold line regions, the reflection of coaxial grating projection light is weak, and the grating pattern may be invisible or have low contrast in the image. Therefore, the focus degree of these regions is not calculated based on the grating pattern, but rather on the existing natural image texture or structural edges in the image. The natural grayscale texture of the rough surface, the outline edge of the gold line, and the local structure of the gold line surface can all produce grayscale changes. If the Laplacian operator is used, the Laplacian response value is calculated for the local neighborhood containing the natural image texture or structural edge, and this response value is used as the focus function value at that focus location. If the variance function is used, the grayscale dispersion formed by the natural image texture or structural edge in the local neighborhood is calculated, and the grayscale dispersion is used as the focus function value at that focus location. When the natural image texture or structural edge is clear, the local grayscale change is strong, and the corresponding focus function value is large; when the natural image texture or structural edge is blurred, the local grayscale change is weak, and the corresponding focus function value is small. Thus, the focus function values ​​of rough surfaces and gold line regions at different focus locations can be obtained.

[0048] It should be noted that the terms "mirror surface," "metal surface," "rough surface," and "gold line region" refer to different image content sources, and do not require different focusing algorithms for different regions. For mirror and metal surfaces, the computational texture formed by the raster pattern is used in the Laplacian operator or variance function; for rough surfaces and gold line regions, the natural image texture or structural edges are used. The focus level can be calculated using a uniform Laplacian operator or variance function for each region; focusing algorithm fusion is not required, nor is it necessary to assign computational weights to different regions.

[0049] Furthermore, information entropy can be used to assist in judging the reliability of focus function values. Specifically, when calculating the focus function value of the same pixel location or the same local neighborhood at a certain focus position, the occurrence probability of each gray level within that pixel location or local neighborhood is simultaneously counted, and the corresponding information entropy is calculated: ; in, For information entropy, Grayscale gray levels within the pixel neighborhood The probability of occurrence. Information entropy is used to reflect the richness of gray-level distribution in the neighborhood of a pixel; when the focus function value is large and the information entropy is also large, it indicates that the edge, texture or raster pattern under the focus position has rich gray-level changes, and the reliability of the focus function value is high; when the focus function value increases locally but the information entropy does not increase synchronously, it indicates that the focus function value may be caused by isolated noise points, local reflection points or single abrupt edge, and the focus function value can be treated as a low reliability response to avoid judging the focus state based on a single abnormal response.

[0050] Using the above methods, step S103 calculates the focus level using a uniform Laplacian operator or variance function for the mirror surface, metal surface, rough surface, and gold line region. The focus function value for the mirror surface and metal surface is obtained based on the computable texture formed by the grating pattern, while the focus function value for the rough surface and gold line region is obtained based on the natural image texture or structural edges, thus achieving consistent focus level calculation results across different material regions.

[0051] Step S104: Input the multi-focus image sequence into the pre-trained height prediction model to obtain the discrete height map of the semiconductor surface to be tested.

[0052] In step S104, the multi-focus image sequence is input into the pre-trained height prediction model to obtain a discrete height map of the semiconductor surface under test. The pre-trained height prediction model is a height prediction model built based on a variational autoencoder and an attention mechanism U-Net network.

[0053] Specifically, the multi-focus image sequence is input into the encoding part of the variational autoencoder, where the multi-focus image sequence is compressed and encoded to obtain latent space features. These latent space features are then input into the U-Net attention mechanism network, which extracts and reconstructs features from the latent space. The U-Net attention mechanism includes an encoding path, a decoding path, and an attention gating module positioned between the encoding and decoding paths. The attention gating module receives the feature map output from the encoding path and the upsampled feature map from the decoding path, and generates a spatial attention weight map based on these two maps. The spatial attention weight map is then applied to the feature map output from the encoding path to enhance focus-related regions and suppress irrelevant background regions. Finally, the feature map output from the U-Net attention mechanism network is input into the decoding part of the variational autoencoder, where decoding and reconstruction are performed, and a discrete height map is output through an activation function.

[0054] In this embodiment, the pre-trained height prediction model can be a VAE-AttentionUNet model. The model input is a multi-focus image sequence, which may include 10 to 20 frames of images acquired at different focus positions. The model output is a discrete height map. In one specific embodiment, the input image can be 512×512 resolution. The encoding part of the variational autoencoder compresses the input image to a 128×128 resolution latent space through multi-layer convolutional downsampling and feature extraction. The number of channels in the latent space features can be 64. The latent space features retain the key focus information in the original image while reducing the data dimensionality, enabling the height prediction model to extract focus features related to surface height in a lower-dimensional feature space.

[0055] Furthermore, the U-Net attention mechanism network takes 128×128×64 latent space features as input and includes an encoding path, a decoding path, and an attention gating module in the skip connections. The encoding path consists of sequentially connected first to fifth encoding layers, with the number of output channels for each layer being 64, 128, 256, 512, and 512, respectively. The first encoding layer takes 128×128×64 latent space features as input, processes them through a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function, and outputs a 128×128×64 feature map. It is then downsampled to 64×64×64 by a 2×2 max pooling layer with a stride of 2. The second encoding layer processes the features through a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function, and outputs a 64×64×128 feature map. It is then downsampled to 32×32×128 by a 2×2 max pooling layer with a stride of 2. The third encoding layer, after being processed by a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function, outputs a 32×32×256 feature map, which is then downsampled to 16×16×256 by a 2×2 max pooling layer with a stride of 2. The fourth encoding layer, after being processed by a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function, outputs a 16×16×512 feature map, which is then downsampled to 8×8×512 by a 2×2 max pooling layer with a stride of 2. The fifth encoding layer, acting as the bottleneck layer, outputs a deep feature map of 8×8×512 after being processed by a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function. Through the encoding path processing, the feature map resolution is gradually reduced from 128×128 to 8×8.

[0056] Furthermore, the decoding path comprises sequentially connected first to fourth decoding layers. Each decoding layer first undergoes a 2x upsampling through transposed convolution, then performs a skip connection concatenation with the feature map of the corresponding layer in the encoding path after being weighted by the attention-gated module, and performs feature fusion through a 3×3 convolutional layer, a batch normalization layer, and a ReLU activation function. The first decoding layer upsamples the 8×8×512 deep features output from the fifth encoding layer to 16×16×512, and concatenates them with the feature map of the fourth encoding layer after being weighted by the attention-gated module, resulting in a 16×16×256 feature map after convolution. The second decoding layer upsamples the feature map to 32×32×256, and concatenates it with the feature map of the third encoding layer after being weighted by the attention-gated module, resulting in a 32×32×128 feature map after convolution. The third decoding layer upsamples the feature map to 64×64×128 and concatenates it with the feature map from the second encoding layer (weighted by an attention-gated module). After convolution, a 64×64×64 feature map is obtained. The fourth decoding layer upsamples the feature map to 128×128×64 and concatenates it with the feature map from the first encoding layer (weighted by an attention-gated module). After convolution, a 128×128×64 feature map is obtained. After the decoding path processing, the feature map resolution is gradually restored from 8×8 to 128×128.

[0057] Simultaneously, an attention gating module is positioned between the outputs of the first to fourth encoding layers and the inputs of the first to fourth decoding layers to perform spatial attention weighting on the feature maps output from the encoding path. Specifically, the attention gating module receives the feature maps output from the encoding path and the upsampled feature maps from the decoding path, and generates a spatial attention weight map based on these two maps. The attention gating module can use the feature map output from the encoding path as a gating signal and the upsampled feature map from the decoding path as a query signal. After mapping both to the same channel space through a 1×1 convolution, they are added together, activated by ReLU, and then activated again by a 1×1 convolution and Sigmoid to generate the spatial attention weight map. The spatial attention weight map is then multiplied element-wise with the feature map output from the encoding path to achieve spatial attention weighting. This processing enhances the feature responses of focus-related regions such as gold lines, solder balls, metal pads, and mirror areas, while suppressing the responses of irrelevant background regions and reducing the interference of background noise on the discrete height map generation results.

[0058] Then, the 128×128×64 feature map output by the U-Net attention mechanism is input into the decoding part of the variational autoencoder for decoding and reconstruction. The decoding part of the variational autoencoder can progressively upsample through multiple transposed convolutional layers to restore the 128×128×64 feature map to a resolution of 512×512, and output a 256-level discrete height map through the Softmax activation function. In the 256-level discrete height map, the gray value of each pixel position is a value between 0 and 255, and each gray value corresponds to a discrete height level. Each discrete height level covers the height range of the semiconductor surface under test from the bottom plate to the top.

[0059] Furthermore, the pre-trained height prediction model is trained through the following steps. First, multiple sets of original multi-focus image sequences are acquired, and the initial focus level of each set of original multi-focus image sequences is calculated using the Laplacian algorithm to obtain an initial height map. The initial height map provides the basis for initial height annotation based on traditional focus evaluation results. Then, the pyramid algorithm is used to perform multi-scale analysis on the initial height map, and suspicious regions in the initial height map are identified based on whether the focus level evaluation results are consistent across different resolution levels. The pyramid algorithm decomposes the initial height map into multiple resolution levels, analyzing the consistency of the focus level distribution layer by layer from low-resolution to high-resolution levels; when the focus level evaluation results of a certain region are inconsistent across different resolution levels, that region is identified as a suspicious region. For example, when a region in a low-resolution level is judged to be well-focused, while the same region in a high-resolution level is judged to be poorly focused, that region is marked as a suspicious region.

[0060] Next, a manual annotation tool displays the original multi-focus image sequence, the initial heightmap, and the marking information of suspicious areas. Based on the changes in sharpness at different focal positions in the original multi-focus image sequence, erroneous points in the suspicious areas are corrected, resulting in a corrected heightmap. The manual annotation tool provides a visual interface, allowing operators to simultaneously observe the original multi-focus image sequence, the initial heightmap, and the marking information of suspicious areas. By observing the changes in sharpness at different focal positions in the original multi-focus image sequence, the correct focal position can be determined, thereby correcting erroneous points in the initial heightmap. The corrected heightmap is used as ground truth annotation in model training.

[0061] Subsequently, the original multi-focus image sequence and the corresponding corrected height map are cropped to establish an initial training sample set. In one specific embodiment, the original multi-focus image sequence and the corrected height map can be cropped to a size of 512×512 pixels to ensure that the input samples and ground truth labels correspond in spatial location. The initial training sample set is then expanded by rotation and mirroring to obtain an expanded sample set. Rotation expansion can include 0°, 90°, 180°, and 270° rotations, and mirroring can include horizontal and vertical mirroring. Through data expansion, the initial training sample set can be expanded to an expanded sample set containing 2000 images, improving the adaptability of the height prediction model to different directional structures and spatial poses. Compared with manual annotation of all regions, this sample construction method automatically completes the annotation judgment of most regions using the Laplacian algorithm and the pyramid algorithm, requiring only manual correction of erroneous points in suspicious regions, which can reduce the annotation workload by about 80%, thereby reducing the cost and time of sample construction.

[0062] Finally, a height prediction model based on a variational autoencoder and an attention mechanism U-Net network is trained using an expanded sample set. During training, the expanded sample set is divided into a training set and a validation set. Multi-focus image sequences from the training set are input into the encoding part of the variational autoencoder to obtain latent space features. These features are then processed by the U-Net network. Finally, a discrete height map is output through the decoding part of the variational autoencoder. During training, mean squared error or mean absolute error can be used as the loss function, and the Adam optimizer is used for parameter updates. The initial learning rate can be set to 1e-4, and the number of training epochs can be set to 200 to 500. During training, the model performance is monitored using a validation set. Training is terminated early when the validation set loss no longer decreases for several consecutive epochs to prevent overfitting.

[0063] During the model deployment phase, TensorRT can be used to accelerate the trained height prediction model through quantization, generating an optimized inference engine. TensorRT supports precision modes such as FP32, FP16, and INT8. In a specific embodiment, FP16 or INT8 quantization can be used to quantize model weights and activation values, while simultaneously performing operator fusion and memory optimization to generate a high-efficiency inference engine. After TensorRT optimization, the single inference speed of the height prediction model can be improved to within 50ms; when combined with a 65-megapixel high-resolution telecentric camera, the computation speed of image acquisition, focus calculation, model inference, and height value processing in the overall detection process can be controlled below 300ms, thus meeting the high-speed inspection requirements of semiconductor production lines.

[0064] Through the above processing, step S104 uses a variational autoencoder to compress the multi-focus image sequence into latent space features, and then uses an attention-based U-Net network to extract and reconstruct highly correlated features from the latent space features. Finally, the decoding part of the variational autoencoder outputs a discrete height map. Compared with performing U-Net processing directly on high-resolution images, feature processing in the latent space can reduce the computational cost to about 1 / 16 of the original resolution, improving training and inference speed. At the same time, latent space features are not sensitive to noise, and combined with the enhancement effect of the attention gating module on focus-related regions, the height prediction continuity of slender structures such as gold lines can be improved, and the interference of complex background regions on the discrete height map generation results can be reduced.

[0065] Step S105: Based on the height value indicated by the discrete height map, extract the focus function value corresponding to the height value from the focus degree and form a focus curve. Calculate the centroid based on the focus function value corresponding to each height position in the focus curve to determine the final height value of the semiconductor surface under test. Perform three-dimensional reconstruction based on the final height value and output the three-dimensional topography measurement result of the semiconductor surface under test.

[0066] In step S105, based on the height values ​​indicated by the discrete height map, the focus function values ​​corresponding to the height values ​​near the focus values ​​are extracted from the focus level to form a focus curve. The centroid is calculated based on the focus function values ​​corresponding to each height position in the focus curve to determine the final height value of the semiconductor surface under test. Three-dimensional reconstruction is performed based on the final height value, and the three-dimensional topography measurement results of the semiconductor surface under test are output. The discrete height map is used to provide the approximate height range for each pixel position, and the focus curve is used to further determine the continuous final height values ​​within this approximate height range.

[0067] Specifically, the discrete height level corresponding to each pixel location is determined based on the discrete height map, and the corresponding height value is determined based on the discrete height level. In one specific embodiment, the discrete height map is a 256-level discrete height map, where the grayscale value of each pixel location corresponds to a discrete height level, which represents the approximate height range of the corresponding pixel location. Since the discrete height map is output by a pre-trained height prediction model, it can limit the range of height corresponding to each pixel location, but the discrete height level is still a hierarchical result and therefore is not directly used as the final height value.

[0068] Furthermore, a height search range is determined among multiple focal positions corresponding to the multi-focus image sequence, centered on the height value. The height search range is determined based on the physical height range corresponding to the discrete height levels; for example, the physical height range corresponding to several adjacent discrete levels near the height value indicated by the discrete height map can be selected as the height search range. In a specific embodiment, the physical height range corresponding to five discrete levels near the height value indicated by the discrete height map can be selected as the height search range. Each height position within the height search range corresponds to a corresponding focal position in the multi-focus image sequence.

[0069] Furthermore, focus function values ​​are extracted from the focus degree at each focus position corresponding to the height search range, and focus curves are formed according to the height order of the focus positions. In step S103, the focus degree of each pixel position or local neighborhood on the semiconductor surface under test at different focus positions has been obtained, and the focus degree is represented by the focus function value at the corresponding focus position. Therefore, focus function values ​​corresponding to each focus position within the height search range can be extracted from the focus degree and arranged according to the height order of each focus position to form the focus curve corresponding to that pixel position or local neighborhood.

[0070] In practical implementation, the traditional Laplacian operator can be recalculated near the height values ​​indicated by the discrete height map to obtain the focus response values ​​at various height positions within the height search range for that pixel location. These focus response values ​​are then used as focus function values ​​to form a focus curve. In other words, the horizontal variable of the focus curve represents the multiple height positions within the height search range, and the response variable is the focus function value corresponding to each height position. When the telecentric camera captures a clear image of the corresponding pixel location on the semiconductor surface under test at a certain height position, the focus function value corresponding to that height position is relatively large; when the telecentric camera deviates from the clear focus position, the focus function value corresponding to that height position decreases.

[0071] Furthermore, the centroid is calculated based on the focus function values ​​corresponding to each height position in the focus curve, and the calculated centroid position is determined as the final height value of the corresponding pixel position on the semiconductor surface under test. During centroid calculation, each height position within the height search range is used as a position variable, and the focus function value corresponding to each height position is used as a response weight. The focus curve is integrated, and the centroid position is calculated based on the response distribution of the focus curve in the height direction. Compared to directly selecting the single height position with the largest focus function value, centroid calculation uses the response distribution of multiple height positions within the height search range to determine the final height value. This reduces the impact of noise, local reflections, or abnormal responses in a single frame on the height determination result, improving the stability and repeatability accuracy of height measurement.

[0072] In one alternative embodiment, when calculating the centroid based on the focus function values ​​corresponding to each height position in the focus curve, the final height value can be determined using the focus centroid response quantity constrained by the discrete height map. For pixel positions... Let the height indicated by the discrete height map be _____. ,by The height search range determined by the center is The first in the height search range The focal position is pixel position In focus The focusing function value is below Then the first The centroid response corresponding to each focusing position is: ; in, pixel position In focus The focused centroid response quantity, pixel position In the high search range The maximum focusing function value within, To prevent extremely small positive numbers with a denominator of zero, For the height search range relative to The maximum height offset. In the formula, the square root term is used to convert the focus function value into a nonlinear response, so that height positions with larger focus function values ​​receive higher responses, while avoiding a single abnormally large focus function value from having an excessive impact on the final height value; the second term is used to constrain the response amount based on the distance between the focus position and the height value indicated by the discrete height map, the distance... The closer, the higher the corresponding response value; the distance... The farther away, the lower the corresponding response value.

[0073] After obtaining the centroid response at each focusing position, the final height value is obtained. Determined by the following formula: ; in, pixel position The final height value. This formula represents the height search range. Within this model, the centroid response of each focusing position is used as the response in the height direction, ensuring that the focusing responses on both sides of the final height value achieve centroidal balance in the height direction. Therefore, the final height value is not directly selected from the single height position with the largest focusing function value, but rather determined by combining the focusing function values ​​of each height position with the degree of deviation of each height position from the height value indicated by the discrete height map within the height search range defined by the discrete height map.

[0074] In this manner, the discrete height map is used to define the search center and search range for the final height value, the focus function value is used to reflect the image sharpness response at each height position, and the focus centroid response is used to combine the two into a response distribution along the height direction. Therefore, this embodiment can reduce the impact of single-frame abnormal response, local reflection response, or discrete height prediction error on the final height value without changing the original "discrete height map constraint + focus curve centroid calculation" process, thereby improving the stability and continuity of height measurement results for complex semiconductor surfaces.

[0075] Finally, a height distribution of the semiconductor surface under test is generated based on the final height values ​​of each pixel location, and a 3D reconstruction is performed based on the height distribution to output the 3D topography measurement results. Specifically, height search range determination, focus function value extraction, focus curve formation, and centroid calculation are performed on multiple pixel locations on the semiconductor surface under test to obtain the final height value corresponding to each pixel location; then, the final height values ​​are organized according to the spatial coordinates of each pixel location in the image to form the height distribution of the semiconductor surface under test; the 3D topography of the semiconductor surface under test is reconstructed based on the height distribution, thereby outputting the 3D topography measurement results.

[0076] In summary, combining Figure 2 The system controls a coaxial grating to project a grating pattern onto the surface of the semiconductor under test, ensuring that the imaging focal plane of the coaxial grating coincides with the focal plane of the telecentric camera. The telecentric camera is then moved along the optical axis to acquire multi-focus image sequences of the semiconductor surface at different focal positions. Based on these multi-focus image sequences, the degree of focus of the semiconductor surface at different focal positions is calculated. For mirror surfaces and metal surfaces, the focus degree is calculated based on the computable texture formed by the grating pattern; for rough surfaces and gold-line regions, the focus degree is calculated based on the natural image texture. The multi-focus image sequences are then input into a pre-trained height prediction model built using a variational autoencoder and an attention mechanism U-Net network to obtain a discrete height map. Finally, based on the height value indicated by the discrete height map, the corresponding focus function value is extracted near that height value to form a focus curve. The final height value is determined by calculating the centroid of the focus curve, and 3D reconstruction is performed based on the final height value to output the 3D topographic measurement results of the semiconductor surface.

[0077] Based on the above scheme, the imaging focal plane of the coaxial grating coincides with the focal plane of the telecentric camera, allowing the sharpness of the grating pattern to vary with the camera's focusing position. This enables mirror surfaces and metallic surfaces lacking natural texture to form image textures usable for focus calculation. Simultaneously, rough surfaces and gold-line regions can still utilize their own natural textures or structural edges for focus calculation, thus achieving continuous and comparable focus responses across different material regions. Furthermore, the pre-trained height prediction model first provides a discrete height range, then extracts focus function values ​​near this range and determines the final height value through centroid calculation. This ensures that the height determination process does not entirely rely on the discrete output of the neural network, nor does it require blindly searching for focus peaks across the entire height range, thereby improving the accuracy, stability, and computational efficiency of three-dimensional topography measurement of complex semiconductor surfaces. This application, through acquiring multi-focus image sequences using a telecentric camera, calculating focus based on image texture, and reconstructing three-dimensional topography, belongs to a visual measurement technology scheme based on image acquisition and image processing.

[0078] Figure 3 This is a structural block diagram of a semiconductor three-dimensional topography measurement system based on displacement imaging according to an embodiment of this application. The system includes at least the following modules: The grating projection module is used to control the coaxial grating to project the grating pattern onto the surface of the semiconductor under test, and to make the imaging focal plane of the coaxial grating coincide with the focal plane of the telecentric camera. The image acquisition module is used to control the telecentric camera to move along the optical axis and acquire multi-focus image sequences of the semiconductor surface under test at different focusing positions; The focusing calculation module is used to calculate the degree of focusing of the semiconductor surface under test at different focusing positions based on the multi-focus image sequence. Specifically, for the mirror surface and metal surface of the semiconductor surface under test, the degree of focusing is calculated based on the computable texture formed by the grating pattern; for the rough surface and gold line area of ​​the semiconductor surface under test, the degree of focusing is calculated based on the natural image texture. The height generation module is used to input multi-focus image sequences into a pre-trained height prediction model to obtain discrete height maps of the semiconductor surface under test. The 3D reconstruction module is used to extract the focus function values ​​corresponding to the height values ​​indicated by the discrete height map and form a focus curve. Based on the focus function values ​​corresponding to each height position in the focus curve, the centroid is calculated to determine the final height value of the semiconductor surface under test. Based on the final height value, 3D reconstruction is performed, and the 3D topography measurement results of the semiconductor surface under test are output.

[0079] For relevant details, please refer to the above method implementation examples.

[0080] Figure 4This is a block diagram of an electronic device provided in one embodiment of this application. The device includes at least a processor 401 and a memory 402.

[0081] Processor 401 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 401 may be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array) to perform image processing of multi-focus image sequences, focus calculation, height calculation, and 3D reconstruction. Processor 401 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), processes the main data during device operation; the coprocessor processes auxiliary computing tasks or data under low-power conditions. In some embodiments, processor 401 may integrate a GPU (Graphics Processing Unit), which can be used to perform parallel processing of multi-focus image sequences, discrete height map generation, and rendering and display of 3D topography measurement results. In some embodiments, processor 401 may further include an AI (Artificial Intelligence) processor, which can be used to perform inference calculations of a pre-trained height prediction model to obtain a discrete height map of the semiconductor surface under test based on a multi-focus image sequence.

[0082] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 402 is used to store at least one instruction, which is executed by the processor 401 to implement the semiconductor three-dimensional topography measurement method based on displacement imaging provided in the method embodiments of this application. This method includes controlling a coaxial grating to project a grating pattern, controlling a telecentric camera to acquire multi-focus image sequences along the optical axis, calculating the degree of focus at different focus positions, generating a discrete height map, calculating and determining the final height value based on the centroid of the focus curve, and outputting the three-dimensional topography measurement result.

[0083] Optionally, this application also provides a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the semiconductor three-dimensional topography measurement method based on displacement imaging as described in the above method embodiments.

[0084] Optionally, this application also provides a computer product including a computer-readable storage medium storing a program, which is loaded and executed by a processor to implement the semiconductor three-dimensional topography measurement method based on displacement imaging described in the above method embodiments.

[0085] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0086] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for measuring the three-dimensional topography of a semiconductor based on displacement imaging, characterized in that, The method includes: Control the coaxial grating to project a grating pattern onto the surface of the semiconductor under test, and make the imaging focal plane of the coaxial grating coincide with the focal plane of the telecentric camera; The telecentric camera is controlled to move along the optical axis to acquire a multi-focus image sequence of the semiconductor surface under test at different focusing positions; The degree of focus of the semiconductor surface under test at different focus positions is calculated based on the multi-focus image sequence. For the mirror surface and metal surface of the semiconductor surface under test, the degree of focus is calculated based on the computable texture formed by the grating pattern. For the rough surface and gold line area of ​​the semiconductor surface under test, the degree of focus is calculated based on the natural image texture. The multi-focus image sequence is input into a pre-trained height prediction model to obtain a discrete height map of the semiconductor surface under test. Based on the height value indicated by the discrete height map, the focus function value corresponding to the height value is extracted from the focus degree and a focus curve is formed. Based on the focus function value corresponding to each height position in the focus curve, the centroid is calculated to determine the final height value of the semiconductor surface under test. Based on the final height value, three-dimensional reconstruction is performed, and the three-dimensional topography measurement result of the semiconductor surface under test is output. The step of calculating the centroid based on the focusing function values ​​corresponding to each height position in the focusing curve to determine the final height value of the semiconductor surface under test includes: The final height value is determined using the focused centroid response constrained by the discrete height map, for each pixel position. Let the height indicated by the discrete height map be _____. ,by The height search range determined by the center is The first in the height search range The focal position is pixel position In focus The focusing function value is below Then the first The centroid response corresponding to each focusing position is: ； in, pixel position In focus The focused centroid response quantity, pixel position In the high search range The maximum focusing function value within, To prevent extremely small positive numbers with a denominator of zero, For the height search range relative to Maximum height offset; After obtaining the centroid response at each focusing position, the final height value is obtained. Determined by the following formula: ； in, pixel position The final height value.

2. The semiconductor three-dimensional topography measurement method based on displacement imaging according to claim 1, characterized in that, The method is applied to a semiconductor inspection system, which includes a controller, a telecentric camera, a multilayer RGB light source, a coaxial grating, and a grating ruler. The controller is communicatively connected to the telecentric camera, the multilayer RGB light source, the coaxial grating, and the grating ruler. The coaxial grating is disposed in the coaxial illumination optical path of the telecentric camera. The coaxial grating is a transmissive grating or a reflective grating, and the coaxial grating is composed of periodically arranged bright and dark stripes. The multi-layer RGB light source includes a high-angle light source, a medium-angle light source, a low-angle light source, and an RGB point coaxial light source. The high-angle light source is used to illuminate the top area of ​​the gold wire, the medium-angle light source is used to illuminate the side wall area of ​​the gold wire, the low-angle light source is used to illuminate the surface area of ​​the substrate, and the RGB point coaxial light source is disposed in the optical path of the coaxial grating and is used to cooperate with the coaxial grating to provide coaxial illumination for the mirror area.

3. The semiconductor three-dimensional topography measurement method based on displacement imaging according to claim 2, characterized in that, The control of the telecentric camera to move along the optical axis and acquire multi-focus image sequences of the semiconductor surface under test at different focusing positions includes: The telecentric camera is controlled to move continuously along the optical axis, and the displacement of the telecentric camera is measured in real time by the grating ruler. The pulse signal output by the grating ruler is connected to the counting module of the controller, and the counting module counts the number of pulses of the pulse signal. When the number of pulses reaches a preset trigger threshold, the controller sends a trigger signal to the telecentric camera, causing the telecentric camera to acquire an image frame at the current position. The displacement interval of the telecentric camera when acquiring images is adjusted by adjusting the trigger threshold, so as to acquire multiple frames of images at different focusing positions and form the multi-focus image sequence.

4. The semiconductor three-dimensional topography measurement method based on displacement imaging according to claim 1, characterized in that, The calculation of the degree of focus of the semiconductor surface under test at different focusing positions based on the multi-focus image sequence includes: The images in the multi-focus image sequence are smoothed and denoised to obtain denoised images; High-frequency components are extracted from the denoised image based on the Laplacian operator or variance function to obtain the degree of focus of the semiconductor surface under test at different focus positions. The degree of focus is represented by the focus function value at the corresponding focus position. For the mirror surface and the metal surface in the semiconductor surface under test, the computable texture formed by the grating pattern in the image is extracted, and the contrast or gradient magnitude of the computable texture is calculated. Based on the contrast or gradient magnitude, the focus function values ​​of the mirror surface and the metal surface at different focus positions are determined. For the rough surface and gold wire region in the semiconductor surface under test, the natural image texture or structural edge in the image is extracted, and the focus function value of the rough surface and the gold wire region at different focus positions is determined based on the natural image texture or the structural edge. At the same pixel location, the image frames corresponding to different focal positions in the multi-focus image sequence are traversed to obtain the degree of focus of the focal function value of the pixel location as the focal position changes.

5. The semiconductor three-dimensional topography measurement method based on displacement imaging according to claim 1, characterized in that, The pre-trained height prediction model is a height prediction model built based on a variational autoencoder and an attention mechanism U-Net network. The step of inputting the multi-focus image sequence into the pre-trained height prediction model to obtain a discrete height map of the semiconductor surface under test includes: The multi-focus image sequence is input into the encoding part of the variational autoencoder, and the multi-focus image sequence is compressed and encoded by the encoding part to obtain latent space features; The latent space features are input into the U-Net attention mechanism network, and the U-Net attention mechanism network is used to extract and reconstruct the latent space features. The U-Net attention mechanism network includes an encoding path, a decoding path, and an attention gating module set between the encoding path and the decoding path. The attention gating module receives the feature map output from the encoding path and the upsampled feature map from the decoding path, and generates a spatial attention weight map based on the feature map output from the encoding path and the upsampled feature map from the decoding path. The feature map output by the encoding path is spatially attention-weighted according to the spatial attention weight map to enhance the focus on relevant regions and suppress irrelevant background regions; The feature map output by the U-Net network with the attention mechanism is input into the decoding part of the variational autoencoder, and the decoding part performs decoding and reconstruction, and outputs the discrete height map through the activation function.

6. The semiconductor three-dimensional topography measurement method based on displacement imaging according to claim 5, characterized in that, The pre-trained height prediction model is trained through the following steps: Multiple sets of original multifocus image sequences are acquired, and the initial focus degree of each set of original multifocus image sequences is calculated using the Laplacian algorithm to obtain the initial height map; The pyramid algorithm is used to perform multi-scale analysis on the initial height map, and the consistency of the results is evaluated based on the degree of focus at different resolution levels to identify suspicious areas in the initial height map. The original multi-focus image sequence, the initial height map, and the marking information of the suspicious areas are displayed using a manual annotation tool. Based on the changes in sharpness of the original multi-focus image sequence at each focal position, the erroneous points in the suspicious areas are corrected to obtain the corrected height map. The original multi-focus image sequence and the corresponding corrected height map are cropped to establish an initial training sample set; The initial training sample set is rotated and mirrored to expand it, resulting in an expanded sample set. The high-precision prediction model, constructed using a U-Net network based on a variational autoencoder and attention mechanism, is trained based on the expanded sample set.

7. The semiconductor three-dimensional topography measurement method based on displacement imaging according to claim 1, characterized in that, The process involves extracting the focus function values ​​corresponding to the height values ​​indicated by the discrete height map from the focus level and forming a focus curve, then calculating the centroid based on the focus function values ​​corresponding to each height position in the focus curve to determine the final height value of the semiconductor surface under test, including: The discrete height level corresponding to each pixel position is determined based on the discrete height map, and the corresponding height value is determined based on the discrete height level. Using the height value as the center, determine the height search range among multiple focal positions corresponding to the multi-focus image sequence; The focus function value is extracted from the focus degree at each focus position corresponding to the height search range, and the focus curve is formed according to the height order of the focus position; Based on the focus function values ​​corresponding to each height position in the focus curve, the centroid is calculated, and the calculated centroid position is determined as the final height value of the corresponding pixel position on the surface of the semiconductor under test. The height distribution of the semiconductor surface under test is generated based on the final height value of each pixel position, and a three-dimensional reconstruction is performed based on the height distribution to output the three-dimensional topography measurement result.

8. A semiconductor three-dimensional topography measurement system based on displacement imaging, characterized in that, include: A grating projection module is used to control a coaxial grating to project a grating pattern onto the surface of a semiconductor under test, and to make the imaging focal plane of the coaxial grating coincide with the focal plane of a telecentric camera. The image acquisition module is used to control the telecentric camera to move along the optical axis and acquire multi-focus image sequences of the semiconductor surface under test at different focusing positions; The focusing calculation module is used to calculate the degree of focusing of the semiconductor surface under test at different focusing positions based on the multi-focus image sequence. Specifically, for the mirror surface and metal surface of the semiconductor surface under test, the degree of focusing is calculated based on the computable texture formed by the grating pattern; for the rough surface and gold line region of the semiconductor surface under test, the degree of focusing is calculated based on the natural image texture. The height generation module is used to input the multi-focus image sequence into a pre-trained height prediction model to obtain a discrete height map of the semiconductor surface under test. The 3D reconstruction module is used to extract the focus function values ​​corresponding to the height values ​​indicated by the discrete height map from the focus degree and form a focus curve; to calculate the centroid based on the focus function values ​​corresponding to each height position in the focus curve, and determine the final height value of the semiconductor surface under test; to perform 3D reconstruction based on the final height value, and output the 3D topography measurement results of the semiconductor surface under test; The step of calculating the centroid based on the focusing function values ​​corresponding to each height position in the focusing curve to determine the final height value of the semiconductor surface under test includes: The final height value is determined using the focused centroid response constrained by the discrete height map, for each pixel position. Let the height indicated by the discrete height map be _____. ,by The height search range determined by the center is The first in the height search range The focal position is pixel position In focus The focusing function value is below Then the first The centroid response corresponding to each focusing position is: ； in, pixel position In focus The focused centroid response quantity, pixel position In the high search range The maximum focusing function value within, To prevent extremely small positive numbers with a denominator of zero, For the height search range relative to Maximum height offset; After obtaining the centroid response at each focusing position, the final height value is obtained. Determined by the following formula: ； in, pixel position The final height value.

9. An electronic device, characterized in that, The device includes a processor and a memory; the memory stores a program that is loaded and executed by the processor to implement a semiconductor three-dimensional topography measurement method based on displacement imaging as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a program that, when executed by a processor, is used to implement a semiconductor three-dimensional topography measurement method based on displacement imaging as described in any one of claims 1 to 7.

Images