A method and system for imaging ultra-micro targets

Abstract

This invention discloses an ultra-micro target imaging method and system. It receives the full-band reflected light field from an ultra-micro target illuminated by a full-band light source, performs high-frequency and low-frequency switching filtering on the reflected light, and collects the intensity information of the filtered reflected light. Fourier transform and wavelet transform are then performed on the intensity information to obtain global and local images. Global features of the global image are extracted, and local features of the local image are extracted. The global and local features are then fused, and an enhanced image of the ultra-micro target is obtained based on the fused features. This invention solves the technical problem of low imaging accuracy in existing ultra-micro target imaging systems, achieving ultra-high resolution, strong noise resistance, and fast imaging speed. It can effectively solve the chip imaging problem during eutectic bonding, and the entire system has a simple structure, low cost, and is easier to maintain.

Description

Technical Field

[0001] This invention relates primarily to the field of optical imaging technology, and specifically to a method and system for imaging ultra-micro targets. Background Technology

[0002] With the continuous development of integrated circuit manufacturing processes, chip integration is becoming increasingly higher, and critical dimensions have entered the deep submicron or even nanometer scale. In order to further improve the performance and reliability of chips, it is necessary to be able to perform ultra-high resolution non-destructive testing and imaging of critical structures inside the chip.

[0003] In recent years, significant progress has been made in super-resolution optical microscopy based on principles such as structured illumination and saturated lasers. These techniques can overcome the diffraction limit of optical systems, achieving higher resolution imaging. Simultaneously, by employing specialized image processing algorithms, high-resolution images can be recovered from low-resolution images—a process known as super-resolution image reconstruction.

[0004] By organically combining these two technologies and employing optimized optical path design, it is expected to achieve rapid, accurate, and non-destructive ultra-high resolution imaging of critical areas of chips, making it a powerful tool for detecting chip defects, process control, and quality control. This represents a significant opportunity for the development of ultra-resolution chip imaging technology.

[0005] The invention patent with publication number CN114137005B proposes a distributed multimode diffraction imaging method. This method uses multiple distributed sub-diffraction systems to acquire images with different fields of view and spectral bands, and obtains high-resolution results through image processing algorithms. However, the matching relationship between different subsystems is not fully considered. The image processing algorithm is not comprehensive enough, and its customization and optimization for diffraction imaging are insufficient, making it difficult to achieve the expected results. Moreover, the algorithm has high complexity, placing high demands on the computing power of the hardware. In addition, the distributed system structure suffers from collinearity of optical axes and stability issues, making its implementation relatively complex.

[0006] Patent application CN116559427A discloses a lateral flow immunoassay chip and device based on CMOS lensless imaging. This method uses a lens-free CMOS image sensor for direct imaging, combined with a printable immunochromatographic membrane to achieve lateral flow immunoassay. However, its ability to control and correct imaging quality in the optical path is weak, relying mainly on back-end image processing to improve the effect. It has poor adaptability to complex scenes, and the CMOS chip used in this method has poor resolution reconstruction, limiting its application scope to lateral flow immunoassay chip applications. Summary of the Invention

[0007] The present invention provides a method and system for imaging ultra-micro targets, which solves the technical problem of low imaging accuracy of existing ultra-micro targets.

[0008] To address the aforementioned technical problems, the present invention proposes an ultra-micro target imaging method comprising:

[0009] It receives the full-band reflected light field after a full-frequency light source illuminates an ultra-micro target, which is a chip.

[0010] The reflected light in the full-band reflected light field is filtered by switching between high and low frequencies, and the intensity information of the filtered reflected light is collected.

[0011] Fourier transform and wavelet transform are performed on the light intensity information to obtain global and local images.

[0012] Extract global features from the global image and local features from the local image.

[0013] Global and local features are fused, and an enhanced image of the ultra-micro target is obtained based on the fused features.

[0014] Furthermore, the full-band reflected light field after the ultra-micro target is illuminated by a full-frequency light source includes:

[0015] It receives the light beam reflected by the light guide after the full-frequency illumination source illuminates the ultra-micro target.

[0016] Based on the reflected beam, a reflected light field model is established, where the specific formula for the reflected light field model is:

[0017]

[0018] Where R1(r,t) represents the full-band reflected light field model corresponding to the position vector r at time t, and R(r,θ1,θ2,λ) represents the reflected light field with position vector r, incident angle θ1, reflection angle θ2, and light wavelength λ. i (r,θ1,λ) represents the incident light field with position vector r, incident angle θ1, and wavelength λ, and O(r,θ1,θ2,λ) represents the modulation function of the imaging object with position vector r, incident angle θ1, reflection angle θ2, and wavelength λ. min ,λ max [ ] represents the wavelength range of light, ω represents the angular frequency of light, and k r Let represent the wave vector corresponding to the position vector r, and let i represent the imaginary unit.

[0019] Based on the reflected light field model, the full-band reflected light field is obtained.

[0020] Furthermore, extracting global features from the global image includes:

[0021] A multi-level graph convolutional network is constructed to extract global features from the global image. The formula for calculating the feature vector extracted by each level of the multi-level graph convolutional network is as follows:

[0022]

[0023] Among them, G j G represents the feature vector output by the j-th level graph convolutional network, M represents the number of pixels in the global image, and G... 0 Y1 represents the input to the multi-level graph convolutional network, and Y1 represents the global image. and These represent the features of the k-th point in the feature vectors output by the j-th and (j+1)-th level graph convolutional networks, respectively. This indicates that when computing the output of the j-th level graph convolutional network, in A local region with K elements is created around it. Let represent the feature of the nth point in the feature vector output by the j-th level graph convolutional network, where 1≤k≤M, 1<K<M, 1≤n≤K, σ represents the non-linear activation function, w is the learnable parameter of the multilayer perceptron, and Max-Pool is the max pooling operation.

[0024] The global features of the global image are obtained based on the feature vectors output by each level of the graph convolutional network. The specific formula is as follows:

[0025]

[0026] in, G represents global features. 1 G 2 and G j represents the feature vectors output by the first, second, and j-th level graph convolutional networks, respectively, and MLP represents the perceptual function of the graph convolutional network.

[0027] Furthermore, when computing the output of the j-th level graph convolutional network, in The surrounding local region, consisting of K elements, includes:

[0028] The specific formula for calculating the feature distance between the k-th point and other points in the feature vector output by the j-th level graph convolutional network is as follows:

[0029]

[0030] in, and Let G represent the feature vectors output by the j-th level graph convolutional network, respectively. j Features of the k-th and m-th points, This represents the feature distance between the k-th and m-th points. Representative to Find the square of the L2 norm, where 1 ≤ k ≤ M and 1 ≤ m ≤ M.

[0031] When selecting K elements whose distance is less than a preset threshold as the output of the j-th level graph convolutional network, A local region with K elements is created around it. And 1 < K < M.

[0032] Furthermore, extracting local features from a local image includes:

[0033] A convolution operation with a preset layer threshold is performed on a local image. The specific formula is as follows:

[0034] F (l) =ReLU(W (l) *F (l-1) +b (l) ),

[0035] Among them, F (l) and F (l-1) W represents the local output features of the l-th and (l-1)-th convolutional layers, respectively. (l) and b (l) Represents the weights and biases of the l-th convolutional layer, respectively, ReLU represents the activation function, and N... th This represents the preset layer threshold, where 0 ≤ l ≤ N. th .

[0036] The local output features of the last convolutional layer are used as the local features of the local image.

[0037] Furthermore, the fusion of global and local features, and the acquisition of an enhanced image of the ultra-micro target based on the fused features, includes:

[0038] Global features are fused with multiple local features.

[0039] Perform an inverse Fourier transform on the fused features.

[0040] An enhanced image of the ultra-micro target is obtained based on the inverse Fourier transform result.

[0041] Furthermore, the full-frequency light source comprises a laser diode, a collimating lens, a polarizer, an aspherical mirror, a ring-shaped laser, and first, second, third, and fourth output lens groups connected in sequence, wherein:

[0042] Laser diodes are used as pump sources.

[0043] A collimating lens is used to adjust the divergence angle and beam quality of a light beam, turning a diverging beam into parallel light.

[0044] A polarizer is used to select and control the polarization state of polarized light.

[0045] Aspherical mirrors are used to convert Gaussian beams into ring beams.

[0046] Ring-shaped lasers are used to suppress diffraction loss and achieve super-resolution focusing and imaging.

[0047] The first, second, third, and fourth output lens groups are used to expand the illumination area.

[0048] Furthermore, the light guide comprises a first plano-convex lens, a relay lens group, and a second plano-convex lens connected in sequence, wherein:

[0049] The first plano-convex lens is used to converge the light output from the full-frequency light source.

[0050] A relay lens group, consisting of a series of cylindrical lenses, is used to converge light rays to the focal point of a second plano-convex lens.

[0051] The second plano-convex lens is used to converge parallel light rays.

[0052] The present invention provides an ultra-micro target imaging system comprising:

[0053] The invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the ultra-micro target imaging method provided by the present invention.

[0054] The proposed method and system for imaging ultra-micro targets involves receiving the full-band reflected light field from an ultra-micro target illuminated by a full-band light source. The reflected light is then filtered using high-frequency and low-frequency switching, and the intensity information of the filtered reflected light is acquired. Fourier transform and wavelet transform are performed on the intensity information to obtain global and local images. Global features of the global image are extracted, and local features of the local image are extracted. The global and local features are then fused, and an enhanced image of the ultra-micro target is obtained based on the fused features. This method solves the problem of low imaging accuracy in existing ultra-micro target imaging systems, achieving ultra-high resolution, strong noise resistance, and fast imaging speed. It effectively solves the chip imaging problem during eutectic bonding, and the entire system has a simple structure, low cost, and is easier to maintain. Furthermore, this invention uses a novel imaging method to replace traditional optical imaging, designing a complete super-resolution chip imaging system capable of acquiring high-quality images of chip microstructures at high speed, effectively solving the imaging challenges in eutectic bonding.

[0055] The beneficial effects of this invention specifically include:

[0056] (1) Multi-level light field capture: Through a finely designed optical path structure, this invention can capture richer light field information, including light waves of different frequencies and angles, laying the foundation for subsequent high-resolution image reconstruction.

[0057] (2) Advanced mathematical modeling: By using complex mathematical modeling of reflected light fields, this invention can more accurately describe the interaction between light and micro-targets, which is the key to realizing nanoscale resolution imaging.

[0058] (3) Wavelet-Fourier imaging algorithm based on cross-domain image generation network: Combining the global feature capture of Fourier transform and the local feature analysis of wavelet transform, the algorithm can extract image information from multiple dimensions and significantly improve the resolution of the reconstructed image.

[0059] (4) Deep learning enhancement: Through deep learning technology, it is possible to effectively reduce noise and improve the signal-to-noise ratio of images during image reconstruction. This is especially important for imaging under low light or high noise conditions.

[0060] (5) Parallel computing framework: By adopting parallel processing technology, including at the hardware and algorithm levels, the invention has significantly improved the speed of image processing. Attached Figure Description

[0061] Figure 1 This is a flowchart of the ultra-micro target imaging method according to Embodiment 2 of the present invention;

[0062] Figure 2 This is a schematic diagram of the full-frequency light source and light guide mirror assembly according to Embodiment 2 of the present invention;

[0063] Figure 3 This is a schematic diagram of the structure of the ultra-micro target imaging device according to Embodiment 2 of the present invention;

[0064] Figure 4 This is a network architecture diagram of Embodiment 2 of the present invention;

[0065] Figure 5 This is a flowchart of the Wave-Fourier imaging algorithm based on a cross-domain image generation network according to Embodiment 2 of the present invention;

[0066] Figure 6 This is a structural block diagram of the ultra-micro target imaging system according to an embodiment of the present invention.

[0067] Figure label:

[0068] 1. Laser diode; 2. Collimating lens; 3. Polarizer; 4. Aspherical mirror; 5. Ring-shaped laser; 6. First output lens group; 7. Second output lens group; 8. Third output lens group; 9. Fourth output lens group; 10. First plano-convex lens; 11. Relay lens group; 12. Second plano-convex lens; 13. Full-frequency illumination source; 14. Light guide; 15. Observation chip; 16. Beam splitter; 17. Bidirectional frequency divider; 18. CCD; 19. Wavelet-Fourier imaging algorithm based on cross-domain image generation network; 20. Memory; 30. Processor. Detailed Implementation

[0069] To facilitate understanding of the present invention, the present invention will be described more fully and in detail below with reference to the accompanying drawings and preferred embodiments, but the scope of protection of the present invention is not limited to the following specific embodiments.

[0070] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0071] Example 1

[0072] The ultra-micro target imaging method provided in Embodiment 1 of the present invention includes:

[0073] Step S101: Receive the full-band reflected light field after the full-frequency light source illuminates the micro target, which is a chip.

[0074] Step S102: High-frequency and low-frequency switching filtering is performed on the reflected light in the full-band reflected light field, and the light intensity information of the filtered reflected light is collected.

[0075] Step S103: Perform Fourier transform and wavelet transform on the light intensity information to obtain the global image and the local image.

[0076] Step S104: Extract global features from the global image and extract local features from the local image.

[0077] Step S105: The global features and local features are fused, and the enhanced image of the ultra-micro target is obtained based on the fused features.

[0078] The ultra-micro target imaging method provided in this invention receives the full-band reflected light field after the ultra-micro target is illuminated by a full-band light source. It performs high-frequency and low-frequency switching filtering on the reflected light in the full-band reflected light field and collects the intensity information of the filtered reflected light. Fourier transform and wavelet transform are then performed on the intensity information to obtain global and local images. Global features of the global image are extracted, and local features of the local image are extracted. The global and local features are then fused, and an enhanced image of the ultra-micro target is obtained based on the fused features. This method solves the technical problem of low imaging accuracy in existing ultra-micro target imaging systems, achieving ultra-high resolution, strong noise resistance, and fast imaging speed. It effectively solves the chip imaging problem in the eutectic bonding process, and the entire system has a simple structure, low cost, and is easier to maintain. Furthermore, this invention uses a novel imaging method to replace traditional optical imaging, designing a complete super-resolution chip imaging system that can acquire high-quality images of chip microstructures at high speed, effectively solving the imaging problem in the eutectic bonding process.

[0079] Example 2

[0080] During the eutectic bonding process of chips, malfunctions often occur due to defects in the chip itself or the placement machine not following the preset process, resulting in unnecessary production losses and material waste. Therefore, it is crucial to design a super-resolution chip imaging system to provide clear and observable images of a series of issues such as whether there are deviations in the relative positions of the chip and the wafer and the degree of solder liquefaction during the bonding process.

[0081] Traditional optical imaging systems struggle to image details such as the chip itself and its minute defects. Traditional optical microscopes, limited by the optical diffraction limit, find it difficult to achieve high-resolution imaging at the nanometer scale. While scanning tunneling microscopy and atomic force microscopy, based on scanning probe technology, can achieve sub-nanometer high-resolution imaging, their small scanning range and slow imaging speed make them unsuitable for large-scale chip imaging and inspection. Therefore, developing novel super-resolution chip imaging technologies has become a key technical challenge.

[0082] Reference Figure 1 The method for achieving ultra-micro target imaging in this embodiment of the invention mainly includes:

[0083] (1) Full-range lighting source:

[0084] Specifically, the structural design of the full-frequency illumination source 13 in this embodiment refers to... Figure 2 and Figure 3This system is designed with a full-frequency illumination source 13 to meet the strong coherence requirements of the illumination light. A laser diode 1 serves as the pump source, and a collimating lens 2 adjusts the divergence angle and beam quality, converting the divergent beam into parallel light. A polarizer 3 selectively controls the polarization state of the light. An aspherical mirror 4 acts as a circular laser generator, converting the Gaussian beam into a ring beam. The ring-shaped laser 5 has a large numerical aperture and a small diffraction limit, suppressing diffraction loss and achieving super-resolution focusing and imaging. Finally, the illumination area is expanded by the first output lens group 6, the second output lens group 7, the third output lens group 8, and the fourth output lens group 9 to meet the illumination requirements of the super-resolution imaging system. The lens group design and the actual optical path structure are referenced. Figure 2 .

[0085] (2) Light guide:

[0086] The light guide 14 in this embodiment is designed with reference to... Figure 2 and Figure 3 Because the intensity and coherence of laser light are affected by attenuation and degradation when propagating in a scattering medium, and the deep network hybrid optimization algorithm requires low transmission loss of the diffracted light field spectrum image and accurate and reliable light wave information, this system designs a separate light guide 14 to ensure the quality of the reconstructed image. The light is converged by the first plano-convex lens 10, and the relay lens group 11, composed of a series of cylindrical lenses, converges the light to the focal point of the second plano-convex lens 12. Finally, the second plano-convex lens 12 converges the parallel light and emits it, thus ensuring higher light transmission efficiency and coherence, significantly improving the quality of the reconstructed image. The lens group design and the actual optical path structure are referenced. Figure 2 .

[0087] (3) Modeling of reflection and reflected light field function:

[0088] In this embodiment, light emitted from the full-frequency illumination source 13 illuminates the observation chip 15 after passing through a light guide, resulting in reflection and forming a reflected light field. The reflected light field is modeled below to provide a more comprehensive description and capture more subtle physical characteristics. Existing general modeling methods struggle to accurately describe the light field in super-resolution imaging to improve image quality and do not allow the imaging system to better adapt to different imaging needs and conditions, especially in the complex and demanding field of chip imaging. Therefore, the reflected light field is modeled here, with R(r,λ,t) representing the reflected light field function. Considering the potential instability of the illumination source during eutectic bonding, and the precision imaging requirements of chip surface circuit patterns, transistors, and other micro-components typically at the micrometer or nanometer scale, edge feature textures are enhanced to achieve multi-scale analysis, improve subsequent image processing capabilities, and enrich neural network training data. The specific model is established as follows:

[0089]

[0090] Where R1(r,t) represents the full-band reflected light field model corresponding to the position vector r at time t, and R(r,θ1,θ2,λ) represents the reflected light field with position vector r, incident angle θ1, reflection angle θ2, and light wavelength λ. i (r,θ1,λ) represents the incident light field with position vector r, incident angle θ1, and wavelength λ, and O(r,θ1,θ2,λ) represents the modulation function of the imaging object with position vector r, incident angle θ1, reflection angle θ2, and wavelength λ. min ,λ max [ ] represents the wavelength range of light, ω represents the angular frequency of light, and k r Let represent the wave vector corresponding to the position vector r, and let i represent the imaginary unit.

[0091] This embodiment enhances imaging capabilities and more accurately captures the microscopic features of the chip through the aforementioned reflection field modeling method. Particularly in high-resolution imaging, the enhanced edge texture and multi-scale analysis provide richer information for subsequent image processing, which is crucial for improving the final image quality. The comprehensiveness of this model allows the imaging system to better adapt to different imaging needs and conditions, especially in the complex and demanding field of chip imaging.

[0092] (4) Optical splitter and bidirectional frequency divider:

[0093] Reference Figure 3 The beam splitter 16 divides the reflected light into two beams, both of which pass through a bidirectional frequency divider 17. The bidirectional frequency divider 17 filters the processed beams. It is worth noting that the bidirectional filter does not simply perform frequency filtering, but rather switches between texture light and structured light at a 500 Hz boundary. Texture light represents the high-frequency components of the beam, and structured light represents the low-frequency components. It is noteworthy that the switching of the bidirectional filter is simultaneous. This is to enable the cross-domain image generation network (XGSN) in the depth wave-Fourier imaging algorithm to achieve better image generation results. The modulation effect of the bidirectional frequency divider can be expressed as:

[0094] R2(r,t)=M(r,t)·R1(r,t) (2)

[0095] The function of M(r,t) is to separate texture light from structure light. At time t0, the bidirectional frequency divider is a high-frequency filter that separates the texture component in the reflected light. At time t0+Δt, it is a low-frequency filter that separates the low-frequency component, i.e., the structural features.

[0096] (5) CCD light field detection:

[0097] like Figure 3The CCD 18 is used to capture light field intensity information. When photons are incident on the surface of the CCD sensor, since the energy of a photon is inversely proportional to its wavelength, the photons are absorbed by the semiconductor material of the CCD sensor, and the photoelectric effect generates electrons. Electrons at each pixel (x,y) gradually accumulate into charge, and the amount of charge is directly proportional to the incident light intensity; brighter areas generate more charge.

[0098] By transferring and reading these charges row by row or column by column, a charge image can be obtained, reflecting the intensity distribution of the reflected light field. The output of a CCD sensor is an image composed of charge quantities, typically represented as a two-dimensional array. Each array element corresponds to a pixel, and its value represents the intensity of the incident light at that location. The overall detected intensity of the reflected light field is represented by Y.

[0099] (6) Wavelet-Fourier imaging algorithm based on cross-domain image generation network:

[0100] Existing imaging algorithms, especially in the field of super-resolution imaging, typically rely on techniques such as structured light illumination, saturated laser technology, or post-processing-based image reconstruction. These techniques aim to improve resolution, but they have significant limitations in terms of processing speed, image quality (especially in noisy environments), and adaptability to complex scenes. Therefore, existing micro-object imaging techniques are more accurately described as image sharpness enhancement techniques and cannot meet the requirements for real-time monitoring of the eutectic patch process.

[0101] The Wavelet-Fourier imaging algorithm 19 based on a cross-domain image generation network proposed in this embodiment employs an optimized deep learning network. This model is optimized for the specific task of eutectic patching and undergoes continuous parameter tuning during training. In practical use, it can complete complex image processing tasks in a relatively short time. These models are typically trained to process specific types of data quickly and efficiently. Furthermore, the algorithm employs parallel feature extraction; Fourier transform and wavelet transform can process image data in parallel, and a bidirectional frequency divider achieves isolated learning of texture and structural information, thereby improving the overall processing speed. This parallel processing can significantly improve the algorithm's operating efficiency. The Wavelet-Fourier imaging algorithm based on a cross-domain image generation network in this embodiment mainly includes:

[0102] I. Perform Fourier transform and wavelet transform on the light intensity information:

[0103] The parameters of the reflected light field can be initialized using formulas (1) and (2), where λ min =400nm,λ max =760nm, other parameters can be initialized and substituted into formulas (1) and (2) to obtain the initial light field R2(r,t).

[0104] The intensity information of the beam after CCD processing is subjected to Fourier transform and used as global image information. The intensity information of another processed beam is first divided into n local intensity information maps according to the region, and each of the n intensity information maps is processed. Taking one of the maps as an example, a continuous wavelet transform (CWT) is first performed to obtain local accurate image information.

[0105]

[0106] Where Y1 represents the Fourier global image information, Y2 represents the wavelet local precise information, a is the scaling parameter used to control the scaling of the wavelet function, b is the translation parameter, by changing b, the characteristics of the signal at different locations can be analyzed, and P(t) is the wavelet basis function. * (t) is the complex conjugate of P(t), and P(t) has the following specific form:

[0107]

[0108] II. Cross-domain image generation network:

[0109] The cross-domain image generation network consists of two branches: a global image branch (Fourier branch) and a local scenic area image branch (wavelet branch). The Fourier branch uses a multi-level GCN to capture the global features of the image, while the wavelet branch extracts the precise features of each local area. After feature extraction, the global features and the precise features of each local area are input into the feature fusion module. Finally, after inverse Fourier transform, the enhanced image is output. The Fourier branch and the wavelet branch will be described in detail below.

[0110] During network training, for the Fourier branch, a sufficiently high-resolution eutectic patch image is used as input. Assuming the original image size is 256*256 pixels, after Fourier transform, we obtain 256*256 complex values, each representing a specific frequency component. These complex values ​​are used as initial feature information. To extract information from surrounding pixels from discrete points, a multi-level graph convolution feature extraction method is employed to gradually refine the extraction of global image features, i.e., the spatial correlation of the image. Assume the features extracted by the j-th level feature extractor are:

[0111]

[0112] The number of layers is set to j, M represents the number of pixels in the global image, and G... j G represents the feature vector output by the j-th level graph convolutional network, M represents the number of pixels in the global image, and G... 0The input to the multi-level graph convolutional network is represented by the 0th-level feature, which is the global image Y1 after Fourier transform. The result of the Fourier transform (i.e., a specific frequency component) is considered as a node in the image. Each node represents a specific frequency component in the image. This represents the feature vector of the k-th point in the output feature vector of the j-th level graph convolutional network. (The last part, "to calculate," appears to be incomplete and requires further context.) Features, in Create a local region Ωj with K elements around it. k ,Right now:

[0113]

[0114] in, and Let G represent the feature vectors output by the j-th level graph convolutional network, respectively. j Features of the k-th and m-th points, This represents the feature distance between the k-th and m-th points. Representative to Find the square of the L2 norm, 1 ≤ k ≤ M, 1 ≤ m ≤ M. Iterate through and calculate all frequency components, where... Indicates except Other elements besides those yielded The smallest first K elements Then, a new calculation is obtained.

[0115]

[0116] in, Let represent the feature of the nth point in the feature vector output by the j-th level graph convolutional network, where 1≤k≤M, 1<K<M, 1≤n≤K, σ represents the non-linear activation function, w is the learnable parameter of the multilayer perceptron, and Max-Pool is the max pooling operation.

[0117] The quality of feature extraction directly affects the quality of the reconstructed image. Common frequency domain feature extraction methods fall into two main categories: directly using the results of Fourier transforms, or using fully connected networks to process frequency domain data. However, these networks cannot effectively simulate the relationships between frequency domain points. These methods have the following drawbacks: (1) They cannot model the correlation between frequency domain information, resulting in insufficient local features; (2) They lose the global structural information of the image, providing insufficient support for reconstruction; (3) The expressive power and hierarchical nature of the extracted features are weak; (4) The final features lack representativeness, offering limited support for improving image quality.

[0118] In this embodiment of the invention, the proposed feature extraction method re-establishes the correlation between points in the frequency domain through a multi-level graph convolutional network of formulas (5), (6), and (7), which can obtain more abstract and higher-order feature representations. This allows for the extraction of richer global image features, which is a highly innovative method. The reason why other existing feature extraction structures are not used here is that most of them are aimed at spatial domain information, while this embodiment processes the frequency domain, and the above-mentioned defects are made up by iteratively aggregating surrounding information and fusing the global frequency domain structure through multi-level network. Therefore, compared with other feature extraction methods, the feature extraction method in this embodiment can achieve better results and provide stronger support for subsequent image reconstruction. The final output frequency domain features fully integrate global information and preserve the overall structure and details of the image, which is also the important value of this design.

[0119] The final extracted Fourier features are The specific expression is as follows:

[0120]

[0121] Among them, G 1,2,3...j This represents the feature information extracted by layers 1, 2, 3...j of the network. MLP stands for Perceptual Function in the feature extraction module of Graph Convolutional Network (GCN). It takes the feature vector as input, passes it through multiple layers and nonlinear transformations, and outputs the extracted features. The complete process is as follows Figure 5 The Fourier branch feature extraction part in the model consists of j GCN modules in its physical structure. The function of a single GCN is shown in formulas (5), (6), and (7). The final feature is output by the j-th and last GCN module.

[0122] For the wavelet branch, the wavelet-transformed Y1 is also used as the initial input. A series of local information after wavelet transformation are shared in the convolutional network layer with the same convolutional kernel as the initial input of the network for convolutional feature extraction.

[0123]

[0124] Where * represents the convolution operation, W (l) With b (l) Represents the weights and biases of the l-th convolutional layer, ReLU represents the activation function, and F... (l) For the output of the l-th convolutional layer, F (l-1) This represents the output of the (l-1)th convolutional layer, where the initial F (0) Y2 is the result of the continuous wavelet transform of formula (3). The complete representation of the convolutional layer operation is as follows:

[0125]

[0126] Where Conv represents the convolution operation and ReLU represents the activation function, the wavelet feature extraction of a certain local image is completed according to formula (4), and n local images are processed to obtain n two-dimensional features.

[0127] It's worth noting that, since the images are localized and the complexity of 2D features decreases continuously during feature extraction, wavelet feature extraction, although a repetitive operation across multiple images, does not affect the imaging speed. The majority of the time is consumed during the training phase, and once the network parameters are adjusted, the algorithm complexity is significantly reduced in practical applications. Specifically, the output size of the convolutional layers from layer 1 to layer 6 is set to 8-8-24-24-16-16. A schematic diagram of the specific network architecture is shown below. Figure 4 The complete process is as follows Figure 5 The wavelet branch.

[0128] Finally, based on the original image S, the features of the Fourier branch and the wavelet branch are fused, and an inverse Fourier transform is performed to obtain the final enhanced image. During training, a loss function is introduced to measure the quality of the generated images:

[0129]

[0130] Loss In the initial training phase, if the loss is too large, adjust each parameter:

[0131]

[0132] in, Here, Γ represents the adjusted parameters, Γ represents the original parameters (including parameters from the light field modeling process, network structure parameters in the Fourier and wavelet branches), and η represents the learning rate, which is set to 6.33 × 10⁻⁶ in this algorithm. -2 Iterative training, when the loss Less than 10 -5 Once the enhanced image is considered to be approximately the same as the original image, training is complete, and the obtained parameters are the final network parameters. Training then ends. The input image can then be replaced with a real-world captured image for image generation.

[0133] In summary, the advantages of this Wave-Fourier imaging algorithm based on a cross-domain image generation network in improving imaging speed mainly stem from its efficient parallel processing capabilities, algorithm optimization, and utilization of modern hardware acceleration technologies. These characteristics make it particularly suitable for applications requiring fast, high-quality imaging in eutectic patches.

[0134] This invention presents a complete super-resolution imaging method, employing a full-frequency illuminator and light guide to provide highly coherent illumination. A reflected light field function model is established to comprehensively describe the light field information. A beam splitter and a bidirectional frequency divider are used to process the reflected light field, extracting structured light and texture light. A CCD captures the light field intensity information and converts it into two-dimensional light intensity distribution data. A deep learning-based wavelet-Fourier imaging algorithm, including Fourier transform, wavelet transform, and a cross-domain image generation network, processes the light intensity data to generate a super-resolution image. The cross-domain image generation network consists of Fourier and wavelet branches, extracting global and local image information respectively, and finally, feature fusion outputs an enhanced image. This method achieves ultra-high resolution, strong noise resistance, and fast imaging speed, effectively solving the chip imaging problem during eutectic bonding. The system has a simple structure, low cost, and is easy to maintain.

[0135] The embodiments of the present invention employ a novel imaging method to replace traditional optical imaging, and design a complete super-resolution chip imaging system that can obtain high-quality images of chip microstructures at high speed, effectively solving the imaging problem in the eutectic bonding process.

[0136] Reference Figure 6 The ultra-micro target imaging system proposed in this embodiment of the invention includes a memory 20, a processor 30, and a computer program stored in the memory 20 and executable on the processor 30. When the processor 30 executes the computer program, it implements the steps of the ultra-micro target imaging method proposed in this embodiment.

[0137] The specific working process and working principle of the ultra-micro target imaging system in this embodiment can be referred to the working process and working principle of the ultra-micro target imaging method in this embodiment.

[0138] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for imaging ultra-micro targets, characterized in that, The method includes: The receiver receives a full-band reflected light field from a micro-target after it is illuminated by a full-band light source, wherein the micro-target is a chip, and the full-band reflected light field from the micro-target includes: It receives the light beam reflected by the light guide after the full-frequency illumination source illuminates the ultra-micro target; Based on the reflected light beam, establish a model of the reflected light field; Based on the reflected light field model, the full-band reflected light field is obtained; The reflected light in the full-band reflected light field is filtered by switching between high frequency and low frequency, and the intensity information of the filtered reflected light is collected. Fourier transform and wavelet transform are performed on the light intensity information to obtain global and local images; Extract global features from the global image and local features from the local image. The extraction of global features from the global image includes: A multi-level graph convolutional network is constructed to extract global features from the global image. The formula for calculating the feature vector extracted by each level of the multi-level graph convolutional network is as follows: ， in, Representing the The feature vector output by the graph convolutional network. Represents the number of pixels in the global image. This represents the input to a multi-level graph convolutional network. Represents the global image. and They represent the first Level and First The first-order feature vector output by a level-1 graph convolutional network Features of each point Indicates the calculation of the first When a graph convolutional network outputs, The surrounding creation with A local region of an element. Indicates the first The first-order feature vector output by a level-1 graph convolutional network Features of points, among which , , , Represents a non-linear activation function. These are the learnable parameters of a multilayer perceptron. This is a max pooling operation; The global features of the global image are obtained based on the feature vectors output by each level of the graph convolutional network. The specific formula is as follows: ， in, Represents global characteristics. , and They represent level 1, level 2, and level 3 respectively. The feature vector output by the graph convolutional network. The perceptual function representing a graph convolutional network; Global and local features are fused, and an enhanced image of the ultra-micro target is obtained based on the fused features.

2. The ultra-micro target imaging method according to claim 1, characterized in that, The specific formula for the reflected light field model is as follows: ， in, express Time, and position vector The corresponding full-band reflected light field model, The position vector is The angle of incidence is The reflection angle is The wavelength of light is The reflected light field, The position vector is The angle of incidence is The wavelength of light is The incident light field, The position vector is The angle of incidence is The reflection angle is The wavelength of light is The modulation function of the imaged object, [ [This refers to the range of light wavelengths.] Represents the angular frequency of light. Represents position vector The corresponding wave vector, It represents the imaginary unit.

3. The ultra-micro target imaging method according to claim 2, characterized in that, Calculate the first When a graph convolutional network outputs, The surrounding creation with The local region of each element includes: Calculate the first The first-order feature vector output by a level-1 graph convolutional network The characteristic distance between a point and other points is given by the following formula: ， in, and They represent the first Feature vectors output by a graph convolutional network The Middle The and the first Features of each point Representing the The and the first Feature distance of each point, Representative to Find the square of the L2 norm. , ; Select those with a distance less than a preset threshold The element is used to calculate the nth element. When a graph convolutional network outputs, The surrounding creation with Local regions of each element ,and .

4. The ultra-micro target imaging method according to claim 3, characterized in that, Extracting local features from a local image includes: A convolution operation with a preset layer threshold is performed on a local image. The specific formula is as follows: ， in, and They represent the first Convolutional layers and the first Local output features of convolutional layers and They represent the first The weights and biases of convolutional layers, Represents the activation function. This indicates the preset layer threshold, and ; The local output features of the last convolutional layer are used as the local features of the local image.

5. The ultra-micro target imaging method according to claim 4, characterized in that, The process of fusing global and local features and obtaining an enhanced image of the ultra-micro target based on the fused features includes: Feature fusion is performed by fusing global features with multiple local features; Perform an inverse Fourier transform on the fused features; An enhanced image of the ultra-micro target is obtained based on the inverse Fourier transform result.

6. The ultra-micro target imaging method according to claim 5, characterized in that, The full-frequency light source comprises a laser diode, a collimating lens, a polarizer, an aspherical mirror, a ring-shaped laser, and first, second, third, and fourth output lens groups connected in sequence, wherein: The laser diode is used as a pump source; The collimating lens is used to adjust the divergence angle and beam quality of the light beam, turning the diverging light beam into parallel light. The polarizer is used to select and control the polarization state of polarized light; The aspherical mirror is used to convert a Gaussian beam into a ring beam. The circular laser is used to suppress diffraction loss and achieve super-resolution focusing and imaging; The first, second, third, and fourth output lens groups are used to expand the illumination area.

7. The ultra-micro target imaging method according to claim 6, characterized in that, The light guide comprises a first plano-convex lens, a relay lens group, and a second plano-convex lens connected in sequence, wherein: The first plano-convex lens is used to converge the light output from the full-frequency light source; The relay lens group is composed of a series of cylindrical lenses, which converges light rays to the focal point of the second plano-convex lens. The second plano-convex lens is used to converge parallel light rays for emission.

8. A microscopic target imaging system, the system comprising: The memory (20), the processor (30), and the computer program stored in the memory (20) and executable on the processor (30) are characterized in that the processor (30) implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.

Images