A micro device pose visual positioning method for a semiconductor substrate

Abstract

The application discloses a micro device pose visual positioning method for a semiconductor substrate, a flexible magnetic positioning subsystem composed of a coil array and a magnetic sensor array arranged on the same flexible substrate is used to perform global six-degree-of-freedom coarse positioning on a magnetic carrier integrated with a permanent magnet mark; then, an imaging probe is guided to reach a target neighborhood based on the coarse positioning pose, and improved YOLO-Pose neural network is used to complete sub-pixel detection of key points of the micro device, the improved network takes CSPDarknet53 as a backbone, embeds deformable convolution and scale self-adaptive modulation, is matched with a dynamic receptive field adjustment module and a hierarchical key point regression head containing multi-scale Fourier feature coding, and finally, six-degree-of-freedom fine positioning pose of the micro device is solved through an EPnP algorithm. The application effectively overcomes the defects of insufficient positioning accuracy, susceptibility to visual interference and easy loss of micro target features of the traditional positioning.

Description

Technical Field

[0001] This invention relates to a visual positioning method for the pose of micro-devices on a semiconductor substrate, belonging to the field of visual positioning in semiconductor packaging. Background Technology

[0002] In the mounting and packaging process of semiconductor substrates, precise positioning of tiny devices is a key step in ensuring packaging quality. As the electronics manufacturing industry moves towards high density and miniaturization, the size of components on circuit boards has entered the submicron and even nanometer scale, which places higher demands on the positioning accuracy and operating efficiency of vision positioning systems.

[0003] Currently, pose localization for micro-devices on circuit boards mainly employs traditional image processing-based visual localization methods and deep learning-based visual localization methods. Traditional image processing-based visual localization methods primarily rely on image processing algorithms for localization, but they exhibit poor robustness in handling complex backgrounds, and their localization accuracy is limited by image resolution, making it difficult to meet sub-micron level precision requirements. While deep learning-based visual localization methods can effectively handle complex background interference, deep learning models are prone to losing feature information of small targets during multiple downsampling processes, and their output bounding box accuracy is limited to the pixel level, failing to directly achieve the sub-pixel accuracy required for industrial applications. Furthermore, these methods demand significant computational resources, making them difficult to deploy in real-time control systems. Summary of the Invention

[0004] The present invention provides a visual positioning method for the pose of micro-devices on a semiconductor substrate in order to solve the problems existing in the prior art.

[0005] The technical solutions adopted in this invention are as follows:

[0006] A method for visually locating the pose of a micro-device on a semiconductor substrate, wherein the micro-device is a chip-type electronic component mounted on the semiconductor substrate, the method comprising the following steps:

[0007] S1: A magnetic carrier with integrated permanent magnet markers used to support a semiconductor substrate is coarsely positioned globally using a magnetic positioning subsystem to obtain the six-degree-of-freedom coarse positioning pose of the magnetic carrier; wherein, the magnetic positioning subsystem includes a coil array for generating an encoded magnetic field and a magnetic sensor array for detecting the magnetic field distribution, and the coil array and the magnetic sensor array are arranged on the same flexible substrate.

[0008] S2: Based on the six-degree-of-freedom coarse positioning pose guidance imaging probe, the probe moves to the target neighborhood where the micro-device is located. The sub-pixel coordinates of key points of the micro-device are detected using an improved YOLO-Pose neural network. The improved YOLO-Pose neural network includes:

[0009] A multi-scale feature extraction backbone network is used to extract multi-scale image features;

[0010] The dynamic receptive field adjustment module is implemented through deformable convolution and modulates the scale-related offset according to the target device scale to adaptively adjust the convolutional receptive field.

[0011] The hierarchical keypoint regression head includes a coarse regression branch that outputs integer coordinates and a fine regression branch that outputs sub-pixel offsets based on multi-scale Fourier feature encoding.

[0012] S3: The six-degree-of-freedom precise positioning pose of the micro-device is obtained by using the sub-pixel coordinates of the key points through the EPnP algorithm.

[0013] Furthermore, in step S2, the key points to be detected include the corners, edges, and pin endpoints of chip-type electronic components.

[0014] Furthermore, the imaging probe is a miniature imaging probe with a diameter of 1~3mm, and the imaging probe includes a miniature camera module or an optical fiber image transmission bundle, and the imaging probe is mounted on a three-degree-of-freedom precision motion platform.

[0015] Furthermore, the multi-scale feature extraction backbone network is a CSPDarknet53 structure. The CSPDarknet53 structure includes an initial downsampling layer and four concatenated feature extraction stages. The four feature extraction stages correspond to feature map outputs at different scales. Deformable convolution and scale-adaptive modulation are set in the four feature extraction stages to achieve multi-scale feature extraction. That is, the multi-scale feature extraction backbone network adopts a CSPDarknet53 structure including a Stem layer and Stage 1 to Stage 4, and deformable convolution and scale-adaptive modulation are introduced in Stage 1 to Stage 4.

[0016] Furthermore, the scale-adaptive modulation is achieved through a modulation factor. This modulation factor, based on the ratio of the target device scale to the reference scale, uses a hyperbolic tangent function to suppress or amplify the scale-related offset. The modulation factor is expressed as:

[0017] ,

[0018] The modulated offset is expressed as:

[0019] ,

[0020] in, For the target device scale, As a reference scale, when Less than When the modulation factor approaches zero, it suppresses the offset; when and At the same order of magnitude or Greater than At this time, the modulation factor approaches one to allow the offset to increase; For learnable offsets of deformable convolution, It is the hyperbolic tangent function. This is a modulated identifier.

[0021] Furthermore, the dynamic receptive field adjustment module estimates the target device size through a scale prediction branch. And based on the target device size and the minimum detectable size The ratio is used to dynamically calculate the optimal void ratio. To adjust the size of the convolutional receptive field, the optimal dilation rate is expressed as:

[0022] ,

[0023] in is the scaling factor, and the receptive field size is proportional to the optimal porosity.

[0024] Furthermore, the multi-scale Fourier feature encoding employs multiple frequency scale components to map continuous coordinates to a high-dimensional feature space. The fine regression branch outputs sub-pixel offsets based on linear regression in the high-dimensional feature space. The Fourier encoding is expressed as follows:

[0025] ,

[0026] in, For Fourier coding functions, Continuous coordinates of key points It is a cosine function. It is a sine function. Pi For matrix transpose, The elements are random matrices that follow a Gaussian distribution. The encoding of the multiple frequency scale components is obtained by concatenating sine and cosine components of different frequency scales.

[0027] Furthermore, the improved YOLO-Pose neural network also includes a geometrically constrained attention module for applying geometrical positional constraints to key points of the microdevice.

[0028] Further, in step S1, the global coarse localization of the magnetic vehicle is initially estimated using a stochastic composite algorithm and iteratively optimized using a Levenberg-Marquardt nonlinear optimization algorithm to obtain the six-degree-of-freedom coarse localization pose of the magnetic vehicle.

[0029] Furthermore, in step S2, the process of guiding the imaging probe to move to the target neighborhood includes: determining the confidence ellipse based on the covariance matrix of the six-degree-of-freedom coarse positioning pose, ensuring that the initial deployment position of the imaging probe ensures that the micro-device falls within the field of view with a probability greater than 99%, and adjusting the optical axis direction of the imaging probe according to the attitude information of the six-degree-of-freedom coarse positioning pose.

[0030] Furthermore, the training of the improved YOLO-Pose neural network employs a weighted loss function, which is expressed as follows:

[0031] ,

[0032] in, For coarse regression loss, For fine regression loss, The loss is calculated using scale, and the weighting coefficients of the fine regression loss are greater than those of the coarse regression loss.

[0033] The present invention has the following beneficial effects:

[0034] (1) By using the fine regression branch based on multi-scale Fourier feature encoding in the hierarchical key point regression head, continuous coordinates are mapped to high-dimensional feature space and sub-pixel offsets are regressed, enabling the network to express fine positional relationships at the sub-pixel level, thus breaking through the bottleneck of traditional deep learning model output being limited to pixel-level precision.

[0035] (2) The magnetic-visual dual-stage architecture enables the vision subsystem to directly enter the local target neighborhood for detection under the guidance of magnetic coarse localization without performing a global search in the full image, thus reducing the invalid computation area; at the same time, the improved neural network structure achieves a balance between the number of parameters and computational efficiency, which can meet the processing speed requirements of the real-time control system.

[0036] (3) The magnetic positioning subsystem first achieves coarse positioning in the global range, providing prior pose information for the ultra-micro vision subsystem, guiding it into the local precise observation area, and avoiding the problems of limited field of view of the high magnification vision subsystem and difficulty in distinguishing small details of the large field of view vision subsystem.

[0037] (4) The coil array and magnetic sensor array of the magnetic positioning subsystem are arranged on the same flexible substrate. They can be bent and bonded according to the geometry of the semiconductor substrate to form a non-planar distributed reconfigurable magnetic field source. Compared with rigid magnetic positioning devices, it can adapt to load-bearing scenarios with curved or non-planar features.

[0038] (5) The dynamic receptive field adjustment module dynamically adjusts the convolutional receptive field according to the actual size of the target device through deformable convolution and scale-adaptive modulation, so that the network can maintain the ability to focus on local details for small-sized devices and maintain the ability to capture the overall structure for large-sized devices, thereby reducing feature loss or detection errors caused by scale differences.

[0039] (6) Magnetic coarse positioning is based on magnetic field measurement and is not affected by visual interference factors such as changes in lighting, surface reflection, silk screen and pad texture. It can provide reliable prior guidance for fine positioning under limited visual conditions or complex backgrounds, and improve the environmental adaptability of the overall positioning system. Attached Figure Description

[0040] Figure 1 This is a hardware system architecture diagram on which the method of the present invention is based.

[0041] Figure 2 This is a schematic diagram of the coarse positioning solution for the magnetic sensor array.

[0042] Figure 3 A schematic diagram of the structure of the improved YOLO-Pose neural network.

[0043] Figure 4 This is a schematic diagram of Fourier feature encoding calculation.

[0044] In the picture:

[0045] 1. Magnetic positioning subsystem; 2. Vision subsystem; 10. Semiconductor substrate; 11. Flexible substrate; 12. Coil array; 13. Magnetic sensor array; 14. Magnetic carrier; 140. Permanent magnet marker. Detailed Implementation

[0046] The invention will now be further described with reference to the accompanying drawings.

[0047] This invention provides a visual positioning method for the pose of micro-devices on semiconductor substrates. It achieves coarse positioning over a global range through a magnetic sensor array, guides the ultra-micro vision subsystem into a local precise observation area, and then achieves sub-micron level fine positioning, thus resolving the contradiction between accuracy and field of view, adaptability and stability in the prior art.

[0048] The hardware system on which this invention is based is such as Figure 1 The method of the present invention will be described in detail below in conjunction with the hardware system.

[0049] First, the corresponding hardware system is deployed, which consists of two parts: magnetic positioning subsystem 1 and vision subsystem 2.

[0050] The magnetic positioning subsystem includes a coil array 12, a magnetic sensor array 13, a magnetic positioning calculation unit, and a magnetic carrier 14 with an integrated permanent magnet marker 140 attached to the flexible substrate 11. The permanent magnet marker 140 is integrated at the bottom of the magnetic carrier 14, which is used to support the semiconductor substrate 10.

[0051] The magnetic sensor array 13 (four magnetic sensors) and the coil array 12 (four coils) are coplanarly integrated on the same flexible substrate 11, as shown in the arrangement diagram. Figure 1 The magnetic sensor array is distributed among the gaps in the coil array, with each magnetic sensor positioned at the geometric center of four adjacent coils. Electromagnetic isolation between the coil array and the magnetic sensor array is achieved through the coil layer of the flexible substrate. The coil array is used to generate the coded magnetic field, while the magnetic sensor array is used to detect the magnetic field distribution.

[0052] The magnetic positioning calculation unit is based on the Randomized Component Analysis (RCA) algorithm and the Levenberg-Marquardt nonlinear optimization algorithm to calculate the three-dimensional position and orientation of the micro magnetic markers in real time.

[0053] The vision subsystem includes an imaging probe, a multispectral illumination module, and a vision processing unit.

[0054] The imaging probe employs a miniature camera module or fiber optic image transmission bundle with a diameter of 1-3 mm, mounted on a three-degree-of-freedom precision motion platform. The multispectral illumination module integrates coaxial illumination and low-angle ring illumination to adapt to the inspection needs of devices with different surface characteristics. The vision processing unit executes an improved neural network visual positioning algorithm, achieving high-precision pose extraction by fusing multiple feature extraction and detection methods.

[0055] The mathematical modeling and coarse localization solution of the magnetic sensor array 13 are as follows:

[0056] like Figure 2 As shown, the coil array consists of... Composed of planar spiral coils, the first The geometric center of each coil is located at... The normal direction is When current flows through the coil At that time, its spatial location The magnetic induction intensity generated at the location for:

[0057] ,

[0058] in The permeability of free space, For the trajectory of the coil wire, For line element vectors, This refers to the position of the line element.

[0059] For radius A circular planar coil, when the observation point A is located on the coil axis at a distance... At this point, the axial magnetic field component can be analytically expressed as:

[0060] ,

[0061] To enhance positioning robustness, this invention employs time-division multiplexing to drive multiple coils to generate a spatiotemporally encoded magnetic field. Let the first... The driving current vector at each moment is Then spatial location The resultant magnetic field at that location is:

[0062] ,

[0063] in, For the first The coil is in position The magnetic field response function of a unit current at a given location.

[0064] The micro-magnetic marker (point B) attached to the magnetic carrier can be modeled as a magnetic dipole model (the micro-magnetic marker is permanently embedded in the magnetic carrier and does not need to be attached to the circuit board). Let the magnetic moment of the magnetic marker be... The location is Then the magnetic induction intensity it produces at its spatial location is:

[0065] ,

[0066] in, The vector pointing from the magnetic marker to the sensor. Right now The model, It is a unit vector.

[0067] The above equation can be expanded into matrix form:

[0068] ,

[0069] in, It is a 3×3 magnetic coupling matrix. It is a third-order identity matrix.

[0070] When the magnetic marker's orientation changes, the direction of its magnetic moment also changes. Let the rotation matrix of the magnetic marker's local coordinate system relative to the global coordinate system be... ,in The azimuth angle is given by the inherent magnetic moment. ,but:

[0071] ,

[0072] Therefore, the complete pose parameters of the magnetic marker are: The magnetic field it generates is a nonlinear function of position and attitude:

[0073] ,

[0074] Magnetic positioning requires inferring the magnetic marker's pose by measuring the magnetic field. Assume the magnetic sensor array contains... The sensor, the first The sensor locations are The measured value is ,Will The measurements from each sensor are constructed into a measurement vector. Let the theoretical model predict the value. The error between the theoretical value and the actual measured value is set as the residual vector. ,but:

[0075] ,

[0076] At this point, pose estimation is transformed into a nonlinear least squares optimization problem, namely:

[0077] ,

[0078] The Levenberg-Marquardt algorithm is used for iterative solution.

[0079] Define the Jacobian matrix : .

[0080] The iterative update formula is: ,in is the damping coefficient.

[0081] To improve computational efficiency, the stochastic composite algorithm (RCA) is used for initial value estimation. RCA constructs multiple low-dimensional linear systems by randomly sampling a subset of sensors, uses median filtering to suppress outliers, and quickly obtains a rough estimate of the pose, which serves as the initial value for the iteration of the LM algorithm.

[0082] The output of magnetic coarse positioning is the six-degree-of-freedom pose estimate. This enables coarse positioning at the sub-millimeter level.

[0083] Based on the prior information provided by magnetic coarse positioning, according to The location information is transformed through a conversion matrix between the magnetic positioning subsystem and the visual subsystem. Transform to the vision subsystem coordinate system and guide the imaging mechanism (which can be a ball screw mechanism) to move the vision probe to the target neighborhood.

[0084] Define the magnetic coarse positioning position confidence ellipse:

[0085] ,

[0086] in The magnetic positioning covariance matrix is... The 99% confidence interval is for a chi-square distribution with 3 degrees of freedom. The initial deployment position of the vision probe should ensure that the target falls within the field of view with a high probability (>99%).

[0087] Then, based on the attitude information provided by magnetic coarse positioning Adjust the optical axis of the vision probe to roughly align it with the normal of the target surface to ensure image clarity.

[0088] This invention proposes a key point detection algorithm based on an improved YOLO-Pose neural network. Key points vary depending on the type of substrate under test, requiring manual pre-labeling of the position and order of key points for each device type. Through training, the neural network can automatically detect key points. For example, for chip-type micro-devices, key points are typically the four corner points, edge points, and pin endpoints.

[0089] After acquiring images of tiny devices on a semiconductor substrate, the key point coordinates of the target device are output by improving the processing of the YOLO-Pose neural network. Then, the precise pose information is obtained from these key point coordinates by the EPnP (Efficient Perspective-n-Point) algorithm.

[0090] Existing YOLO-Pose networks suffer from drawbacks when processing ultra-miniature devices, including high error rates in detecting small targets, inability to represent fine positional relationships at the sub-pixel level, and fixed receptive fields. This invention addresses these shortcomings by optimizing and improving the network structure. The network structure mainly includes a multi-scale feature extraction backbone, a sub-pixel position encoding module, a dynamic receptive field adjustment module, a geometrically constrained attention module, and a hierarchical keypoint regression head. The improved YOLO-Pose neural network structure is as follows: Figure 3 As shown.

[0091] The backbone network adopts an improved CSPDarknet53 structure, which extracts high-level semantic features step by step through 5 stages, as shown in Table 1.

[0092] Table 1

[0093]

[0094] This invention introduces deformable convolution, allowing sampling points to adaptively adjust according to the target shape. The output of standard convolution is located at the image position. The value at that location is:

[0095] ,

[0096] in The sampling grid for the convolution kernel, This is the offset within the grid. The learnable offsets for deformable convolutions are learned from the input feature map through additional convolutional layers, thus enabling image positioning. The convolution output at point is:

[0097] ,

[0098] For micro-devices on semiconductor substrates, excessive offset can lead to feature loss. This invention introduces a modulation factor. This makes the offset correlated with the target scale. Let the final offset be... ,but:

[0099] ,

[0100] in, The target scale is provided by the scale prediction branch; For reference scale; It is the hyperbolic tangent function. This is a modulated identifier.

[0101] When the device size is very small , The offset is suppressed; when the device size is large, This allows for greater deformation. The final output image position... The value at that location is:

[0102] ,

[0103] The backbone network outputs feature maps at three scales:

[0104] P3 (80×80×128): Preserves high-resolution details, suitable for small target positioning;

[0105] P4 (40×40×256): Medium resolution, suitable for medium-sized targets;

[0106] P5 (20×20×512): Low resolution but high semantics, suitable for positioning large targets (BGA package).

[0107] To accommodate devices of different sizes, the network needs to dynamically adjust its receptive field. First, the scale prediction branch estimates the size of the target device:

[0108] ,

[0109] in For feature maps, The scale prediction convolutional layer outputs a positive scale value after an exponential transformation. The loss function is:

[0110] ,

[0111] in This represents the actual dimensions of the device.

[0112] To expand the receptive field, dilated convolutions are introduced into the scale prediction convolutional layer, and the optimal dilation rate is dynamically calculated based on the target scale. :

[0113] ,

[0114] in, It is the smallest detectable scale. This is the scaling factor (taken as 2.5 based on experience).

[0115] After obtaining the void ratio, sense the size of the wild and Proportional to the feature map, dilated convolution is performed:

[0116] ,

[0117] Feel the size of the wild This is proportional to the size of the sensor, allowing for a smaller field of view for small components, focusing on local details, and a larger field of view for large components, capturing the overall structure.

[0118] The hierarchical keypoint regression (head network) is as follows:

[0119] The coarse regression branch outputs integer coordinates, which serve as anchor points for subsequent fine regression.

[0120] ,

[0121] The loss function used is L1 loss:

[0122] ,

[0123] in, For the first Predicted integer coordinates of key points For the first The true integer coordinates of each key point.

[0124] Integer coordinates cannot express positional relationships with sub-pixel precision. This invention introduces Fourier feature encoding to map continuous coordinates to a high-dimensional feature space, and then obtains the sub-pixel offset through linear regression. Its working principle is as follows: Figure 4 As shown. For continuous coordinates Fourier coding is defined as:

[0125] ,

[0126] in It is a random matrix whose elements follow a Gaussian distribution. ; For Fourier coding functions, It is a cosine function. It is a sine function. Pi This is the matrix transpose.

[0127] To capture information at different frequencies, this invention uses multiple frequency scales:

[0128] ,

[0129] in , (Corresponding to a 1-pixel period) .

[0130] Subpixel offset regression takes features around the coarse coordinates as input and outputs the subpixel offset through linear regression:

[0131] ,

[0132] in For learnable weight matrix, For bias.

[0133] Fine regression loss uses smoothed L1 loss:

[0134] ,

[0135] in, The loss function is a combination of L1 loss and L2 loss, meaning that L2 loss is used for small errors and L1 loss is used for large errors. Indicates the first The true offset of each key point relative to the coarse coordinates Indicates the first Predicted integer coordinates of key points.

[0136] The final loss function of the neural network is the weighted sum of the above loss functions, that is:

[0137] ,

[0138] The empirical values ​​for the weighting coefficients are: , , .

[0139] The improved YOLO-Pose neural network outputs the coordinate information of key points. Using this key point coordinate data, the present invention uses the EPnP (Efficient Perspective-n-Point) algorithm to finally measure the pose information of the micro device.

[0140] To verify the improved technical effect of this invention, a comparison was made between existing technologies and their effects on the pose positioning of micro-devices on semiconductor substrates with different sizes, such as 0.16 × 0.08 mm, 0.6 × 0.3 mm, and 1.0 × 0.5 mm. The results are shown in Table 2. As can be seen from the table, the method proposed in this invention has higher accuracy and faster operating efficiency.

[0141] Table 2 Comparison of the effects of micro-device pose localization methods

[0142]

[0143] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements without departing from the principle of the present invention, and these improvements should also be considered within the scope of protection of the present invention.

Claims

1. A method for visually locating the pose of a micro-device on a semiconductor substrate, wherein the micro-device is a chip-type electronic component mounted on the semiconductor substrate, characterized in that... The method includes the following steps: S1: A magnetic carrier with integrated permanent magnet markers used to support a semiconductor substrate is coarsely positioned globally using a magnetic positioning subsystem to obtain the six-degree-of-freedom coarse positioning pose of the magnetic carrier; wherein, the magnetic positioning subsystem includes a coil array for generating an encoded magnetic field and a magnetic sensor array for detecting the magnetic field distribution, and the coil array and the magnetic sensor array are arranged on the same flexible substrate. S2: Based on the six-degree-of-freedom coarse positioning pose guidance imaging probe, the probe moves to the target neighborhood where the micro-device is located. The sub-pixel coordinates of key points of the micro-device are detected using an improved YOLO-Pose neural network. The improved YOLO-Pose neural network includes: A multi-scale feature extraction backbone network is used to extract multi-scale image features; The dynamic receptive field adjustment module is implemented through deformable convolution and modulates the scale-related offset according to the target device scale to adaptively adjust the convolutional receptive field. The hierarchical keypoint regression head includes a coarse regression branch that outputs integer coordinates and a fine regression branch that outputs sub-pixel offsets based on multi-scale Fourier feature encoding. S3: The six-degree-of-freedom precise positioning pose of the micro-device is obtained by using the sub-pixel coordinates of the key points through the EPnP algorithm.

2. The method for visual positioning of micro-devices on a semiconductor substrate as described in claim 1, characterized in that, In step S2, the key points for detection include the corners, edges, and pin endpoints of chip-type electronic components.

3. The method for visual positioning of micro-devices on a semiconductor substrate as described in claim 1, characterized in that, The multi-scale feature extraction backbone network is a CSPDarknet53 structure. The CSPDarknet53 structure includes an initial downsampling layer and four concatenated feature extraction stages. The four feature extraction stages correspond to feature map outputs at different scales. Deformable convolution and scale-adaptive modulation are set in the four feature extraction stages to achieve multi-scale feature extraction.

4. The method for visual positioning of micro-devices on a semiconductor substrate as described in claim 3, characterized in that, The scale-adaptive modulation is achieved through a modulation factor, which, based on the ratio of the target device scale to the reference scale, uses a hyperbolic tangent function to suppress or amplify the scale-related offset. The modulation factor is expressed as: ， The modulated offset is expressed as: ， in, For the target device scale, As a reference scale, when Less than When the modulation factor approaches zero, it suppresses the offset; when and At the same order of magnitude or Greater than At this time, the modulation factor approaches one to allow the offset to increase; For learnable offsets of deformable convolution, It is the hyperbolic tangent function. This is a modulated identifier.

5. The method for visual positioning of micro-devices on a semiconductor substrate as described in claim 1, characterized in that, The dynamic receptive field adjustment module estimates the target device size through a scale prediction branch. And based on the target device size and the minimum detectable size The ratio is used to dynamically calculate the optimal void ratio. To adjust the size of the convolutional receptive field, the optimal dilation rate is expressed as: ， in is the scaling factor, and the receptive field size is proportional to the optimal porosity.

6. The method for visual positioning of micro-devices on a semiconductor substrate as described in claim 1, characterized in that, The multi-scale Fourier feature encoding employs multiple frequency scale components to map continuous coordinates to a high-dimensional feature space. The fine regression branch outputs sub-pixel offsets based on linear regression in the high-dimensional feature space. The multi-scale Fourier feature encoding is expressed as follows: ， in, For Fourier coding functions, Continuous coordinates of key points It is a cosine function. It is a sine function. Pi For matrix transpose, The elements are random matrices that follow a Gaussian distribution. The encoding of the multiple frequency scale components is obtained by concatenating sine and cosine components of different frequency scales.

7. The method for visual positioning of micro-devices on a semiconductor substrate as described in claim 1, characterized in that, The improved YOLO-Pose neural network also includes a geometrically constrained attention module for applying geometrical positional constraints to key points of the microdevice.

8. The method for visual positioning of micro-devices on a semiconductor substrate as described in claim 1, characterized in that, In step S1, the global coarse localization of the magnetic vehicle is performed by initial value estimation through a stochastic composite algorithm and iterative optimization through a Levenberg-Marquardt nonlinear optimization algorithm to obtain the six-degree-of-freedom coarse localization pose of the magnetic vehicle.

9. The method for visual positioning of micro-devices on a semiconductor substrate as described in claim 1, characterized in that, In step S2, the process of guiding the imaging probe to move to the target neighborhood includes: determining the confidence ellipse based on the covariance matrix of the six-degree-of-freedom coarse positioning pose, ensuring that the initial deployment position of the imaging probe ensures that the micro-device falls within the field of view with a probability greater than 99%, and adjusting the optical axis direction of the imaging probe according to the attitude information of the six-degree-of-freedom coarse positioning pose.

10. The method for visual positioning of micro-devices on a semiconductor substrate as described in claim 1, characterized in that, The improved YOLO-Pose neural network is trained using a weighted loss function, which is expressed as follows: ， in, For coarse regression loss, For fine regression loss, The loss is calculated using scale, and the weighting coefficients of the fine regression loss are greater than those of the coarse regression loss.

Images