A semiconductor chip gold wire segmentation method and system based on deep learning

Abstract

The application relates to the technical field of semiconductor package detection, and provides a semiconductor chip gold wire segmentation method and system based on deep learning. A multi-view projection image of a gold wire is acquired; image features are extracted and cross-view fusion is carried out to obtain a three-dimensional feature body; a three-dimensional probability graph of the gold wire is generated; a center line point set is extracted and a three-dimensional space curve is reconstructed; and the curve is projected to each image to obtain a two-dimensional segmentation result. Through multi-view image fusion and three-dimensional space curve reconstruction, accurate positioning of the gold wire in a three-dimensional space and consistent segmentation in a two-dimensional image are realized, complex scenes such as small gold wires, reflection and shielding can be effectively coped with, and the method has the advantages of high segmentation precision, good robustness and strong generalization ability.

Description

Technical Field

[0001] This application relates to semiconductor packaging inspection technology, and more specifically, to a method and system for gold wire segmentation of semiconductor chips based on deep learning. Background Technology

[0002] In semiconductor chip packaging, gold wires (also known as bonding wires) are used to connect chip electrodes to the lead frame, enabling the transmission of electrical signals between the chip and external circuits. The quality of the gold wires directly affects the reliability and lifespan of the chip; therefore, the visual inspection of the gold wires is a crucial step in the packaging process.

[0003] Traditional gold wire inspection mainly relies on manual visual inspection or machine vision methods based on two-dimensional images. Manual inspection is inefficient, highly subjective, and difficult to meet the needs of large-scale production. Machine vision methods based on two-dimensional images typically segment gold wires using a single image, but gold wires are small in diameter (usually between 15-50 micrometers), have strong surface reflectivity, are easily confused with the background, and suffer from severe occlusion and overlap problems in multi-chip stacking or high-density packaging scenarios, resulting in inaccurate single-view segmentation results.

[0004] In recent years, multi-view imaging technology has been introduced into the field of gold wire inspection. This involves simultaneously acquiring projected images from different angles using multiple image acquisition devices positioned around the chip package, and then performing 3D reconstruction based on stereo vision principles. However, existing methods typically separate 3D reconstruction from gold wire segmentation: first, 3D reconstruction is performed to obtain the 3D point cloud or volume data of the gold wire, and then segmentation is performed based on the reconstruction results. This separate processing has the following problems: First, the 3D reconstruction process does not fully consider the geometric prior information of the gold wire (such as the catenary morphology), resulting in poor reconstruction of occluded areas and a tendency to produce breaks or distortions; second, when extracting the centerline of the gold wire from the 3D volume data, existing methods struggle to effectively handle skeleton bifurcation and burrs, leading to inaccurate centerline extraction; third, the final gold wire segmentation result is output in 3D form, which is incompatible with traditional 2D inspection processes and makes it difficult to compare and analyze with chip package layout design data.

[0005] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0006] The purpose of this application is to provide a method and system for segmenting gold wires in semiconductor chips based on deep learning. This method has the advantages of enabling the collaborative processing of three-dimensional reconstruction and two-dimensional segmentation of gold wires in multi-view images, thereby improving the robustness, continuity and generalization ability of the segmentation results.

[0007] Firstly, this application provides a deep learning-based method for gold wire segmentation in semiconductor chips, comprising: Multiple projection images of the gold wire at different angles are acquired, and the projection images are simultaneously acquired by multiple image acquisition devices arranged around the chip package. The multiple projected images are input into a feature extraction network to obtain a feature map corresponding to each projected image; Cross-view feature fusion is performed on the feature maps of multiple projected images to obtain a fused three-dimensional feature volume; the cross-view feature fusion includes aligning and aggregating features of different projected images at the same depth layer to generate the three-dimensional feature volume; A three-dimensional probability map of the gold wire is generated based on the three-dimensional feature volume, and each voxel value in the three-dimensional probability map represents the probability that the position belongs to the surface of the gold wire. The centerline point set of the gold line is extracted based on the three-dimensional probability map, and the three-dimensional spatial curve of the gold line is reconstructed based on the centerline point set. The reconstructed three-dimensional spatial curve is projected onto the imaging plane of each projected image to obtain the two-dimensional segmentation result of the gold line on each projected image.

[0008] Furthermore, the feature extraction network includes a first encoding branch and a second encoding branch, wherein the number of convolutional layers in the first encoding branch is less than the number of convolutional layers in the second encoding branch; the step of inputting the multiple projected images into the feature extraction network to obtain a feature map corresponding to each projected image includes: Each projected image is input into the first encoding branch and the second encoding branch respectively to obtain a first feature sequence and a second feature sequence. The first feature sequence is the detail feature map output by the front-level convolutional layer of the feature extraction network, and the second feature sequence is the semantic feature map output by the back-level convolutional layer of the feature extraction network. A lateral connection is established between the first feature sequence and the second feature sequence. The lateral connection is used to perform channel concatenation of the detail feature map in the first feature sequence and the semantic feature map of the corresponding resolution in the second feature sequence to obtain the feature map of each projected image.

[0009] Furthermore, the cross-view feature fusion of the feature maps from multiple projected images to obtain the fused three-dimensional feature body includes: Based on the optical center position, imaging angle, and lens focal length of each image acquisition device, establish the mapping relationship between pixel coordinates and three-dimensional spatial coordinates of each projected image; Based on the mapping relationship, the feature maps of each projected image are back-projected onto a unified three-dimensional spatial grid to obtain the three-dimensional feature sub-volumes corresponding to each projected image; The three-dimensional feature volume is obtained by weighted averaging of the feature vectors of multiple three-dimensional feature sub-volumes at the same spatial location. The weights of the weighted average are determined by the imaging quality scores of each projected image at that spatial location.

[0010] Furthermore, the generation of a three-dimensional probability map of the gold wire based on the three-dimensional feature volume includes: The three-dimensional feature volume is input into a three-dimensional convolutional network, which includes multiple cascaded three-dimensional convolutional layers, each of which is followed by a non-linear activation layer. The output of the three-dimensional convolutional network is connected to a voxel classification layer, which is used to classify each voxel in the three-dimensional feature volume and output the three-dimensional probability map.

[0011] Further, the step of extracting the centerline point set of the gold line based on the three-dimensional probability map and reconstructing the three-dimensional spatial curve of the gold line based on the centerline point set includes: The three-dimensional probability map is segmented by a threshold, and points with probability values ​​higher than a preset threshold are taken as target points; The target point is divided into multiple point sets according to its spatial location, and the geometric center of each point set is calculated to obtain a sequence of center points; The sequence of center points is sorted according to their spatial location and connected sequentially to form segmented straight lines, thus obtaining the three-dimensional spatial curve.

[0012] Furthermore, the reconstructed three-dimensional spatial curve is projected onto the imaging plane of each projected image to obtain the two-dimensional segmentation results of the gold lines on each projected image, including: Based on the optical center position, imaging angle and lens focal length of each image acquisition device, establish the projection transformation relationship between the three-dimensional space curve and the imaging plane of each projected image. Based on the projection transformation relationship, the projection segmented lines of the three-dimensional space curve on each projection image are calculated. Taking each segment of the projection segmented lines as the center line, the lines are expanded to both sides with a preset width along the normal direction to generate multiple rectangular segmented regions. The multiple rectangular segmented regions are merged to obtain a continuous strip segmented region, which is the two-dimensional segmentation result of the gold line.

[0013] Furthermore, taking each segment of the projected segmented line as a center line, a predetermined width is extended to both sides along the normal direction to generate a strip-shaped segmented region, including: Calculate the inclination angle of each segment of the projected segmented straight line, where the inclination angle is the angle between the segment and the horizontal direction; The tilt angle is compared with multiple preset angle intervals. Each angle interval corresponds to a fixed expansion width. The smaller the tilt angle, the larger the corresponding expansion width, and the larger the tilt angle, the smaller the corresponding expansion width. Using each line segment as the center line, the rectangular segmented region is generated by expanding to both sides along the normal direction with the corresponding expansion width.

[0014] Secondly, this application also proposes a deep learning-based semiconductor chip gold wire segmentation system, comprising: The image acquisition module is used to acquire multiple projected images of the gold wire at different angles. The projected images are acquired synchronously by multiple image acquisition devices arranged around the chip package. The feature extraction module is used to input the multiple projected images into the feature extraction network to obtain the feature map corresponding to each projected image; The feature fusion module is used to perform cross-view feature fusion on the feature maps of multiple projected images to obtain a fused three-dimensional feature volume; the cross-view feature fusion includes aligning and aggregating features of different projected images on the same depth layer to generate the three-dimensional feature volume; The probability generation module is used to generate a three-dimensional probability map of the gold wire based on the three-dimensional feature volume, wherein each voxel value in the three-dimensional probability map represents the probability that the position belongs to the surface of the gold wire; The curve reconstruction module is used to extract the centerline point set of the gold line according to the three-dimensional probability map, and reconstruct the three-dimensional spatial curve of the gold line based on the centerline point set. The projection segmentation module is used to project the reconstructed three-dimensional spatial curve onto the imaging plane of each projection image to obtain the two-dimensional segmentation result of the gold line on each projection image.

[0015] Thirdly, this application also proposes an electronic device comprising: one or more processors, and a memory for storing one or more computer programs; the computer programs are configured to be executed by the one or more processors, and the programs include steps for performing the deep learning-based semiconductor chip gold wire segmentation method as described in the first aspect.

[0016] Fourthly, this application also proposes a storage medium storing a computer program; the program is loaded and executed by a processor to implement the steps of the deep learning-based semiconductor chip gold wire segmentation method as described in the first aspect.

[0017] As can be seen from the above, the semiconductor chip gold wire segmentation method and system provided in this application, based on deep learning, solves the problem of decreased segmentation accuracy caused by occlusion, reflection and deformation in multi-view gold wire image segmentation by constructing a deformable three-dimensional space template to guide feature fusion, cross-view back projection and weighted aggregation, and piecewise straight line reconstruction and dynamic width expansion. It has the advantages of being able to achieve high-precision three-dimensional reconstruction and two-dimensional projection segmentation of gold wire morphology under complex packaging structures, and improving the accuracy and robustness of semiconductor chip gold wire detection. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the steps of the deep learning-based semiconductor chip gold wire segmentation method disclosed in an embodiment of the present invention. Figure 2 This is a flowchart illustrating the steps of performing cross-view feature fusion on feature maps of multiple projected images to obtain a fused three-dimensional feature body, as disclosed in an embodiment of the present invention. Figure 3 This is a flowchart illustrating the steps of the method disclosed in this embodiment of the invention for extracting the centerline point set of a gold line based on the three-dimensional probability map and reconstructing the three-dimensional spatial curve of the gold line based on the centerline point set. Figure 4 This is a flowchart illustrating the method steps of projecting the reconstructed three-dimensional spatial curve onto the imaging plane of each projected image to obtain the two-dimensional segmentation result of the gold line on each projected image, as disclosed in an embodiment of the present invention. Figure 5 This is a schematic diagram of the semiconductor chip gold wire segmentation system based on deep learning disclosed in an embodiment of the present invention. Detailed Implementation

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which these embodiments belong; the terminology used herein and in the specification of the application is for the purpose of describing particular embodiments only and is not intended to limit these embodiments; the terms "comprising" and "having," and any variations thereof, in the specification of these embodiments and the foregoing drawings, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification of these embodiments and the foregoing drawings are used to distinguish different objects, not to describe a particular order.

[0021] The implementation details of the technical solution in this embodiment are described in detail below: This application proposes a deep learning-based method for gold wire segmentation in semiconductor chips, such as... Figure 1 As shown, the method includes: S101, acquire multiple projected images of the gold wire at different angles, the projected images being synchronously acquired by multiple image acquisition devices arranged around the chip package.

[0022] Specifically, in this embodiment, multiple projected images of the gold wire at different angles are acquired. This is achieved by simultaneously acquiring projected images of the gold wire from different perspectives using multiple image acquisition devices arranged around the chip package under the same trigger signal. This synchronous acquisition method ensures the spatiotemporal consistency between the images, providing accurate spatial correspondence for subsequent multi-view feature fusion. For example, four or eight high-resolution industrial cameras can be evenly arranged around the chip package, with the optical axes of the cameras aligned with the center of the chip package. Each camera exposes simultaneously after the trigger signal is issued, acquiring projected images of the gold wire at different angles.

[0023] Synchronous acquisition can be achieved by connecting each camera to the same controller via hardware trigger lines. After the controller sends a pulse signal, each camera begins exposure simultaneously. The exposure time can be set according to the motion state of the gold wire and the lighting conditions. For example, when the gold wire is in a static detection scene, the exposure time can be set to 10ms; when the gold wire is in a dynamic transmission process, the exposure time needs to be shortened to less than 1ms to eliminate motion blur.

[0024] More specifically, the image acquisition device layout also needs to consider the baseline distance and field-of-view overlap between the cameras. The baseline distance, the spatial distance between the optical centers of two adjacent cameras, directly affects the depth resolution of subsequent 3D reconstruction. A larger baseline distance results in higher depth resolution, but reduces the overlap area. The baseline distance is typically balanced based on the size of the chip package and the required depth-direction detection accuracy. For example, for a scenario requiring a depth-direction detection accuracy of 0.1mm, the baseline distance can be set between 50mm and 100mm. Field-of-view overlap requires that the common field of view of all cameras completely cover the entire chip package, especially the area containing the gold lines, to ensure that each spatial point on the gold lines is captured by at least two cameras. This is a prerequisite for achieving multi-view feature fusion.

[0025] S102, the multiple projected images are input into the feature extraction network to obtain the feature map corresponding to each projected image.

[0026] Furthermore, the feature extraction network includes a first encoding branch and a second encoding branch, wherein the number of convolutional layers in the first encoding branch is less than the number of convolutional layers in the second encoding branch. In this embodiment, multiple projected images are input into a feature extraction network to obtain a feature map corresponding to each projected image. This involves extracting features from images at various viewpoints using a pre-trained deep learning network. The feature extraction network can employ an encoder-decoder structure, where the encoder extracts semantic features of the image progressively through multiple convolutional operations, and the decoder restores the spatial resolution of the feature map through upsampling operations. In practical applications, the feature extraction network can include two parallel encoding branches. The first encoding branch has fewer convolutional layers to preserve detailed image features, while the second encoding branch has more convolutional layers to extract high-level semantic features. The outputs of the two branches are concatenated laterally, resulting in a final feature map that simultaneously contains both detailed and semantic information. This is crucial for the accurate segmentation of elongated structures like gold lines.

[0027] In step S102, the step of inputting the multiple projected images into a feature extraction network to obtain a feature map corresponding to each projected image includes: Each projected image is input into the first encoding branch and the second encoding branch respectively to obtain a first feature sequence and a second feature sequence. The first feature sequence is the detail feature map output by the front-level convolutional layer of the feature extraction network, and the second feature sequence is the semantic feature map output by the back-level convolutional layer of the feature extraction network. A lateral connection is established between the first feature sequence and the second feature sequence. The lateral connection is used to perform channel concatenation of the detail feature map in the first feature sequence and the semantic feature map of the corresponding resolution in the second feature sequence to obtain the feature map of each projected image.

[0028] Specifically, in this embodiment, the feature extraction network adopts a dual-coding branch structure. The first encoding branch has a smaller number of convolutional layers, for example, it can contain three convolutional layers. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function to extract detailed features from the image. Since the diameter of gold wire is usually between 15-50μm, occupying only a few pixels in width in the image, the preservation of detailed features is crucial for the accurate localization of the gold wire boundary. In the first encoding branch, the kernel size of the first convolutional layer can be set to 3×3, with a stride of 1 and 64 output channels; the kernel size of the second convolutional layer is 3×3, with a stride of 2 and 128 output channels. Downsampling is achieved through convolution with a stride of 2, while expanding the receptive field; the kernel size of the third convolutional layer is 3×3, with a stride of 1 and 128 output channels. After processing by the first encoding branch, the resulting first feature sequence contains detail feature maps at three scales, with resolutions of 1 / 1, 1 / 2, and 1 / 2 of the original image (the resolution is halved after convolution with a stride of 2).

[0029] The second encoding branch has more convolutional layers than the first encoding branch, for example, it can contain five convolutional layers to extract higher-level semantic features. The first three convolutional layers of the second encoding branch can share the same structure and parameters as the first three convolutional layers of the first encoding branch. Two additional convolutional layers are added on top of this: the fourth convolutional layer has a kernel size of 3×3, a stride of 2, and 256 output channels; the fifth convolutional layer has a kernel size of 3×3, a stride of 1, and 256 output channels. After processing by the second encoding branch, the resulting second feature sequence contains semantic feature maps at five scales, with resolutions of 1 / 1, 1 / 2, 1 / 2, 1 / 4, and 1 / 4 of the original image, respectively. Because the second encoding branch has a deeper network and a larger receptive field, it can capture the overall direction and spatial context information of the gold thread, which is crucial for handling complex situations such as gold thread occlusion and breakage.

[0030] Specifically, a lateral connection is established between the first and second feature sequences. This involves concatenating the detail feature map output from the first encoding branch with the semantic feature map of the same resolution in the second encoding branch along the channel dimension. Specifically, a detail feature map with a resolution of 1 / 1 (64 channels) in the first feature sequence is concatenated with a semantic feature map with a resolution of 1 / 1 (64 channels) in the second feature sequence to obtain a feature map with 128 channels; similarly, a detail feature map with a resolution of 1 / 2 (128 channels) in the first feature sequence is concatenated with a semantic feature map with a resolution of 1 / 2 (128 channels) in the second feature sequence to obtain a feature map with 256 channels. For higher-level semantic feature maps (1 / 4 resolution) in the second feature sequence, since there are no detail feature maps of the corresponding resolution in the first feature sequence, they can be directly retained as the output at that scale.

[0031] In practical applications, lateral connections can be implemented using a 1×1 convolutional layer to adjust the number of channels in the feature maps, ensuring that the number of channels in the two feature maps matches before concatenation. For example, before concatenation, a 1×1 convolution can be performed on the detail feature map in the first feature sequence to transform its channel count to match that of the corresponding semantic feature map; or a 1×1 convolution can be performed on the semantic feature map in the second feature sequence to compress its channel count before concatenation. The concatenated feature map is then fused using a 3×3 convolutional layer to obtain the final feature map at that scale. This lateral connection approach ensures that the final feature map simultaneously possesses the high-resolution detail information of the first coding branch and the strong semantic information of the second coding branch.

[0032] For feature maps of different scales, you can choose to retain all or only the highest resolution feature map as input for subsequent processing, depending on the actual application scenario. For example, in the gold line segmentation task, since the gold line width is small and the requirement for detailed information is high, you can choose to retain the highest resolution (1 / 1) fused feature map as input for subsequent cross-view feature fusion. The resolution of this feature map is the same as the original image, the number of channels is 128 (64+64), and the feature vector at each pixel location contains the local texture details and global semantic context information at that location.

[0033] This application's solution, through a dual-branch encoding structure and lateral connections, effectively resolves the contradiction between detail and semantics in gold line segmentation. The first encoding branch preserves high-resolution detail features, enabling precise localization of the gold line boundaries; simultaneously, the second encoding branch extracts rich semantic features through a deeper network structure, maintaining the overall continuity of the gold line. The fusion of these two branches ensures that the final feature map accurately identifies the edges of the gold line while distinguishing it from background noise, thus providing high-quality input features for subsequent multi-view fusion and 3D reconstruction. Compared to traditional single-path feature extraction networks, this dual-branch fusion structure is more effective at handling elongated and easily occluded targets like gold lines.

[0034] S103, perform cross-view feature fusion on the feature maps of multiple projected images to obtain a fused three-dimensional feature body; the cross-view feature fusion includes aligning and aggregating features of different projected images on the same depth layer to generate the three-dimensional feature body.

[0035] Furthermore, in S103, the feature maps of multiple projected images are fused across different viewpoints to obtain a fused three-dimensional feature body, such as... Figure 2 As shown, the specific steps include the following: S2001, Based on the optical center position, imaging angle and lens focal length of each image acquisition device, establish the mapping relationship between pixel coordinates and three-dimensional spatial coordinates of each projected image; S2002, Based on the mapping relationship, the feature maps of each projected image are back-projected onto a unified three-dimensional spatial grid to obtain the three-dimensional feature sub-volumes corresponding to each projected image; S2003, the feature vectors of the multiple three-dimensional feature sub-voids at the same spatial location are weighted and averaged to obtain the three-dimensional feature volume, and the weight of the weighted average is determined by the imaging quality score of each projected image at that spatial location.

[0036] Specifically, in this embodiment, S2001, establishing the mapping relationship between the pixel coordinates and three-dimensional spatial coordinates of each projected image based on the optical center position, imaging angle and lens focal length of each image acquisition device, means obtaining the intrinsic parameter matrix and extrinsic parameter matrix of each image acquisition device through camera calibration, and using these parameters to construct the projection equation from three-dimensional spatial points to two-dimensional image points.

[0037] Here, the optical center position corresponds to the translation vector t = (tx, ty, tz) in the extrinsic parameter matrix, representing the three-dimensional coordinates of the camera's optical center in the world coordinate system; the imaging angle corresponds to the rotation matrix R in the extrinsic parameter matrix, describing the orientation of the camera coordinate system relative to the world coordinate system; the lens focal length corresponds to fx and fy in the intrinsic parameter matrix K, in pixels, representing the ratio of the lens's optical focal length to the pixel size, which directly affects the scaling ratio of the projection point coordinates. For example, for a camera with a resolution of 4000×3000 pixels, 15-20 chessboard images with different poses are acquired using the Zhang Zhengyou calibration method. The intrinsic parameter matrix K is calculated using the OpenCV calibration toolbox as follows: K = [fx, 0, cx; 0, fy, cy; 0, 0, 1] = [3520, 0, 2000; 0, 3515, 1500; 0, 0, 1], where the lens focal length fx = 3520 pixels, fy = 3515 pixels, cx = 2000 pixels, and cy = 1500 pixels are the principal point coordinates. In the extrinsic parameter matrix, the translation vector t = (0mm, 0mm, 100mm) corresponding to the optical center position indicates that the camera optical center is located 100mm directly above the chip package; the rotation matrix R corresponding to the imaging angle is composed of Euler angles (-90°, 0°, 0°), that is, the camera rotates -90 degrees around the X-axis so that the optical axis is vertically downward aligned with the center of the package.

[0038] When establishing the mapping relationship, for any point P(X,Y,Z) in 3D space, its projection onto the image point p(u,v) depends entirely on the optical center position t, the imaging angle R, and the lens focal lengths fx and fy. First, the world coordinates are converted to camera coordinates based on the optical center position and imaging angle: Pcam = R·P + t. Let the coordinates of point P be (2mm, 3mm, 0.8mm), and substitute them into the above extrinsic parameters: Pcam = [1,0,0; 0,0,1; 0,-1,0]·(2,3,0.8) + (0,0,100) = (2, 0.8, -3) + (0,0,100) = (2, 0.8, 97). Here, the Z component of the translation vector t, 100mm, determines the working distance, and the rotation matrix R determines the imaging angle. Together, they determine the depth value of point P in the camera coordinate system, 97mm. Then normalization is performed to obtain the normalized image coordinates (x,y) = (Xcam / Zcam, Ycam / Zcam) = (2 / 97, 0.8 / 97) ≈(0.02062, 0.00825).

[0039] Next, the normalized coordinates are distorted using the radial distortion coefficients k1=-0.05, k2=0.02, k3=-0.01 and the tangential distortion coefficients p1=0.001, p2=0.0005 obtained from calibration. r is then calculated. 2 = x 2 +y 2 ≈ 0.000425 + 0.000068 = 0.000493, the radial distortion correction factor is 1 + k1·r 2 +k2·r 4 +k3·r 6 ≈ 1 - 0.05 × 0.000493 ≈ 0.999975, Tangential distortion correction: Δx = 2p1xy + p2(r 2 +2x 2 ) ≈ 2×0.001×0.02062×0.00825 + 0.0005×(0.000493+2×0.000425) ≈ 0.00000034 + 0.00000067 = 0.00000101; Δy = p1(r 2 +2y 2 ) + 2p2xy ≈ 0.001×(0.000493+2×0.000068) + 2×0.0005×0.02062×0.00825 ≈ 0.00000063 + 0.00000017 = 0.00000080.

[0040] The corrected normalized coordinates are xcorrected = 0.02062×0.999975 + 0.00000101 ≈0.02062, ycorrected = 0.00825×0.999975 + 0.00000080 ≈ 0.00825.

[0041] Finally, the pixel coordinates are calculated based on the lens focal length parameters fx and fy in the intrinsic parameter matrix: u = fx·xcorrected + cx = 3520×0.02062 + 2000 ≈ 2072.6; v = fy·ycorrected + cy = 3515×0.00825 + 1500 ≈ 1529.0.

[0042] As can be seen, the lens focal lengths fx and fy directly determine the scaling ratio from normalized coordinates to pixel coordinates—the larger the focal length, the greater the distance the same spatial point moves on the image, and the higher the spatial resolution. Thus, the three-dimensional spatial point (2mm, 3mm, 0.8mm) is precisely projected onto the image point (2072.6, 1529.0) through the combined effect of the optical center position t, the imaging angle R, and the lens focal lengths fx and fy.

[0043] S2002, based on the above mapping relationship, the feature maps of each projected image are back-projected onto a unified three-dimensional spatial grid to obtain the three-dimensional feature sub-volumes corresponding to each projected image. First, a unified spatial grid is defined in three-dimensional space, and the grid range should completely cover the chip package and the area where the gold wire is located. For example, for a chip package with dimensions of 10mm×10mm×2mm, the range of the three-dimensional spatial grid can be defined as -5mm to 5mm in the X direction, -5mm to 5mm in the Y direction, and -1mm to 3mm in the Z direction. The voxel size of the grid can be set according to the detection accuracy requirements. If the diameter of the gold wire is approximately 25μm, the voxel size can be set to 10μm, and the grid size is 1000×1000×400 voxels. For each voxel center point P(X,Y,Z) in the three-dimensional spatial grid, its corresponding pixel coordinates on each projected image are calculated according to the above projection equation. Then, the feature vector at that coordinate is extracted from the feature map of the projected image through bilinear interpolation and filled into the corresponding position of the voxel in the three-dimensional feature sub-volume corresponding to the projected image. The formula for bilinear interpolation is as follows: Let the target pixel coordinates be (u, v), and its four surrounding integer pixels be (u0, v0), ... 0+1 ,v0)(u0,v 0+1 ), (u 0+1 ,v 0+1 The corresponding feature vector is f.00 f 10 f 01 f 11 Then the interpolated eigenvector is f = (1-α)(1-β)f 00 + α(1-β)f 10 + (1-α)βf 01 + αβf 11 , where α = u - u0, β = v - v0. For voxels that are occluded or outside the field of view at certain perspectives, their corresponding eigenvectors can be set to all zero vectors.

[0044] S2003, a weighted average is performed on the feature vectors of multiple 3D feature sub-volules at the same spatial location to obtain a 3D feature volume. The weights of the weighted average are determined by the imaging quality scores of each projected image at that spatial location. The imaging quality score can be calculated based on the gradient magnitude of the voxel location on each projected image. Let Gi be the gradient magnitude at the image pixel corresponding to a certain voxel location under the i-th viewpoint. The gradient magnitude can be calculated using the Sobel operator: G = sqrt(Gx 2 +Gy 2 Let Gx and Gy be the gradients of the image in the x and y directions, respectively. The image quality score wi for the i-th viewpoint can be defined as the ratio of Gi to the sum of the gradient magnitudes of all viewpoints: wi = Gi / ΣGj. A larger gradient magnitude indicates clearer texture and sharper edges at that location in the image, resulting in higher image quality, and therefore a higher weight is assigned. For a voxel location, assuming three viewpoints can observe that location with gradient magnitudes G1=120, G2=85, and G3=45, the weights for each viewpoint are w1=120 / (120+85+45)=0.48, w2=85 / 250=0.34, and w3=45 / 250=0.18, respectively. Let the feature vectors extracted from these three viewpoints be f1, f2, and f3, then the weighted average feature vector is F = 0.48×f1 + 0.34×f2 + 0.18×f3. This weighted averaging method effectively suppresses the interference of low-quality image features on the fusion result and highlights the role of high-quality image features.

[0045] The scheme in this application, through the aforementioned cross-view feature fusion method, can effectively integrate feature maps from multiple two-dimensional perspectives into a single three-dimensional feature volume. It is precisely because specific parameters such as the optical center position t=(0mm,0mm,100mm), the Euler angles corresponding to the imaging angle R (-90°,0°,0°), and the lens focal lengths fx=3520 pixels and fy=3515 pixels, as defined in S2001, and the establishment of an accurate projection equation including distortion correction, that features from different perspectives can be accurately aligned to the same spatial position. It is precisely because bilinear interpolation is used for feature sampling in S2002 that the continuity and accuracy of feature extraction are guaranteed. It is precisely because a gradient magnitude-based weighted averaging mechanism for imaging quality scores is introduced in S2003 that the fusion process can automatically select high-quality image features and suppress the influence of low-quality or occluded areas. Compared to simple averaging or maximum value fusion, this fusion method can better utilize the complementarity of multi-view information, providing more reliable and accurate feature input for subsequent three-dimensional probabilistic map generation.

[0046] S104, Generate a three-dimensional probability map of the gold wire based on the three-dimensional feature body, wherein each voxel value in the three-dimensional probability map represents the probability that the position belongs to the surface of the gold wire.

[0047] Specifically, in S104, generating a three-dimensional probability map of gold lines based on the three-dimensional feature volume includes: inputting the three-dimensional feature volume into a three-dimensional convolutional network, the three-dimensional convolutional network including multiple cascaded three-dimensional convolutional layers, each three-dimensional convolutional layer being connected to a non-linear activation layer; the output end of the three-dimensional convolutional network is connected to a voxel classification layer, used to classify each voxel in the three-dimensional feature volume and output the three-dimensional probability map.

[0048] Specifically, in this embodiment, the three-dimensional feature volume is input into a three-dimensional convolutional network, that is, the fused three-dimensional feature volume obtained in the previous step is used as the input data of the three-dimensional convolutional network. The dimension of the three-dimensional feature volume can be represented as H×W×D×C, where H, W, and D represent the number of voxels in the height, width, and depth directions of the three-dimensional spatial grid, respectively, and C represents the feature vector dimension of each voxel. For example, for a detection space of 10mm×10mm×2mm, if the voxel size is set to 10μm, the size of the three-dimensional feature volume is 1000×1000×200 voxels. If the number of channels in the output feature map of the feature extraction network is 128, then the feature vector dimension of each voxel is 128. Therefore, the data scale of the three-dimensional feature volume is approximately 1000×1000×200×128, or approximately 25.6 billion floating-point numbers. In practical applications, this can be optimized by cropping the detection area, reducing the voxel resolution, or using half-precision floating-point numbers.

[0049] The 3D convolutional network comprises multiple cascaded 3D convolutional layers, each followed by a nonlinear activation layer. The kernel size of the 3D convolutional layers can be designed based on the spatial distribution characteristics of the gold wires; for example, the kernel size can be set to 3×3×3 or 5×5×5. Since gold wires exhibit a slender structure in 3D space, their cross-sectional dimensions (diameter) are typically around 25μm. If the voxel size is set to 10μm, the gold wire cross-section occupies approximately 2-3 voxels in 3D space. Therefore, using a 3×3×3 convolutional kernel can effectively capture the local spatial features of the gold wires. The stride of the convolutional layer can be set to 1, and the padding method can be "same padding" to ensure consistent input and output dimensions. The nonlinear activation layer can use the ReLU function, expressed as f(x) = max(0,x), to introduce nonlinear transformation capabilities. The 3D convolutional network can contain 3 to 5 3D convolutional layers. By stacking convolutional layers, the receptive field is gradually expanded, enabling the network to learn the overall orientation and spatial context information of the gold wires.

[0050] In practical applications, after the feature map output by the last 3D convolutional layer of the 3D convolutional network, a voxel classification layer is connected. The voxel classification layer can be a 1×1×1 convolutional layer with a kernel size of 1×1×1 and one output channel, used to map the feature vector of each voxel to a scalar value. This scalar value is then converted into a probability value between 0 and 1 after passing through the Sigmoid activation function, representing the probability that the voxel belongs to the gold wire surface. The expression for the Sigmoid function is σ(x) = 1 / (1+e^(-x)). (-x) When the input value x is large, the output approaches 1; when x is small, the output approaches 0. Therefore, the output of the voxel classification layer can be constructed as a three-dimensional probability map with the same size as the input 3D feature volume, and its dimensions are H×W×D×1. The value of each voxel position is the probability that the position belongs to the gold line surface. For example, suppose that the scalar value output by the voxel classification layer is 2.5, and after Sigmoid activation, the probability value is 0.924, indicating that there is a 92.4% probability that the position belongs to the gold line surface; another voxel position outputs a scalar value of -1.8, and after activation, the probability value is 0.142, indicating that the position has a higher probability of belonging to the background. In this way, a probability distribution map of the gold lines in the entire 3D space can be obtained, in which the regions where the gold lines are located will show higher probability values, while the background regions will show lower probability values.

[0051] S105, extract the centerline point set of the gold line according to the three-dimensional probability map, and reconstruct the three-dimensional spatial curve of the gold line based on the centerline point set.

[0052] Further, in S105, the step of extracting the centerline point set of the gold line based on the three-dimensional probability map and reconstructing the three-dimensional spatial curve of the gold line based on the centerline point set is as follows: Figure 3As shown, the specific steps include the following: S3001, perform threshold segmentation on the three-dimensional probability map, and take points with probability values ​​higher than a preset threshold as target points; S3002, the target point is divided into multiple point sets according to its spatial location, and the geometric center of each point set is calculated to obtain a sequence of center points; S3003, the center point sequence is sorted according to its spatial position and connected sequentially to form a segmented straight line, thereby obtaining the three-dimensional spatial curve.

[0053] Specifically, in this embodiment, in S3001, threshold segmentation is performed on the three-dimensional probability map. This involves setting a probability threshold, marking voxels with probability values ​​higher than the threshold as target points, and marking voxels with probability values ​​lower than or equal to the threshold as background points. The preset threshold can be adjusted according to the actual application scenario. For example, it can be set to 0.5, meaning voxels with a probability value greater than 50% are considered to belong to the gold wire surface. In practical applications, the threshold setting is related to the distribution characteristics of the three-dimensional probability map. If the probability values ​​in the gold wire region are generally high and concentrated, the threshold can be set to 0.6 or 0.7 to filter out low-probability noise; if the probability values ​​in the gold wire region are relatively dispersed, the threshold can be set to 0.4 to improve recall. For example, for a three-dimensional probability map with dimensions of 1000×1000×200, assuming the number of voxels corresponding to the gold wire diameter is approximately 5 (if the voxel size is 5μm, the gold wire diameter is 25μm), after threshold segmentation, the resulting set of target points includes all positions with probability values ​​higher than the threshold. These target points are roughly distributed in and around the strip-shaped region where the gold wire is located.

[0054] In step S3002, target points are divided into multiple point sets based on their spatial location, i.e., clustering target points based on their spatial connectivity. Since the gold line is a continuous structure in three-dimensional space, target points belonging to the gold line are spatially connected, while target points generated by background noise are often isolated or form small clusters. Therefore, connected component analysis algorithms, such as region growing algorithms based on 26-neighborhood connectivity, can be used to group interconnected target points into the same point set. Assuming that approximately 50,000 target points are obtained after thresholding, connected component analysis reveals that these target points form two connected components: one component contains approximately 48,000 target points, corresponding to the main body of the gold line; the other component contains approximately 2,000 target points, corresponding to noise or interference regions. For each connected component point set, its geometric center is calculated, i.e., the average of the three-dimensional coordinates of all target points in that point set. Suppose a connected component contains n target points with coordinates (x1, y1, z1), (x2, y2, z2), ..., (xn, yn, zn). Then the geometric center coordinates of this component are: xcenter = (x1 + x2 + ... + xn) / n, ycenter = (y1 + y2 + ... + yn) / n, zcenter = (z1 + z2 + ... + zn) / n. For the connected component corresponding to the main body of the gold line, its geometric center coordinates are approximately located near the central axis of the gold line. For the connected component corresponding to noise, its geometric center may deviate from the actual position of the gold line and can be filtered out or ignored in subsequent processing.

[0055] In practical applications, in S3003, the center point sequence is sorted according to its spatial position, that is, the multiple geometric center points obtained in the previous step are arranged according to their spatial position order. Since the gold wire usually extends from the chip electrode to the pin frame, it has a clear start and end point, so an appropriate sorting criterion can be selected according to the actual direction of the gold wire. For example, it can be sorted according to the coordinate values ​​of each center point in the Z-axis direction (height direction) from smallest to largest, because the gold wire usually arches upward from the chip electrode (lower height) and then descends to the pin frame (higher or lower height). After sorting, an ordered center point sequence is obtained, where the first center point corresponds to the start point of the gold wire and the last center point corresponds to the end point of the gold wire. Then, the adjacent center points after sorting are connected sequentially with straight line segments to form a segmented straight line composed of multiple straight line segments. For example, if the center point sequence contains 10 center points, then the 1st-2nd point, the 2nd-3rd point, ..., the 9th-10th point are connected sequentially to obtain 9 straight line segments. These straight line segments are connected end to end to form a broken line, which serves as the three-dimensional spatial curve of the gold wire.

[0056] This application's scheme, through threshold segmentation, connected component analysis, geometric center extraction, and sorted connections, efficiently extracts the centerline point set of a gold wire from a 3D probabilistic map and reconstructs the 3D spatial curve. The use of connected component analysis to cluster target points effectively distinguishes the gold wire from background noise, avoiding interference from noise points in centerline extraction. Furthermore, using the geometric center as the representative point of each connected component allows the centerline point set to concisely represent the spatial orientation of the gold wire, avoiding the skeleton bifurcation and burr problems common in traditional thinning algorithms. This geometric center-based centerline extraction method, compared to skeletonization-based methods, offers higher computational efficiency and more stable results, and is particularly suitable for slender, continuous, and branchless structures like gold wire.

[0057] S106, the reconstructed three-dimensional spatial curve is projected onto the imaging plane of each projected image to obtain the two-dimensional segmentation result of the gold line on each projected image.

[0058] Further, in S106, the reconstructed three-dimensional spatial curve is projected onto the imaging plane of each projected image to obtain the two-dimensional segmentation result of the gold line on each projected image, such as... Figure 4 As shown, the specific steps include the following: S4001, Based on the optical center position, imaging angle and lens focal length of each image acquisition device, establish the projection transformation relationship between the three-dimensional space curve and the imaging plane of each projected image; S4002, based on the projection transformation relationship, calculate the projection segmented straight lines of the three-dimensional space curve on each projection image, and take each segment of the projection segmented straight line as the center line, expand the preset width to both sides along the normal direction to generate multiple rectangular segmented regions; S4003, merge the multiple rectangular segmented regions to obtain a continuous strip segmented region, which is the two-dimensional segmentation result of the gold line.

[0059] In S4002, taking each segment of the projected segmented line as the center line, a preset width is extended to both sides along the normal direction to generate a strip-shaped segmented region, including: Calculate the tilt angle of each line segment of the projected segmented line, where the tilt angle is the angle between the line segment and the horizontal direction; compare the tilt angle with multiple preset angle intervals, where each angle interval corresponds to a fixed expansion width, the smaller the tilt angle, the larger the corresponding expansion width, and the larger the tilt angle, the smaller the corresponding expansion width; using each line segment as the center line, expand to both sides along the normal direction according to the corresponding expansion width to generate the rectangular segmented region.

[0060] Specifically, in this embodiment, S4001 establishes the projection transformation relationship between the three-dimensional space curve and the imaging plane of each projected image based on the optical center position, imaging angle and lens focal length of each image acquisition device. This means using the intrinsic and extrinsic parameter matrices obtained from the previous camera calibration to construct the projection equation from the three-dimensional space point to the two-dimensional image point.

[0061] First, each image acquisition device was calibrated using Zhang Zhengyou's calibration method to obtain the following parameters: (1) Optical center position: that is, the coordinates of the camera's optical center in the world coordinate system, represented by the translation vector t = (tx, ty, tz) in the extrinsic parameter matrix. For example, the calibration yielded t = (0mm, 0mm, 100mm), indicating that the camera's optical center is located 100mm directly above the chip package. (2) Imaging angle: that is, the orientation of the camera coordinate system relative to the world coordinate system, represented by the rotation matrix R in the extrinsic parameter matrix. For example, the calibration yielded R corresponding to the Euler angle (-90°, 0°, 0°), that is, the camera rotates -90 degrees around the X-axis so that the optical axis is vertically downward aligned with the center of the package, and its matrix form is R = [1, 0, 0; 0, 0, 1; 0, -1, 0]. (3) Lens focal length: that is, the ratio of the lens's optical focal length to the pixel size, represented by fx and fy in the intrinsic parameter matrix K. For example, the calibration yields the intrinsic parameter matrix K = [fx, 0, cx; 0, fy, cy; 0, 0, 1] = [3520, 0, 2000; 0, 3515, 1500; 0, 0, 1], where fx = 3520 pixels and fy = 3515 pixels are the lens focal length parameters, cx = 2000 pixels and cy = 1500 pixels are the principal point coordinates.

[0062] Based on the above calibration parameters, the process of constructing the projection equation consists of three steps: The first step is to establish the transformation relationship from world coordinates to camera coordinates based on the optical center position and imaging angle. For any point P(X,Y,Z) on the 3D spatial curve, its camera coordinates Pcam = (Xcam, Ycam, Zcam) are obtained through the following transformation: Pcam = R·P + t. Where R is the rotation matrix and t is the translation vector. This transformation converts the point in the world coordinate system to the camera coordinate system with the camera optical center as the origin and the optical axis direction as the Z-axis. For example, take a point P(2mm, 3mm, 0.8mm) on the 3D spatial curve and substitute it with the above extrinsic parameters: Pcam = [1,0,0; 0,0,1; 0,-1,0]·(2,3,0.8) +(0,0,100) = (2, 0.8, -3) + (0,0,100) = (2, 0.8, 97). Here, the Z component of the translation vector, 100mm, determines the working distance, and the rotation matrix determines the imaging angle. Together, they determine the depth value of point P in the camera coordinate system, Zcam = 97mm.

[0063] The second step is to establish a projection relationship from camera coordinates to normalized image coordinates based on the lens focal length. Normalizing the camera coordinates yields the normalized image coordinates (x, y) = (Xcam / Zcam, Ycam / Zcam). This step essentially utilizes the physical meaning of focal length—the intersection of the line connecting a spatial point to the optical center and the imaging plane. For the above Pcam=(2,0.8,97), normalization yields (x,y)=(2 / 97, 0.8 / 97)≈(0.02062, 0.00825). Here, (x,y) represents the normalized position of the spatial point on the image plane under an ideal, distortion-free imaging model.

[0064] The third step is to convert the normalized image coordinates into pixel coordinates based on the lens focal length parameters fx and fy. Considering lens distortion, the normalized coordinates are first corrected for distortion, then multiplied by the focal length parameters and the principal point coordinates are added to obtain the final pixel coordinates (u, v). u = fx·xcorrected + cx; v = fy·ycorrected + cy; The distortion correction is based on the radial distortion coefficients k1, k2, k3 and the tangential distortion coefficients p1, p2 obtained from calibration, and the correction formula is as follows: xcorrected = x(1+k1·r 2 +k2·r 4 +k3·r 6 ) + (2p1xy + p2(r 2 +2x 2 )); ycorrected = y(1+k1·r 2 +k2·r 4 +k3·r 6 ) + (p1(r 2 +2y 2 ) + 2p2xy); Where r 2 =x 2 +y 2 Taking point P as an example, we calculate xcorrected≈0.02062 and ycorrected≈0.00825. Substituting these values ​​into the focal length parameter: u = 3520×0.02062 + 2000 ≈ 2072.6; v = 3515×0.00825 + 1500 ≈ 1529.0; Thus, the projection equation is fully established: for any point in three-dimensional space, its precise pixel coordinates on the image plane can be obtained through the above three-step transformation.

[0065] Applying the above projection equation to a three-dimensional spatial curve establishes the projection transformation relationship between the three-dimensional spatial curve and the imaging planes of each projected image. Assume the reconstructed three-dimensional spatial curve consists of three straight lines: the first line runs from point P1 (1mm, 2mm, 0.5mm) to P2 (3mm, 2mm, 0.5mm); the second line runs from P2 (3mm, 2mm, 0.5mm) to P3 (3mm, 4mm, 1.2mm); and the third line runs from P3 (3mm, 4mm, 1.2mm) to P4 (1mm, 4mm, 0.8mm). Using the projection equation, the projected coordinates of each endpoint are calculated, resulting in P1 (2036, 1518), P2 (2980, 1518), P3 (3050, 2150), and P4 (1080, 1980). Connecting P1 and P2 yields the projected line segment L1, connecting P2 and P3 yields L2, and connecting P3 and P4 yields L3. In this way, each straight line on the three-dimensional curve is precisely mapped to a projected segmented straight line on the image plane.

[0066] In S4002, for each projected line segment, its tilt angle is first calculated. For L1, the starting point is (2036, 1518), the ending point is (2980, 1518), du=944, dv=0, the line segment length L1=944 pixels, and the tilt angle θ1=arctan(0 / 944)=0°, which belongs to the first angle range (0°≤θ<30°). For L2, the starting point is (2980, 1518), the ending point is (3050, 2150), du=70, dv=632, L2=sqrt(70^2+632^2)≈636.0 pixels, and θ2=arctan(632 / 70)≈83.7°, which belongs to the third angle range (60°≤θ≤90°). For L3, the starting point is (3050, 2150), the ending point is (1080, 1980), du = -1970, dv = -170, and L3 = sqrt(1970). 2 +170 2 The angle is approximately 1977.3 pixels, and θ3 = arctan(170 / 1970) ≈ 4.9°, which falls within the first angle range. The preset correspondence between angle ranges and expansion widths is as follows: first range (0° ≤ θ < 30°) corresponds to expansion width W = 15 pixels, second range (30° ≤ θ < 60°) corresponds to W = 10 pixels, and third range (60° ≤ θ ≤ 90°) corresponds to W = 7 pixels. Therefore, L1 takes W1 = 15 pixels, L2 takes W2 = 7 pixels, and L3 takes W3 = 15 pixels.

[0067] For L1, the unit direction vector d = (1,0) and the unit normal vector n = (0,1). The two boundary points at the starting point R0 (2036,1518) are R0left = R0 + W·n = (2036,1533) and R0right = R0 - W·n = (2036,1503); the two boundary points at the ending point R1 (2980,1518) are R1left = (2980,1533) and R1right = (2980,1503). The rectangular region enclosed by these four points is the rectangular segmentation region corresponding to L1. For L2, the unit direction vector d≈(0.110,0.994), the unit normal vector n≈(-0.994,0.110), the two boundary points at the starting point R0(2980,1518) are R0left≈(2973,1518.8) and R0right≈(2987,1517.2); the two boundary points at the ending point R1(3050,2150) are R1left≈(3043,2150.8) and R1right≈(3057,2149.2). For L3, the unit direction vector d≈(-0.996,-0.086), the unit normal vector n≈(0.086,-0.996), the two boundary points at the starting point R0(3050,2150) are R0left≈(3051.3,2135.1) and R0right≈(3048.7,2164.9); the two boundary points at the ending point R1(1080,1980) are R1left≈(1081.3,1965.1) and R1right≈(1078.7,1994.9).

[0068] In S4003, multiple rectangular segmented regions are merged to obtain a continuous strip-shaped segmented region. First, a binary mask image with the same size as the original image is created, with all pixels initially set to 0. The rectangular regions corresponding to L1, L2, and L3 are traversed, and all pixels within each rectangle are marked as 1. Since the rectangular regions of L1 and L2 overlap at their connection point (near (2980, 1518)) and the rectangular regions of L2 and L3 overlap at their connection point (near (3050, 2150)), the three rectangular regions are naturally connected through the overlapping parts in the mask image obtained after direct marking. For any possible small gaps, morphological closing operations are used to fill them: a 3×3 pixel circular structuring element is used to dilate the mask image, expanding each rectangular region outward by one pixel, connecting the gap regions; then, the same structuring element is used to erode the region, restoring its original width. After the closing operation, the originally separated rectangular regions are connected into a complete connected region. Finally, all pixels within the connected region are output as the two-dimensional segmentation result of the gold line on the image at that viewpoint.

[0069] The solution in this application, through the aforementioned projection transformation and dynamic expansion methods, can accurately map three-dimensional spatial curves back to two-dimensional images. It is precisely because S4001 specifies the specific parameters such as the optical center position t=(0mm,0mm,100mm), the imaging angle R corresponding to the Euler angles (-90°,0°,0°), and the lens focal lengths fx=3520 pixels and fy=3515 pixels, and establishes a precise projection equation including distortion correction, that the sub-pixel accuracy of the projection point coordinates is guaranteed. It is precisely because S4002 introduces the correspondence between the tilt angle and the expansion width that the width of the segmented region can adaptively compensate for width changes caused by different projection directions—when the line... When the line segment is close to horizontal (e.g., L1 and L3, with tilt angles of approximately 0° and 5°), the perspective compression effect is significant; using a larger expansion width of 15 pixels can avoid segmentation breaks. When the line segment is close to vertical (e.g., L2, with a tilt angle of approximately 84°), the projected width is close to the actual width; using a smaller expansion width of 7 pixels can reduce background interference. Because S4003 uses morphological closing operations for region merging, the segmented rectangular regions can be stitched together into a continuous strip-like segmentation result, avoiding discontinuities caused by segmentation. This dynamic expansion method based on tilt angle effectively improves the accuracy and continuity of gold line 2D segmentation, especially in scenarios with significant changes in gold line orientation.

[0070] Secondly, this application also proposes a deep learning-based semiconductor chip gold wire segmentation system, such as... Figure 5 As shown, it includes: The image acquisition module 501 is used to acquire multiple projected images of the gold wire at different angles. The projected images are acquired synchronously by multiple image acquisition devices arranged around the chip package. The feature extraction module 502 is used to input the multiple projection images into the feature extraction network to obtain the feature map corresponding to each projection image; The feature fusion module 503 is used to perform cross-view feature fusion on the feature maps of multiple projection images to obtain a fused three-dimensional feature volume; the cross-view feature fusion includes aligning and aggregating the features of different projection images on the same depth layer to generate the three-dimensional feature volume; The probability generation module 504 is used to generate a three-dimensional probability map of the gold wire based on the three-dimensional feature body, wherein each voxel value in the three-dimensional probability map represents the probability that the position belongs to the surface of the gold wire. The curve reconstruction module 505 is used to extract the centerline point set of the gold line according to the three-dimensional probability map, and reconstruct the three-dimensional spatial curve of the gold line based on the centerline point set. The projection segmentation module 506 is used to project the reconstructed three-dimensional spatial curve onto the imaging plane of each projection image to obtain the two-dimensional segmentation result of the gold line on each projection image.

[0071] This system can be used to execute the deep learning-based semiconductor chip gold wire segmentation method described in the first aspect, which will not be elaborated further here.

[0072] The solution proposed in this application achieves fully automated processing of gold wire segmentation from multi-view image acquisition to 3D reconstruction and 2D segmentation through the collaborative work of the aforementioned six modules. The image acquisition module 501, employing synchronous triggering and precise arrangement, ensures temporal consistency and spatial correspondence of multi-view images. The feature extraction module 502, using dual-coding branches and lateral connections, ensures that the extracted feature maps simultaneously possess detailed and semantic information. The feature fusion module 503, establishing precise projection transformation and introducing image quality weighted averaging, enables the 3D feature volume to fuse high-quality features from multiple perspectives. The probability generation module 504, using a 3D convolutional network for voxel classification, ensures that the 3D probability map accurately reflects the spatial distribution of the gold wire. The curve reconstruction module 505, extracting the centerline based on the geometric center and reconstructing the curve, avoids the skeleton bifurcation problem of traditional thinning algorithms. The projection segmentation module 506, dynamically adjusting the expansion width and merging regions according to the line segment tilt angle, allows the 2D segmentation results to adaptively conform to the actual projection shape of the gold wire. This systematic modular design enables the gold wire segmentation method to operate efficiently and accurately in actual semiconductor packaging inspection.

[0073] Thirdly, an electronic device is also proposed, comprising: one or more processors, and a memory for storing one or more computer programs; the computer programs are configured to be executed by the one or more processors, the programs including steps for performing the deep learning-based semiconductor chip gold wire segmentation method as described in the first aspect.

[0074] Specifically, in this embodiment, the electronic device may be an industrial control computer, an embedded industrial control computer, or a server, which includes a processor and a memory. The memory stores a computer program, which, after being loaded by the processor, executes the deep learning-based semiconductor chip gold wire segmentation method as described in the first aspect.

[0075] Fourthly, this application also proposes a storage medium storing a computer program; the program is loaded and executed by a processor to implement the steps of the deep learning-based semiconductor chip gold wire segmentation method as described in the first aspect.

[0076] Specifically, in this embodiment, the storage medium can be a non-volatile storage medium such as a read-only memory (ROM), random access memory (RAM), flash memory, solid-state drive (SSD), hard disk drive (HDD), or optical disc (such as CD-ROM, DVD). The storage medium stores a computer program, which, when loaded and executed by a processor, enables the implementation of the deep learning-based semiconductor chip gold wire segmentation method described in the first aspect. In practical applications, the storage medium can be pre-installed in the control system of a semiconductor packaging and testing equipment, or distributed as a standalone software installation package.

[0077] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for gold wire segmentation of semiconductor chips based on deep learning, characterized in that, include: Multiple projection images of the gold wire at different angles are acquired, and the projection images are simultaneously acquired by multiple image acquisition devices arranged around the chip package. The multiple projected images are input into a feature extraction network to obtain a feature map corresponding to each projected image; Cross-view feature fusion is performed on the feature maps of multiple projected images to obtain a fused three-dimensional feature volume; the cross-view feature fusion includes aligning and aggregating features of different projected images at the same depth layer to generate the three-dimensional feature volume; A three-dimensional probability map of the gold wire is generated based on the three-dimensional feature volume, and each voxel value in the three-dimensional probability map represents the probability that the position belongs to the surface of the gold wire. The centerline point set of the gold line is extracted based on the three-dimensional probability map, and the three-dimensional spatial curve of the gold line is reconstructed based on the centerline point set. The reconstructed three-dimensional spatial curve is projected onto the imaging plane of each projected image to obtain the two-dimensional segmentation result of the gold line on each projected image.

2. The deep learning-based semiconductor chip gold wire segmentation method according to claim 1, characterized in that, The feature extraction network includes a first encoding branch and a second encoding branch, wherein the number of convolutional layers in the first encoding branch is less than the number of convolutional layers in the second encoding branch; the step of inputting the multiple projected images into the feature extraction network to obtain a feature map corresponding to each projected image includes: Each projected image is input into the first encoding branch and the second encoding branch respectively to obtain a first feature sequence and a second feature sequence. The first feature sequence is the detail feature map output by the front-level convolutional layer of the feature extraction network, and the second feature sequence is the semantic feature map output by the back-level convolutional layer of the feature extraction network. A lateral connection is established between the first feature sequence and the second feature sequence. The lateral connection is used to perform channel concatenation of the detail feature map in the first feature sequence and the semantic feature map of the corresponding resolution in the second feature sequence to obtain the feature map of each projected image.

3. The semiconductor chip gold wire segmentation method based on deep learning according to claim 1, characterized in that, The cross-view feature fusion of the feature maps from multiple projected images to obtain a fused three-dimensional feature volume includes: Based on the optical center position, imaging angle, and lens focal length of each image acquisition device, establish the mapping relationship between pixel coordinates and three-dimensional spatial coordinates of each projected image; Based on the mapping relationship, the feature maps of each projected image are back-projected onto a unified three-dimensional spatial grid to obtain the three-dimensional feature sub-volumes corresponding to each projected image; The three-dimensional feature volume is obtained by weighted averaging of the feature vectors of multiple three-dimensional feature sub-volumes at the same spatial location. The weights of the weighted average are determined by the imaging quality scores of each projected image at that spatial location.

4. The deep learning-based semiconductor chip gold wire segmentation method according to claim 1, characterized in that, The generation of a three-dimensional probability map of gold lines based on the three-dimensional feature volume includes: The three-dimensional feature volume is input into a three-dimensional convolutional network, which includes multiple cascaded three-dimensional convolutional layers, each of which is followed by a non-linear activation layer. The output of the three-dimensional convolutional network is connected to a voxel classification layer, which is used to classify each voxel in the three-dimensional feature volume and output the three-dimensional probability map.

5. The deep learning-based semiconductor chip gold wire segmentation method according to claim 1, characterized in that, The step of extracting the centerline point set of the gold line based on the three-dimensional probability map and reconstructing the three-dimensional spatial curve of the gold line based on the centerline point set includes: The three-dimensional probability map is segmented by a threshold, and points with probability values ​​higher than a preset threshold are taken as target points; The target point is divided into multiple point sets according to its spatial location, and the geometric center of each point set is calculated to obtain a sequence of center points; The sequence of center points is sorted according to their spatial location and connected sequentially to form segmented straight lines, thus obtaining the three-dimensional spatial curve.

6. The deep learning-based semiconductor chip gold wire segmentation method according to claim 5, characterized in that, The reconstructed three-dimensional spatial curve is projected onto the imaging plane of each projected image to obtain the two-dimensional segmentation results of the gold lines on each projected image, including: Based on the optical center position, imaging angle and lens focal length of each image acquisition device, establish the projection transformation relationship between the three-dimensional space curve and the imaging plane of each projected image. Based on the projection transformation relationship, the projection segmented lines of the three-dimensional spatial curve on each projection image are calculated. Taking each segment of the projection segmented lines as the center line, the lines are extended to both sides with a preset width along the normal direction to generate multiple rectangular segmented regions. The multiple rectangular segmented regions are merged to obtain a continuous strip segmented region, which is the two-dimensional segmentation result of the gold line.

7. The deep learning-based semiconductor chip gold wire segmentation method according to claim 6, characterized in that, Using each segment of the projected segmented line as a center line, a predetermined width is extended to both sides along the normal direction to generate a strip-shaped segmented region, including: Calculate the inclination angle of each segment of the projected segmented straight line, where the inclination angle is the angle between the segment and the horizontal direction; The tilt angle is compared with multiple preset angle intervals. Each angle interval corresponds to a fixed expansion width. The smaller the tilt angle, the larger the corresponding expansion width, and the larger the tilt angle, the smaller the corresponding expansion width. Using each line segment as the center line, the rectangular segmented region is generated by expanding to both sides along the normal direction with the corresponding expansion width.

8. A semiconductor chip gold wire segmentation system based on deep learning, characterized in that, include: The image acquisition module is used to acquire multiple projected images of the gold wire at different angles. The projected images are acquired synchronously by multiple image acquisition devices arranged around the chip package. The feature extraction module is used to input the multiple projected images into the feature extraction network to obtain the feature map corresponding to each projected image; The feature fusion module is used to perform cross-view feature fusion on the feature maps of multiple projected images to obtain a fused three-dimensional feature volume; the cross-view feature fusion includes aligning and aggregating features of different projected images on the same depth layer to generate the three-dimensional feature volume; The probability generation module is used to generate a three-dimensional probability map of the gold wire based on the three-dimensional feature volume, wherein each voxel value in the three-dimensional probability map represents the probability that the position belongs to the surface of the gold wire; The curve reconstruction module is used to extract the centerline point set of the gold line according to the three-dimensional probability map, and reconstruct the three-dimensional spatial curve of the gold line based on the centerline point set. The projection segmentation module is used to project the reconstructed three-dimensional spatial curve onto the imaging plane of each projection image to obtain the two-dimensional segmentation result of the gold line on each projection image.

9. An electronic device, the electronic device comprising: One or more processors, a memory for storing one or more computer programs; characterized in that the computer programs are configured to be executed by the one or more processors, the programs including steps for performing the deep learning-based semiconductor chip gold wire segmentation method as described in any one of claims 1-7.

10. A storage medium storing a computer program; characterized in that, The program is loaded and executed by a processor to implement the steps of the deep learning-based semiconductor chip gold wire segmentation method as described in any one of claims 1-7.

Images