A gold wire height detection method, system, device and medium of an optical module

Abstract

The present application relates to a kind of gold line height detection method, system, equipment and medium of optical module, its method includes: optical module image is extracted to multiple scale convolution feature, and the V channel feature of image is fused into convolution feature;Convolution feature map is input into corresponding multi-scale detection head and gold line positioning detection is carried out, obtains the two-dimensional coordinate information of each gold line;Based on two-dimensional coordinate information, the region of interest corresponding to each gold line is intercepted in optical module image, and pixel-level depth detection is carried out to the region of interest, obtains the relative depth map and reference depth value of each gold line;According to outlier rejection of relative depth map according to the Laidar criterion, and based on the optimal relative depth map obtained, in combination with two-dimensional coordinate information and corresponding reference depth value, the actual height of gold line relative to substrate is obtained.The present application improves gold line height detection precision, especially for dense arrangement, staggered overlap and slender form gold line all have higher detection effect.

Description

Technical Field

[0001] This invention relates to the field of optical module testing technology, and in particular to a method, system, device and medium for detecting the gold wire height of an optical module. Background Technology

[0002] Gold wires, as critical connecting components between the internal chips and external circuits of optical modules, play a vital role in transmitting electrical signals from the substrate to the laser or detector chip and feeding back signals from the chip to the substrate. Abnormal gold wire height directly impacts the reliability of the optical module, causing signal transmission interference and reducing production yield. Therefore, accurate detection of gold wire height is crucial for identifying process defects, preventing subsequent failures, and ensuring the long-term performance stability of optical modules.

[0003] Currently, gold wire height detection mainly relies on traditional methods such as manual visual inspection, laser triangulation, confocal microscopy, and white light interferometry. In recent years, with the development of 3D laser imaging and artificial intelligence technologies, a technical approach based on scanning the gold wire surface with line or surface lasers to generate a 3D point cloud, then reconstructing the gold wire morphology and calculating its height, has gradually become mainstream. This type of method, combined with deep learning algorithms, can achieve automatic recognition of the 3D contour and height measurement of gold wires, exhibiting high detection efficiency.

[0004] In existing technologies, such as the patent with patent number CN115482224B, a method for detecting dense gold wires on a chip surface is disclosed. This method segments the gold wires and fits them into straight line segments, then performs Gaussian curve fitting along the radial cross-section to extract the center point, and finally achieves gold wire connection and identification through straight line fitting. However, this method has certain limitations in practical applications: when the gold wires are distorted due to oxidation, contamination, intersection, or imaging noise, Gaussian fitting is prone to center point positioning deviation, especially when the spacing between gold wires is smaller than the imaging resolution, which may mistakenly identify multiple gold wires as a single wire, affecting detection accuracy. In addition, for arc-shaped or curved gold wires, segmented straight line fitting is difficult to accurately restore their true three-dimensional contour, leading to height measurement errors.

[0005] Therefore, existing gold wire height detection methods still have certain shortcomings in terms of detection accuracy, real-time performance, and versatility. There is an urgent need for a gold wire height detection solution that can balance high accuracy, high efficiency, and strong adaptability. Summary of the Invention

[0006] (a) Technical problems to be solved In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a method, system, device and medium for gold wire height detection of optical modules, aiming to achieve high-precision identification, high-speed detection and integrated gold wire height detection for multiple models.

[0007] (II) Technical Solution To achieve the above objectives, the main technical solutions adopted by the present invention include: In a first aspect, embodiments of the present invention provide a method for detecting the height of gold wires in an optical module, comprising: Multi-scale convolutional feature extraction is performed on the image of the optical module, and the V channel features of the image in the HSV color space are fused into the convolutional features to obtain a multi-scale convolutional feature map. The convolutional feature map is input into the corresponding multi-scale detection head for gold line localization and detection, and the two-dimensional coordinate information of each gold line is obtained. The multi-scale detection head is optimized by the bounding box regression loss function and the minimum scale detection head is strengthened based on the positive and negative sample allocation strategy. Based on two-dimensional coordinate information, the region of interest corresponding to each gold line is extracted from the optical module image, and pixel-level depth detection is performed on the region of interest to obtain the relative depth map and reference depth value of each gold line. Outliers were removed from the relative depth map according to the Laida criterion, and the actual height of the gold wire relative to the substrate was obtained based on the obtained optimal relative depth map, combined with two-dimensional coordinate information and the corresponding reference depth value.

[0008] Optionally, before performing multi-scale convolutional feature extraction on the image of the optical module and fusing the V channel features of the image in the HSV color space into the convolutional features to obtain a multi-scale convolutional feature map, the method further includes: The pixels in the gold line edge region of the input optical module image are locked, and the locked optical module image is denoised using a Gaussian filter. Adaptive histogram equalization is performed on the denoised optical module image to obtain an illuminated optical module image. The light module image with uniform illumination is normalized in terms of pixel values ​​to obtain the light module image used as input for multi-scale convolution feature extraction.

[0009] Optionally, multi-scale convolutional feature extraction is performed on the image of the optical module, and the V channel features of the image in the HSV color space are fused into the convolutional features to obtain multi-scale convolutional feature maps, including: An improved convolutional network was used to extract multi-scale convolutional features from the optical module image, resulting in three convolutional feature maps at different scales. The maximum-scale convolutional feature map and the minimum-scale convolutional feature map are sampled to the same spatial dimension as the intermediate-scale convolutional feature map to obtain a multi-scale feature map with the same spatial dimension. The V channel features of the optical module image in the HSV color space are fused with feature maps at each scale to obtain a multi-scale fused feature map. The improved convolutional network is constructed by removing the bottleneck layer in the CSP structure and replacing all 3×3 convolutional layers with two sets of cascaded 1×3 grouped convolutional layers.

[0010] Optionally, the V channel features of the optical module image in the HSV color space are fused with feature maps at each scale to obtain a multi-scale fused feature map, including: The image of the optical module is converted into an image in the HSV color space, and the V channel information is extracted and normalized to obtain the V channel features; The spatial dimension of the V-channel features is compressed to a size similar to the corresponding feature size using a convolutional network. Figure 1 The processed V-channel features are then added pixel-by-pixel to the feature maps at each scale to obtain the initial multi-scale fused feature map. By using a pre-defined attention module, channel and spatial attention weights are applied to the V-channel features in the initial multi-scale fusion feature map to obtain the multi-scale fusion feature map.

[0011] Optionally, the convolutional feature map is input into the corresponding multi-scale detection head for gold line localization and detection, obtaining the two-dimensional coordinate information of each gold line, including: Add a penalty term for the aspect ratio of the gold line to the obtained enhanced crossover ratio loss function to obtain the bounding box regression loss function; The initial multi-scale detector head is trained using the bounding box regression loss function, and the bounding box threshold corresponding to the smallest scale detector head is dynamically adjusted based on the positive and negative sample allocation strategy to obtain the optimized multi-scale detector head. Each layer of convolutional feature map is input into the corresponding multi-scale detection head for gold line localization and detection. The smallest scale detection head uses a cross-layer skip connection method to fuse the V channel features after 2x downsampling. Based on the detection results, the two-dimensional coordinate information of each gold wire is output, including the center position coordinates and angle information of the gold wire.

[0012] Optionally, based on two-dimensional coordinate information, regions of interest corresponding to each gold line are extracted from the image of the optical module, and pixel-level depth detection is performed on the regions of interest to obtain the relative depth map and reference depth value of each gold line, including: Based on two-dimensional coordinate information, the region between the upper left and lower right points of each gold line is taken as the region of interest, and the region of interest image corresponding to each gold line is cropped from the optical module image. The images of each region of interest are input into the preset teacher model and student model respectively. The teacher model generates pseudo-label depth maps based on the encoder and decoder, and the student model is trained based on the pseudo-label depth maps through attention alignment loss and output layer distillation to obtain the optimized depth prediction results. The depth prediction results output by the student model are processed by the sigmoid activation function and multiplied by the preset maximum depth value to obtain the relative depth measurement values ​​of each pixel in the region of interest. Select a specified reference position in the optical module image, and extract the depth measurement value at the reference position from the depth prediction result as the reference depth value of the gold line.

[0013] Optionally, outlier removal is performed on the relative depth map according to the Laida criterion, and based on the obtained optimal relative depth map, combined with two-dimensional coordinate information and the corresponding reference depth value, the actual height of the gold wire relative to the substrate is obtained, including: The relative depth measurements of each pixel in the relative depth map are filtered out, and outliers that deviate from the average value by more than three standard deviations are removed to obtain the optimal relative depth map. The reference depth value is used as the relative height of the substrate, and the difference is calculated by combining the optimal relative depth map to obtain the depth map of the gold wire relative to the substrate. Based on two-dimensional coordinate information, the depth map is divided into multiple depth image blocks with the same width, and the average depth of each depth image block is obtained. Traverse all depth averages and determine the maximum depth average as the actual height of the current gold wire relative to the substrate.

[0014] Secondly, embodiments of the present invention provide a gold wire height detection system for an optical module, comprising: The multi-scale visual feature fusion module is used to extract multi-scale convolutional features from the image of the light module and fuse the V channel features of the image in the HSV color space into the convolutional features to obtain a multi-scale convolutional feature map. The multi-scale gold line localization module is used to input the convolutional feature map into the corresponding multi-scale detection head for gold line localization and detection, and obtain the two-dimensional coordinate information of each gold line. The multi-scale detection head is obtained by optimizing the bounding box regression loss function and strengthening the minimum scale detection head based on the positive and negative sample allocation strategy. The gold line region depth perception module is used to extract the region of interest corresponding to each gold line in the light module image based on two-dimensional coordinate information, and perform pixel-level depth detection on the region of interest to obtain the relative depth map and reference depth value of each gold line. The gold wire height calculation module is used to remove outliers from the relative depth map according to the Laida criterion, and based on the obtained optimal relative depth map, combined with two-dimensional coordinate information and the corresponding reference depth value, to obtain the actual height of the gold wire relative to the substrate.

[0015] Thirdly, embodiments of the present invention provide an electronic device, comprising: processor; The memory stores a gold wire height detection method for the processor to control the optical module described above.

[0016] Fourthly, embodiments of the present invention provide a computer-readable medium having computer-executable instructions stored thereon, wherein the executable instructions, when executed by a processor, implement the above-described method for detecting the gold wire height of an optical module.

[0017] (III) Beneficial Effects This invention utilizes multi-scale convolutional feature extraction and fusion with HSV color space V channel features to effectively capture key features of gold threads under different scales and lighting conditions. This enhances the ability to identify gold threads that are small, have low contrast, or are subject to reflective interference, significantly improving the accuracy of two-dimensional localization. Furthermore, a multi-scale detection head optimized with bounding box regression loss and a positive / negative sample allocation strategy is employed to further improve the accuracy of gold thread localization.

[0018] Furthermore, this invention automatically extracts the region of interest for the gold line based on precise two-dimensional coordinates and performs pixel-level depth detection, avoiding computational redundancy caused by global image processing. Simultaneously, by combining a relative depth map with a reference depth value, it achieves rapid and automatic conversion from two-dimensional positioning to three-dimensional height information, reducing the reliance on manual intervention and complex calibration found in traditional measurement methods. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a method for detecting the gold wire height of an optical module according to an embodiment of the present invention. Figure 2 A flowchart illustrating the algorithm for a gold wire height detection method for an optical module according to an embodiment of the present invention; Figure 3 A flowchart of an algorithm for gold thread positioning and detection provided in an embodiment of the present invention; Figure 4 This is a diagram illustrating the effect of extracting the region of interest from a gold line according to an embodiment of the present invention. Figure 5 A flowchart of an image pixel-level depth detection algorithm provided in an embodiment of the present invention; Figure 6 This is intended to demonstrate the effect of relative depth measurement values ​​provided in an embodiment of the present invention. Detailed Implementation

[0020] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] refer to Figures 1 to 6As shown in the embodiment of the present invention, a method for detecting the height of gold wires in an optical module includes: extracting multi-scale convolutional features from an image of the optical module and fusing the V channel features of the image in the HSV color space into the convolutional features to obtain a multi-scale convolutional feature map; inputting the convolutional feature map into a corresponding multi-scale detection head for gold wire localization detection to obtain two-dimensional coordinate information of each gold wire; the multi-scale detection head is obtained by optimizing the bounding box regression loss function and strengthening the minimum scale detection head based on a positive and negative sample allocation strategy; based on the two-dimensional coordinate information, extracting the region of interest corresponding to each gold wire in the image of the optical module and performing pixel-level depth detection on the region of interest to obtain the relative depth map and reference depth value of each gold wire; removing outliers from the relative depth map according to the Laida criterion, and obtaining the actual height of the gold wire relative to the substrate based on the obtained optimal relative depth map, combined with the two-dimensional coordinate information and the corresponding reference depth value.

[0022] This embodiment extracts features through multi-scale convolution and fuses them with V-channel features from the HSV color space. This effectively captures key features of gold threads under different scales and lighting conditions, enhancing the ability to identify gold threads that are small, have low contrast, or are subject to reflective interference, and significantly improving the accuracy of two-dimensional localization. Furthermore, a multi-scale detection head optimized with bounding box regression loss and a positive / negative sample allocation strategy is employed to further improve the accuracy of gold thread localization.

[0023] Furthermore, this embodiment automatically extracts the region of interest for the gold line based on precise two-dimensional coordinates and performs pixel-level depth detection, avoiding computational redundancy caused by global image processing. Simultaneously, by combining a relative depth map with a reference depth value, it achieves rapid and automatic conversion from two-dimensional positioning to three-dimensional height information, reducing the reliance on manual intervention and complex calibration found in traditional measurement methods.

[0024] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0025] Specifically, refer to Figure 1 and Figure 2 As shown, this embodiment proposes a method for detecting the height of gold wires in an optical module. This method is based on an improved YOLOv10 model and a Depth Anything V2 model, and may include the following steps S100 to S400: S100. Perform multi-scale convolution feature extraction on the image of the optical module, and fuse the V channel features of the image in the HSV color space into the convolution features to obtain a multi-scale convolution feature map.

[0026] In this embodiment, by extracting features through multi-scale convolution and fusing them with the V channel features of the HSV color space, the key features of gold wires under different scales and lighting conditions can be effectively captured. This effectively identifies dense or interlaced gold wires, enhances the ability to identify gold wires under fine, low-contrast, or reflective interference conditions, and significantly improves the accuracy of two-dimensional positioning of gold wires.

[0027] In this embodiment, before step S100, the following steps G100 to G300 may also be included: G100 locks the pixels in the gold line edge region of the input optical module image and uses a Gaussian filter to denoise the locked optical module image.

[0028] Furthermore, in this embodiment, before denoising the image of the optical module, the pixels in the gold line edge region of the image are locked. This ensures that the gold line edge details are preserved while the image is smoothed using Gaussian filtering, avoiding edge blurring or loss of detail due to the smoothing operation. Specifically, the gold line edge region can be located first using an edge detection algorithm or threshold segmentation technique to generate a corresponding binary mask. In the subsequent Gaussian filtering process, the pixels in the edge region identified by the mask retain their original values ​​or are processed with a very small filtering intensity, while non-edge regions are treated with conventional Gaussian filtering to effectively suppress noise. This strategy achieves a good balance between noise reduction and edge preservation, contributing to improved accuracy in subsequent image analysis and processing.

[0029] G200 performs adaptive histogram equalization on the denoised optical module image to obtain an illuminated optical module image.

[0030] G300 performs pixel value normalization processing on the light module image with equal illumination to obtain the light module image used as input for multi-scale convolution feature extraction.

[0031] It is worth mentioning that during the model training phase, when the dataset is small, in addition to using the above steps G100 to G300 to preprocess the dataset, the samples can also be expanded through operations such as rotation, scaling, and horizontal flipping to augment the dataset and thus improve the model's generalization ability.

[0032] In this embodiment, reference Figure 3 As shown, step S100 may further include the following steps S110 to S130: S110. Using an improved convolutional network, multi-scale convolutional feature extraction is performed on the optical module image to obtain three convolutional feature maps at different scales. The improved convolutional network is constructed by removing the bottleneck layer in the CSP structure and replacing all 3×3 convolutional layers with two sets of cascaded 1×3 grouped convolutional layers.

[0033] Furthermore, the improvements to the convolutional network in this embodiment mainly include network depth reduction and convolutional kernel splitting. Network depth reduction: The convolutional layers are reduced from 12 to 10, removing two redundant CSP bottleneck layers (each bottleneck layer containing 2 convolutional layers) from the backbone network, retaining 10 key convolutional layers, forming a convolutional network structure with an input layer, 10 convolutional layers, and feature output. Convolutional kernel splitting: All 3×3 convolutional layers in the original convolutional network are converted into two cascaded 1×3 grouped convolutional structures, with both groups using 2-way grouped convolutions, thereby reducing computational cost.

[0034] S120. Sample the maximum-scale convolutional feature map and the minimum-scale convolutional feature map to the same spatial dimension as the intermediate-scale convolutional feature map to obtain a multi-scale feature map with the same spatial dimension.

[0035] Furthermore, the spatial dimensions of the convolutional feature maps at different scales are 80×80×256 for the C3 feature extraction module (detecting short gold lines), 40×40×512 for the C4 feature extraction module (detecting dense gold lines), and 20×20×1024 for the C5 feature extraction module (detecting long gold lines). Then, the convolutional feature map of the C3 feature extraction module is downsampled, and the convolutional feature map of the C5 feature extraction module is upsampled to the spatial dimension of the C4 feature extraction module. The V channel features of the light module image in the HSV color space are also sampled to the C4 scale. Finally, the V channel features are fused with the feature maps of each scale through the FPN feature pyramid network to obtain a multi-scale fused feature map.

[0036] S130. The V channel features of the optical module image in the HSV color space are fused with the feature maps at each scale to obtain a multi-scale fused feature map.

[0037] Furthermore, step SS130 may also include the following steps S131 to S133: S131. Convert the optical module image into an image in the HSV color space, extract the V channel information and perform normalization processing to obtain the V channel features.

[0038] S132. Compress the spatial dimension of the V-channel features to a size similar to the corresponding feature size using a convolutional network. Figure 1 The processed V-channel features are then added pixel-by-pixel to the feature maps at each scale to obtain the initial multi-scale fused feature map.

[0039] S133. Using a preset attention module, channel and spatial attention weights are applied to the V channel features in the initial multi-scale fusion feature map to obtain the multi-scale fusion feature map.

[0040] To further explain, this embodiment employs a V-channel feature fusion strategy within the HSV channel to enhance detection performance. By strengthening the key features of gold lines under different scales and lighting conditions, it effectively identifies dense or intersecting gold lines through feature fusion. First, the input optical module image is converted to HSV format, and the V-channel information is extracted and normalized. Next, after each feature layer output, the number of V channels is compressed to match the current feature layer using a 1×1 convolution (e.g., if the feature layer has 256 channels, the V channels are mapped to 256 channels via a 1×1 convolution). Then, a feature addition and attention weighting mechanism is used to add the multi-scale feature map and the V-channel features pixel by pixel. Finally, a lightweight channel attention module weights the fused features, enhancing the weight of high-brightness regions of the gold lines in the V channel, thereby suppressing background interference.

[0041] S200. Input the convolutional feature map into the corresponding multi-scale detection head for gold line localization and detection, and obtain the two-dimensional coordinate information of each gold line. The multi-scale detection head is obtained by optimizing the bounding box regression loss function and strengthening the minimum scale detection head based on the positive and negative sample allocation strategy.

[0042] In this embodiment, a multi-scale detection head enhanced by bounding box regression loss optimization and positive / negative sample allocation strategy is used for gold wire localization and detection, effectively solving the problems of missed detection and localization deviation caused by the small size, varied shape, and strong background interference of gold wires. By multi-scale feature fusion and targeted enhancement of the smallest scale detection head, the model's ability to perceive gold wires of different scales is significantly improved, and high localization accuracy and robustness are maintained, especially in densely arranged scenarios.

[0043] In this embodiment, reference Figure 3 As shown, step S200 may further include the following steps S210 to S240: S210. Add a penalty term for the aspect ratio of the gold line to the obtained enhanced crossover ratio loss function to obtain the bounding box regression loss function.

[0044] Furthermore, a penalty term for the aspect ratio of the gold line is added to the enhanced crossover ratio loss function to construct a bounding box regression loss function. If the aspect ratio of the predicted box deviates significantly from the actual value, the loss is increased.

[0045] S220. The preset initial multi-scale detector head is trained using the bounding box regression loss function, and the bounding box threshold corresponding to the smallest scale detector head is dynamically adjusted based on the positive and negative sample allocation strategy to obtain the optimized multi-scale detector head.

[0046] Furthermore, the positive and negative sample allocation strategy is a dynamic label allocation strategy targeting the characteristics of small gold line targets (long gold lines). Candidate boxes with an intersection-union ratio greater than 0.2 (other scales are set to 0.3) are judged as positive samples, thereby improving the sample utilization rate of small targets.

[0047] S230. Input the convolutional feature maps of each layer into the corresponding multi-scale detection head for gold line localization and detection. The smallest scale detection head uses a cross-layer skip connection method to fuse the V channel features after 2x downsampling.

[0048] Furthermore, this embodiment employs Feature Pyramid Network (FPN) adaptation optimization to ensure that the slender gold line feature is not lost during multi-scale transmission. For the smallest scale feature, i.e., the small gold line target, a cross-layer skip connection is added to directly introduce the V channel feature of the input image after a 2x downsampling. Enhancement is applied under FPN, with two sets of cascaded 1×3 grouped convolutional layers added to each layer, ensuring that the slender gold line feature can be completely captured when detecting the smallest scale feature.

[0049] S240. Based on the detection results, output the two-dimensional coordinate information of each gold wire. The two-dimensional coordinate information includes the center position coordinates and angle information of the gold wire.

[0050] S300: Based on two-dimensional coordinate information, extract the region of interest corresponding to each gold line in the optical module image, and perform pixel-level depth detection on the region of interest to obtain the relative depth map and reference depth value of each gold line.

[0051] In this embodiment, the region of interest for the gold line is automatically extracted based on precise two-dimensional coordinates, and pixel-level depth detection is performed, avoiding computational redundancy caused by global image processing. Simultaneously, by combining a relative depth map with a reference depth value, rapid and automatic conversion from two-dimensional positioning to three-dimensional height information is achieved, reducing the reliance on manual intervention and complex calibration found in traditional measurement methods.

[0052] In this embodiment, reference Figure 4 As shown, step S300 may further include the following steps S310 to S340.

[0053] S310. Based on two-dimensional coordinate information, the region between the upper left point and the lower right point of each gold line is taken as the region of interest, and the region of interest image corresponding to each gold line is cropped from the optical module image.

[0054] For example, in a specific embodiment, the region of interest image corresponding to each gold line is as follows: Figure 5 As shown.

[0055] S320. Input the images of each region of interest into the preset teacher model and student model respectively. The teacher model generates pseudo-label depth maps based on the encoder and decoder. The student model is trained based on the pseudo-label depth maps through attention alignment loss and output layer distillation to obtain the optimized depth prediction results.

[0056] Furthermore, the teacher model uses DINOv2-G as the encoder, paired with a DPT decoder to generate depth maps. The teacher model exhibits extremely high accuracy on synthetic data, and the quality of the generated pseudo-label depth maps is close to that of the true depth maps. The student model employs a lightweight DINOv2 variant through feature mimicry: an attention alignment loss is applied to the encoder layer, forcing the student model's self-attention distribution to match that of the teacher model; output distillation training ensures that the student model's depth predictions simultaneously satisfy pseudo-label supervision and semantic consistency.

[0057] S330. The depth prediction results output by the student model are processed by the sigmoid activation function and multiplied by the preset maximum depth value to obtain the relative depth measurement values ​​of each pixel in the region of interest.

[0058] Furthermore, the decoded feature layer is activated using a sigmoid function and multiplied by a preset maximum depth value (default 10 mm) to convert it into the relative depth of each pixel position. Then, the coordinates of the region of interest are mapped back to the original image for display, such as... Figure 6 As shown, the camera is a line scan camera with a lens magnification of 20x. The imaging accuracy is 1µm in the horizontal X and Y directions and 7µm in the depth direction. The figure visualizes the 3D information of a pixel on a gold line in the middle position, where x, y, and z are in mm.

[0059] S340. Select a specified reference position in the optical module image, and extract the depth measurement value at the reference position from the depth prediction result as the reference depth value of the gold line.

[0060] S400. Based on the Laida criterion, outliers are removed from the relative depth map. Based on the obtained optimal relative depth map, combined with two-dimensional coordinate information and the corresponding reference depth value, the actual height of the gold wire relative to the substrate is obtained.

[0061] In this embodiment, step S400 may further include the following steps S410 to S440.

[0062] S410. Filter the relative depth measurement values ​​of each pixel in the relative depth map, remove outliers that deviate from the average value by more than three standard deviations, and obtain the optimal relative depth map.

[0063] Furthermore, in actual use of relative depth maps, a certain number of outliers exist in the relative depth measurements. For example, inaccurate focusing can cause blurring at the edges of gold lines, resulting in relative depth measurements that are far lower than the actual values; lighting angle issues can lead to strong reflections and overexposure of the gold lines, causing outliers; oxide layers and contaminants on the gold line surface can cause outliers in depth estimation; and prediction errors within the model itself can also cause outliers. Therefore, this embodiment uses the Laida criterion to remove outliers and ensure the reliability of the depth values. First, the mean depth μ and standard deviation σ are calculated for the relative depth measurements of each pixel in the relative depth map. Then, if the relative depth measurement x satisfies |x-μ|>3σ, the relative depth measurement x is determined to be an outlier and removed from the relative depth map. Finally, after removing outliers, the remaining relative depth measurements are used to re-statistically analyze the features to obtain the optimal relative depth map.

[0064] S420. The reference depth value is used as the relative height of the substrate, and the difference is calculated by combining the optimal relative depth map to obtain the depth map of the gold wire relative to the substrate.

[0065] S430. Based on the two-dimensional coordinate information, the depth map is divided into multiple depth image blocks with the same width, and the average depth of each depth image block is obtained.

[0066] Furthermore, based on the two-dimensional coordinate information, the depth map in the corresponding area is divided into depth image blocks with a width of 10um. Then, the depth value data in each depth image block is sorted in descending order, the top 5% of points are removed, and finally the average value of the top 10% of depth values ​​is calculated and determined as the depth value of the current depth image block.

[0067] S440: Traverse all depth averages and determine the maximum depth average as the actual height of the current gold wire relative to the substrate.

[0068] Furthermore, this embodiment also provides a gold wire height detection system for an optical module, including: The multi-scale visual feature fusion module is used to extract multi-scale convolutional features from the light module image and fuse the V channel features of the image in the HSV color space into the convolutional features to obtain a multi-scale convolutional feature map.

[0069] The multi-scale gold line localization module is used to input the convolutional feature map into the corresponding multi-scale detection head for gold line localization and detection, and obtain the two-dimensional coordinate information of each gold line. The multi-scale detection head is obtained by optimizing the bounding box regression loss function and strengthening the minimum scale detection head based on the positive and negative sample allocation strategy.

[0070] The gold line region depth perception module is used to extract the region of interest corresponding to each gold line in the light module image based on two-dimensional coordinate information, and perform pixel-level depth detection on the region of interest to obtain the relative depth map and reference depth value of each gold line.

[0071] The gold wire height calculation module is used to remove outliers from the relative depth map according to the Laida criterion, and based on the obtained optimal relative depth map, combined with two-dimensional coordinate information and the corresponding reference depth value, to obtain the actual height of the gold wire relative to the substrate.

[0072] Furthermore, this embodiment also provides an electronic device, including: a processor; and a memory for storing a gold wire height detection method for the processor to control the optical module described above.

[0073] Finally, this embodiment also provides a computer-readable medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method for detecting the gold wire height of an optical module.

[0074] In summary, this invention proposes a method, system, device, and medium for detecting the height of gold wires in optical modules. By fusing multi-scale visual features and HSV color space information, combined with an optimized multi-scale detection head, this invention achieves high-precision two-dimensional positioning of the gold wires. Based on this, an efficient depth estimation model is used to obtain the depth information of the gold wire region, and the depth information is optimized based on the Laida criterion. Finally, a high-precision calculation strategy is used to calculate the actual height of the gold wire relative to the substrate. This invention achieves automated and high-precision measurement of gold wire height, effectively overcoming the limitations of existing technologies in adapting to changes in gold wire morphology, differences in imaging conditions, and insufficient background interference. It performs particularly well in identifying densely arranged, overlapping, and slender gold wires, significantly improving detection accuracy, efficiency, and system reliability, providing strong technical support for quality control and process optimization in the production of optical modules.

[0075] Since the systems / devices described in the above embodiments of the present invention are systems / devices used to implement the methods of the above embodiments of the present invention, those skilled in the art can understand the specific structure and modifications of the systems / devices based on the methods described in the above embodiments of the present invention, and therefore will not be repeated here. All systems / devices used in the methods of the above embodiments of the present invention fall within the scope of protection of the present invention.

[0076] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0078] It should be noted that in the description of this invention, the word "a" or "an" preceding a component does not exclude the existence of multiple such components. This invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. The use of terms such as first, second, third, etc., is merely for convenience and does not indicate any order. These terms can be understood as part of the component names.

[0079] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0080] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning of the basic inventive concept, can make other changes and modifications to these embodiments.

[0081] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from the spirit and scope of the invention.

Claims

1. A method for detecting the height of gold wires in an optical module, characterized in that, include: Multi-scale convolutional feature extraction is performed on the image of the optical module, and the V channel features of the image in the HSV color space are fused into the convolutional features to obtain a multi-scale convolutional feature map. The convolutional feature maps are input into the corresponding multi-scale detector head for gold line localization and detection, obtaining the two-dimensional coordinate information of each gold line. The multi-scale detector head is optimized by the bounding box regression loss function and the minimum scale detector head is enhanced based on the positive and negative sample allocation strategy. This includes: adding a penalty term for the aspect ratio of the gold line to the obtained enhanced intersection-over-union loss function to obtain the bounding box regression loss function; training the preset initial multi-scale detector head using the bounding box regression loss function, and dynamically adjusting the bounding box threshold corresponding to the minimum scale detector head based on the positive and negative sample allocation strategy to obtain the optimized multi-scale detector head; inputting the convolutional feature maps of each layer into the corresponding multi-scale detector head for gold line localization and detection, wherein the minimum scale detector head uses a cross-layer skip connection method to fuse the V channel features after 2x downsampling; and outputting the two-dimensional coordinate information of each gold line based on the detection results, including the center position coordinates and angle information of the gold line. Based on two-dimensional coordinate information, the region of interest corresponding to each gold line is extracted from the optical module image, and pixel-level depth detection is performed on the region of interest to obtain the relative depth map and reference depth value of each gold line. Outlier removal is performed on the relative depth map according to the Laida criterion. Based on the obtained optimal relative depth map, combined with two-dimensional coordinate information and the corresponding reference depth value, the actual height of the gold wire relative to the substrate is obtained. This includes: filtering the relative depth measurement values ​​of each pixel in the relative depth map, removing outliers that deviate from the average value by more than three standard deviations, and obtaining the optimal relative depth map; using the reference depth value as the relative height of the substrate, and performing difference calculations with the optimal relative depth map to obtain the depth map of the gold wire relative to the substrate; dividing the depth map into multiple depth image blocks of equal width based on the two-dimensional coordinate information, and obtaining the average depth value of each depth image block; traversing all the average depth values, and determining the largest average depth value as the current actual height of the gold wire relative to the substrate.

2. The method as described in claim 1, characterized in that, Before performing multi-scale convolutional feature extraction on the optical module image and fusing the V channel features of the image in the HSV color space into the convolutional features to obtain a multi-scale convolutional feature map, the following steps are also included: The pixels in the gold line edge region of the input optical module image are locked, and the locked optical module image is denoised using a Gaussian filter. Adaptive histogram equalization is performed on the denoised optical module image to obtain an illuminated optical module image. The light module image with uniform illumination is normalized in terms of pixel values ​​to obtain the light module image used as input for multi-scale convolution feature extraction.

3. The method as described in claim 1, characterized in that, Multi-scale convolutional feature extraction is performed on the image of the optical module, and the V channel features of the image in the HSV color space are fused into the convolutional features to obtain multi-scale convolutional feature maps, including: An improved convolutional network was used to extract multi-scale convolutional features from the optical module image, resulting in three convolutional feature maps at different scales. The maximum-scale convolutional feature map and the minimum-scale convolutional feature map are sampled to the same spatial dimension as the intermediate-scale convolutional feature map to obtain a multi-scale feature map with the same spatial dimension. The V channel features of the optical module image in the HSV color space are fused with feature maps at each scale to obtain a multi-scale fused feature map. The improved convolutional network is constructed by removing the bottleneck layer in the CSP structure and replacing all 3×3 convolutional layers with two sets of cascaded 1×3 grouped convolutional layers.

4. The method as described in claim 3, characterized in that, The V channel features of the optical module image in the HSV color space are fused with feature maps at each scale to obtain a multi-scale fused feature map, including: The image of the optical module is converted into an image in the HSV color space, and the V channel information is extracted and normalized to obtain the V channel features; The spatial dimension of the V-channel features is compressed to match the feature map at the corresponding scale by using a convolutional network. The processed V-channel features are then added pixel by pixel to the feature maps at each scale to obtain the initial multi-scale fused feature map. By using a pre-defined attention module, channel and spatial attention weights are applied to the V-channel features in the initial multi-scale fusion feature map to obtain the multi-scale fusion feature map.

5. The method as described in claim 1, characterized in that, Based on two-dimensional coordinate information, regions of interest corresponding to each gold line are extracted from the image of the optical module, and pixel-level depth detection is performed on the regions of interest to obtain the relative depth map and reference depth value of each gold line, including: Based on two-dimensional coordinate information, the region between the upper left and lower right points of each gold line is taken as the region of interest, and the region of interest image corresponding to each gold line is cropped from the optical module image. The images of each region of interest are input into the preset teacher model and student model respectively. The teacher model generates pseudo-label depth maps based on the encoder and decoder, and the student model is trained based on the pseudo-label depth maps through attention alignment loss and output layer distillation to obtain the optimized depth prediction results. The depth prediction results output by the student model are processed by the sigmoid activation function and multiplied by the preset maximum depth value to obtain the relative depth measurement values ​​of each pixel in the region of interest. Select a specified reference position in the optical module image, and extract the depth measurement value at the reference position from the depth prediction result as the reference depth value of the gold line.

6. A gold wire height detection system for an optical module, characterized in that, include: The multi-scale visual feature fusion module is used to extract multi-scale convolutional features from the image of the light module and fuse the V channel features of the image in the HSV color space into the convolutional features to obtain a multi-scale convolutional feature map. A multi-scale gold line localization module is used to input convolutional feature maps into the corresponding multi-scale detector head for gold line localization and detection, obtaining the two-dimensional coordinate information of each gold line. The multi-scale detector head is obtained by optimizing the bounding box regression loss function and strengthening the minimum scale detector head based on the positive and negative sample allocation strategy. This includes: adding a penalty term for the aspect ratio of the gold line to the obtained enhanced intersection-over-union loss function to obtain the bounding box regression loss function; training the preset initial multi-scale detector head using the bounding box regression loss function, and dynamically adjusting the bounding box threshold corresponding to the minimum scale detector head based on the positive and negative sample allocation strategy to obtain the optimized multi-scale detector head; inputting the convolutional feature maps of each layer into the corresponding multi-scale detector head for gold line localization and detection, wherein the minimum scale detector head uses a cross-layer skip connection method to fuse the V-channel features after 2x downsampling; and outputting the two-dimensional coordinate information of each gold line based on the detection results, including the center position coordinates and angle information of the gold line. The gold line region depth perception module is used to extract the region of interest corresponding to each gold line in the light module image based on two-dimensional coordinate information, and perform pixel-level depth detection on the region of interest to obtain the relative depth map and reference depth value of each gold line. The gold wire height calculation module is used to remove outliers from the relative depth map according to the Laida criterion, and based on the obtained optimal relative depth map, combined with two-dimensional coordinate information and the corresponding reference depth value, to obtain the actual height of the gold wire relative to the substrate. This includes: filtering the relative depth measurement values ​​of each pixel in the relative depth map, removing outliers that deviate from the average value by more than three standard deviations, and obtaining the optimal relative depth map; using the reference depth value as the relative height of the substrate, and performing difference calculations with the optimal relative depth map to obtain the depth map of the gold wire relative to the substrate; dividing the depth map into multiple depth image blocks of equal width based on the two-dimensional coordinate information, and obtaining the average depth value of each depth image block; traversing all the average depth values, and determining the largest average depth value as the current actual height of the gold wire relative to the substrate.

7. An electronic device, characterized in that, include: processor; The memory stores a method for detecting the gold wire height of an optical module as described in any one of claims 1-5, used by the processor.

8. A computer-readable medium having computer-executable instructions stored thereon, characterized in that, When the executable instructions are executed by the processor, they implement a method for detecting the gold wire height of an optical module as described in any one of claims 1-5.

Images