Method, system, equipment, and medium for defect detection of optical communication chips based on continuous imaging and local resolution calculation.

Abstract

This invention relates to a method, system, device, and medium for detecting defects in optical communication chips based on continuous imaging and local sharpness calculation. The method includes: preprocessing a sequence of optical communication chip images acquired through continuous imaging; using a target detection model to infer the effective key detection regions of the optical communication chip from each preprocessed image frame; performing multi-dimensional dynamic sharpness evaluation on the effective key detection regions to obtain a comprehensive sharpness score; selecting the image with the highest comprehensive sharpness score as the optimal sharp image; inputting the optimal sharp image into the defect detection model for defect detection, and generating a defect detection result by combining the bounding box coordinates of the effective key detection regions. This invention significantly improves the accuracy and robustness of optical communication chip defect detection through a closed-loop defect detection framework encompassing target region localization, multi-dimensional dynamic sharpness evaluation, optimal sharp image selection, and defect detection.

Description

Technical Field

[0001] This invention relates to the field of chip defect technology, and in particular to a method, system, device and medium for detecting defects in optical communication chips based on continuous imaging and local sharpness calculation. Background Technology

[0002] Optical communication chips are core components for data transmission in optical networks, and their manufacturing processes typically achieve precision down to the micrometer or even nanometer level. Surface defects on chips, such as scratches, cracks, breaks, and exposed substrate, despite their minute size, can lead to optical signal attenuation, transmission delays, and even communication interruptions. Therefore, high-precision and efficient detection of these defects during chip manufacturing is a crucial step in ensuring product yield and reliability.

[0003] Currently, automated inspection methods based on machine vision have become the mainstream approach for defect detection in optical communication chips. However, image sharpness directly affects the quality of defect feature extraction. Blurred images may obscure the true edges and textures of minute defects or produce false contours, leading to missed or false detections. Although existing technologies exist that link image sharpness evaluation with defect detection in a sequential inspection process, they generally suffer from the following shortcomings in practical applications: 1. Existing methods often evaluate the sharpness of the entire image uniformly, ignoring the fact that different regions of the optical communication chip have different detection priorities, resulting in a waste of computing resources and making it difficult to provide targeted sharpness judgments for key detection areas.

[0004] 2. Fixed-weight sharpness evaluation metrics are difficult to adapt to changes in image quality under different lighting and contrast conditions, affecting the robustness and adaptability of the evaluation results.

[0005] 3. Clarity evaluation and defect detection are often treated as independent modules, lacking an effective linkage mechanism between them. Blurry images are easily mistakenly included in the clear image set and enter the defect detection stage, increasing the risk of false detection and missed detection.

[0006] Therefore, those skilled in the art urgently need an efficient defect detection method that can accurately calculate the sharpness of local areas, dynamically adapt to changes in image quality, and be deeply integrated with the defect detection process. Summary of the Invention

[0007] (a) Technical problems to be solved In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a method, system, device and medium for detecting defects in optical communication chips based on continuous photography and local sharpness calculation. It solves the technical problems of wasted computing power and insufficient targeting of sharpness judgment caused by directly calculating the sharpness of the whole image in the existing continuous photography detection.

[0008] (II) Technical Solution To achieve the above objectives, the main technical solutions adopted by the present invention include: In a first aspect, embodiments of the present invention provide a method for detecting defects in optical communication chips based on continuous imaging and local sharpness calculation, including: Acquire the image sequence of the optical communication chip captured by continuous photography, and preprocess each frame of the image; The pre-set target detection model is used to infer the key detection areas of the optical communication chip in each pre-processed image, and the effective key detection areas are screened based on the pre-set confidence threshold. A multi-dimensional dynamic sharpness evaluation is performed on the effective key detection region in each frame of the image to obtain the comprehensive sharpness score of the effective key detection region. Based on the overall sharpness score, the image with the highest score is selected as the optimal sharpness image from the preprocessed optical communication chip image sequence; The optimal clear image is input into the preset defect detection model for defect detection, and the defect detection results of the optical communication chip are generated by combining the bounding box coordinates of the effective key detection area.

[0009] Optionally, acquiring a sequence of images of the optical communication chip through continuous image capture, and preprocessing each frame of the image, includes: The optical communication chip image sequence is acquired by a continuous imaging device, and each frame in the optical communication chip image sequence is adjusted to the same image format. Histogram equalization is performed on the optical communication chip image sequence after image formatting to obtain the equalized optical communication chip image sequence. The equalized optical communication chip image sequence was denoised using Gaussian filtering to obtain the denoised optical communication chip image sequence.

[0010] Optionally, a preset target detection model is used to infer the key detection regions of the optical communication chip for each preprocessed image, and effective key detection regions are selected based on a preset confidence threshold, including: The preset target detection model is trained based on the image data training set of the key detection areas of the optical communication chip. The trained target detection model is used to infer the key detection areas and corresponding bounding box coordinates of the optical communication chip in each frame of the preprocessed optical communication chip image sequence. The confidence level of the key detection area is compared with the preset confidence threshold, and the key detection areas with a confidence level not less than the preset confidence threshold are determined as valid key detection areas; When multiple valid key detection regions exist in the same frame of an image, the valid key detection region with the highest confidence level is determined as the final valid key detection region of the current image.

[0011] Optionally, a multi-dimensional dynamic sharpness evaluation is performed on the effective key detection regions in each frame image to obtain a comprehensive sharpness score for the effective key detection regions, including: The effective key detection region in each frame image is cropped, and the cropped image of the effective key detection region is processed into grayscale to obtain the grayscale image of the effective key detection region. The Laplacian variance and Sobel gradient magnitude of the grayscale image are calculated to obtain the Laplacian variance and Sobel gradient magnitude of the effective key detection region. Based on the image brightness and contrast of the current frame, the initial weights of the Laplacian variance and the Sobel gradient magnitude are linearly adjusted to generate dynamic weights. After normalizing the Laplacian variance and Sobel gradient magnitude, dynamic weights are used to perform weighted fusion of the normalized Laplacian variance and Sobel gradient magnitude to obtain the comprehensive sharpness score of the effective key detection area.

[0012] Optionally, the initial weights of the Laplacian variance and the Sobel gradient magnitude are linearly adjusted based on the image brightness and contrast of the current frame to generate dynamic weights, including: Obtain the average brightness value L and contrast value C of the effective key detection area in the current frame; According to the preset brightness threshold L min L max and contrast threshold C th The weighting coefficients ω1 for the Laplace variance and ω2 for the Sobel gradient magnitude are linearly adjusted. If L <L min If ω1 increases linearly, then ω1 increases linearly. If L>L max Then ω1 decreases linearly. If C <C th If ω increases linearly by 2; Otherwise, set ω1=ω2=0.5; Where ω1+ω2=1.

[0013] Optionally, based on the overall sharpness score, the image with the highest score is selected as the optimal sharpness image from the preprocessed optical communication chip image sequence, including: The average comprehensive sharpness score of the optical communication chip image sequence is obtained by combining the comprehensive sharpness scores of the effective key detection areas in each frame of the image. Images with a comprehensive sharpness score lower than the average comprehensive sharpness score are removed from the optical communication chip image sequence to obtain the optical communication chip image sequence with the best sharpness. The image with the highest score is selected from the image sequence of the optical communication chip as the optimal clear image, and the remaining images are stored as candidate clear images.

[0014] Optionally, the optimal clear image is input into a preset defect detection model for defect detection, and the defect detection results of the optical communication chip are generated by combining the bounding box coordinates of the effective key detection area, including: The pre-set defect detection model is trained based on the image data training set labeled with the defect types of optical communication chips; The best clear image is input into the trained defect detection model to perform defect detection, and the initial defect detection result is output. The system compares the bounding box coordinates of the effective key detection area with the defect positions in the initial defect detection results. Based on the comparison results, the validity of the defects is determined. Defects located within the key detection area are retained as valid defects, while falsely detected defects located outside the key detection area are removed. Defects that cross the boundary of the key detection area are marked. Finally, the defect detection results after area verification are output.

[0015] In a second aspect, embodiments of the present invention provide a defect detection system for optical communication chips based on continuous imaging and local sharpness calculation, comprising: The image preprocessing module is used to acquire the image sequence of the optical communication chip captured by continuous photography and to preprocess each frame of the image. The target detection module is used to infer the key detection areas of the optical communication chip using a preset target detection model, and to filter the effective key detection areas based on a preset confidence threshold. The sharpness calculation module is used to perform multi-dimensional dynamic sharpness evaluation on the effective key detection areas in each frame of the image to obtain the comprehensive sharpness score of the effective key detection areas. The clear image filtering module is used to select the image with the highest score from the preprocessed optical communication chip image sequence as the optimal clear image based on the comprehensive clarity score. The defect detection module is used to input the best clear image into the preset defect detection model for defect detection, and generate the defect detection results of the optical communication chip by combining the bounding box coordinates of the effective key detection area.

[0016] Thirdly, embodiments of the present invention provide an electronic device, comprising: At least one processor; and memory that is communicatively connected to at least one processor; The memory stores instructions that can be executed by at least one processor. These instructions are executed by at least one processor to enable the at least one processor to perform the aforementioned method for detecting defects in optical communication chips based on continuous imaging and local resolution calculation.

[0017] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the aforementioned method for detecting defects in optical communication chips based on continuous imaging and local sharpness calculation.

[0018] (III) Beneficial Effects The beneficial effects of this invention are as follows: The optical communication chip defect detection method based on continuous imaging and local sharpness calculation proposed in this invention has the following significant advantages over existing technologies due to its technical framework of closed-loop defect detection throughout the entire process, which includes target area localization, multi-dimensional dynamic sharpness evaluation, optimal sharpness image selection, and defect detection: First, this invention pre-extracts key detection areas of the optical communication chip using a target detection model, and performs multi-dimensional dynamic sharpness evaluation only on these local areas, rather than uniformly processing the entire image. This not only avoids interference from non-critical areas such as background and pins on the evaluation results, but also significantly reduces the computational load, allowing computing resources to focus on the core chip components that require the most attention.

[0019] Secondly, this invention evaluates the clarity of the effective key detection area in a multi-dimensional and dynamically weighted manner, and selects the image with the highest score from the image sequence as the optimal clear image to be sent to the defect detection module. This can adapt to changes in image quality under different lighting and imaging conditions, and avoid low-quality images caused by inaccurate focus, motion blur, etc. from entering the defect detection stage, thereby reducing the risk of missed detection and false detection caused by image blur.

[0020] Finally, this invention combines the defect detection results of the optimal clear image with the bounding box coordinates of the effective key detection region to generate a defect detection result constrained by the region. This ensures that only defects located within the key detection region are ultimately output, while false detections outside the region are automatically filtered out. Therefore, the closed-loop design of this invention—first locating the region, then selecting the best image, and finally verifying the region—improves the accuracy and reliability of the defect detection results. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a method for detecting defects in optical communication chips based on continuous imaging and local sharpness calculation, provided in an embodiment of the present invention. Figure 2A schematic diagram of the algorithm flow for a defect detection method for optical communication chips based on continuous imaging and local sharpness calculation, provided in an embodiment of the present invention; Figure 3 A schematic diagram of the algorithm flow for optimal clear image selection provided in an embodiment of the present invention; Figure 4 A schematic diagram showing the comprehensive sharpness score of each effective key detection area in a set of optical communication chip image sequences provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the algorithm flow of a defect detection method provided in an embodiment of the present invention; Figure 6 This is intended to illustrate the defect detection results provided in one embodiment of the present invention. Detailed Implementation

[0022] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] refer to Figures 1 to 6 As shown in the embodiment of the present invention, a defect detection method for optical communication chips based on continuous imaging and local sharpness calculation is proposed. The method includes: acquiring a sequence of optical communication chip images captured by continuous imaging, and preprocessing each frame of the image; using a preset target detection model to infer the key detection regions of each preprocessed image frame, extracting the key detection regions of the optical communication chip, and filtering the effective key detection regions based on a preset confidence threshold; performing multi-dimensional dynamic sharpness evaluation on the effective key detection regions in each frame of the image to obtain a comprehensive sharpness score for the effective key detection regions; selecting the image with the highest score from the preprocessed optical communication chip image sequence as the optimal sharp image based on the comprehensive sharpness score; inputting the optimal sharp image into a preset defect detection model for defect detection, and generating the defect detection result of the optical communication chip by combining the bounding box coordinates of the effective key detection regions.

[0024] This embodiment pre-extracts key detection areas of the optical communication chip using a target detection model, performing multi-dimensional dynamic sharpness evaluation only on these local areas, rather than uniformly processing the entire image. This not only avoids interference from non-critical areas such as background and pins on the evaluation results but also significantly reduces computational load, allowing computational resources to focus on the core chip components that require the most attention. Secondly, this embodiment also performs multi-dimensional and dynamically weighted sharpness evaluation on the effective key detection areas, selecting the image with the highest score from the image sequence as the optimal sharp image and sending it to the defect detection module. This adapts to changes in image quality under different lighting and imaging conditions, preventing low-quality images caused by inaccurate focus or motion blur from entering the defect detection stage, thereby reducing the risk of missed and false detections caused by image blur. Finally, this embodiment combines the defect detection results of the optimal sharp image with the bounding box coordinates of the effective key detection areas to generate defect detection results constrained by the region. This ensures that only defects located within the key detection areas are ultimately output, while false detections outside the regions are automatically filtered out. Therefore, the closed-loop design adopted in this invention, which first locates the region, then selects the best imaging method, and finally verifies the region, improves the accuracy and reliability of the defect detection results.

[0025] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0026] Specifically, refer to Figure 1 and 2 As shown in this embodiment, a defect detection method for optical communication chips based on continuous imaging and local sharpness calculation is proposed. Addressing the high-precision requirements of optical communication chip defect detection, it designs a closed-loop defect detection framework encompassing target area localization, multi-dimensional sharpness evaluation, optimal sharpness image selection, and defect detection. First, the key detection area (ROI) of the optical communication chip is accurately extracted using a target detection model. Then, multi-dimensional sharpness evaluations based on Laplacian variance (VoL) and gradient magnitude (GM) are fused to select the sharpest image of the ROI from the dataset. Finally, high-precision defect detection of the optical communication chip is achieved based on this image, avoiding missed or false detections caused by blurred images. The method includes the following steps S100 to S500: S100: Acquire the image sequence of the optical communication chip captured by continuous photography, and preprocess each frame of the image.

[0027] In this embodiment, step S100 may include the following sub-steps S110 to S130: S110. Acquire an image sequence of the optical communication chip using a continuous imaging device, and adjust each frame of the optical communication chip image sequence to the same image format.

[0028] Furthermore, after acquiring the image sequence of the optical communication chip through a continuous imaging device, all images in the image sequence are converted to BGR format and uniformly resolved to a resolution of 4992×2670 to avoid the image size difference affecting subsequent target recognition and defect detection.

[0029] S120. Perform histogram equalization on the optical communication chip image sequence after image formatting to obtain the equalized optical communication chip image sequence.

[0030] Furthermore, this embodiment performs histogram equalization illumination normalization processing on the image sequence of the optical communication chip to solve the problem of local image blurring caused by uneven illumination.

[0031] S130. Use Gaussian filtering to denoise the equalized optical communication chip image sequence to obtain the denoised optical communication chip image sequence.

[0032] Furthermore, this embodiment uses Gaussian filtering (cv2.GaussianBlur(gray,(3,3),0)) to smooth high-frequency noise generated by exposure and the sensor itself, so as to avoid noise interference with subsequent sharpness assessment and defect detection.

[0033] S200: Use a preset target detection model to infer the key detection area of ​​each preprocessed image frame, extract the key detection area of ​​the optical communication chip, and filter the effective key detection area based on a preset confidence threshold.

[0034] In this embodiment, the most relevant regions on the optical communication chip are accurately and efficiently located from the continuously acquired preprocessed image sequence, and interference from low-quality or non-target regions is eliminated, providing accurate regions of interest (ROIs) for subsequent local sharpness calculations and defect detection. Specifically, step S200 may include the following sub-steps S210 to S240: S210. Based on the image data training set of the key detection areas of the labeled optical communication chip, train the preset target detection model.

[0035] Furthermore, the object detection model employs the MobileNet-SSD network, which is lightweight, efficient, and suitable for real-time processing. The image training set consists of images with key ROI regions of the optical communication chip annotated using the LabelImg annotation tool. 80% of the images in the training set are used for training, and 20% for validation. In subsequent model training, the training parameters are configured as follows: input image size 960×960, batch size=16, learning rate 0.001, and 300 training epochs.

[0036] S220. Using the trained target detection model, reason about each frame of the preprocessed optical communication chip image sequence to obtain the key detection area of ​​the optical communication chip and the corresponding bounding box coordinates.

[0037] S230. Compare the confidence level of the key detection area with the preset confidence threshold, and determine the key detection area with a confidence level not less than the preset confidence threshold as the effective key detection area.

[0038] Furthermore, only valid key detection areas (ROIs) with a confidence level ≥ 0.85 are retained to avoid falsely detected areas affecting the sharpness assessment.

[0039] S240. When there are multiple valid key detection regions in the same frame image, the valid key detection region with the highest confidence is determined as the final valid key detection region of the current image.

[0040] Furthermore, when multiple valid key detection regions exist in the same frame of an image, i.e., multiple ROI regions with a confidence level ≥ 0.85, they are sorted according to their confidence levels, and the region with the highest confidence level is selected as the output. Additionally, the ROI regions of each frame in the optical communication chip image sequence are obtained. Some images are filtered by confidence level; if no ROI region exists, the corresponding image data is removed from the optical communication chip image sequence.

[0041] S300: Perform multi-dimensional dynamic sharpness evaluation on the effective key detection areas in each frame of the image to obtain the comprehensive sharpness score of the effective key detection areas.

[0042] In this embodiment, a multi-dimensional dynamic sharpness assessment is performed on the effective key detection area (ROI). Based on the actual brightness and contrast of the effective key detection area in the current frame, the weights of the Laplacian variance (VoL) and the Sobel gradient magnitude (GM) in the sharpness score are linearly adjusted. The weight allocation is automatically optimized under low-light or low-contrast conditions, enabling the sharpness evaluation to adapt to different lighting environments and changes in image quality. This results in more objective and accurate evaluation results, solving the problem of poor adaptability of fixed-weight methods. Specifically, refer to... Figure 3As shown, step S300 may include the following sub-steps S310 to S340: S310. Cropping the effective key detection region in each frame image and performing grayscale processing on the cropped image of the effective key detection region to obtain a grayscale image of the effective key detection region.

[0043] S320. Calculate the Laplacian variance and Sobel gradient magnitude of the grayscale image to obtain the Laplacian variance and Sobel gradient magnitude of the effective key detection area.

[0044] Furthermore, the Laplacian variance (VoL) is calculated: using a 3×3 kernel adapted to the fine texture of the optical communication chip, the variance of the Laplacian response of the ROI region is calculated. The Sobel gradient magnitude (GM) is calculated: using a 3×3 kernel, the Euclidean distance between the horizontal and vertical gradients is calculated, and the total gradient magnitude is statistically analyzed to make comparisons suitable for ROIs of different sizes.

[0045] S330. Based on the image brightness and contrast of the current frame, linearly adjust the initial weights of the Laplacian variance and the Sobel gradient magnitude to generate dynamic weights.

[0046] Furthermore, firstly, the average brightness value L and contrast value C of the effective key detection region in the current frame are obtained; then, based on the preset brightness threshold L... min L max and contrast threshold C th The weighting coefficients ω1 for the Laplace variance and ω2 for the Sobel gradient magnitude are linearly adjusted: If L <L min Then ω1 increases linearly, and the calculation formula is: ω1=min(0.9,0.5+(L min -L) / L min ·0.4), In the formula, min(·) is the upper limit cutoff function; If L>L max Then ω1 decreases linearly, and the calculation formula is: ω1=max(0.1,0.5+(LL max ) / (255-L max )·0.4), In the formula, min(·) is the lower limit cutoff function; If C <C th Then ω2 increases linearly, and the calculation formula is: ω2=min(0.9,0.5+(C th -C) / C th ·0.4), Otherwise, set ω1=ω2=0.5; Where ω1+ω2=1.

[0047] S340. After normalizing the Laplacian variance and Sobel gradient magnitude, dynamic weights are used to perform weighted fusion of the normalized Laplacian variance and Sobel gradient magnitude to obtain the comprehensive sharpness score of the effective key detection area.

[0048] Furthermore, since different ROIs have different sizes and texture densities, VoL and GM need to be normalized to obtain a comprehensive sharpness score after unifying the weights. This embodiment uses Min-Max normalization to map VoL and GM to the [0,1] interval: ; In the formula, X norm The sharpness index after normalization. X These are the original resolution metrics (VoL and GM). X min It is the global minimum value. X max This is the global maximum value.

[0049] The formula for calculating the overall sharpness score is: Score = ω1·VoL norm + ω2·GM norm ; In the formula, VoL norm For the normalized Laplace variance, GM norm This is the normalized Sobel gradient magnitude.

[0050] S400. Based on the comprehensive sharpness score, select the image with the highest score from the preprocessed optical communication chip image sequence as the optimal sharpness image.

[0051] In this embodiment, among the continuously acquired preprocessed images, based on the calculated comprehensive sharpness score of the effective key detection regions in each frame, the frame with the highest quality is automatically selected for subsequent defect detection, while candidate images are retained for verification, thereby improving the robustness and fault tolerance of the overall detection system. Specifically, step S400 may include the following sub-steps S410 to S430: S410. Obtain the average comprehensive sharpness score of the optical communication chip image sequence by calculating the comprehensive sharpness score of the effective key detection area in each frame image.

[0052] S420. Images with a comprehensive sharpness score lower than the average comprehensive sharpness score are removed from the optical communication chip image sequence to obtain the optical communication chip image sequence with the best sharpness.

[0053] S430: Select the image with the highest score from the image sequence of the optical communication chip as the optimal clear image, and store the remaining images as candidate clear images.

[0054] In one specific embodiment, a multi-dimensional dynamic sharpness evaluation is performed on the effective key detection regions in each frame of a set of communication chip image sequences to obtain a comprehensive sharpness score for the effective key detection regions, such as... Figure 4 As shown, based on the overall sharpness score and visual perception, the image with an overall sharpness score of 0.872 is the sharpest. The average overall sharpness score is 0.3324. Images with an overall sharpness score higher than 0.3324 are stored as candidate sharp images, and the remaining images are discarded.

[0055] S500: Input the best clear image into the preset defect detection model to perform defect detection, and generate the defect detection results of the optical communication chip by combining the bounding box coordinates of the effective key detection area.

[0056] In this embodiment, in a sequence of continuously captured images, the clearest image is automatically selected based on the overall sharpness score and sent to the defect detection model. Furthermore, the defect detection results are verified by checking the boundaries of key detection areas to eliminate false defects outside these areas. This embodiment employs a linked mechanism of first selecting the clearest image, then detecting, and finally verifying, preventing blurry images from entering the detection process at the source and effectively filtering out false detections, greatly improving the reliability and accuracy of defect detection. Specifically, refer to... Figure 5 As shown, step S500 may include the following sub-steps S510 to S530: S510. Train the preset defect detection model based on the image data training set labeled with the defect types of optical communication chips.

[0057] Furthermore, the defect detection model employs the CenterNet model. The training set for this model consists of image data annotated with the defect types (such as dirt, breakage, scratches, and plating overflow) of the optical communication chip using the LabelImg annotation tool. In the subsequent inference configuration, the input size is set to 960×960, consistent with the optimal clear image, with a confidence threshold of 0.85 and an IOU threshold of 0.45.

[0058] S520. Input the best clear image into the trained defect detection model to perform defect detection, and output the initial defect detection result.

[0059] S530. Compare the bounding box coordinates of the effective key detection area with the defect positions in the initial defect detection results. Based on the comparison results, determine the validity of the defects, retain the defects located within the key detection area as valid defects, remove false detection defects located outside the key detection area, and mark the defects that cross the boundary of the key detection area. Finally, output the defect detection results after area verification.

[0060] In one specific embodiment, the output defect detection result is referenced. Figure 6 As shown, the optical communication chip was detected as a damaged defect type from the highest resolution image, and the damaged area was marked.

[0061] Furthermore, this embodiment also proposes an optical communication chip defect detection system based on continuous imaging and local sharpness calculation, including: The image preprocessing module is used to acquire the image sequence of the optical communication chip captured by continuous photography and to preprocess each frame of the image.

[0062] The target detection module is used to infer the key detection areas of the optical communication chip by using a preset target detection model for each preprocessed image, and to filter the effective key detection areas based on a preset confidence threshold.

[0063] The sharpness calculation module is used to perform multi-dimensional dynamic sharpness evaluation on the effective key detection areas in each frame of the image, and obtain the comprehensive sharpness score of the effective key detection areas.

[0064] The clear image filtering module is used to select the image with the highest score from the preprocessed optical communication chip image sequence as the optimal clear image based on the comprehensive clarity score.

[0065] The defect detection module is used to input the best clear image into the preset defect detection model for defect detection, and generate the defect detection results of the optical communication chip by combining the bounding box coordinates of the effective key detection area.

[0066] Furthermore, this embodiment also proposes an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute the above-described method for detecting defects in optical communication chips based on continuous imaging and local sharpness calculation.

[0067] Finally, this embodiment also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method for detecting defects in optical communication chips based on continuous imaging and local sharpness calculation.

[0068] In summary, the optical communication chip defect detection method, system, device, and medium proposed in this invention, based on continuous imaging and local sharpness calculation, effectively solves the problem of missed or false detections of defects due to image blurring in the prior art through a closed-loop design of first locating the key detection area, then selecting optimal imaging, and finally verifying the area. It has the following significant advantages: First, it adapts to the structural characteristics and testing requirements of optical communication chips, making up for the shortcomings of existing testing methods in terms of specificity. Compared with traditional chips, optical communication chips have unique geometric structures, and their surface defects are easily affected by image clarity. Existing testing methods mostly adopt general image processing strategies without being optimized for the testing scenarios of optical communication chips. This embodiment achieves scenario-based optimization of the testing process through a closed-loop process of identifying and locating ROI, calculating local clarity, selecting the optimal image, and identifying defects, thereby improving the accuracy and stability of defect detection and meeting the high reliability and high consistency testing requirements of optical communication chips.

[0069] Secondly, by using multi-dimensional dynamic clarity assessment to select the best image, the problem of missed detection and false detection caused by blurry images is avoided from the source. Combined with the enhanced recognition capability of the defect detection model for minor defects, it can accurately identify various defect types such as dirt, damage, scratches, and over-plating, so that the core detection accuracy can reach the high standard requirements.

[0070] Third, the standardized data preprocessing process eliminates interference factors such as size differences, uneven lighting, and noise. The complementary design of multi-dimensional clarity evaluation indicators and the outlier filtering mechanism ensure that the solution can output stable results in different detection scenarios.

[0071] Fourth, the lightweight MobileNet-SSD model enables rapid and accurate ROI extraction, and the closed-loop design of the entire process reduces the time spent on invalid data processing. At the same time, it can output detailed inspection reports containing defect type, quantity, and location coordinates, providing reliable data support for subsequent defect analysis and process optimization, and meeting the high-precision and high-efficiency defect detection requirements in the production process of optical communication chips.

[0072] Since the systems / devices described in the above embodiments of the present invention are systems / devices used to implement the methods of the above embodiments of the present invention, those skilled in the art can understand the specific structure and modifications of the systems / devices based on the methods described in the above embodiments of the present invention, and therefore will not be repeated here. All systems / devices used in the methods of the above embodiments of the present invention fall within the scope of protection of the present invention.

[0073] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0074] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0075] It should be noted that in the description of this invention, the word "a" or "an" preceding a component does not exclude the existence of multiple such components. This invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. The use of terms such as first, second, third, etc., is merely for convenience and does not indicate any order. These terms can be understood as part of the component names.

[0076] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0077] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning of the basic inventive concept, can make other changes and modifications to these embodiments.

[0078] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from the spirit and scope of the invention.

Claims

1. A method for detecting defects in optical communication chips based on continuous imaging and local sharpness calculation, characterized in that, include; Acquire the image sequence of the optical communication chip captured by continuous photography, and preprocess each frame of the image; The pre-set target detection model is used to infer the key detection areas of the optical communication chip in each pre-processed image, and the effective key detection areas are screened based on the pre-set confidence threshold. A multi-dimensional dynamic sharpness evaluation is performed on the effective key detection region in each frame of the image to obtain the comprehensive sharpness score of the effective key detection region. Based on the overall sharpness score, the image with the highest score is selected as the optimal sharpness image from the preprocessed optical communication chip image sequence; The optimal clear image is input into the preset defect detection model for defect detection, and the defect detection results of the optical communication chip are generated by combining the bounding box coordinates of the effective key detection area.

2. The method as described in claim 1, characterized in that, Acquire a sequence of images from the optical communication chip captured by continuous imaging, and preprocess each frame of the image, including: The optical communication chip image sequence is acquired by a continuous imaging device, and each frame in the optical communication chip image sequence is adjusted to the same image format. Histogram equalization is performed on the optical communication chip image sequence after image formatting to obtain the equalized optical communication chip image sequence. The equalized optical communication chip image sequence was denoised using Gaussian filtering to obtain the denoised optical communication chip image sequence.

3. The method as described in claim 1, characterized in that, The pre-defined target detection model is used to infer the key detection regions of the optical communication chip for each pre-processed image frame, and the effective key detection regions are selected based on a pre-set confidence threshold, including: The preset target detection model is trained based on the image data training set of the key detection areas of the optical communication chip. The trained target detection model is used to infer the key detection areas and corresponding bounding box coordinates of the optical communication chip in each frame of the preprocessed optical communication chip image sequence. The confidence level of the key detection area is compared with the preset confidence threshold, and the key detection areas with a confidence level not less than the preset confidence threshold are determined as valid key detection areas; When multiple valid key detection regions exist in the same frame of an image, the valid key detection region with the highest confidence level is determined as the final valid key detection region of the current image.

4. The method as described in claim 1, characterized in that, A multi-dimensional dynamic sharpness evaluation is performed on the effective key detection regions in each frame of the image to obtain a comprehensive sharpness score for the effective key detection regions, including: The effective key detection region in each frame image is cropped, and the cropped image of the effective key detection region is processed into grayscale to obtain the grayscale image of the effective key detection region. The Laplacian variance and Sobel gradient magnitude of the grayscale image are calculated to obtain the Laplacian variance and Sobel gradient magnitude of the effective key detection region. Based on the image brightness and contrast of the current frame, the initial weights of the Laplacian variance and the Sobel gradient magnitude are linearly adjusted to generate dynamic weights. After normalizing the Laplacian variance and Sobel gradient magnitude, dynamic weights are used to perform weighted fusion of the normalized Laplacian variance and Sobel gradient magnitude to obtain the comprehensive sharpness score of the effective key detection area.

5. The method as described in claim 4, characterized in that, Based on the image brightness and contrast of the current frame, the initial weights of the Laplacian variance and Sobel gradient magnitude are linearly adjusted to generate dynamic weights, including: Obtain the average brightness value L and contrast value C of the effective key detection area in the current frame; According to the preset brightness threshold L min L max and contrast threshold C th The weighting coefficients ω1 for the Laplace variance and ω2 for the Sobel gradient magnitude are linearly adjusted. If L <L min If ω1 increases linearly, then ω1 increases linearly. If L>L max Then ω1 decreases linearly. If C <C th If ω increases linearly by 2; Otherwise, set ω1=ω2=0.5; Where ω1+ω2=1.

6. The method as described in claim 1, characterized in that, Based on the overall sharpness score, the image with the highest score is selected as the optimal sharpness image from the preprocessed optical communication chip image sequence, including: The average comprehensive sharpness score of the optical communication chip image sequence is obtained by combining the comprehensive sharpness scores of the effective key detection areas in each frame of the image. Images with a comprehensive sharpness score lower than the average comprehensive sharpness score are removed from the optical communication chip image sequence to obtain the optical communication chip image sequence with the best sharpness. The image with the highest score is selected from the image sequence of the optical communication chip as the optimal clear image, and the remaining images are stored as candidate clear images.

7. The method as described in claim 1, characterized in that, The optimal, clearest image is input into a preset defect detection model for defect detection. The defect detection results for the optical communication chip are then generated by combining the bounding box coordinates of the effective key detection regions. The pre-set defect detection model is trained based on the image data training set labeled with the defect types of optical communication chips; The best clear image is input into the trained defect detection model to perform defect detection, and the initial defect detection result is output. The system compares the bounding box coordinates of the effective key detection area with the defect positions in the initial defect detection results. Based on the comparison results, the validity of the defects is determined. Defects located within the key detection area are retained as valid defects, while falsely detected defects located outside the key detection area are removed. Defects that cross the boundary of the key detection area are marked. Finally, the defect detection results after area verification are output.

8. A defect detection system for optical communication chips based on continuous imaging and local sharpness calculation, characterized in that, include; The image preprocessing module is used to acquire the image sequence of the optical communication chip captured by continuous photography and to preprocess each frame of the image. The target detection module is used to infer the key detection areas of the optical communication chip using a preset target detection model, and to filter the effective key detection areas based on a preset confidence threshold. The sharpness calculation module is used to perform multi-dimensional dynamic sharpness evaluation on the effective key detection areas in each frame of the image to obtain the comprehensive sharpness score of the effective key detection areas. The clear image filtering module is used to select the image with the highest score from the preprocessed optical communication chip image sequence as the optimal clear image based on the comprehensive clarity score. The defect detection module is used to input the best clear image into the preset defect detection model for defect detection, and generate the defect detection results of the optical communication chip by combining the bounding box coordinates of the effective key detection area.

9. An electronic device, characterized in that, include: At least one processor; and memory that is communicatively connected to at least one processor; The memory stores instructions that can be executed by at least one processor, which are executed by at least one processor to enable the at least one processor to perform the optical communication chip defect detection method based on continuous imaging and local sharpness calculation as described in any one of claims 1-7.

10. A computer-readable storage medium storing computer-executable instructions thereon, characterized in that, When the computer-executable instructions are executed by the processor, they implement the optical communication chip defect detection method based on continuous imaging and local sharpness calculation as described in any one of claims 1-7.

Images