Image deblurring method, system, device and medium for optical communication chip based on convolutional neural network

Abstract

The application relates to an image deblurring method, system, device and medium based on a convolutional neural network optical communication chip, which comprises the following steps: performing optical feature pre-extraction on an input optical communication chip image based on an improved pyramid network, performing parallel feature fusion through a multi-size convolution kernel, and obtaining a receptive field expansion feature map; performing multi-scale feature extraction on the receptive field expansion feature map according to an encoder, introducing a dual-domain attention mechanism to perform feature enhancement on the extracted feature vector, and obtaining a multi-scale feature image; adopting a jump connection improved by a hollow convolution group to perform resolution-adaptive receptive field matching fusion on different level feature maps of the multi-scale feature image and a decoder; inputting the fused features into the decoder for progressive upsampling processing, synchronously adopting a multi-scale composite loss function to optimize multi-level features output by the decoder, and reconstructing a clear optical communication chip image. The application improves the efficiency and precision of optical communication chip image deblurring processing.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image deblurring method, system, device, and medium based on an optical communication chip using a convolutional neural network. Background Technology

[0002] In the field of optical communication chips, image quality is crucial for performance evaluation and defect detection. However, due to the complex structure and precision manufacturing process of optical communication chips, their images often suffer from blurring. This blurring can be caused by a variety of factors, including imperfections in the optical imaging system, minute defects on the chip surface, vibration of the chip stage or jitter of the laser scanning mirror leading to image blurring, defocusing of the optical system or defocusing effect caused by height differences in the microstructures on the chip surface, and the difficulty in distinguishing submicron features such as electrode stripes in blurred images.

[0003] Blurred images not only reduce the accuracy of chip detection but also increase the difficulty of subsequent processing. Image deblurring technology for optical communication chips faces numerous challenges. First, there is a wide variety of chips with complex and diverse appearances, and different chips have significantly different optical characteristics. Second, defects on the chip surface are small in size, with only a slight contrast difference between them and the background area, making it difficult to accurately identify and separate defects in blurred images. Furthermore, the instability of chip manufacturing processes leads to a wide variety of defects, and the same defect may exhibit multiple different feature forms, further increasing the difficulty of deblurring and defect detection.

[0004] Traditional image deblurring methods primarily rely on manually adjusted parameters and preset image processing algorithms. These methods often struggle to adapt to complex blur patterns and diverse chip backgrounds when processing images of optical communication chips. Because chip images exhibit a variety of blur types (such as motion blur, defocus blur, and loss of microstructural features), traditional methods cannot effectively distinguish between different types of blur, resulting in unsatisfactory deblurring effects. Furthermore, traditional methods have high computational complexity when processing high-resolution images, making them difficult to meet real-time requirements.

[0005] Furthermore, the existing image acquisition process involves significant manual intervention, relying on the operator's experience and subjective judgment regarding image blurriness. Due to the complexity of optical communication chip images, manually adjusting parameters and re-acquiring images is not only time-consuming and labor-intensive, but also makes it difficult to guarantee the consistency of deblurring results. Especially for the detection of minute defects, manual operation makes it difficult to accurately quantify the degree of blurriness, and if the image is unclear, it can easily lead to missed detections and false detections. Summary of the Invention

[0006] (a) Technical problems to be solved

[0007] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides an image deblurring method, system, device and medium for optical communication chips based on convolutional neural networks, which solves the technical problem of low deblurring quality in complex scenes in existing optical communication chip image deblurring.

[0008] (II) Technical Solution

[0009] To achieve the above objectives, the main technical solutions adopted by the present invention include:

[0010] In a first aspect, embodiments of the present invention provide an image deblurring method for optical communication chips based on convolutional neural networks, comprising:

[0011] Based on a pre-constructed multi-scale optical distortion compensation pyramid network, optical features are pre-extracted from the input optical communication chip image. Through parallel feature fusion of multi-size convolutional kernels, a receptive field extension feature map with cross-scale light intensity distribution characteristics is obtained.

[0012] Based on the encoder in the convolutional neural network, multi-scale feature extraction is performed on the receptive field expansion feature map, and a cascaded channel-first dual-domain attention mechanism is introduced to enhance the extracted feature vectors, thereby obtaining multi-scale feature images of the optical communication chip.

[0013] An improved skip connection using dilated convolutional groups with multi-level adaptive dilation rate is adopted to fuse feature maps of different levels of multi-scale feature images with the decoder through resolution-adaptive receptive field matching.

[0014] The fused features are input to the decoder and progressively upsampled. Simultaneously, a multi-scale composite loss function that fuses pixel-level, feature-level, and frequency-domain reconstructions is used to optimize the multi-level features output by the decoder, thereby reconstructing a clear image of the optical communication chip.

[0015] Optionally, before performing optical feature pre-extraction on the input optical communication chip image based on a pre-constructed multi-scale optical distortion compensation pyramid network, and obtaining a receptive field extension feature map with cross-scale light intensity distribution characteristics through parallel feature fusion of multi-size convolutional kernels, the method further includes:

[0016] The acquired blurred optical communication chip image data is preprocessed with image adjustment and feature enhancement to obtain an image training set;

[0017] A multi-scale optical distortion compensation pyramid network is set up in the input layer of the original convolutional neural network, and the generated improved convolutional neural network is subjected to lightweight processing such as model pruning, quantization compression, and knowledge distillation.

[0018] The lightweight improved convolutional neural network is trained based on the image training set, and the training results are evaluated in multiple dimensions through a set multi-scale composite loss function. The optimal convolutional neural network is output based on the evaluation results.

[0019] Optionally, the acquired blurred optical communication chip image data is preprocessed with image adjustment and feature enhancement to obtain an image training set including:

[0020] The acquired optical communication chip image data is blurred.

[0021] Image adjustment is performed on the blurred optical communication chip image data by performing multi-size transformation and normalization to obtain normalized image data output according to a set size set;

[0022] By performing noise suppression, region of interest cropping, and data augmentation on normalized image data, a training set of images with enhanced features is obtained.

[0023] Optionally, the training results can be evaluated in multiple dimensions using a set multi-scale composite loss function, and the optimal convolutional neural network can be output based on the evaluation results, including:

[0024] The pixel difference between the predicted image output by the convolutional neural network and the actual image is calculated using mean square error, and multi-scale pixel difference measurement is achieved by combining multi-scale content loss function.

[0025] The Manhattan distance between the predicted image and the actual image is calculated in the same frequency domain using a multi-scale frequency reconstruction loss function.

[0026] A pre-trained convolutional neural network is used to extract feature vectors at the same location as the predicted image and the actual image. The similarity between the extracted feature vectors is then compared using a perceptual loss function to obtain the feature similarity between the predicted image and the actual image.

[0027] The frequency domain constraints of the convolutional neural network are obtained by weighting the frequency components above a set frequency threshold in the spectrograms of the predicted and actual images using the weighted Fourier frequency domain loss function.

[0028] Based on multi-scale pixel difference values, multi-scale Manhattan distance, feature similarity, and frequency domain constraints, the training results are evaluated in multiple dimensions by combining the weight ratio of multi-dimensional loss assessment, and the optimal convolutional neural network is output based on the evaluation results.

[0029] Optionally, optical features are pre-extracted from the input optical communication chip image based on a pre-constructed multi-scale optical distortion compensation pyramid network. Through parallel feature fusion using multi-size convolutional kernels, a receptive field extension feature map with cross-scale light intensity distribution characteristics is obtained, including:

[0030] By using multiple convolutional branches of different scales set in parallel, optical feature response maps with different receptive fields are extracted from the optical communication chip image. The optical feature response maps include local defect features caused by edge scattering of optical structures, non-uniform transmission, or differences in material properties.

[0031] An upsampling operation is performed on each optical feature response map to upsample it to the original spatial resolution of the optical communication chip image, thereby obtaining feature maps corresponding to different original receptive fields;

[0032] By using the parallel features of multi-size convolutional kernels, the feature maps of different original receptive fields after unifying the resolution are cascaded and fused in the channel dimension to obtain an extended receptive field feature map with cross-scale light intensity distribution characteristics.

[0033] Optionally, based on the encoder in the convolutional neural network, multi-scale feature extraction is performed on the receptive field expansion feature map, and a cascaded channel-first dual-domain attention mechanism is introduced to enhance the extracted feature vectors, thereby obtaining a multi-scale feature image of the optical communication chip, including:

[0034] The encoder is optimized by using a multi-scale composite loss function that integrates pixel-level, feature-level, and frequency-domain reconstructions. Multi-scale feature extraction is performed on the receptive field extended feature map to obtain the initial multi-scale feature image of the optical communication chip.

[0035] Based on the cross-scale light intensity distribution characteristics of the receptive field extended feature map, the contribution weight of each feature channel is dynamically adjusted by combining the channel domain attention module in the dual-domain attention mechanism, so as to adaptively compensate for the feature expression deviation caused by uneven illumination in the initial multi-scale feature image.

[0036] Based on the waveguide structure information of the optical communication chip, the spatial domain attention module in the dual-domain attention mechanism is used to enhance the pixel features located in the waveguide edge region and the adjacent weak texture region in the channel-enhanced multi-scale feature image, thereby obtaining a multi-scale feature image with enhanced waveguide structure recognition in the spatial dimension.

[0037] Among them, the channel domain attention module dynamically adjusts the saliency response intensity of each feature channel through channel weight mapping guided by light intensity gradient; the spatial domain attention module generates a spatially sensitive attention mask based on the edge gradient distribution characteristics of the distorted region, enhancing the sub-pixel level detail features of the blurred boundary.

[0038] Optionally, skip connections improved by using dilated convolutional groups with multi-level adaptive dilation rates are employed to fuse feature maps from different levels of multi-scale feature images with the decoder through resolution-adaptive receptive field matching, including:

[0039] In the skip connection path between the decoder and encoder, a multi-level dilated convolutional group with adaptive dilation rate is introduced to improve the skip connection mechanism;

[0040] By using an improved skip connection mechanism, feature vectors from different levels in the multi-scale feature image extracted by the encoder are fused with the decoder through resolution-adaptive receptive field matching, thereby obtaining multi-scale fused features that can simultaneously capture local details and global contextual information.

[0041] Secondly, embodiments of the present invention provide an image deblurring system based on a convolutional neural network for an optical communication chip, comprising:

[0042] The optical feature pre-extraction module is used to pre-extract optical features from the input optical communication chip image based on a pre-constructed multi-scale optical distortion compensation pyramid network. Through parallel feature fusion of multi-size convolutional kernels, a receptive field extension feature map with cross-scale light intensity distribution characteristics is obtained.

[0043] The encoding module is used to extract multi-scale features from the receptive field extended feature map based on the encoder in the convolutional neural network, and to introduce a cascaded channel-first dual-domain attention mechanism to enhance the extracted feature vectors, thereby obtaining multi-scale feature images of the optical communication chip.

[0044] The adaptive receptive field matching module is used to improve the skip connections by using dilated convolution groups with multi-level adaptive dilation rates to fuse the receptive field matching of different levels of feature maps of multi-scale feature images with the decoder in a resolution-adaptive manner.

[0045] The decoding module is used to progressively upsample the fused features input to the decoder and simultaneously optimize the multi-level features output by the decoder using a multi-scale composite loss function that fuses pixel-level, feature-level, and frequency-domain reconstructions, thereby reconstructing a clear image of the optical communication chip.

[0046] Thirdly, embodiments of the present invention provide an electronic device, comprising:

[0047] processor;

[0048] The memory stores the steps of the image deblurring method for the optical communication chip based on the convolutional neural network described above, which are used by the processor to control the chip.

[0049] Fourthly, embodiments of the present invention provide a computer-readable medium storing computer-executable instructions, characterized in that the executable instructions, when executed by a processor, implement the steps of the image deblurring method for optical communication chips based on convolutional neural networks described above.

[0050] (III) Beneficial Effects

[0051] The beneficial effects of this invention are as follows: This invention employs a convolutional neural network to deblur images of optical communication chips, which can automatically extract high-level semantic features from the image without the need for manually designed feature extractors, thus greatly improving the efficiency and accuracy of image deblurring. Furthermore, this invention also pre-extracts optical features from the input optical communication chip image through a pre-constructed multi-scale optical distortion compensation pyramid network to expand the receptive field of cross-scale light intensity distribution. Additionally, this invention introduces a cascaded channel-first dual-domain attention mechanism in the encoder part of the convolutional neural network to filter the extracted features, improving the recognition of sub-pixel level detail features in optical communication chip images. Finally, this invention uses multi-level dilated convolutional groups with adaptive dilation rates during skip connections to enhance multi-resolution detail feature processing, which can further improve the efficiency and accuracy of deblurring. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating an image deblurring method for an optical communication chip based on a convolutional neural network, provided in an embodiment of the present invention.

[0053] Figure 2 This is a schematic diagram of the structure of a convolutional neural network provided in an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of the structure of an improved pyramid provided in an embodiment of the present invention;

[0055] Figure 4 This is a schematic diagram of the structure of a dual-domain attention mechanism provided in an embodiment of the present invention;

[0056] Figure 5 This is a schematic diagram of the structure of a channel domain attention module provided in an embodiment of the present invention;

[0057] Figure 6 This is a schematic diagram of the spatial domain attention module provided in an embodiment of the present invention;

[0058] Figure 7 This is a schematic diagram of the structure of a dilated convolutional group provided in an embodiment of the present invention;

[0059] Figures 8 to 13 The images provided in this embodiment of the invention show the visual effects of six sets of optical communication chip images before and after deblurring. Detailed Implementation

[0060] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0061] Prior to this, in order to facilitate understanding of the technical solutions provided by this invention, some basic information related to the technical solutions of this application will be introduced below.

[0062] PSNR (Peak Signal-to-Noise Ratio): PSNR is calculated based on the mean square error (MSE). The higher the PSNR value, the better the image quality, that is, the smaller the difference between images.

[0063] SSIM (Structural Similarity): SSIM takes into account that the human eye is more sensitive to the structural information of an image. It evaluates the similarity of images from three aspects: brightness, contrast and structure. The closer the SSIM value is to 1, the more similar the images are.

[0064] refer to Figure 1-7 As shown in the embodiment of the present invention, an image deblurring method for optical communication chips based on convolutional neural networks is proposed. The method includes: First, pre-extracting optical features from the input optical communication chip image based on a pre-constructed multi-scale optical distortion compensation pyramid network, and obtaining a receptive field extension feature map with cross-scale light intensity distribution characteristics through parallel feature fusion of multi-size convolutional kernels; Next, extracting multi-scale features from the receptive field extension feature map based on the encoder in the convolutional neural network, and introducing a cascaded channel-first dual-domain attention mechanism to enhance the extracted feature vectors, thereby obtaining a multi-scale feature image of the optical communication chip; Then, using a skip connection improved by a multi-level adaptive dilation rate dilated convolutional group, fusing different levels of feature maps of the multi-scale feature image with the decoder through resolution-adaptive receptive field matching; Finally, inputting the fused features into the decoder for progressive upsampling processing, and simultaneously using a multi-scale composite loss function that fuses pixel-level, feature-level, and frequency-domain reconstructions to optimize the multi-level features output by the decoder, thereby reconstructing a clear image of the optical communication chip.

[0065] This embodiment employs a convolutional neural network (CNN) to deblur the image of an optical communication chip. It automatically extracts high-level semantic features from the image, eliminating the need for manually designed feature extractors and significantly improving the efficiency and accuracy of image deblurring. Furthermore, this embodiment uses a pre-constructed multi-scale optical distortion compensation pyramid network to pre-extract optical features from the input optical communication chip image, expanding the receptive field for cross-scale light intensity distribution. Additionally, this embodiment introduces a cascaded channel-first dual-domain attention mechanism in the encoder part of the CNN to filter the extracted features, improving the recognition of sub-pixel level detail features in the optical communication chip image. Finally, this embodiment uses multi-level dilated convolutional groups with adaptive dilation rates during skip connections to enhance multi-resolution detail feature processing, further improving the efficiency and accuracy of deblurring.

[0066] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0067] Specifically, refer to Figure 1 As shown, this embodiment proposes a method based on Figure 2 The method shown, which uses a convolutional neural network to deblur images of optical communication chips, may include:

[0068] S100: Based on a pre-constructed multi-scale optical distortion compensation pyramid network, optical features are pre-extracted from the input optical communication chip image. Through parallel feature fusion of multi-size convolutional kernels, a receptive field extension feature map with cross-scale light intensity distribution characteristics is obtained.

[0069] In this embodiment, the following is adopted: Figure 3 The multi-scale optical distortion compensation pyramid network shown performs optical feature pre-extraction on the input optical communication chip image: the input is an H*W*3 RGB image, and after fusing convolutional kernels with different receptive fields (1*1, 3*3, 5*5, 7*7) to obtain feature maps, it is upsampled and fused, and finally the features of different scales are concatenated along the channel dimension to output an H*W*7 receptive field extended feature map. This embodiment uses a pre-constructed multi-scale optical distortion compensation pyramid network to pre-extract optical features from the input optical communication chip image, thereby expanding the receptive field across scale light intensity distributions and further improving the efficiency of feature vector extraction by the encoder in the subsequent convolutional neural network.

[0070] In this embodiment, before step S100, method steps G100 to G300 may also be executed:

[0071] G100 performs image adjustment and feature enhancement preprocessing on the acquired blurred optical communication chip image data to obtain an image training set.

[0072] Furthermore, addressing the issues of varying degrees of blur in different images, limited pairs of blurred and sharp data, and diverse noise interference, this embodiment employs a unique image preprocessing method. This method includes a series of image adjustment and feature enhancement preprocessing operations such as scale transformation, normalization, noise suppression, ROI (Region of Interest) cropping, and data augmentation. This aims to improve image quality, preserve the integrity of key information as much as possible, and amplify the amount of defective data. Specifically, step G100 may include the following sub-steps G110 to G130:

[0073] G110: Blur the acquired optical communication chip image data.

[0074] G120 performs multi-size transformation and normalization processing on the blurred optical communication chip image data to obtain normalized image data output according to a set size set.

[0075] Size Transformation: The aspect ratio of images from optical communication chips is 5:1. During deep learning training, the length and width of the images are scaled to the same proportional size. To accommodate this scaling change, this embodiment expands and scales the original data. Different chip images also have aspect ratios of 3:1 and 4:1, so these variations must be considered when transforming the original image. For example, a 1500*300 image, after expansion, will have image sizes of 1200*300, 900*300, 300*900, 512*512, 256*256, and 1024*1024, etc.

[0076] Normalization: There are many types of optical communication chips. During the chip manufacturing process, due to the instability of chip technology, the thickness and surface roughness of the chips vary, resulting in changes in color and brightness in the image. Even chips from the same batch will have certain differences in image quality. Therefore, this implementation uses Z-score normalization technology to preprocess the image to stabilize the training process and improve the adaptability of the deblurring algorithm model to different optical communication chip images. Specifically, the Z-score normalization method includes: first, performing grayscale transformation on the image, and then converting the data points of the grayscale image into a standard normal distribution with a mean of 0 and a standard deviation of 1. The calculation formula (1) is as follows:

[0077]

[0078] In equation (1), X is the original data point, μ is the mean of the grayscale image, and σ1 is the standard deviation of the dataset.

[0079] G130 performs noise suppression, region of interest cropping, and data augmentation on normalized image data to obtain an image training set with enhanced features.

[0080] Noise Suppression: In actual industrial inspection scenarios, the optical communication images acquired by the camera contain Gaussian noise or Poisson noise to a certain extent. Depending on the degree of noise, a slight Gaussian filter (σ2 = 0.5~1.0) is applied to reduce noise interference and avoid excessive smoothing, so as not to lose high-frequency details during the noise suppression process. The specific implementation method is: to perform a convolution operation on the image using a Gaussian function (normal distribution function) to achieve smoothing of the image. The core of Gaussian filtering is to use the weights of the Gaussian function to perform a weighted average of the pixels in the image, so that the noise in the image is smoothed out, while preserving the edge information of the image as much as possible. The calculation formula (2) is as follows:

[0081]

[0082] In equation (2), x and y are the coordinates of the pixel in the image, σ² is the standard deviation of the Gaussian function, which determines the width and smoothness of the Gaussian kernel, and exp is the natural exponential function.

[0083] Region of Interest (ROI) cropping: Since the detection area of ​​optical communication chips is usually concentrated in a specific location, different regions of the chip are first located using Canny / Sobel edge detection or template matching. Irrelevant other areas are removed, focusing on the ROI. After cropping, the image is input into the deep learning network, reducing computational load, improving training efficiency, and enhancing model robustness. The cropped image can be further resized to a fixed size of 256*256 or 512*512 to adapt to the input requirements of different application scenarios.

[0084] Data augmentation: Enhancement methods for data quality include image contrast adjustment, horizontal and vertical edge enhancement overlay, boosting high-frequency information, and reducing low-frequency information. Enhancement methods for data quantity include horizontal flipping, vertical flipping, translation, rotation (not 90 degrees), saturation adjustment, local non-linear deformation, and random partial area filling.

[0085] G200 sets up a multi-scale optical distortion compensation pyramid network in the input layer of the original convolutional neural network, and performs lightweight processing such as model pruning, quantization compression, and knowledge distillation on the generated improved convolutional neural network.

[0086] Model pruning includes channel pruning and structured pruning. Channel pruning: Based on the importance assessment of convolutional channels at different gradients, redundant convolutional channels are removed; Structured pruning: Key channels of multi-scale Inception modules are preferentially retained to avoid disrupting multi-scale feature fusion.

[0087] Quantization compression: Convert FP32 (32-bit floating-point) weights to INT8 (8-bit integers) to reduce memory usage and increase model inference speed; use Dibit quantization, i.e. QAT (quantization-aware training), to reduce accuracy loss.

[0088] Knowledge distillation: Using UNet (convolutional neural network) as a teacher model to guide the model to learn more compact feature representations.

[0089] The G300 trains the lightweight, improved convolutional neural network using an image training set, and evaluates the training results in multiple dimensions using a set multi-scale composite loss function, outputting the optimal convolutional neural network based on the evaluation results.

[0090] Furthermore, step G300 may include the following sub-steps G310 to G150.

[0091] G310 uses mean squared error to calculate the pixel difference between the predicted image output by the convolutional neural network and the actual image, and combines it with a multi-scale content loss function to achieve multi-scale pixel difference measurement.

[0092] Mean squared error is used to measure the pixel difference between the predicted image and the real clear image. It can effectively optimize the overall similarity of the image. The calculation formula (3) is as follows:

[0093]

[0094] In equation (3), y i For the true value, Unpredicted value, where n is the sample size.

[0095] The multi-scale content loss function can better capture the details and structural information of the image by calculating the difference between the predicted image and the real image at multiple scales. The calculation formula (4) is as follows:

[0096]

[0097] In equation (4), and Let λ represent the features of the real image and the predicted image at the k-th scale, respectively. k These are the weighting coefficients.

[0098] G320 calculates the multi-scale Manhattan distance between the predicted image and the actual image in the same frequency domain using a multi-scale frequency reconstruction loss function.

[0099] The multi-scale frequency reconstruction loss function effectively improves the model's ability to reconstruct high-frequency information and reduces the difference in frequency space by measuring the L1 (Manhattan distance) between the multi-scale real image and the deblurred image in the frequency domain. The calculation formula (5) is as follows:

[0100]

[0101] In equation (5), F is the Fourier transform.

[0102] G330 uses a pre-trained convolutional neural network to extract feature vectors at the same positions in the predicted image and the actual image, and uses a perceptual loss function to compare the similarity of the extracted feature vectors to obtain the feature similarity between the predicted image and the actual image.

[0103] The perceptual loss function optimizes the model by comparing the similarity between the predicted image and the real image in the feature space. In this embodiment, a pre-trained convolutional neural network is used to extract features, which can better reflect the perceptual characteristics of the human visual system. The calculation formula (6) is as follows:

[0104]

[0105] In equation (6), VGG is a pre-trained convolutional neural network.

[0106] G340: The frequency components above a set frequency threshold in the spectrograms of the predicted and actual images are weighted using the weighted Fourier frequency domain loss function to obtain the frequency domain constraints of the convolutional neural network.

[0107] The weighted Fourier frequency domain loss function processes the spectrum after Fourier transform in a weighted manner, making the pre-trained model pay more attention to high-frequency components. By designing a weight function based on the distance to the center point, the model's ability to learn high-frequency information can be enhanced. The calculation formula (7) is as follows:

[0108]

[0109] In equation (7), W i For frequency-based weights.

[0110] G350 performs multi-dimensional evaluation of the training results based on multi-scale pixel difference values, multi-scale Manhattan distance, feature similarity, and frequency domain constraints, combined with the weight ratio of multi-dimensional loss evaluation, and outputs the optimal convolutional neural network based on the evaluation results.

[0111] By combining mean squared error, multi-scale content loss, multi-scale frequency reconstruction loss, perceptual loss, and weighted Fourier frequency domain loss to optimize the overall similarity and detail information of the image, the generalization ability of the pre-trained model can be further improved. The total loss function (8) is as follows:

[0112] L total =a1L MSE +a2L MSC +a3LMSFR +a4L PL +a5L WFL (8)

[0113] In equation (8), a1, a2, a3, a4, and a5 represent the weighting coefficients for different losses.

[0114] In this embodiment, step S100 may include the following sub-steps S110 to S130:

[0115] S110. Using multiple convolutional branches of different scales set in parallel, optical feature response maps with different receptive fields are extracted from the optical communication chip image. The optical feature response maps include local defect features caused by edge scattering of optical structures, non-uniform transmission, or differences in material properties.

[0116] S120. Perform an upsampling operation on each optical feature response map, upsampling to the original spatial resolution of the optical communication chip image to obtain feature maps corresponding to different original receptive fields.

[0117] S130 uses the parallel features of multi-size convolutional kernels to cascade and fuse feature maps of different original receptive fields at a unified resolution along the channel dimension, thereby obtaining an extended receptive field feature map with cross-scale light intensity distribution characteristics.

[0118] S200: Based on the encoder in the convolutional neural network, multi-scale feature extraction is performed on the receptive field expansion feature map, and a cascaded channel-first dual-domain attention mechanism is introduced to enhance the extracted feature vectors, thereby obtaining multi-scale feature images of the optical communication chip.

[0119] In this embodiment, reference Figure 4 The cascaded channel-priority dual-domain attention mechanism structure diagram shown below includes step S200, which may include the following sub-steps S210 to S230:

[0120] S210: The encoder is optimized by using a multi-scale composite loss function that integrates pixel-level, feature-level, and frequency-domain reconstruction. Multi-scale feature extraction is performed on the receptive field extended feature map to obtain the initial multi-scale feature image of the optical communication chip.

[0121] S220. Based on the cross-scale light intensity distribution characteristics of the receptive field extended feature map, the contribution weight of each feature channel is dynamically adjusted in combination with the channel domain attention module in the dual-domain attention mechanism to adaptively compensate for the feature expression deviation caused by uneven illumination in the initial multi-scale feature image.

[0122] Further, refer to Figure 5The channel domain attention module shown in the diagram dynamically adjusts the saliency response intensity of each feature channel through channel weight mapping guided by light intensity gradient, and then feeds the channel weighted feature vector into the spatial domain attention module after fusing it with the original feature vector under the channel attention mechanism.

[0123] S230. Based on the waveguide structure information of the optical communication chip, the spatial domain attention module in the dual-domain attention mechanism is used to enhance the pixel features located in the waveguide edge region and the adjacent weak texture region in the channel-enhanced multi-scale feature image, thereby obtaining a multi-scale feature image with enhanced spatial recognition of the waveguide structure.

[0124] Further, refer to Figure 6 The spatial domain attention module shown in the diagram generates a spatially sensitive attention mask based on the edge gradient distribution characteristics of the distorted region, enhancing sub-pixel level details of the blurred boundaries. First, spatial weights are assigned to the channel-fused features to generate spatially weighted features. Then, the spatially weighted features are fused again with the original input features to output the final features.

[0125] S300 employs a skip connection improved by using dilated convolutional groups with multi-level adaptive dilation rates to fuse feature maps of different levels of multi-scale feature images with the decoder through resolution-adaptive receptive field matching.

[0126] In this embodiment, by using skip connections, the detailed features extracted by the encoder can be directly passed to the decoder, thereby compensating for information loss that may occur during upsampling. Simultaneously, without increasing the kernel size or the number of parameters, this embodiment expands the receptive field by adjusting the dilation rate, improving the processing efficiency for multi-scale features such as the microstructure, edges, and textures of the optical communication chip. This allows the convolutional branches to simultaneously capture local details and global contextual information. This embodiment uses a multi-level adaptive dilation rate dilated convolution group to improve the skip connections. The dilation rate of the dilated convolution is adaptively adjusted according to the feature distribution (such as target scale and texture complexity) of the optical communication chip feature image to obtain the optimal expanded receptive field. This prevents the receptive field from being too small to capture long-distance dependencies, and prevents it from being too large, resulting in sparse sampling points and loss of local information. The dilated convolution group references... Figure 7 As shown, Figure (a) represents a normal 3*3 convolution with a receptive field of 3*3, Figure (b) represents a dilated convolution with an inflation rate of 2, where the receptive field of each red dot is 3*3 and the total receptive field is 7*7, and Figure (c) represents a dilated convolution with an inflation rate of 4, where the receptive field of each red dot is 5*5 and the total receptive field is 15*15.

[0127] In this embodiment, step S300 may include the following sub-steps S310 to S320:

[0128] S310. In the skip connection path between the decoder and encoder, a multi-level dilated convolutional group with adaptive dilation rate is introduced to improve the skip connection mechanism.

[0129] S320: Through an improved skip connection mechanism, feature vectors at different levels in the multi-scale feature image extracted by the encoder are fused with the decoder through resolution-adaptive receptive field matching to obtain multi-scale fused features that can simultaneously capture local details and global contextual information.

[0130] S400: The fused features are input to the decoder and progressively upsampled. Simultaneously, a multi-scale composite loss function that fuses pixel-level, feature-level, and frequency-domain reconstructions is used to optimize the multi-level features output by the decoder, thereby reconstructing a clear image of the optical communication chip.

[0131] This embodiment proposes an image reconstruction framework for optical communication chips based on multi-scale feature fusion and composite loss optimization. Through a progressive upsampling structure in the decoder, combined with multi-modal monitoring signals at the pixel, feature, and frequency domains, high-fidelity reconstruction of the chip's micro / nano structure is achieved. This method effectively solves the detail blurring problem of traditional super-resolution techniques in high-noise, low-contrast industrial scenarios.

[0132] In one specific embodiment, the optical communication chip image deblurring method described in steps S100 to S400 is used to... Figures 8 to 13 Six sets of optical communication chip images were deblurred. The performance of the deblurring results in terms of PSNR and SSIM was analyzed, and the visual effects were combined to evaluate the deblurring results. Figures 8 to 11 In the image, the left side is the original image, and the right side is the deblurred image. Figures 12 to 13 In the image, the original image is shown at the top, and the image after blurring is shown at the bottom.

[0133] By comparing the visual effects of six sets of optical communication chip images before and after processing, a significant difference can be clearly seen. In the unprocessed images, the details of the optical communication chip are blurred, and the overall contrast of the chip's edges and internal structure is low, making it impossible to clearly present the chip's key features and defects. Furthermore, the images contain considerable noise, interfering with the observation of chip details and making it difficult to distinguish the chip's minute structures and features. In contrast, the images processed using the method described in this embodiment show a significant improvement. The processed images have richer layers and significantly improved contrast, allowing for clear distinction between different parts of the chip. The chip's edge contours become sharper, and internal structures and minute features (such as circuit lines and solder joints) are clearly visible, resulting in richer detail. Simultaneously, noise is effectively suppressed, achieving a good balance between high-frequency information (such as subtle textures and structures) and noise, significantly improving the overall image quality. In addition, the processed images exhibit more natural and accurate color reproduction, more realistically reflecting the actual appearance of the optical communication chip. This improvement in image quality not only aids in the visual inspection and quality assessment of the chip but also provides a more reliable foundation for subsequent image analysis, fault detection, and automated inspection. Overall, the method in this embodiment performs excellently in improving the image quality of optical communication chips, and can provide a more reliable foundation for the visual inspection, quality assessment and automated testing of chips.

[0134] Image quality metrics (PSNR and SSIM) for six groups of optical communication chips were calculated, and the specific metrics data shown in Table 1 were obtained. The convolutional neural network model achieved excellent performance of PSNR>28dB and SSIM>0.9 on most samples. When PSNR>28dB, it indicates excellent deblurring effect, and when SSIM>0.9, it indicates excellent structural similarity and complete preservation of details.

[0135] Table 1. PSNR and SSIM metrics on different datasets

[0136] Figure 8 28.00 0.9056 Figure 9 32.94 0.9081 Figure 10 31.48 0.9424 Figure 11 27.21 0.8945 Figure 12 29.18 0.9150 Figure 13 29.35 0.9145

[0137] Furthermore, this embodiment also proposes an image deblurring system for an optical communication chip based on a convolutional neural network, comprising:

[0138] The optical feature pre-extraction module is used to pre-extract optical features from the input optical communication chip image based on a pre-constructed multi-scale optical distortion compensation pyramid network. Through parallel feature fusion of multi-size convolutional kernels, a receptive field extension feature map with cross-scale light intensity distribution characteristics is obtained.

[0139] The encoding module is used to extract multi-scale features from the receptive field extended feature map based on the encoder in the convolutional neural network, and to introduce a cascaded channel-first dual-domain attention mechanism to enhance the extracted feature vectors, thereby obtaining multi-scale feature images of the optical communication chip.

[0140] The adaptive receptive field matching module is used to improve the skip connections by using dilated convolution groups with multi-level adaptive dilation rates to fuse the receptive field matching of different levels of feature maps of multi-scale feature images with the decoder in a resolution-adaptive manner.

[0141] The decoding module is used to progressively upsample the fused features input to the decoder and simultaneously optimize the multi-level features output by the decoder using a multi-scale composite loss function that fuses pixel-level, feature-level, and frequency-domain reconstructions, thereby reconstructing a clear image of the optical communication chip.

[0142] Furthermore, this embodiment also proposes an electronic device, including: a processor; and a memory storing steps S100 to S400 of the image deblurring method based on a convolutional neural network optical communication chip described above for the processor to control.

[0143] Finally, this embodiment also proposes a computer-readable medium storing computer-executable instructions, which, when executed by a processor, implement steps S100 to S400 of the image deblurring method for optical communication chips based on convolutional neural networks described above.

[0144] In summary, this embodiment proposes an image deblurring method, system, device, and medium for optical communication chips based on convolutional neural networks. Through efficient deblurring, it significantly improves image quality, providing a clearer and more accurate image foundation for subsequent defect detection and classification. First, considering the characteristics of optical communication chips—variable types, inconsistent aspect ratios, diverse imaging color and brightness, and uneven defect distribution—a unique image adjustment and feature enhancement preprocessing method is designed to mitigate the impact of data issues on model accuracy and robustness. Next, before entering the encoder of the convolutional neural network, an improved multi-scale optical distortion compensation pyramid network is used to perform preliminary feature extraction on the input image, expanding the receptive field. Then, a dual-domain attention mechanism is introduced in the encoder part of the convolutional neural network to filter the extracted features, enhancing important features and transferring unimportant features. Simultaneously, multi-level adaptive dilation rate dilated convolutions are used during skip connections to enhance multi-resolution detail feature processing. Finally, progressive upsampling processing is performed through the decoder to reconstruct a clear image of the optical communication chip. Furthermore, this embodiment proposes a combination of five loss mechanisms: mean squared error, multi-scale content loss, multi-scale frequency reconstruction loss, perceptual loss, and weighted Fourier frequency domain loss, thereby enhancing the model's generalization ability. Therefore, this embodiment can effectively distinguish between different types of blur in optical communication chips, adapt to complex chip image backgrounds, and improve the efficiency and accuracy of deblurring, providing an efficient and reliable solution for the quality inspection of optical communication chips.

[0145] Since the systems / devices described in the above embodiments of the present invention are systems / devices used to implement the methods of the above embodiments of the present invention, those skilled in the art can understand the specific structure and modifications of the systems / devices based on the methods described in the above embodiments of the present invention, and therefore will not be repeated here. All systems / devices used in the methods of the above embodiments of the present invention fall within the scope of protection of the present invention.

[0146] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0147] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0148] It should be noted that in the description of this invention, the word "a" or "an" preceding a component does not exclude the existence of multiple such components. This invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. The use of terms such as first, second, third, etc., is merely for convenience and does not indicate any order. These terms can be understood as part of the component names.

[0149] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0150] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning of the basic inventive concept, can make other changes and modifications to these embodiments.

[0151] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from the spirit and scope of the invention.

Claims

1. An image deblurring method for optical communication chips based on convolutional neural networks, characterized in that, include: Based on a pre-constructed multi-scale optical distortion compensation pyramid network, optical features are pre-extracted from the input optical communication chip image. Through parallel feature fusion of multi-size convolutional kernels, a receptive field extension feature map with cross-scale light intensity distribution characteristics is obtained. Based on the encoder in the convolutional neural network, multi-scale feature extraction is performed on the receptive field expansion feature map, and a cascaded channel-first dual-domain attention mechanism is introduced to enhance the extracted feature vectors, thereby obtaining multi-scale feature images of the optical communication chip. An improved skip connection using dilated convolutional groups with multi-level adaptive dilation rate is adopted to fuse feature maps of different levels of multi-scale feature images with the decoder through resolution-adaptive receptive field matching. The fused features are input to the decoder and progressively upsampled. Simultaneously, a multi-scale composite loss function that fuses pixel-level, feature-level, and frequency-domain reconstructions is used to optimize the multi-level features output by the decoder, thereby reconstructing a clear image of the optical communication chip.

2. The method as described in claim 1, characterized in that, Before performing optical feature pre-extraction on the input optical communication chip image based on a pre-constructed multi-scale optical distortion compensation pyramid network, and obtaining a receptive field extension feature map with cross-scale light intensity distribution characteristics through parallel feature fusion of multi-size convolutional kernels, the following steps are also included: The acquired blurred optical communication chip image data is preprocessed with image adjustment and feature enhancement to obtain an image training set; A multi-scale optical distortion compensation pyramid network is set up in the input layer of the original convolutional neural network, and the generated improved convolutional neural network is subjected to lightweight processing such as model pruning, quantization compression, and knowledge distillation. The lightweight improved convolutional neural network is trained based on the image training set, and the training results are evaluated in multiple dimensions through a set multi-scale composite loss function. The optimal convolutional neural network is output based on the evaluation results.

3. The method as described in claim 2, characterized in that, The acquired blurred optical communication chip image data is preprocessed with image adjustment and feature enhancement to obtain an image training set including: The acquired optical communication chip image data is blurred. Image adjustment is performed on the blurred optical communication chip image data by performing multi-size transformation and normalization to obtain normalized image data output according to a set size set; By performing noise suppression, region of interest cropping, and data augmentation on normalized image data, a training set of images with enhanced features is obtained.

4. The method as described in claim 2, characterized in that, The training results are evaluated from multiple dimensions using a defined multi-scale composite loss function. Based on the evaluation results, the optimal convolutional neural network is output, including: The pixel difference between the predicted image output by the convolutional neural network and the actual image is calculated using mean square error, and multi-scale pixel difference measurement is achieved by combining multi-scale content loss function. The Manhattan distance between the predicted image and the actual image is calculated in the same frequency domain using a multi-scale frequency reconstruction loss function. A pre-trained convolutional neural network is used to extract feature vectors at the same location as the predicted image and the actual image. The similarity between the extracted feature vectors is then compared using a perceptual loss function to obtain the feature similarity between the predicted image and the actual image. The frequency domain constraints of the convolutional neural network are obtained by weighting the frequency components above a set frequency threshold in the spectrograms of the predicted and actual images using the weighted Fourier frequency domain loss function. Based on multi-scale pixel difference values, multi-scale Manhattan distance, feature similarity, and frequency domain constraints, the training results are evaluated in multiple dimensions by combining the weight ratio of multi-dimensional loss assessment, and the optimal convolutional neural network is output based on the evaluation results.

5. The method as described in claim 1, characterized in that, Based on a pre-constructed multi-scale optical distortion compensation pyramid network, optical features are pre-extracted from the input optical communication chip image. Parallel feature fusion using multi-size convolutional kernels yields a receptive field extension feature map with cross-scale light intensity distribution characteristics, including: By using multiple convolutional branches of different scales set in parallel, optical feature response maps with different receptive fields are extracted from the optical communication chip image. The optical feature response maps include local defect features caused by edge scattering of optical structures, non-uniform transmission, or differences in material properties. An upsampling operation is performed on each optical feature response map to upsample it to the original spatial resolution of the optical communication chip image, thereby obtaining feature maps corresponding to different original receptive fields; By using the parallel features of multi-size convolutional kernels, the feature maps of different original receptive fields after unifying the resolution are cascaded and fused in the channel dimension to obtain an extended receptive field feature map with cross-scale light intensity distribution characteristics.

6. The method as described in claim 1, characterized in that, Based on the encoder in a convolutional neural network, multi-scale feature extraction is performed on the receptive field expansion feature map. A cascaded channel-first dual-domain attention mechanism is introduced to enhance the extracted feature vectors, resulting in multi-scale feature images of the optical communication chip, including: The encoder is optimized by using a multi-scale composite loss function that integrates pixel-level, feature-level, and frequency-domain reconstructions. Multi-scale feature extraction is performed on the receptive field extended feature map to obtain the initial multi-scale feature image of the optical communication chip. Based on the cross-scale light intensity distribution characteristics of the receptive field extended feature map, the contribution weight of each feature channel is dynamically adjusted by combining the channel domain attention module in the dual-domain attention mechanism, so as to adaptively compensate for the feature expression deviation caused by uneven illumination in the initial multi-scale feature image. Based on the waveguide structure information of the optical communication chip, the spatial domain attention module in the dual-domain attention mechanism is used to enhance the pixel features located in the waveguide edge region and the adjacent weak texture region in the channel-enhanced multi-scale feature image, thereby obtaining a multi-scale feature image with enhanced waveguide structure recognition in the spatial dimension. Among them, the channel domain attention module dynamically adjusts the saliency response intensity of each feature channel through channel weight mapping guided by light intensity gradient; the spatial domain attention module generates a spatially sensitive attention mask based on the edge gradient distribution characteristics of the distorted region, enhancing the sub-pixel level detail features of the blurred boundary.

7. The method as described in claim 1, characterized in that, An improved skip connection using dilated convolutional groups with multi-level adaptive dilation rates is employed to fuse feature maps from different levels of multi-scale feature images with the decoder through resolution-adaptive receptive field matching. In the skip connection path between the decoder and encoder, a multi-level dilated convolutional group with adaptive dilation rate is introduced to improve the skip connection mechanism; By using an improved skip connection mechanism, feature vectors from different levels in the multi-scale feature image extracted by the encoder are fused with the decoder through resolution-adaptive receptive field matching, thereby obtaining multi-scale fused features that can simultaneously capture local details and global contextual information.

8. An image deblurring system based on a convolutional neural network for an optical communication chip, characterized in that, include: The optical feature pre-extraction module is used to pre-extract optical features from the input optical communication chip image based on a pre-constructed multi-scale optical distortion compensation pyramid network. Through parallel feature fusion of multi-size convolutional kernels, a receptive field extension feature map with cross-scale light intensity distribution characteristics is obtained. The encoding module is used to extract multi-scale features from the receptive field extended feature map based on the encoder in the convolutional neural network, and to introduce a cascaded channel-first dual-domain attention mechanism to enhance the extracted feature vectors, thereby obtaining multi-scale feature images of the optical communication chip. The adaptive receptive field matching module is used to improve the skip connections by using dilated convolution groups with multi-level adaptive dilation rates to fuse the receptive field matching of different levels of feature maps of multi-scale feature images with the decoder in a resolution-adaptive manner. The decoding module is used to progressively upsample the fused features input to the decoder and simultaneously optimize the multi-level features output by the decoder using a multi-scale composite loss function that fuses pixel-level, feature-level, and frequency-domain reconstructions, thereby reconstructing a clear image of the optical communication chip.

9. An electronic device, characterized in that, include: processor; A memory that stores steps of an image deblurring method for a convolutional neural network-based optical communication chip as described in any one of claims 1-7.

10. A computer-readable medium having computer-executable instructions stored thereon, characterized in that, When the executable instructions are executed by the processor, they implement the steps of the image deblurring method for optical communication chips based on convolutional neural networks as described in any one of claims 1-7.

Images