Semiconductor laser chip damage detection method and system based on mamba-yolo

Abstract

The application provides a semiconductor laser chip damage detection method and system based on Mamba-YOLO, which comprises the following steps: collecting semiconductor laser chip damage images to construct a training data set; constructing a damage detection model based on a Mamba neural network and a YOLOv8 model, improving the YOLOv8 model structure, more effectively fusing feature information of different scales, and enhancing the image feature capturing ability and target recognition ability of the model; and training the model by using the training data set to obtain the damage detection model. The damage detection model significantly improves the accuracy and reliability of semiconductor laser chip damage detection.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, and in particular to a method and system for detecting damage to semiconductor laser chips based on Mamba-YOLO. Background Technology

[0002] With the rapid development of fiber optic communication and semiconductor technology, semiconductor laser chips have become a core component of fiber optic communication systems. Due to their advantages such as wide wavelength range, superior transmission performance, high photoelectric conversion efficiency, and ease of modulation, laser chips are widely used in communications, industrial processing, medical diagnostics, damage detection, and military equipment. However, if a laser chip fails, it can severely impact the entire communication system. Therefore, accurately detecting defects in laser chips, analyzing their failure causes and damage mechanisms, optimizing manufacturing processes, and extending their lifespan have become crucial pathways to improving the reliability of semiconductor lasers.

[0003] Currently, traditional laser chip damage detection mainly relies on manual inspection, which is time-consuming, inefficient, and costly, failing to meet the high-efficiency requirements of industrial inspection. Traditional computer vision-based detection methods rely on predefined feature templates for pattern recognition, requiring manual feature selection and extraction. This process is cumbersome, has poor adaptability, and is easily affected by the environment, limiting the generalization ability of the detection system and making it difficult to achieve a balance between detection speed and performance.

[0004] Deep learning technology has been introduced into damage detection. Leveraging a general object detection framework, models can automatically learn and extract features, flexibly handling complex and ever-changing damage patterns. The YOLO (You Only Look Once) algorithm, as one of the mainstream object detection technologies, boasts advantages such as high computational speed, low computational complexity, and high detection accuracy. Compared to its predecessor YOLOv5, YOLOv8 further optimizes the structure and performance, for example, by introducing a C2f module and an improved loss function, significantly improving the regression accuracy of bounding box prediction. Although the YOLO algorithm performs excellently in object detection, its ability to learn damage features in the specific scenario of laser chip damage detection still has room for improvement due to the high diversity and complexity of damage features. Existing technologies struggle to balance detection accuracy and computational efficiency when handling complex damage patterns, especially in the case of small targets and complex scenes, where performance still requires further improvement. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a semiconductor laser chip damage detection method and system based on Mamba-YOLO, so as to eliminate or improve one or more defects existing in the prior art.

[0006] On one hand, the present invention provides a method for damage detection of semiconductor laser chips based on Mamba-YOLO, the method comprising the following steps:

[0007] Acquire an image of the semiconductor laser chip to be inspected;

[0008] The image of the semiconductor laser chip is input into the pre-trained damage detection model to generate detection results;

[0009] The training method for the damage detection model includes the following steps:

[0010] Acquire and preprocess semiconductor laser chip damage image data to construct a training dataset;

[0011] An initial damage detection model is constructed, which includes a feature extraction module, a path aggregation feature module, and a detection output module connected in sequence.

[0012] The initial damage detection model is trained using the training dataset. The training dataset is input into the initial damage detection model. In the feature extraction module, the SimpleStem module extracts low-level features, which are then input into the VSSBlock module for multi-scale feature extraction and fusion. The fused features are then input into the VisionClueMerge module for cross-layer feature fusion, generating cross-layer fused features, which are then input into the path aggregation feature module. In the path aggregation feature module, the XSSBlock module groups the cross-layer fused features, processes them on different paths, and then fuses them to generate multi-scale features, which are then input into the detection output module. In the detection output module, the XSSBlock module performs multi-scale prediction on the multi-scale features to generate detection results. The detection results include target category, bounding box position, and confidence score. Training stops when the number of iterations reaches a preset value or the performance meets a preset condition, resulting in the damage detection model.

[0013] In some embodiments of the present invention, acquiring and preprocessing semiconductor laser chip damage image data includes:

[0014] The semiconductor laser chip damage image data is preprocessed, including at least size normalization, image flipping, random cropping, and mirroring.

[0015] In some embodiments of the present invention, the damage detection model adopts the YOLOv8 framework, and the original Conv and C2f modules used in the feature extraction module are improved to the SimpleStem module, VSSBlock module, and VisionClueMerge module; in the feature extraction module, the method further includes:

[0016] The training dataset is input into the initial damage detection model to capture damage feature information and model the long-distance dependencies of damage images.

[0017] The SimpleStem module extracts low-level features from the semiconductor laser chip damage image data, calculated as follows:

[0018] F out1 =σ(BN(Conv(F) in ,W,b)));

[0019] Where σ represents the activation function; BN represents batch normalization; Conv represents the convolution operation; F in This represents the damage image data of the semiconductor laser chip; W represents the weight; b represents the bias.

[0020] The low-level features are input into the VSSBlock module for multi-scale feature extraction and fusion, calculated as follows:

[0021] Y k =Conv k (F out1 W k );

[0022] F out2 =∑ k α k Y k ,α k =softmax(φ(Y) k ));

[0023] Where k represents the kernel size; W k Represents the convolution kernel weights; α k Represents the dynamic fusion weights; φ represents the feature transformation function;

[0024] The fused features are input into the VisionClueMerge module for cross-layer feature fusion, generating cross-layer fused features. The calculation formula is as follows:

[0025]

[0026] Where, β i F represents the weights calculated by Softmax. out2 represents the features fused by the VSSBlock module; g represents the function for learning the importance of features.

[0027] In some embodiments of the present invention, the method further includes, in the feature extraction module:

[0028] Input the training dataset into the feature extraction module;

[0029] Reduce the height and width of the input features and increase the number of channels to reduce computational complexity and enhance the expressive power of damage features;

[0030] Set the number of channels for each stage, and the number of channels expands with each stage;

[0031] As the network layers become deeper, downsampling is used in each stage to reduce the feature map resolution and the number of patch blocks;

[0032] By continuously increasing the number of channels, deeper semantic information of the semiconductor laser chip damage image is gradually extracted, thereby capturing coarse and fine granular information of the damage image for modeling long-distance dependencies of the damage image.

[0033] In some embodiments of the present invention, the damage detection model adopts the YOLOv8 framework, and the original C2f module used in the path aggregation feature module is improved to the XSSBlock module; in the path aggregation feature module, the method further includes:

[0034] The cross-layer fusion features are input into the path aggregation feature module, and the cross-layer fusion features are divided into multiple groups according to the number of channels; transformation processing is performed on different paths to capture multi-level and multi-scale features; the transformation processing includes at least multi-scale convolution, attention enhancement and lightweight transformation processing;

[0035] In the multi-scale convolutional processing, multiple convolutional kernels are applied to each group of features to capture contextual information at different scales;

[0036] In the attention enhancement process, a lightweight attention mechanism is introduced to weight each set of features;

[0037] In the lightweight variation processing, depthwise separable convolutions are used to reduce computational complexity.

[0038] In some embodiments of the present invention, the method further includes, in the path aggregation feature module:

[0039] The output features of all paths are fused by weighted summation or feature concatenation.

[0040] When using the weighted summation method, the calculation formula is:

[0041]

[0042] Where Y represents the multi-scale feature finally generated by the path aggregation feature module; β i Y represents the weighting coefficient of the i-th path; iThis represents the output feature of the i-th path;

[0043] When using the feature-based concatenation method, the calculation formula is:

[0044] Y = Concat(Y1, Y2, ..., Y) G );

[0045] Among them, the multi-scale feature Y∈R C′×H×W C′ represents the new number of channels; H and W are the height and width of the image, respectively.

[0046] In some embodiments of the present invention, the method further includes, in the path aggregation feature module:

[0047] After fusing and generating the multi-scale features, the XSSBlock module introduces residual connections to directly add the cross-layer fused features to the multi-scale features, thereby stabilizing training and preserving the input features.

[0048] In some embodiments of the present invention, the damage detection model adopts the YOLOv8 framework, and the original C2f module used in the detection output module is improved to the XSSBlock module; in the detection output module, the method further includes:

[0049] For each feature scale, predict the target category, bounding box location, and confidence level;

[0050] The formulas for predicting the coordinates of the bounding box center point, the width, and the height of the bounding box are as follows:

[0051] x=(σ(t x )+c x )·s,y=(σ(t y )+c y )·s;

[0052] w = p w ·exp(t w ), h = p h ·exp(t h );

[0053] Among them, t x ,t y ,t w ,t h The network prediction value is σ; σ represents the activation function; (c x ,c y ) represents the coordinates of the top-left corner of the grid cell; s represents the grid cell step size; p w Indicates the anchor point width; p h Indicates the anchor point height;

[0054] The XSSBlock module optimizes the network predictions using an attention mechanism, calculated as follows:

[0055] t x ,t y =Attention(P) i ′)+t′ x ,t′ y ;

[0056] The prediction formulas for the target category and the confidence level using the joint attention mechanism are as follows:

[0057] p=σ(W p ·Concat(P i ′,GAP(P i ′))+b p );

[0058] Where σ represents the activation function; W p The shared weights represent the target category and the confidence level; GAP represents global average pooling; P i ′ represents the feature map after feature extraction and fusion; b p This represents the bias parameter.

[0059] In some embodiments of the present invention, the method further includes, during the training of the damage detection model:

[0060] Construct the complete intersection-union loss and distributed focus loss between the predicted bounding boxes and the ground truth bounding boxes;

[0061] The damage detection model is trained by weighting the complete intersection-union loss and the distribution focus loss to construct a joint loss and minimizing the joint loss.

[0062] On the other hand, the present invention also provides a semiconductor laser chip damage detection system based on Mamba-YOLO, which, when executed, implements the steps of any of the methods mentioned above. This invention provides a method and system for semiconductor laser chip damage detection based on Mamba-YOLO, comprising: collecting semiconductor laser chip damage images to construct a training dataset; constructing a damage detection model based on the Mamba neural network and the YOLOv8 model, integrating the long-range dependency capture capability and linear computational complexity of the Mamba neural network into the YOLOv8 architecture, and improving the modules of the YOLOv8 architecture. In the feature extraction module, modules such as SimpleStem, VSSBlock, and VisionClueMerge are combined to progressively extract and fuse multi-level and multi-scale features of the image, enabling the model to learn deeper and more comprehensive features of the image; in the path aggregation feature module, the feature connection method in the Concat module is improved, and the aggregation of cross-layer features is optimized, making the fusion of shallow and deep features more effective; the C2f module is improved to the XSSBlock module to better and faster combine features of different scales; in the detection output module, the XSSBlock module is integrated into each feature scale to further optimize the performance after feature fusion; finally, the damage detection model is trained. This damage detection model can effectively capture both coarse-grained and fine-grained features of semiconductor laser chip damage, significantly improving the accuracy and reliability of detection.

[0063] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0064] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0065] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:

[0066] Figure 1 This is a schematic diagram of the steps of a semiconductor laser chip damage detection method based on Mamba-YOLO in one embodiment of the present invention.

[0067] Figure 2 This is a schematic diagram of the overall structure of the damage detection model in one embodiment of the present invention.

[0068] Figure 3 This is a schematic diagram of the SimpleStem model in one embodiment of the present invention.

[0069] Figure 4 This is a schematic diagram of the VSSBlock module in one embodiment of the present invention.

[0070] Figure 5 This is a schematic diagram of the VisionClueMerge module in one embodiment of the present invention.

[0071] Figure 6 This is a schematic diagram of the detection results of the damage detection model in one embodiment of the present invention. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0073] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0074] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0075] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0076] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0077] It should be emphasized here that the step markers mentioned below are not a limitation on the order of the steps, but should be understood as meaning that the steps can be executed in the order mentioned in the embodiments, or in a different order than in the embodiments, or several steps can be executed simultaneously.

[0078] To address the problems of insufficient accuracy and low computational efficiency in existing laser chip damage detection technologies, this invention provides a semiconductor laser chip damage detection method based on Mamba-YOLO, such as... Figure 1 As shown, the method includes the following steps S101 to S102:

[0079] Step S101: Obtain an image of the semiconductor laser chip to be inspected.

[0080] Step S102: Input the semiconductor laser chip image into the pre-trained damage detection model to generate detection results.

[0081] In step S102, the training method for the damage detection model may further include the following steps S2011 to S2013:

[0082] Step S1021: Acquire semiconductor laser chip damage image data and preprocess it to construct a training dataset.

[0083] Step S1022: Construct an initial damage detection model, which includes a feature extraction module, a path aggregation feature module, and a detection output module connected in sequence.

[0084] Step S1023: Train the initial damage detection model using the training dataset; input the training dataset into the initial damage detection model. In the feature extraction module, the SimpleStem module extracts low-level features, and inputs the low-level features into the VSSBlock module for multi-scale feature extraction and fusion. The fused features are then input into the VisionClueMerge module for cross-layer feature fusion, generating cross-layer fused features, which are then input into the path aggregation feature module. In the path aggregation feature module, the XSSBlock module groups the cross-layer fused features, processes them on different paths, and then fuses them to generate multi-scale features, which are then input into the detection output module. In the detection output module, the XSSBlock module performs multi-scale prediction on the multi-scale features to generate detection results. The detection results include target category, bounding box position, and confidence score. When the number of iterations reaches a preset value or the performance meets a preset condition, training is stopped, and the damage detection model is obtained.

[0085] First, we will further explain the training method of the damage detection model.

[0086] In step S1021, a training dataset for model training is constructed.

[0087] Obtain available semiconductor laser chip damage image data, and for example, use digital image processing techniques to enhance and sharpen the images to improve image quality, and construct a dataset.

[0088] In some embodiments, the acquired semiconductor laser chip damage image data can be used not only as a training dataset, but also divided into a training sample set, a validation sample set, and a test sample set according to a preset ratio to validate and test the trained model. Preferably, the preset ratio is 6:2:2.

[0089] In some embodiments, the annotation information in the training dataset is transformed into YOLO format.

[0090] In some embodiments, preprocessing operations are performed on the training dataset, typically including size normalization, image flipping, random cropping, and mirroring.

[0091] Specifically, the original resolution of each image in the training dataset is adjusted to a uniform size, such as 640×640. Data augmentation is then performed on each sample after resizing, such as horizontally or vertically flipping the image to enhance the diversity of the dataset. Random cropping can be used to randomly select a sub-region from the original image, increasing sample diversity. Mirroring operations typically refer to horizontal flipping, but also include mirroring transformations of image color or brightness. For example, if the image pixel value range is [0, 255], mirroring can be represented by inverting the pixel values.

[0092] The preprocessing and enhancement operations described above can improve sample diversity, enabling the model to better adapt to different scenarios and feature variations. Furthermore, these methods can reduce the risk of overfitting, improve the model's generalization ability, and ultimately enhance its ability to detect targets in real-world scenarios.

[0093] In step S1022, a damage detection model is constructed. It should be noted that, to distinguish the order of training, the damage detection model before training is referred to as the initial damage detection model in this application, and its model structure is consistent with the final damage detection model.

[0094] like Figure 2 As shown, the damage detection model combines the characteristics of the Mamba network with the architecture of YOLOv8, forming a new fusion model, referred to as Mamba-YOLO in this application. Mamba-YOLO integrates the long-range dependency capture capability and linear computational complexity of the Mamba network into the YOLOv8 architecture, and further improves upon it. The original Conv and C2f modules in the YOLOv8 feature extraction module (Backbone) are improved to SimpleStem, VSSBlock, and VisionClueMerge modules; the feature connection method in the Concat module of the path aggregation feature module (PAFPN) is improved, and the original C2f module is improved to an XSSBlock module; the original C2f module in the detection output module is also improved to an XSSBlock module. Specifically, in... Figure 2 In the path aggregation feature module, U represents the upsampling operation and C represents the connection operation.

[0095] The following section provides further explanation of the functional and structural improvements of each module.

[0096] The training dataset is input into the damage detection model. It first enters the feature extraction module to capture coarse and fine granular information of semiconductor laser chip damage images, which is used to model the long-distance dependencies of semiconductor laser chip damage images.

[0097] For example, the size of the input semiconductor laser chip damage image is 640×640×3, i.e., the initial number of channels is 3 (RGB image). Reducing the height and width of the input features can lower computational complexity, while increasing the number of channels can enhance the expressive power of the damage features. The number of channels is set for each stage, expanding with each stage. Preferably, the number of channels for each stage is [C, 2C, 4C, 8C, 8C], where C is set to 128. As the network depth increases, downsampling is used in each stage to reduce the feature map resolution and the number of patches. Each stage of downsampling halves the feature map resolution, and the number of patches decreases with the square of the resolution. By continuously increasing the number of channels (e.g., expanding from 3 channels to 1024 channels), deeper semantic information of the semiconductor laser chip damage image is gradually extracted, thereby capturing coarse and fine-grained information of the damage image for modeling long-distance dependencies in the damage image.

[0098] The following section explains the specific structure of the model.

[0099] The training dataset is input into the damage detection model, which then proceeds to the feature extraction module. For example... Figure 3 The diagram shown is a structural diagram of the SimpleStem model. The SimpleStem model simplifies the initial convolution and quickly extracts low-level features from the input image. The calculation formula is shown in Equation (1):

[0100] F otu1 =σ(BN(Conv(F) in ,W,b))); (1)

[0101] Where σ represents the activation function; BN represents batch normalization; Conv represents the convolution operation; F in This represents the input semiconductor laser chip damage image data; W represents the weight; b represents the bias.

[0102] like Figure 4 The diagram shows the structure of the VSSBlock module. The low-level features extracted from the SimpleStem model are input into the VSSBlock module for multi-scale feature extraction and fusion to enhance feature representation capabilities. First, multi-scale features are extracted using convolutional kernels of different sizes, as shown in formula (2):

[0103] Y k =Conv k (F out1W k (2)

[0104] Where k represents the kernel size; Y k Represents the extracted multi-scale features; Conv represents the convolution operation; F out1 Represents low-level features; W k This represents the weights of the convolution kernel.

[0105] Then, dynamic fusion is performed, and the calculation formula is shown in formula (3):

[0106] F out2 =∑ k α k Y k ,α k =softmax(φ(Y) k (3)

[0107] Where, α k φ represents the dynamic fusion weights; φ represents the feature transformation function.

[0108] like Figure 5 The diagram shows the structure of the VisionClueMerge module. The features fused by the VSSBlock module are input into the VisionClueMerge module for cross-layer feature fusion. Features from different layers are integrated through an attention mechanism to generate cross-layer fused features, calculated as shown in formula (4).

[0109]

[0110] Where, β i F represents the weights calculated by Softmax. out2 This represents the features fused by the VSSBlock module; g represents the function that learns the importance of the features.

[0111] Through processing by the SimpleStem, VSSBlock, and VisionClueMerge modules, the feature extraction module finally outputs cross-layer fusion features, which are then input into the path aggregation feature module.

[0112] Compared to the Conv and C2f modules used in YOLOv8, the SimpleStem module optimizes the shallow feature extraction module, resulting in fewer parameters and higher efficiency. The VSSBlock module replaces the original CSP-like module, improving computational efficiency and feature extraction capabilities. The VisionClueMerge module is used for cross-layer feature fusion and enhancement, further improving feature integration. These replacement modules can more effectively fuse feature information at different scales, enhancing the model's ability to capture image features and recognize targets.

[0113] In the path aggregation feature module, this application improves the original C2f module to an XSSBlock module. The C2f module uses a feature extraction module with dense residual connections for partial cross-stage connections, combining shallow and deep features. The XSSBlock module divides the input features into different groups, performs transformation processing on different paths to capture multi-level and multi-scale features, and finally fuses the features processed by different paths.

[0114] In some embodiments, the cross-layer fusion feature input path aggregation feature module divides the cross-layer fusion features into multiple groups according to the number of channels, and the calculation formula is shown in formula (5):

[0115]

[0116] Among them, X i Let i represent the i-th feature group, where i ∈ G.

[0117] In some embodiments, the transformation process includes at least multi-scale convolution, attention enhancement, and lightweight transformation.

[0118] In multi-scale convolution processing, multiple convolution kernels are applied to each set of features to capture contextual information at different scales. For example, with convolution kernels of 1×1, 3×3, and 5×5, the calculation formula is shown in formula (6):

[0119] Y i,k =Conv k (X i ),k∈{1,3,5}; (6)

[0120] In the attention enhancement process, a lightweight attention mechanism is introduced to weight each set of features, and the calculation formula is shown in Equation (7):

[0121] Attention Weight: α i =σ(W·GAP(X) i )+b);

[0122] Y i ′=α i ·X i (7)

[0123] Where, α i σ represents the weights; σ represents the activation function; W and b are trainable parameters; GAP represents global average pooling.

[0124] In lightweight transformation processing, depthwise separable convolution is used to reduce computational complexity, as shown in formula (8):

[0125] Yi ′=σ(W d *X i +b d (8)

[0126] Where σ represents the activation function; W d Indicates a depthwise convolution kernel; * indicates a depthwise convolution operation; b d These are trainable parameters.

[0127] In some embodiments, the output features of all paths are fused by weighted summation or feature concatenation.

[0128] If a weighted summation method is used, the calculation formula is shown in formula (9):

[0129]

[0130] Where Y represents the multi-scale feature finally generated by the path aggregation feature module; β i The weighting coefficients for the i-th path can be learned through training; Y i This represents the output feature of the i-th path.

[0131] If feature splicing is used, the calculation formula is as shown in formula (10):

[0132] Y = Concat(Y1, Y2, ..., Y) G (10)

[0133] Among them, the multi-scale feature Y∈R C′×H×W C′ represents the new number of channels; H and W are the height and width of the image, respectively.

[0134] In some embodiments, after fusing and generating multi-scale features, the XSSBlock module introduces residual connections to directly add cross-layer fused features to the multi-scale features in order to stabilize training and preserve input features.

[0135] Compared to YOLOv8's C2f module, the XSSBlock module's multi-branch design enhances diversity, its fusion attention mechanism improves feature selection capabilities, and its lightweight operations reduce computational costs. Through these improvements, the XSSBlock module can enhance feature extraction capabilities and model performance while reducing computational complexity.

[0136] The original YOLOv8 detection head generates multi-scale features (P3 / 8, P4 / 16, P5 / 32) based on feature maps of different resolutions, namely P3, P4, and P5. P3, P4, and P5 correspond to feature maps of different scales, capable of capturing targets of different sizes. The P3 feature map has high resolution and is suitable for detecting small targets; the P4 feature map has moderate resolution and is suitable for detecting medium-sized targets; and the P5 feature map has low resolution and is suitable for detecting large targets. The class, position offset, and confidence level are predicted for each feature scale. The output tensor shape is shown in formula (11).

[0137] Output = (B, N) anchors ,H,W,Z+5); (11)

[0138] Where B represents the batch size; N anchors The anchor number for each pixel is usually 1; H and W represent the height and width of the feature map, respectively; Z is the number of categories, and 5 represents the position offset (4) and confidence (1).

[0139] In this application, the detection output module has been optimized in terms of feature processing and multi-scale prediction. The XSSBlock module is introduced to replace the C2f module in the original YOLOv8. By introducing multi-scale convolution to capture information from different receptive fields, and by assigning higher weights to important features through an attention mechanism, the detection output is improved by using smaller convolutional kernels or grouped convolutions and by improving the bounding box prediction formula.

[0140] The original YOLOv8 center point coordinates and width / height prediction method are shown in formulas (12) to (13):

[0141] x=(σ(t x )+c x )·s;y=(σ(t y )+c y )·s; (12)

[0142] w = p w ·exp(t w h = p h ·exp(t h (13)

[0143] Where (x,y) represents the coordinates of the center point; w,h represent the width and height of the bounding box; t x ,t y ,t w ,t h The network prediction value is σ; σ represents the activation function; (c x ,c y ) represents the coordinates of the top-left corner of the grid cell; s represents the grid cell step size; p wIndicates the anchor point width; p h Indicates the anchor point height.

[0144] In this application, the XSSBlock module optimizes the network prediction values ​​through an attention mechanism, thereby enhancing the accuracy of offset prediction. The calculation formula is shown in Equation (14):

[0145] t x ,t y =Attention(P′) i )+t′ x ,t′ y (14)

[0146] The prediction formula for target category and confidence level through the joint attention mechanism is shown in Equation (15):

[0147] p=σ(W p ·Concat(P′ i ,GAP(P′ i ))+b p (15)

[0148] Where σ represents the activation function; W p Represents the shared weights of the target category and confidence level; GAP represents global average pooling; P i ′ represents the feature map after feature extraction and fusion; b p This represents the bias parameter.

[0149] In the detection output module, the introduction of the XSSBlock module enhances feature representation, especially for small targets and complex scenes; the attention mechanism and weighted fusion optimize the combination of features at different levels; lightweight operations such as grouped convolution reduce computational complexity; and enhanced bounding box prediction improves the accuracy of positional offset and width / height prediction. These improvements enable the Mamba-YOLO detection head to achieve a better balance between efficiency and accuracy.

[0150] In step S1023, the initial damage detection model is trained using a training dataset. The training dataset is input into the initial damage detection model in batches and processed sequentially by the feature extraction module, the path aggregation feature module, and the detection output module. Finally, the detection result is output. When the number of iterations reaches a preset value or the performance meets a preset condition, the training stops and the damage detection model is obtained.

[0151] In some embodiments, a complete intersection-union loss (Complete-IoU Loss) and a distribution focal loss are constructed between the predicted bounding box and the ground truth bounding box. The complete intersection-union loss and the distribution focal loss are weighted to construct a joint loss, and the damage detection model is trained with the goal of minimizing the joint loss.

[0152] In some embodiments, the training dataset is fed into the initial damage detection model for training, the batch size is set to 4, the image input size is set to 640×640, and the training process is stopped when the number of iterations reaches a set value (e.g., 500) or when the model accuracy does not improve after 30 consecutive iterations.

[0153] In some embodiments, the validation sample set mentioned in the embodiment of step S1021 is input into the trained damage detection model (Mamba-YOLO fusion model), and the detection performance of the validation sample set is calculated using common target detection metrics such as mean accuracy (mAP). The model parameter information when the detection performance on the validation sample set is optimal is saved, and the model is adjusted.

[0154] The training of the damage detection model is now complete. In practical applications, simply input the image of the semiconductor laser chip to be detected into the damage detection model trained above to generate detection results, which include the category of the target damage, the location of the bounding box, and the confidence level.

[0155] In some embodiments, the predicted target bounding box detection results of the semiconductor laser chip damage image can be saved in JPG format.

[0156] The following section provides further explanation of this application and its technical effects based on specific experimental results.

[0157] The experimental environment consisted of an Ubuntu 22.04 operating system, a 2.5GHz processor, a single NVIDIA GeForce RTX 3060 graphics card, and a 12th Gen Intel(R) Core(TM) i9-12900H (CPU). The modeling tools used included Visual Studio Code, Python 3.11, PyTorch 2.1.2, and CUDA 12.1.

[0158] As shown in Table 1 and Figure 6 As shown, the damage detection model (Mamba-YOLO) provided in this application has significantly improved detection accuracy compared with other target detection models when using the same dataset.

[0159] Table 1

[0160]

[0161]

[0162] Corresponding to the above method, the present invention also provides a semiconductor laser chip damage detection system based on Mamba-YOLO, which, when executed, implements the steps of the semiconductor laser chip damage detection method based on Mamba-YOLO.

[0163] Corresponding to the above method, the present invention also provides an electronic device including a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the electronic device performs the steps of the method as described above.

[0164] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0165] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0166] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0167] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0168] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting damage to semiconductor laser chips based on Mamba-YOLO, characterized in that, The method includes the following steps: Acquire an image of the semiconductor laser chip to be inspected; The image of the semiconductor laser chip is input into the pre-trained damage detection model to generate detection results; The training method for the damage detection model includes the following steps: Acquire and preprocess semiconductor laser chip damage image data to construct a training dataset; An initial damage detection model is constructed, which includes a feature extraction module, a path aggregation feature module, and a detection output module connected in sequence. The initial damage detection model is trained using the training dataset. The training dataset is input into the initial damage detection model. In the feature extraction module, the SimpleStem module extracts low-level features, which are then input into the VSSBlock module for multi-scale feature extraction and fusion. The fused features are then input into the VisionClueMerge module for cross-layer feature fusion, generating cross-layer fused features, which are then input into the path aggregation feature module. In the path aggregation feature module, the XSSBlock module groups the cross-layer fused features, processes them on different paths, and then fuses them to generate multi-scale features, which are then input into the detection output module. In the detection output module, the XSSBlock module performs multi-scale prediction on the multi-scale features to generate detection results. The detection results include target category, bounding box position, and confidence score. Training stops when the number of iterations reaches a preset value or the performance meets a preset condition, resulting in the damage detection model. The damage detection model employs the YOLOv8 framework, and the original Conv and C2f modules used in the feature extraction module are improved to SimpleStem, VSSBlock, and VisionClueMerge modules; the method in the feature extraction module further includes: The training dataset is input into the initial damage detection model to capture damage feature information and model the long-distance dependencies of damage images. The SimpleStem module extracts low-level features from the semiconductor laser chip damage image data, calculated as follows: ； in, Indicates the activation function; Indicates batch normalization; Indicates the convolution operation; This represents the damage image data of the semiconductor laser chip; Indicates weight; Indicates bias; The low-level features are input into the VSSBlock module for multi-scale feature extraction and fusion, calculated as follows: ； ； in, Indicates the kernel size; Indicates the convolution kernel weights; Indicates dynamic fusion weights; Represents the characteristic transformation function; The fused features are input into the VisionClueMerge module for cross-layer feature fusion, generating cross-layer fused features. The calculation formula is as follows: ； in, This represents the weights calculated using Softmax; This represents the features fused by the VSSBlock module; A function representing the importance of learned features; The damage detection model uses the YOLOv8 framework, and the original C2f module in the path aggregation feature module is improved to the XSSBlock module; in the path aggregation feature module, the method further includes: The cross-layer fusion features are input into the path aggregation feature module, and the cross-layer fusion features are divided into multiple groups according to the number of channels; transformation processing is performed on different paths to capture multi-level and multi-scale features; the transformation processing includes at least multi-scale convolution, attention enhancement and lightweight transformation processing; In the multi-scale convolutional processing, multiple convolutional kernels are applied to each group of features to capture contextual information at different scales; In the attention enhancement process, a lightweight attention mechanism is introduced to weight each set of features; In the lightweight variation processing, depthwise separable convolution is used to reduce computational complexity; The damage detection model adopts the YOLOv8 framework, and the original C2f module used in the detection output module is improved to the XSSBlock module; in the detection output module, the method further includes: For each feature scale, predict the target category, bounding box location, and confidence level; The formulas for predicting the coordinates of the bounding box center point, the width, and the height of the bounding box are as follows: ， ； ， ； in, This is the network's predicted value; Indicates the activation function; Indicates the coordinates of the top-left corner of the grid cell; Indicates the grid cell step size; Indicates the width of the anchor point; Indicates the anchor point height; The XSSBlock module optimizes the network predictions using an attention mechanism, calculated as follows: ； The prediction formulas for the target category and the confidence level using the joint attention mechanism are as follows: ； in, Indicates the activation function; This represents the shared weight between the target category and the confidence level; Indicates global average pooling; This represents the feature map after feature extraction and fusion. This represents the bias parameter.

2. The semiconductor laser chip damage detection method based on Mamba-YOLO according to claim 1, characterized in that, Acquire and preprocess semiconductor laser chip damage image data, including: The semiconductor laser chip damage image data is preprocessed, including at least size normalization, image flipping, random cropping, and mirroring.

3. The semiconductor laser chip damage detection method based on Mamba-YOLO according to claim 1, characterized in that, In the feature extraction module, the method further includes: Input the training dataset into the feature extraction module; Reduce the height and width of the input features and increase the number of channels to reduce computational complexity and enhance the expressive power of damage features; Set the number of channels for each stage, and the number of channels expands with each stage; As the network layers become deeper, downsampling is used in each stage to reduce the feature map resolution and the number of patch blocks; By continuously increasing the number of channels, deeper semantic information of the semiconductor laser chip damage image is gradually extracted, thereby capturing coarse and fine granular information of the damage image for modeling long-distance dependencies of the damage image.

4. The semiconductor laser chip damage detection method based on Mamba-YOLO according to claim 1, characterized in that, In the path aggregation feature module, the method further includes: The output features of all paths are fused by weighted summation or feature concatenation. When using the weighted summation method, the calculation formula is: ； in, This represents the multi-scale features ultimately generated by the path aggregation feature module; Indicates the first The weighting coefficients for each path; Indicates the first Output characteristics of each path; When using the feature-based concatenation method, the calculation formula is: ； Among them, multi-scale features , Indicates the new number of channels; These represent the height and width of the image, respectively.

5. The semiconductor laser chip damage detection method based on Mamba-YOLO according to claim 1, characterized in that, In the path aggregation feature module, the method further includes: After fusing and generating the multi-scale features, the XSSBlock module introduces residual connections to directly add the cross-layer fused features to the multi-scale features, thereby stabilizing training and preserving the input features.

6. The semiconductor laser chip damage detection method based on Mamba-YOLO according to claim 1, characterized in that, The method further includes the following steps when training the damage detection model: Construct the complete intersection-union loss and distributed focus loss between the predicted bounding boxes and the ground truth bounding boxes; The damage detection model is trained by weighting the complete intersection-union loss and the distribution focus loss to construct a joint loss and minimizing the joint loss.

7. A semiconductor laser chip damage detection system based on Mamba-YOLO, characterized in that, When the system is executed, it implements the steps of the method as described in any one of claims 1 to 6.

Images