Image content recognition and classification method based on deep learning

Abstract

The application discloses a deep learning-based image content recognition and classification method, and relates to the technical field of computer vision and artificial intelligence. The method comprises the following steps: collecting PCB images and preprocessing; expanding a data set and labeling by using an improved generative adversarial network; constructing an improved deep learning recognition and classification model, including using an improved ResNet-50 network with embedded deep separable convolution layers and attention mechanisms for feature extraction, fusing multi-scale features through a feature weight distribution algorithm, using a lightweight Transformer to mine global features, and combining an improved SVM and a category calibration algorithm for classification calibration; training the model by using a hybrid loss function; inputting the preprocessed image to be detected into the model to output a defect recognition result; visualizing the output and feeding back to an industrial control system, and continuously optimizing the model through incremental training. The application can accurately identify micro defects, has strong generalization ability, takes a short time to identify a single image, is suitable for industrial real-time detection scenarios, and reduces detection cost.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, specifically to a method for image content recognition and classification based on deep learning. Background Technology

[0002] In the industrial manufacturing sector, printed circuit boards (PCBs) are core components of electronic devices, and their quality directly determines the reliability and lifespan of these devices. Therefore, PCB defect detection is a crucial aspect of industrial quality control. Currently, image recognition technology based on deep learning has been gradually applied to PCB defect detection, but existing technologies still have significant shortcomings and cannot meet the high-precision, high-efficiency, and highly generalized detection requirements of industrial production. These shortcomings are as follows:

[0003] 1. Insufficient accuracy and generalization ability in defect identification: Existing deep learning-based PCB defect detection methods mostly use traditional convolutional neural networks (such as CNN and VGG) as the core of feature extraction and classification. These networks have limited ability to extract features of small defects on the PCB surface (such as line width deviation, small short circuits, and pinholes), and have poor adaptability to changes in lighting, image noise, and PCB batch differences, which easily leads to missed detections and false detections. At the same time, existing methods rely on a large number of labeled samples for model training, but PCB defect samples are scarce and labeling costs are high in industrial scenarios, which causes the generalization ability of the model to drop sharply in small sample scenarios, making it difficult to adapt to the inspection needs of PCBs with different specifications and processes.

[0004] 2. Low identification and classification efficiency, making it difficult to adapt to the real-time inspection needs of industry: Existing methods mostly adopt a serial architecture of "feature extraction-classification and recognition" without targeted optimization of the network structure. The model parameters are redundant and the computational load is large, resulting in a long time for identification and classification of a single PCB image (usually more than 100ms), which cannot meet the real-time inspection requirements of 3-5 images per second in industrial production lines. In addition, existing methods do not have a targeted defect category calibration mechanism, and the classification accuracy of similar defects (such as short circuits and copper scraps, open circuits and line width deviations) is low, requiring secondary manual verification, which further reduces the inspection efficiency. Summary of the Invention

[0005] The technical problem to be solved by this invention is to overcome the defects of the above-mentioned technologies and provide an image content recognition and classification method based on deep learning. This method solves the technical problems of low defect recognition accuracy, poor generalization ability, insufficient detection efficiency, and poor small sample adaptability of existing methods, and achieves the goal of rapid and accurate identification and classification of PCB defects, adapting to industrial real-time detection scenarios.

[0006] To address the aforementioned technical problems, the present invention provides a deep learning-based image content recognition and classification method, comprising the following steps:

[0007] S1. Image Acquisition and Preprocessing: PCB surface images are acquired using an industrial camera to construct a PCB image dataset. The dataset includes normal PCB images and defective PCB images. Defect types include short circuits, open circuits, line width deviations, pinholes, and stray copper. The acquired PCB images are preprocessed to obtain standardized images.

[0008] S2: Dataset augmentation and annotation: The standardized images obtained in step S1 are augmented using an improved generative adversarial network (GAN) to compensate for the scarcity of defect samples; the augmented dataset is annotated, including the defect location and defect type, and an annotation file is generated; the augmented dataset is divided into training set, validation set, and test set in a ratio of 7:2:1.

[0009] S3: Constructing an improved deep learning recognition and classification model: The model includes a feature extraction module, a feature fusion module, a defect recognition module, and a classification calibration module. The feature extraction module uses an improved ResNet-50 network, embedding depthwise separable convolutional layers in the residual blocks of the ResNet-50 network, and adding an attention mechanism module at the network input. The feature fusion module employs a multi-scale feature fusion strategy, fusing features from different levels, including shallow texture features, mid-level contour features, and deep semantic features, through a feature weight allocation algorithm. The defect recognition module uses a lightweight Transformer architecture to mine global features from the fused features and identify defect regions. The classification calibration module combines an improved support vector machine (SVM) and a category calibration algorithm to classify defect types and correct classification results.

[0010] S4: Model Training and Optimization: Input the training set divided in step S2 into the improved deep learning recognition and classification model, use a hybrid loss function to train the model, adjust the model parameters through an adaptive learning rate optimization algorithm, use the validation set to validate the model, iterate and optimize until the model converges, and obtain the trained recognition and classification model.

[0011] S5: Image recognition and classification: After the PCB image to be detected is preprocessed in step S1, it is input into the trained recognition and classification model. The model outputs the defect recognition result, including no defect / defective, defect area coordinates, defect type and classification confidence.

[0012] S6: Result Output and Feedback: Visualize the identification and classification results of step S5 and feed them back to the industrial control system. Mark defective PCBs for subsequent processing. Regularly collect detection data and incrementally train the identification and classification model to continuously improve the model's generalization ability.

[0013] Further, in step S1, the preprocessing step includes grayscale conversion, noise removal, size normalization, and contrast enhancement, wherein noise removal employs an adaptive median filtering algorithm, and the calculation formula is as follows:

[0014]

[0015] in, For the filtered image in The pixel value of the location, For the original image in The pixel value of the location, For the original image in The median of pixels within the location's neighborhood. It is the minimum value of the pixels in the neighborhood. It represents the maximum value of the pixels in the neighborhood.

[0016] Furthermore, in step S2, the generator of the improved generative adversarial network (GAN) adopts a U-Net architecture, the discriminator adopts a lightweight CNN architecture, and the loss function of the generator is calculated as follows:

[0017]

[0018] in, For generator loss, noise vector The distribution Let be the probability that the discriminator classifies the generated image. This is the weighting coefficient, with a value ranging from 0.8 to 1.2. for loss, The image generated by the generator, This is a real defect image.

[0019] Further, in step S3, the calculation formula for the feature weight allocation algorithm is as follows:

[0020]

[0021]

[0022] in, For the first Weights of layer features =1,2,3, corresponding to shallow texture features, mid-level contour features, and deep semantic features, respectively. For the first Significance score of layer features For the first Layer feature map, This is the fused feature map.

[0023] Further, in step S4, the hybrid loss function includes the cross-entropy loss function and The loss function is calculated using the following formula:

[0024]

[0025] in, For mixed loss, This is the weighting coefficient, with a value ranging from 0.6 to 0.8. Let cross-entropy be the loss function. for Loss function;

[0026] The formula for calculating the loss function is:

[0027]

[0028] Where A represents the actual defect area and B represents the defect area predicted by the model;

[0029] The formula for calculating the cross-entropy loss function is:

[0030]

[0031] in, For the sample size, For the first The true label of each sample For the model to the first The predicted probability of a sample.

[0032] Further, in step S3, the calculation formula for the category calibration algorithm is as follows:

[0033]

[0034] in, For the calibrated first Classification probability of class defects The classification probability before calibration. For the first The calibration coefficient for this type of defect is adaptively adjusted based on historical detection data. =1,2,3,4,5, corresponding to short circuit, open circuit, line width deviation, pinhole, and stray copper, respectively.

[0035] Furthermore, in step S1, size normalization uniformly adjusts the PCB image to 512×512 pixels, and contrast enhancement adopts an adaptive histogram equalization algorithm.

[0036] Furthermore, in step S4, the adaptive learning rate optimization algorithm uses the Adam optimizer, with an initial learning rate set to 0.001, and the learning rate decays to 0.5 every 50 iterations.

[0037] Furthermore, in step S5, the threshold for the classification confidence level is set to 0.85. When the classification confidence level is lower than 0.85, it is marked as a suspected defect, prompting manual review.

[0038] Furthermore, in step S6, the incremental training period is 7 days, and the sample size of each incremental training is not less than 5% of the existing training set sample size. During the training process, the original model parameters remain unchanged, and only the parameters of the classification calibration module and the feature fusion module are updated.

[0039] The advantages of this invention compared to the prior art are:

[0040] 1. High recognition accuracy and strong generalization ability: This invention enhances the extraction capability of minute defect features by embedding deep separable convolutional layers and attention mechanism modules into the ResNet-50 network, enabling accurate identification of minute defects such as linewidth deviations and pinholes on PCB surfaces. Simultaneously, an improved GAN is used to expand the dataset, effectively solving the problem of scarce defect samples in industrial scenarios and improving the model's generalization ability in small sample scenarios, adapting to the inspection needs of PCBs with different specifications and processes. Furthermore, through a multi-scale feature fusion strategy, shallow texture features, mid-level contour features, and deep semantic features are integrated, improving the representation ability of defect features, reducing false negative and false positive rates, and increasing the accuracy of defect recognition, especially the accuracy of minute defect recognition.

[0041] 2. High detection efficiency, adaptable to real-time industrial inspection needs: This invention adopts a lightweight network architecture, embedding depthwise separable convolutional layers on the ResNet-50, reducing model parameters and computational load. The time for recognition and classification of a single PCB image is shortened to less than 50ms, meeting the real-time inspection requirements of 3-5 images per second in industrial production lines. At the same time, a classification calibration module is designed to correct the classification results through a category calibration algorithm, reducing the misclassification rate of similar defects, reducing the workload of manual secondary verification, and further improving detection efficiency.

[0042] 3. High adaptability and practicality: The preprocessing step of this invention adopts an adaptive median filtering algorithm, which can effectively remove image noise and adapt to PCB images acquired under different lighting conditions, thus improving the model's environmental adaptability. The model training adopts a hybrid loss function, which takes into account both the localization accuracy and classification accuracy of defect identification, further optimizing the model performance. In addition, through the incremental training mechanism, the detection data is collected regularly to optimize the model, which can continuously improve the model's generalization ability and adaptability, and meet the needs of process upgrades and new defect types in industrial production.

[0043] 4. Low cost and easy industrial deployment: This invention expands the dataset through an improved GAN, reducing the workload of manual sample annotation and lowering the sample annotation cost; the lightweight network architecture reduces the requirements for hardware devices, eliminating the need for high-performance GPU clusters, and can be deployed on embedded devices in industrial settings. The deployment cost is low, making it easy to promote and apply. It can effectively replace traditional manual and semi-automatic inspection methods, reduce the labor cost of industrial quality inspection, and improve production efficiency. Attached Figure Description

[0044] Figure 1 This is an overall flowchart of the deep learning-based image content recognition and classification method of the present invention.

[0045] Figure 2 This is a module diagram of an improved deep learning recognition and classification model for image content recognition and classification based on deep learning, as described in this invention.

[0046] Figure 3 This is a schematic diagram of the improved ResNet-50 network structure of the deep learning-based image content recognition and classification method of this invention.

[0047] Figure 4 This is a block diagram of the multi-scale feature fusion and weight allocation logic of the deep learning-based image content recognition and classification method of this invention.

[0048] Figure 5 This is a flowchart of the model training and incremental training process for the deep learning-based image content recognition and classification method of this invention.

[0049] Figure 6 This is a schematic diagram of the recognition result visualization and operation interface of the deep learning-based image content recognition and classification method of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0051] Example 1, in conjunction with Appendix Figure 1-6 A deep learning-based image content recognition and classification method includes the following steps:

[0052] S1. Image Acquisition and Preprocessing: PCB surface images are acquired using an industrial camera. A global shutter CMOS camera with a resolution of at least 12 megapixels is selected, with an acquisition frame rate of 10 frames per second and a fixed acquisition distance of 15cm. A ring-shaped shadowless light source is used for supplementary lighting to avoid the impact of lighting and shadows on image quality. A PCB image dataset is constructed, including normal PCB images and defective PCB images. Defect types include short circuits, open circuits, trace width deviations, pinholes, and copper spurs. The initial sample size of the dataset is no less than 1000 images, with defective samples accounting for no less than 30%. The acquired PCB images are preprocessed to obtain standardized images. Preprocessing steps include grayscale conversion, noise removal, size normalization, and contrast enhancement. Noise removal uses an adaptive median filtering algorithm, calculated using the following formula:

[0053]

[0054] in, For the filtered image in The pixel value of the location, For the original image in The pixel value of the location, For the original image in The median of pixels within the location's neighborhood, with an initial neighborhood window size of 3×3 and a maximum window size of 7×7. It is the minimum value of the pixels in the neighborhood. The maximum value of the pixels in the neighborhood;

[0055] Grayscale conversion uses a weighted average method, with weights allocated as follows: red channel 0.299, green channel 0.587, and blue channel 0.114. This converts the color PCB image into a single-channel grayscale image. Size normalization adjusts the PCB image to a uniform 512×512 pixels. Bilinear interpolation is used for size scaling to maintain the image texture features without distortion. Contrast enhancement uses an adaptive histogram equalization algorithm, with the block size set to 8×8 and the contrast limit threshold set to 0.03.

[0056] S2: Dataset Augmentation and Annotation: An improved Generative Adversarial Network (GAN) is used to augment the standardized images obtained in step S1 by a factor of 4, resulting in at least 1200 defect samples after augmentation. The generated images have the same resolution as the standardized images (512×512 pixels) to address the scarcity of defect samples. The augmented dataset is then annotated manually using the LabelImg annotation tool in PASCALVOC format. Annotators must have relevant experience in PCB inspection. After annotation, a second verification is performed by professionals to ensure accuracy. The annotation content includes defect location and defect type, and an annotation file is generated. The augmented dataset is then divided into training, validation, and test sets in a 7:2:1 ratio. Stratified random sampling is used during the partitioning process to ensure that the proportion of each type of defect sample in the training, validation, and test sets is consistent with the total dataset.

[0057] The improved generative adversarial network (GAN) uses a U-Net architecture for its generator, which consists of 4 downsampled convolutional layers and 4 upsampled deconvolutional layers. The activation function is LeakyReLU with a negative slope of 0.2. The discriminator uses a lightweight CNN architecture, consisting of 3 depthwise separable convolutional layers, 1 global average pooling layer, and 1 fully connected layer. The output dimension is 1. The loss function of the generator is calculated as follows:

[0058]

[0059] in, For generator loss, noise vector Distribution of noise vector It has a dimension of 128 and follows a standard normal distribution. Let be the probability that the discriminator classifies the generated image. This is a weighting coefficient, with a value ranging from 0.8 to 1.2, and a preferred value of 1.0. for loss, The image generated by the generator, Images of actual defects;

[0060] S3: Constructing an improved deep learning recognition and classification model: The model includes a feature extraction module, a feature fusion module, a defect recognition module, and a classification calibration module. The feature extraction module uses an improved ResNet-50 network, embedding depthwise separable convolutional layers into the residual blocks of the ResNet-50 network. Standard convolutions in the original residual blocks are replaced with depthwise separable convolutions, maintaining the kernel size at 3×3, while the stride and padding parameters remain unchanged. Simultaneously, an attention mechanism module is added to the network input, using the CBAM module to calculate channel attention and spatial attention sequentially. The channel attention compression ratio is set to 16, and the spatial attention convolution kernel size is 7×7. The feature fusion module employs a multi-scale feature fusion strategy, fusing features from different levels through a feature weight allocation algorithm, extracting conv2 and conv2 from the ResNet-50 network respectively. 3. The output features of the conv4 layer correspond to shallow texture features, mid-level contour features, and deep semantic features, with feature map sizes of 256×256, 128×128, and 64×64, respectively, including shallow texture features, mid-level contour features, and deep semantic features. The defect recognition module adopts a lightweight Transformer architecture, with the Transformer encoder having 2 layers, the multi-head attention heads having 4 layers, the embedding dimension having 256, and the feedforward network hidden layer having 512 dimensions. It is used to mine global features of the fused features and identify defect regions. The classification calibration module combines an improved support vector machine (SVM) and a category calibration algorithm. The improved SVM uses a radial basis kernel function with kernel coefficients set to 0.001 and a penalty coefficient C set to 1.0. It is used to classify defect types and correct classification results.

[0061] The calculation formula for the feature weight allocation algorithm is as follows:

[0062]

[0063]

[0064] in, For the first Weights of layer features =1,2,3, corresponding to shallow texture features, mid-level contour features, and deep semantic features, respectively. For the first The saliency score of a layer feature is obtained by calculating the information entropy of the feature map; the higher the information entropy, the higher the saliency score. For the first Layer feature map, The fused feature map has a uniform size of 64×64 and 256 channels.

[0065] The calculation formula for the category calibration algorithm is as follows:

[0066]

[0067] in, For the calibrated first Classification probability of class defects The classification probability before calibration. For the first The calibration coefficients for different defect types are adaptively adjusted based on historical detection data. The initial calibration coefficient is set to 1.0. After every 1000 detections, the calibration coefficient is updated based on the correct identification rate of each defect type; a higher correct identification rate results in a larger calibration coefficient. =1,2,3,4,5, corresponding to short circuit, open circuit, line width deviation, pinhole, and stray copper, respectively;

[0068] S4: Model Training and Optimization: Input the training set divided in step S2 into the improved deep learning recognition and classification model. Set the training batch size to 16, use the PyTorch deep learning framework in the training environment, and configure the hardware to a single RTX 3090 graphics card with 24GB of video memory. Use a hybrid loss function for model training, adjust the model parameters through an adaptive learning rate optimization algorithm, and validate the model using a validation set. After each training epoch, evaluate the model using the validation set. Evaluation metrics include precision, recall, and F1 score. Iterate and optimize until the model converges. The convergence criterion is that the decrease in the validation set loss value is less than 0.001 for 20 consecutive epochs. The trained recognition and classification model is then obtained.

[0069] Among them, the hybrid loss function includes the cross-entropy loss function and The loss function is calculated using the following formula:

[0070]

[0071] in, For mixed loss, This is a weighting coefficient, with a value ranging from 0.6 to 0.8, and a preferred value of 0.7. Let cross-entropy be the loss function. for Loss function;

[0072] The formula for calculating the loss function is:

[0073]

[0074] Where A is the actual defect area and B is the defect area predicted by the model. The coordinates of the area are represented by the coordinates of the top left corner and the width and height, and are uniformly converted to pixel-level coordinates during calculation.

[0075] The formula for calculating the cross-entropy loss function is:

[0076]

[0077] in, For the sample size, For the first The true labels for each sample are generated using one-hot encoding. The label for no defect is [1,0,0,0,0,0], and the five types of defects correspond to the following five dimensions. For the model to the first The predicted probability of each sample;

[0078] The adaptive learning rate optimization algorithm uses the Adam optimizer. The initial learning rate is set to 0.001, and the learning rate decays to 0.5 every 50 iterations. The first moment estimate exponential decay rate β1 of the optimizer is set to 0.9, the second moment estimate exponential decay rate β2 is set to 0.999, and the weight decay coefficient is set to 1e-5.

[0079] S5: Image recognition and classification: After the PCB image to be detected is preprocessed in step S1, it is input into the trained recognition and classification model. The single image inference process adopts a half-precision calculation mode to improve the inference speed. The model outputs the defect recognition result, including no defect / defective, defect area coordinates, defect type and classification confidence.

[0080] The classification confidence threshold is set to 0.85. When the classification confidence is below 0.85, it is marked as a suspected defect. The suspected defect image is stored separately and an alarm message is generated and pushed to the detection terminal to remind manual review.

[0081] S6: Result Output and Feedback: The identification and classification results from step S5 are visualized. The visualization interface is developed using PyQt. Defect areas are marked with red rectangles on the PCB image, and the defect type and confidence level are noted beside them. At the same time, the results are fed back to the industrial control system. The detection results are transmitted to the industrial PLC via the Modbus TCP communication protocol, with a transmission delay of no more than 10ms. Defective PCBs are marked to facilitate subsequent processing. Detection data is collected regularly to incrementally train the identification and classification model. The collected detection data must be screened to remove blurry, distorted, and invalid images, retaining only clear and valid labeled data to continuously improve the model's generalization ability.

[0082] The incremental training cycle is 7 days. The sample size of each incremental training is no less than 5% of the existing training set sample size. During the training process, the original model parameters remain unchanged, all parameters of the feature extraction module and the defect recognition module are frozen, and only the parameters of the classification calibration module and the feature fusion module are updated. The incremental training rounds are set to 30 rounds, and the learning rate is set to 1 / 10 of the initial training learning rate.

[0083] The present invention and its embodiments have been described above. This description is not restrictive. The accompanying drawings are only one embodiment of the present invention. If those skilled in the art are inspired by this description and design similar embodiments without departing from the spirit of the present invention, they should all fall within the protection scope of the present invention.

Claims

1. A deep learning-based image content recognition and classification method, characterized in that, Includes the following steps: S1. Image Acquisition and Preprocessing: PCB surface images are acquired using an industrial camera to construct a PCB image dataset. The dataset includes normal PCB images and defective PCB images. Defect types include short circuits, open circuits, line width deviations, pinholes, and stray copper. The acquired PCB images are preprocessed to obtain standardized images. S2: Dataset augmentation and annotation: The standardized images obtained in step S1 are augmented using an improved generative adversarial network (GAN) to compensate for the scarcity of defect samples; the augmented dataset is annotated, including the defect location and defect type, and an annotation file is generated; the augmented dataset is divided into training set, validation set, and test set in a ratio of 7:2:

1. S3: Constructing an improved deep learning recognition and classification model: The model includes a feature extraction module, a feature fusion module, a defect recognition module, and a classification calibration module. The feature extraction module uses an improved ResNet-50 network, embedding depthwise separable convolutional layers in the residual blocks of the ResNet-50 network, and adding an attention mechanism module at the network input. The feature fusion module employs a multi-scale feature fusion strategy, fusing features from different levels, including shallow texture features, mid-level contour features, and deep semantic features, through a feature weight allocation algorithm. The defect recognition module uses a lightweight Transformer architecture to mine global features from the fused features and identify defect regions. The classification calibration module combines an improved support vector machine (SVM) and a category calibration algorithm to classify defect types and correct classification results. S4: Model Training and Optimization: Input the training set divided in step S2 into the improved deep learning recognition and classification model, use a hybrid loss function to train the model, adjust the model parameters through an adaptive learning rate optimization algorithm, use the validation set to validate the model, iterate and optimize until the model converges, and obtain the trained recognition and classification model. S5: Image recognition and classification: After the PCB image to be detected is preprocessed in step S1, it is input into the trained recognition and classification model. The model outputs the defect recognition result, including no defect / defective, defect area coordinates, defect type and classification confidence. S6: Result Output and Feedback: Visualize the identification and classification results of step S5 and feed the results back to the industrial control system. Mark defective PCBs for subsequent processing. Regularly collect detection data and incrementally train the identification and classification model to continuously improve the model's generalization ability.

2. The image content recognition and classification method based on deep learning according to claim 1, characterized in that, In step S1, the preprocessing step includes grayscale conversion, noise removal, size normalization, and contrast enhancement. Noise removal employs an adaptive median filtering algorithm, calculated using the following formula: in, For the filtered image in The pixel value of the location, For the original image in The pixel value of the location, For the original image in The median of pixels within the location's neighborhood. It is the minimum value of the pixels in the neighborhood. It represents the maximum value of the pixels in the neighborhood.

3. The image content recognition and classification method based on deep learning according to claim 1, characterized in that, In step S2, the generator of the improved generative adversarial network (GAN) adopts the U-Net architecture, the discriminator adopts the lightweight CNN architecture, and the loss function of the generator is calculated as follows: in, For generator loss, noise vector The distribution Let be the probability that the discriminator classifies the generated image. This is the weighting coefficient, with a value ranging from 0.8 to 1.

2. for loss, The image generated by the generator, This is a real defect image.

4. The image content recognition and classification method based on deep learning according to claim 1, characterized in that, In step S3, the calculation formula for the feature weight allocation algorithm is as follows: in, For the first Weights of layer features =1,2,3, corresponding to shallow texture features, mid-level contour features, and deep semantic features, respectively. For the first Significance score of layer features For the first Layer feature map, This is the fused feature map.

5. The image content recognition and classification method based on deep learning according to claim 1, characterized in that, In step S4, the hybrid loss function includes the cross-entropy loss function and The loss function is calculated using the following formula: in, For mixed loss, This is the weighting coefficient, with a value ranging from 0.6 to 0.

8. Let cross-entropy be the loss function. for Loss function; The formula for calculating the loss function is: Where A represents the actual defect area and B represents the defect area predicted by the model; The formula for calculating the cross-entropy loss function is: in, For the sample size, For the first The true label of each sample For the model to the first The predicted probability of a sample.

6. The image content recognition and classification method based on deep learning according to claim 1, characterized in that, In step S3, the calculation formula for the category calibration algorithm is as follows: in, For the calibrated first Classification probability of class defects The classification probability before calibration. For the first The calibration coefficient for this type of defect is adaptively adjusted based on historical detection data. =1,2,3,4,5, corresponding to short circuit, open circuit, line width deviation, pinhole, and stray copper, respectively.

7. The image content recognition and classification method based on deep learning according to claim 1, characterized in that, In step S1, size normalization adjusts the PCB image to a uniform 512×512 pixels, and contrast enhancement uses an adaptive histogram equalization algorithm.

8. The image content recognition and classification method based on deep learning according to claim 1, characterized in that, In step S4, the adaptive learning rate optimization algorithm uses the Adam optimizer, with an initial learning rate of 0.001, which decays to 0.5 every 50 iterations.

9. The image content recognition and classification method based on deep learning according to claim 1, characterized in that, In step S5, the threshold for the classification confidence level is set to 0.

85. When the classification confidence level is lower than 0.85, it is marked as a suspected defect, prompting manual review.

10. The image content recognition and classification method based on deep learning according to claim 1, characterized in that, In step S6, the incremental training period is 7 days, and the sample size of each incremental training is no less than 5% of the existing training set sample size. During the training process, the original model parameters remain unchanged, and only the parameters of the classification calibration module and the feature fusion module are updated.

Images