PLC process defect end-to-end detection method based on Officient-YOLOv10s

Abstract

A PLC process defect end-to-end detection method based on Officient-YOLOv10s comprises the following steps that S1, a high-definition camera is used for capturing PLC process defect data set images of a PLC laboratory, an open source data set is combined for integration, and the images are screened and sorted; s2, obtaining a plurality of PLC process defect data set pictures; s3, labeling the data set by using Labelimg, labeling the data set into an xml format, converting the data set into a txt file, and dividing the txt file into a training set, a verification set and a test set according to a certain proportion; step S4, constructing and obtaining an Officient-YOLOv10s network model, wherein the Officient-YOLOv10s network model comprises a backbone network, a neck network and a head network; step S5, the obtained data set is put into an Officient-YOLOv10s model, the obtained PLC process defect detection image is identified in combination with an Officient-YOLOv10s network model, and a precision recall rate, an average precision mean value and the number of transmission frames per second are used as evaluation indexes; and S6, using the trained model to carry out image prediction and accurate identification on the detection image.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to an end-to-end detection method for PLC process defects based on Efficient-YOLOv10s. Background Art

[0002] PLC is an important industrial control device. If a fault occurs, it may cause the entire industrial system to shut down or suffer losses. Therefore, detecting whether the PLC is faulty is a crucial task. The identification of PLC board process defects mainly relies on advanced automated detection technology and methods. The present invention aims to detect the defects of the PLC board in a timely manner, check the fault phenomenon of the PLC, and determine the specific fault type. Thereby reducing the probability of failure during PLC operation. It is convenient for later experimental personnel to conduct research.

[0003] Since the defects on the surface of PLC boards are diverse and small in size, relying on manual visual inspection is not only inefficient, but also easy to miss problems, resulting in a high missed detection rate. With the continuous advancement of technology and algorithms, many researchers have developed a series of new methods for PLC board defect recognition using deep learning. These innovative algorithms significantly improve the accuracy of defect detection and the convenience of operation by adopting complex machine learning architectures, such as convolutional neural networks. A type of defect detection method based on traditional convolutional neural networks. These defect detection methods based on highly complex networks and a large number of parameters improve the accuracy of detection, can automatically extract features, have good generalization capabilities, and are suitable for large-scale data, but often the detection process is relatively slow due to the huge amount of calculation. In contrast, the YOLO series of detection methods represent another type of technology widely used in the field of defect recognition. Today's YOLOV10 algorithm has achieved real-time detection, which is not only highly accurate, but also omits the non-maximum suppression (NMS) process, greatly reducing the speed of image inference, and greatly reducing the average speed of processing each picture, which can achieve real-time detection of PLC board defects.

[0004] In the existing process defect detection technology for PCL, there are the following technical defects. First, the detection speed of existing detection technology is often not ideal and cannot meet the needs of modern industry for efficient production. This bottleneck of detection speed will lead to a decrease in production line efficiency, especially when facing the rapid detection of a large number of products, the existing technology may be difficult to keep up with the high-frequency production rhythm. Secondly, there are also shortcomings in detection accuracy. As the core component of the control system, the complexity of the PLC industrial control board determines the strict requirements for precision. However, the existing detection technology may not be able to achieve sufficiently high accuracy in identifying subtle defects and detecting minor faults. This will cause potential fault hazards to not be discovered in time, thereby affecting the stability and safety of the system. Therefore, improving the detection speed and detection accuracy is the main challenge of the current PLC industrial control board detection technology. In the future, more advanced sensor technology, intelligent algorithms and automated testing methods will be needed to solve these problems to ensure the efficient and accurate operation of the industrial control system.

[0005] YOLOV10 is divided into five models: YOLOV10 n / s / l / m / x, among which YOLOV10s is the model with the best comprehensive performance in terms of various indicators such as accuracy, parameter quantity, and speed; the present invention mainly improves the average accuracy, speed, and parameter quantity performance of the model, and therefore adopts YOLOV10s as the baseline algorithm, and makes improvements on this basis. The improved algorithm has a very broad application prospect and practical significance in the rapid and real-time identification of PLC process defects. Summary of the invention

[0006] The purpose of the present invention is to solve the technical problems of poor detection speed and detection accuracy in the existing defect detection technology for PLC industrial control boards, and to propose an end-to-end detection technology for PLC process defects based on Efficient-YOLOv10s.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A PLC process defect end-to-end detection method based on Efficient-YOLOv10s includes the following steps:

[0009] Step S1: Use a high-definition camera to capture the PLC process defect dataset images of the PLC laboratory, integrate them with the open source dataset, and screen and organize the images;

[0010] Step S2: Obtain several PLC process defect data set images through image preprocessing steps such as smoothing filtering, Gaussian filtering, and noise processing;

[0011] Step S3: Use Labelimg to annotate the dataset in XML format, convert it into a txt file, and divide it into training set, validation set, and test set according to a certain ratio;

[0012] Step S4: construct and obtain an Efficient-YOLOv10s network model, which includes a backbone network, a neck network, and a head network;

[0013] Step S5: Put the obtained data set into the Efficient-YOLOv10s model, combine the Efficient-YOLOv10s network model to recognize the acquired PLC process defect detection image, and use the precision recall rate, average precision mean and transmission frames per second as evaluation indicators;

[0014] Step S6: Use the trained model to perform image prediction on the detection image for accurate recognition.

[0015] In step S1, the following steps are included:

[0016] Step S1-1: photographing a damaged PLC board in a PLC laboratory using a high-definition camera to obtain an original data set;

[0017] Step S1-2: Find an open source data set on an open source data website, compare it with the original data set, select high-resolution images to expand the data set to obtain the original data set;

[0018] In step S2, the image is preprocessed on the pycharm platform. The data preprocessing method includes smoothing filtering, Gaussian filtering, and noise processing to obtain several PLC process defect data set images and the final data set.

[0019] In step S3, the following steps are included:

[0020] S3-1: Use labelimg image annotation tool to annotate the collected PLC process defect images. The defects are divided into three categories: insufficient solder, missing component, and solder short circuit. Insufficient solder is "Insufficient solder", missing component is "Missing component", and solder short circuit is "Soldering shorted". Get PascalVOC format data labels, that is, XML tags, and then convert them into txt files suitable for training through data format conversion code written in python. The file includes most of the data, including the x-coordinate, y-coordinate, width and height of the center point of the bounding box.

[0021] S3-2: After completing the labeling work, divide the data set into training set, validation set, and test set according to a certain ratio.

[0022] In step S4, the backbone network is specifically:

[0023] The image is input to the first GSConv module, the output of the first GSConv module is connected to the input of the second GSConv module, the output of the second GSConv module is connected to the input of the first C2f module, the output of the first C2f module is connected to the input of the third GSConv module, the output of the third GSConv module is connected to the input of the second C2f module, the output of the second C2f module is connected to the input of the first CSDown module, the output of the first CSDown module is connected to the input of the third C2f module, the output of the third C2f module is connected to the input of the second CSDown module, the output of the second CSDown module is connected to the input of the fourth C2f module, the output of the fourth C2f module is connected to the input of the SPPF module, the output of the SPPF module is connected to the input of the SE module, and the output of the SE module is connected to the neck network;

[0024] The structure of the neck network is as follows: the output of the SE module is connected to the input of the first Upsample module and the input of the C2fCIB module, the output of the first Upsample module is connected to the input of the first Concat module, the output of the first Concat module is connected to the input of the fifth C2f module, the output of the fifth C2f module is connected to the input of the second Upsample module, the output of the second Upsample module is connected to the input of the second Concat module, the output of the second Concat module is connected to the input of the sixth C3f module, the output of the sixth C3f module is connected to the input of the first v10Detect detection head and the input of the ordinary convolution module, the output of the ordinary convolution module is connected to the input of the third Concat module, the output of the third Concat module is connected to the input of the seventh C2f module, the output of the seventh C2f module is connected to the input of the second v10Detect detection head and the input of the third CSDown module, the output of the third CSDown module is connected to the input of the fourth Concat module, the output of the fourth Concat module is connected to the input of the C2fCIB module, and the output of the C2fCIB module is connected to the input of the third v10Detect detection head;

[0025] The head network includes the first v10Detect detection head, the second v10Detect detection head, and the third v10Detect detection head.

[0026] The C2fCIB modules are as follows:

[0027] The C2fCIB module includes a first Conv module, the input of the first Conv module is used to input a feature map, the output of the first Conv module is connected to the Spilt module and the input of the second Conv module, the output of the Spilt module is connected to the first CIB module and the input of the second Conv module, the output of the first CIB module is connected to the second CIB module and the input of the second Conv module, the output of the second CIB module is connected to the third CIB module and the input of the second Conv module, the output of the third CIB module is connected to the Concat module and the input of the second Conv module, the output of the Concat module is connected to the input of the second Conv module, and the output of the second Conv module is used to output features.

[0028] The C2f modules are as follows:

[0029] The C2f module includes the first Conv module, the Spilt module, the Bottleneck module, the Concat module, and the second Conv module; wherein the Bottleneck module includes n serially connected Bottleneck units, and the output of each Bottleneck unit is connected to the input of the second Conv module;

[0030] The input of the first Conv module is used to input the feature map, the output of the first Conv module is connected to the Spilt module and the input of the second Conv module, the output of the Spilt module is connected to the Bottleneck module and the input of the second Conv module, the output of the Bottleneck module is connected to the input of the Concat module and the input of the second Conv module, and the output of the second Conv module is used to output features.

[0031] The structure of the GSConv module is:

[0032] The input feature map has a channel number of C1, and a common convolution operation (Conv operation) is performed on the feature map to obtain a first feature map with a channel number of C2 / 2. The first feature map with a channel number of C2 / 2 is input to the DW convolution module to obtain a second feature map with a channel number of C2 / 2. The first feature map and the second feature map are Figure 1 The output of the splicing module is input to the concatenation module, and the output of the concatenation module is shuffled to finally obtain a feature map with a channel number of C2.

[0033] The CSDown module is as follows: the input feature x_in is input to the CV1 module, the output of the CV1 module is connected to the input of the CV2 module, and the output of the CV2 module obtains the output feature x_out;

[0034] Among them, the stride of the CV1 module is 1, the convolution kernel is 1×1, the stride of the CV2 module is 2, the convolution kernel is 3×3, and 1 pixel of padding is added to the edge of the feature map.

[0035] The SE modules are as follows:

[0036] The SE module includes an Inception module, the input of the Inception module is used to input input features, the output of the Inception module is connected to the input of the Global pooling module and the input of the Scale module, the output of the Global pooling module is connected to the input of the first FC module, the output of the first FC module is connected to the input of the ReLU module, the output of the ReLU module is connected to the input of the second FC module, the output of the second FC module is connected to the input of the Sigmoid module, the output of the Sigmoid module is connected to the input of the Scale module, and the final output features of the SE module are obtained from the output of the Scale module;

[0037] Among them, the number of channels of the output of the Inception module is H*W*C, the number of channels of the output of the Global pooling module is 1*1*C, the number of channels of the output of the first FC module is 1*1*C / r, the number of channels of the output of the ReLU module is 1*1*C / r, the number of channels of the output of the second FC module is 1*1*C, and the number of channels of the output of the Sigmoid module is H*W*C.

[0038] Compared with the prior art, the present invention has the following technical effects:

[0039] 1) Overall, the model of the present invention has significant advantages over existing detection models in terms of accuracy, speed, and lightness. In particular, for small targets and targets at a long distance, the present model shows lower missed detection rates and false positive rates, and can accurately identify PLC process defects. Compared with the baseline model, the model of the present invention significantly improves the recognition accuracy and detection speed on the basis of lightness, and has practical application value for system-side deployment. In addition, the dataset of the present invention not only provides rich samples for the training of deep learning models, but also ensures that the model can be effectively evaluated on a variety of data during the verification and testing phases, thereby enhancing its generalization ability and performance. It is worth mentioning that the present model omits the non-maximum suppression (NMS) step and realizes end-to-end detection. This optimization significantly improves the detection efficiency and has broad promotion significance;

[0040] 2) In terms of details, the present invention has obvious innovations in the detailed structure. First, all the ordinary convolution modules of the backbone network in the original structure, namely the Conv modules, are replaced with lightweight GSConv modules; the SE attention mechanism is added to the end of the backbone network to replace the PSA module; the Inner-IoU loss function is used as the loss function of the model proposed in this paper and combined with the original YOLOv10s model to construct the Efficient-YOLOv10s network structure.

[0041] 3) Adding the GSConv lightweight network module to the YOLOv10s backbone network, replacing all the ordinary convolution modules of the backbone network, the technical effect produced is: the technical effect produced on the network model, sublimating what technical effect the detection of "PLC process defects" brings; GSConv (Ghost Squeeze Convolution) is an improved convolution that aims to improve computational efficiency and reduce model complexity. It reduces the amount of computation and maintains similar feature expression capabilities by replacing ordinary convolution operations with a portion of actual convolution and more cheap linear operations. Using GSConv to replace ordinary convolution in YOLOv10 will improve the computational efficiency of the model: GSConv can significantly reduce the floating point operations (FLOPs) in convolution operations, which means that the computational overhead of YOLOv10 in forward propagation is reduced, thereby speeding up the inference speed. This is particularly useful on resource-constrained devices (such as embedded systems or mobile devices), and can improve the speed of real-time detection without significantly compromising performance. GSConv can also reduce the number of model parameters: GSConv optimizes the storage requirements of YOLOv10 by reducing the number of parameters that need to be learned. This reduction in parameters can help the model be more lightweight in memory usage, making it suitable for deployment on hardware platforms with limited memory.

[0042] 4) The SE module is relatively simple and only aggregates and re-weights global information in the channel dimension, while the PSA module is more complex and involves an attention mechanism in the spatial dimension. By replacing PSA with SE, the model structure can be simplified, the amount of calculation and the number of parameters can be reduced, thereby reducing the complexity of the model. This is particularly useful for the lightweight YOLOv10s model, which needs to take into account both real-time performance and computational efficiency. The SE module has a small computational overhead. It mainly performs feature weighting in the channel dimension through global average pooling and a small number of fully connected layers, which is more efficient than the multi-scale spatial attention calculation of the PSA module. Due to the low computational cost of the SE module, using SE instead of PSA can improve the inference speed. This advantage is particularly obvious when deployed on resource-constrained devices (such as mobile devices or embedded systems).

[0043] 5) CIoU takes into account the matching degree of IoU, center point distance and aspect ratio, and is suitable for the detection of larger targets. However, in small target detection or complex scenes, CIoU may perform poorly in some cases. Inner-IoU pays more attention to the matching of the internal area of ​​the target, and is particularly suitable for processing fine-grained matching of local areas, which can better improve the positioning accuracy in the detection of small targets or overlapping objects. At the same time, CIoU will be severely penalized when the IoU value of the two boxes is small, especially when the center points of the two boxes are far apart. Inner-IoU focuses on the internal overlapping area of ​​the box. When the two boxes partially overlap, Inner-IoU will not excessively penalize the deviation of the bounding box. This has certain advantages for bounding box regression optimization in model training, and can be optimized more effectively when the initial error of the bounding box is large. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0045] Figure 1 This is the Efficient-YOLOv10s network diagram of the present invention;

[0046] Figure 2 It is a flow chart of the present invention;

[0047] Figure 3 It is a schematic diagram of the bottleneck structure of the C2fCIB module;

[0048] Figure 4 It is the bottleneck structure diagram of C2f module;

[0049] Figure 5 Schematic diagram of SE attention mechanism;

[0050] Figure 6 This is the schematic diagram of GSConv;

[0051] Figure 7 This is the schematic diagram of the CSDown module;

[0052] Figure 8 This is an example diagram of insufficient solder detection on a PLC industrial control board;

[0053] Fig. 9 This is an example diagram of welding short circuit detection on PLC industrial control board;

[0054] Fig.10 This is an example diagram of detecting missing components on a PLC industrial control board; DETAILED DESCRIPTION

[0055] like Figure 2As shown, a PLC process defect end-to-end detection method based on Efficient-YOLOv10s is characterized by comprising the following steps:

[0056] Step S1: Use a high-definition camera to capture the PLC process defect dataset images of the PLC laboratory, integrate them with the open source dataset, and screen and organize the images;

[0057] Step S2: image preprocessing steps such as smoothing filtering, Gaussian filtering, and noise processing to obtain 1791 PLC process defect data set images;

[0058] Step S3: Use Labelimg to annotate the dataset in XML format and convert it into a txt file. Then divide it into a training set, a validation set, and a test set in a ratio of 8:1:1, with 1432 images in the training set, 179 images in the validation set, and 180 images in the test set.

[0059] Step S4: construct and obtain an Efficient-YOLOv10s network model, which includes a backbone network, a neck network, and a head network;

[0060] Step S5: putting the obtained data set into the Efficient-YOLOv10s model, combining the Efficient-YOLOv10s network model to recognize the acquired PLC process defect detection image, and using precision (Pre), recall (Rec), mean average precision (mAP) and frames per second (FPS) as evaluation indicators;

[0061] Step S6: Use the trained model to perform image prediction on the detection image for accurate recognition.

[0062] In step S1, the following steps are included:

[0063] Step S1-1: photographing a damaged PLC board in a PLC laboratory using a high-definition camera to obtain an original data set;

[0064] Step S1-2: Find an open source data set on an open source data website, compare it with the original data set, select high-resolution images to expand the data set to obtain the original data set;

[0065] In step S2, the image is preprocessed on the pycharm platform. The data preprocessing method includes smoothing filtering, Gaussian filtering, and noise processing to obtain several PLC process defect data set images and the final data set.

[0066] In step S3, the following steps are included:

[0067] S3-1: Use the labelimg image annotation tool to annotate the collected PLC process defect images. The defects are divided into three categories: insufficient solder, missing component, and soldering short circuit. Insufficient solder is "Insufficient solder", missing component is "Missing component", and soldering short circuit is "Soldering shorted". Get the PascalVOC format data label, that is, XML label, and then convert it into a txt file suitable for training through the data format conversion code written in python. The file includes most of the data, including the x-coordinate, y-coordinate, width and height of the center point of the bounding box.

[0068] S3-2: After completing the labeling work, divide the data set into training set, validation set, and test set according to a certain ratio.

[0069] like Figure 1 As shown, in step S4, the backbone network is specifically:

[0070] The image is input to the first GSConv module, the output of the first GSConv module is connected to the input of the second GSConv module, the output of the second GSConv module is connected to the input of the first C2f module, the output of the first C2f module is connected to the input of the third GSConv module, the output of the third GSConv module is connected to the input of the second C2f module, the output of the second C2f module is connected to the input of the first CSDown module, the output of the first CSDown module is connected to the input of the third C2f module, the output of the third C2f module is connected to the input of the second CSDown module, the output of the second CSDown module is connected to the input of the fourth C2f module, the output of the fourth C2f module is connected to the input of the SPPF module, the output of the SPPF module is connected to the input of the SE module, and the output of the SE module is connected to the neck network;

[0071] The structure of the neck network is: the output of the SE module is connected to the input of the first Upsample module and the input of the C2fCIB module, the output of the first Upsample module is connected to the input of the first Concat module, the output of the first Concat module is connected to the input of the fifth C2f module, the output of the fifth C2f module is connected to the input of the second Upsample module, the output of the second Upsample module is connected to the input of the second Concat module, the output of the second Concat module is connected to the input of the sixth C3f module, the output of the sixth C3f module is connected to the input of the first v10Detect detection head and the input of the ordinary convolution module, the output of the ordinary convolution module is connected to the input of the third Concat module, the output of the third Concat module is connected to the input of the seventh C2f module, the output of the seventh C2f module is connected to the input of the second v10Detect detection head and the input of the third CSDown module, the output of the third CSDown module is connected to the input of the fourth Concat module, the output of the fourth Concat module is connected to the input of the C2fCIB module, and the output of the C2fCIB module is connected to the input of the third v10Detect detection head.

[0072] The head network includes the first v10Detect detection head, the second v10Detect detection head, and the third v10Detect detection head.

[0073] like Figure 3 As shown, the C2fCIB module is specifically:

[0074] The C2fCIB module includes a first Conv module, the input of the first Conv module is used to input a feature map, the output of the first Conv module is connected to the Spilt module and the input of the second Conv module, the output of the Spilt module is connected to the first CIB module and the input of the second Conv module, the output of the first CIB module is connected to the second CIB module and the input of the second Conv module, the output of the second CIB module is connected to the third CIB module and the input of the second Conv module, the output of the third CIB module is connected to the Concat module and the input of the second Conv module, the output of the Concat module is connected to the input of the second Conv module, and the output of the second Conv module is used to output features.

[0075] like Figure 4 As shown, the C2f module is specifically:

[0076] The C2f module includes the first Conv module, the Spilt module, the Bottleneck module, the Concat module, and the second Conv module; wherein the Bottleneck module includes n serially connected Bottleneck units, and the output of each Bottleneck unit is connected to the input of the second Conv module;

[0077] The input of the first Conv module is used to input the feature map, the output of the first Conv module is connected to the Spilt module and the input of the second Conv module, the output of the Spilt module is connected to the Bottleneck module and the input of the second Conv module, the output of the Bottleneck module is connected to the input of the Concat module and the input of the second Conv module, and the output of the second Conv module is used to output features.

[0078] like Figure 6 As shown, the structure of the GSConv module is:

[0079] The input feature map has a channel number of C1, and a common convolution operation (Conv operation) is performed on the feature map to obtain a first feature map with a channel number of C2 / 2. The first feature map with a channel number of C2 / 2 is input to the DW convolution module to obtain a second feature map with a channel number of C2 / 2. The first feature map and the second feature map are Figure 1 The output of the splicing module is input to the concatenation module, and the output of the concatenation module is shuffled to finally obtain a feature map with a channel number of C2.

[0080] like Figure 7 As shown, the CSDown module is specifically as follows: the input feature x_in is input to the CV1 module, the output of the CV1 module is connected to the input of the CV2 module, and the output of the CV2 module obtains the output feature x_out;

[0081] Among them, the stride of the CV1 module is 1, the convolution kernel is 1×1, the stride of the CV2 module is 2, the convolution kernel is 3×3, and 1 pixel of padding is added to the edge of the feature map.

[0082] like Figure 5 As shown, the SE module is specifically:

[0083] The SE module includes an Inception module, the input of the Inception module is used to input input features, the output of the Inception module is connected to the input of the Global pooling module and the input of the Scale module, the output of the Global pooling module is connected to the input of the first FC module, the output of the first FC module is connected to the input of the ReLU module, the output of the ReLU module is connected to the input of the second FC module, the output of the second FC module is connected to the input of the Sigmoid module, the output of the Sigmoid module is connected to the input of the Scale module, and the final output features of the SE module are obtained from the output of the Scale module;

[0084] Among them, the number of channels of the output of the Inception module is H*W*C, the number of channels of the output of the Global pooling module is 1*1*C, the number of channels of the output of the first FC module is 1*1*C / r, the number of channels of the output of the ReLU module is 1*1*C / r, the number of channels of the output of the second FC module is 1*1*C, and the number of channels of the output of the Sigmoid module is H*W*C.

[0085] In order to facilitate those skilled in the art to better understand the present invention, the following embodiments are provided:

[0086] Example:

[0087] A PLC process defect end-to-end detection method based on Efficient-YOLOv10s: comprising the following steps:

[0088] S1. Use a high-definition camera to capture the PLC process defect dataset images of the PLC laboratory, integrate them with the open source dataset, and screen and organize the images;

[0089] S2, using image preprocessing steps such as smoothing filtering, Gaussian filtering, and noise processing to obtain 1791 PLC process defect data set images;

[0090] S3. Labelimg was used to annotate the dataset in XML format and converted into a txt file. The dataset was divided into a training set, a validation set, and a test set in a ratio of 8:1:1. The training set contained 1432 images, the validation set contained 179 images, and the test set contained 180 images.

[0091] S4. Add the GSConv lightweight network module to the YOLOv10s backbone network to replace all the ordinary convolutional layers of the backbone network; add the SE attention mechanism module to the end of the backbone network to replace the PSA module; combine the Inner-IoU loss function as the loss function of the model proposed in this article with the original YOLOv10s model to construct the Efficient-YOLOv10s network structure;

[0092] S5. Put the obtained data set into the Efficient-YOLOv10s model, combine the model to identify the acquired PLC process defect detection images, and use precision (Pre), recall (Rec), mean average precision (mAP) and frames per second (FPS) as evaluation indicators;

[0093] S6. Use the trained model to predict the detection image, accurately identify it, analyze the advantages of the model and promote it.

[0094] Specifically, step S1 includes:

[0095] S1.1: Use a high-definition camera to shoot the damaged PLC board in the PLC laboratory to obtain the original data set. When shooting, the shooting light conditions are good, the equipment is clear, and the reliability is high.

[0096] S1.2: Find an open source dataset on an open source data website, compare it with the original dataset, select high-resolution images to expand the dataset, and then obtain the original dataset.

[0097] Specifically, step S2 includes:

[0098] The images were preprocessed on the pycharm platform. The data preprocessing methods included smoothing filtering, Gaussian filtering, noise processing and other steps. 1791 PLC process defect data set images were obtained to obtain the final data set.

[0099] Specifically, step S3 includes:

[0100] S3.1: Use the labelimg image annotation tool to annotate the collected PLC process defect images. The defects are divided into three categories: insufficient solder, missing component, and soldering short circuit. Insufficient solder is “Insufficient solder”, missing component is “Missing component”, and soldering short circuit is “Soldering shorted”. Get the PascalVOC format data label (XML tag), and then convert it into a txt file suitable for training through the data format conversion code written in python. The file includes most of the data, including the x-coordinate, y-coordinate, width, height and other annotation information of the center point of the bounding box.

[0101] S3.2: After the labeling work is completed, the data set is divided into training set, validation set and test set in a ratio of 8:1:1. There are 1432 images in the training set, 179 images in the validation set and 180 images in the test set.

[0102] Specifically, step S4 includes:

[0103] Starting from the original input module, the original input module is connected to the first GSConv module, and the GSConv lightweight network module is added to the YOLOv10s backbone network, replacing all the ordinary convolution modules of the backbone network.

[0104] GSConv combines three parts: standard convolution (SC), depthwise separable convolution (DSC) and channel shuffle. First, ordinary convolution reduces the dimension of data by downsampling the input, thereby reducing the computational complexity. GSConv is applied to YOLOv10 for defect detection of PLC industrial control boards, which mainly improves detection efficiency and real-time performance by reducing the amount of calculation and achieving model lightweight. This improvement is particularly suitable for defect identification in complex environments of industrial control boards, such as welding defects, component damage, etc. GSConv optimizes the feature extraction process, so that the model can quickly process large amounts of image data while maintaining high detection accuracy, enhancing the ability to detect small targets and subtle defects, thereby providing a more reliable solution for industrial automation; it can also help the model capture features in a larger range; secondly, deep convolution is an operation that processes the results after downsampling. Deep convolution can increase the depth of the network while keeping the number of parameters unchanged, thereby improving the expressive power of the model; then use the splicing operation to merge the results of ordinary convolution and depthwise separable convolution, so that the model can use the features extracted by ordinary convolution and deep convolution at the same time; finally, use the Shuffle operation to rearrange the spliced ​​results, so that the corresponding number of channels of the previous two convolutions are connected together, thereby improving the diversity of features.

[0105] The structure of the constructed YOLOv10s backbone network is as follows: the image is input to the first GSConv module, the output of the first GSConv module is connected to the input of the second GSConv module, the output of the second GSConv module is connected to the input of the first C2f module, the output of the first C2f module is connected to the input of the third GSConv module, the output of the third GSConv module is connected to the input of the second C2f module, the output of the second C2f module is connected to the input of the first CSDown module, the output of the first CSDown module is connected to the input of the third C2f module, the output of the third C2f module is connected to the input of the second CSDown module, the output of the second CSDown module is connected to the input of the fourth C2f module, the output of the fourth C2f module is connected to the input of the SPPF module, the output of the SPPF module is connected to the input of the SE module, and the output of the SE module is connected to the neck network;

[0106] The structure of the neck network is: the output of the SE module is connected to the input of the first Upsample module and the input of the C2fCIB module, the output of the first Upsample module is connected to the input of the first Concat module, the output of the first Concat module is connected to the input of the fifth C2f module, the output of the fifth C2f module is connected to the input of the second Upsample module, the output of the second Upsample module is connected to the input of the second Concat module, the output of the second Concat module is connected to the input of the sixth C3f module, the output of the sixth C3f module is connected to the input of the first v10Detect detection head and the input of the ordinary convolution module, the output of the ordinary convolution module is connected to the input of the third Concat module, the output of the third Concat module is connected to the input of the seventh C2f module, the output of the seventh C2f module is connected to the input of the second v10Detect detection head and the input of the third CSDown module, the output of the third CSDown module is connected to the input of the fourth Concat module, the output of the fourth Concat module is connected to the input of the C2fCIB module, and the output of the C2fCIB module is connected to the input of the third v10Detect detection head.

[0107] The head network includes the first v10Detect detection head, the second v10Detect detection head, and the third v10Detect detection head.

[0108] The SE attention mechanism module is added to the end of the backbone network to replace the PSA module. The main idea of ​​SE is to improve the performance of the model by compressing and exciting the input features. Specifically, the SE attention mechanism consists of two steps: Squeeze and Excitation. In the Squeeze step, the input feature map is compressed into a vector through a global average pooling operation, and then mapped to a smaller vector through a fully connected layer. In the Excitation step, a sigmoid function is used to compress each element in this vector to between 0 and 1, and it is multiplied with the original input feature map to obtain a weighted feature map. Through the SE attention mechanism, the model can adaptively learn the importance of each channel, thereby improving the performance of the model. In practical applications, the SE attention mechanism has been widely used in the fields of industrial defect detection such as PLC industrial control boards and PCB circuit boards, and has achieved good results.

[0109] Figure 3 Schematic diagram of the structure of the C2fCIB module. In the layer with smaller feature map scale, the C2f module is replaced by the C2fCIB module, and the Bottleneck is replaced by CIB (compact inverted block). The depth-wise separable convolution is used to reduce the amount of calculation. Its structure is as follows: the output of the first Conv module is connected to the input of the Spilt module of the second Conv module with a feature map size of H×W×C, the output of the Spilt module is connected to the input of the second Conv module with a feature map size of 0.5H×W×C and the input of the first CIB module, the output of the first CIB module is connected to the input of the second Conv module with a feature map size of 0.5H×W×C and the input of the second CIB module, there are n CIB modules, and the above operation is repeated n times. The output of the nth CIB module is connected to the input of the second Conv module with a feature map size of 0.5H×W×C and the input of the Concat module, and the output of the Concat module and Figure 4 The output of the second Conv is directly connected to the input of the second Conv module without passing through the trunk, and the output of the second Conv module has been processed by the entire C2f module.

[0110] Figure 4Schematic diagram of the structure of the C2f module: The C2f module divides the input data into two branches for processing. One branch is directly passed to the output, and the other branch is processed by multiple Bottleneck modules. Such a branch design helps to increase the nonlinear ability and representation ability of the network, thereby improving the network's ability to model complex data. Finally, the features of the two branches are spliced ​​in the channel dimension to achieve feature fusion. Assuming that the size of the input feature map is H×W×C, where H and W are the height and width of the feature map respectively, and C is the number of channels, its structure is: the output of the first Conv module is connected to the input of the Spilt module with a feature map of size H×W×C, the input of the second Conv module with a feature map of size 0.5H×W×C, the input of the first Bottleneck module with a feature map of size 0.5H×W×C, the input of the second Conv module with a feature map of size 0.5H×W×C, and the input of the second Bottleneck module. There are n Bottleneck modules. Repeat the above operation n times. The output of the nth Bottleneck module is connected to the input of the second Conv module with a feature map of size 0.5H×W×C and the input of the Concat module. The output of the Concat module and Figure 4 The output of the second Conv is directly connected to the input of the second Conv module without passing through the trunk, and the output of the second Conv module has been processed by the entire C2f module.

[0111] Figure 6 The structure of the GSConv module is as follows: the input is a feature map with C1 channels, a normal convolution operation (Conv operation) is performed on the input, the output after the operation is connected to the first feature map with C2 / 2 channels, the output of the previous step is connected to the DW convolution module to obtain the second feature map with C2 / 2 channels, the output of the first feature map and the output of the second feature map are connected to the input of the splicing module, the output of the splicing module is connected to the output after the shuffle operation, and the output number of channels is C2.

[0112] Figure 7 The structure of the CSDown module is as follows: the input is connected to the input of the CV1 module with a step size of 1 and a convolution kernel of 1×1; the output of the CV1 module is connected to the input of the CV2 module with a step size of 2 and a convolution kernel of 3×3, and 1 pixel of padding is added to the edge of the feature map; the output of the CV2 module is the entire process of the CSDown module processing.

[0113] The structure of the SE attention mechanism module is as follows Figure 5 Shown:

[0114] In a convolutional neural network, the SE module usually operates after the output of a convolutional layer, and then passes its result to the next module (such as the next convolutional layer). Assume that the size of the input feature map is H×W×C, where H and W are the height and width of the feature map, respectively, and C is the number of channels).

[0115] Input: Features output by the convolutional layer Figure X , size is H×W×C.

[0116] Squeeze: Compress H×W×C into a vector z of length C through global average pooling.

[0117] Excitation: Generate channel weights w through two fully connected layers and ReLU and Sigmoid activation functions.

[0118] Scale: Weight the original feature map channel by channel to generate a new feature map Y.

[0119] Output: The weighted feature map Y has the same size as the input, still H×W×C.

[0120] Its specific structure is: Output features Figure X The input of the Inception module is connected, the output of the Inception module is connected to the input of the Scale module and the input of the Global Pooling module of size H×W×C, the output of the Global Pooling module is connected to the input of the first FC module of size 1×1×C, the output of the first FC module is connected to the input of the ReLU module of size 1×1×C / r, the output of the ReLU module is connected to the input of the second FC module of size 1×1×C, the output of the second FC module is connected to the input of the Sigmoid module of size 1×1×C, the output of the Sigmoid module is connected to the input of the Scale module of size H×W×C, and the output of the Scale module completes the processing process of the SE attention mechanism.

[0121] S4.3: Use the Inner-IoU loss function as the bounding box loss function of the model proposed in this paper.

[0122] The border loss function used by the original YOLOv10 model is the CIoU loss function.

[0123] The CIoU loss function calculation formula is as follows:

[0124]

[0125] In the formula, b and b gt are the center points of the predicted box and the true box respectively; ρ 2Indicates the calculation of the Euclidean distance between the two center points; c is the diagonal length of the minimum bounding box covering the predicted box and the true box; α is a positive trade-off parameter used to balance the impact of the aspect ratio; v is used to measure the consistency of the aspect ratio of the predicted box and the true box. If the aspect ratios of the two boxes are the same, v = 0.

[0126] CIoU introduces the similarity of center point distance and aspect ratio, which can effectively accelerate model convergence and improve detection performance. However, bounding box regression based on IoU still accelerates convergence by adding new loss terms, without considering the rationality of IoU loss itself. This paper uses Inner-IoU to replace CIoU.

[0127] The main idea of ​​Inner-IoU is to improve the accuracy of bounding box regression (BBR) in object detection, especially when dealing with highly overlapping objects. The traditional IoU (Intersection over Union) calculation method considers the overall overlapping area of ​​the predicted bounding box and the true bounding box, while Inner-IoU focuses on the overlapping part inside the bounding box. It calculates the loss function by introducing auxiliary bounding boxes, which are reduced versions of the original bounding boxes. Inner-IoU provides a more accurate evaluation of the overlapping area by focusing on the core part of the bounding box rather than the whole.

[0128] Inner-IoU is an IoU loss function based on the auxiliary bounding box. The scale factor ratio is introduced to control the size of the auxiliary bounding box, and then the loss is calculated to accelerate the convergence of the model. The definition of Inner-IoU is as follows:

[0129]

[0130]

[0131] Union=(w gt *h gt )*ratio 2 +(w*h)*ratio 2 -Inter (25)

[0132]

[0133] In the formula, and is the inner center point of the real frame; x c and c is the internal center point of the prediction box; are the left, right, top and bottom boundaries of the real box, which are calculated by the center point, width, height and scale factor ratio of the real box; b l , b r , b t , bb It is the left, right, top and bottom boundaries of the prediction box, which is calculated by the center point, width, height and scale factor ratio of the prediction box; Inter represents the intersection of the real box and the prediction box; Union represents the union of the real box and the prediction box; Inner-IoU is calculated by the ratio of the two.

[0134] Inner-IoU has the following advantages over the CIoU original loss function:

[0135] 1) Adjusting Scale: By controlling the size of the auxiliary bounding boxes, Inner-IoU allows fine-tuning for different datasets and detection tasks.

[0136] 2) Improve generalization ability: Experiments have shown that Inner-IoU shows better generalization performance than traditional IoU on different datasets.

[0137] 3) Handling high and low IoU samples: For high IoU samples, using a smaller auxiliary box can accelerate model learning; while for low IoU samples, using a larger auxiliary box can improve regression performance.

[0138] In summary, Inner-IoU is a more detailed and more focused performance evaluation metric for target center, which improves the accuracy and efficiency of target detection tasks by adjusting the scale of auxiliary boxes.

[0139] Specifically, step S5 includes:

[0140] S5.1: The PLC process defect image dataset obtained in S3 is put into the Efficient-YOLOv10s model reconstructed by the present invention for training, the number of training times is 200, best.pt is obtained, and the original model is trained, and the comprehensive performance indicators are compared for analysis.

[0141] S5.2: The Efficient-YOLOv10s model constructed by the present invention is evaluated by using recall (Rec), mean average precision (mAP), frames per second (FPS), and parameters.

[0142] Specifically, step S6 includes:

[0143] The trained model is used to identify and locate images of PLC defects collected on site, and can accurately distinguish the types of various defects, so that experimenters can quickly and accurately judge possible PLC faults and take corresponding measures.

[0144] CN118071733A improves the YOLOv8 algorithm to build a recognition model for PCB boards; however, the model does not give the detection speed, so the real-time detection needs to be verified, and the model is not lightweight, which is a big problem for actual deployment; CN118247235A improves the YOLOv5 model to identify defects on PCBs. The YOLOv5 model used is slower in detection speed than YOLOv10 and cannot meet the requirements of real-time detection. Compared with the solutions proposed by CN118071733A and CN118247235A, the present invention can provide faster detection speed and higher detection accuracy, and provides a more feasible solution for real-time monitoring of PLC process defects.

[0145] Multiple different target detection models were used to verify on the same data set, and a comparison result table was obtained. Compared with other models, the model of the present invention is much higher in accuracy and speed than other detection models, has a stronger ability to handle PLC defects, has a lower missed detection rate and false positive rate, has strong generalization and robustness, and can meet the requirements of PLC defect detection.

[0146] The following is a table of analysis of the results compared with the original model

[0147] Table 1 Comparative test results

[0148]

[0149] The platform used for the test of the present invention is Pycharm11.0.15, the model framework is Pytorch1.13.1, the CPU is 15vCPU Intel(R)Xeon(R)Platinum 8358P CPU@2.60GHz, and the GPU is RTX3090, 24GB.

[0150] When setting hyperparameters, the number of training rounds was set to 200, the optimizer used was SGD, and the batch size was set to 16.

[0151] Depend on Figure 8 It can be seen that if there is insufficient solder on the PLC industrial control board, it can be accurately detected.

[0152] Depend on Fig. 9 It can be seen that if a welding short circuit occurs on the PLC industrial control board, it can be accurately detected.

[0153] Depend on Fig.10 It can be seen that if there are missing components on the PLC industrial control board, it can be accurately detected.

[0154] In summary, the Efficient-YOLOv10s model constructed by the present invention can quickly and accurately identify the process defects of PLC, reduce the risk of system failure due to various short-circuit defects of PLC, play an important prerequisite role in the orderly operation of the PLC system, and can meet the real-time detection, greatly reducing the failures caused by PLC process defects.

Claims

1. A PLC process defect end-to-end detection method based on Efficient-YOLOv10s, characterized in that: The following steps are involved: Step S1: Use a high-definition camera to capture the PLC process defect dataset images of the PLC laboratory, integrate them with the open source dataset, and screen and organize the images; Step S2: Obtain several PLC process defect data set images through image preprocessing steps such as smoothing filtering, Gaussian filtering, and noise processing; Step S3: Use Labelimg to annotate the dataset in XML format, convert it into a txt file, and divide it into training set, validation set, and test set according to a certain ratio; Step S4: construct and obtain an Efficient-YOLOv10s network model, which includes a backbone network, a neck network, and a head network; Step S5: Put the obtained data set into the Efficient-YOLOv10s model, combine the Efficient-YOLOv10s network model to recognize the acquired PLC process defect detection image, and use the precision recall rate, average precision mean and transmission frames per second as evaluation indicators; Step S6: Use the trained model to perform image prediction on the detection image for accurate recognition.

2. The method according to claim 1, characterized in that In step S1, the following steps are included: Step S1-1: photographing a damaged PLC board in a PLC laboratory using a high-definition camera to obtain an original data set; Step S1-2: Find an open source data set on an open source data website, compare it with the original data set, select high-resolution images to expand the data set to obtain the original data set; In step S2, the image is preprocessed on the pycharm platform. The data preprocessing method includes smoothing filtering, Gaussian filtering, and noise processing to obtain several PLC process defect data set images and the final data set.

3. The method according to claim 1, characterized in that: In step S3, the following steps are included: S3-1: Use labelimg image annotation tool to annotate the collected PLC process defect images. The defects are divided into three categories: insufficient solder, missing component, and solder short circuit. Insufficient solder is "Insufficient solder", missing component is "Missing component", and solder short circuit is "Soldering shorted". Get PascalVOC format data labels, that is, XML tags, and then convert them into txt files suitable for training through data format conversion code written in python. The file includes most of the data, including the x-coordinate, y-coordinate, width and height of the center point of the bounding box. S3-2: After completing the labeling work, divide the data set into training set, validation set, and test set according to a certain ratio.

4. The method according to claim 1, characterized in that In step S4, the backbone network is specifically: The image is input to the first GSConv module, the output of the first GSConv module is connected to the input of the second GSConv module, the output of the second GSConv module is connected to the input of the first C2f module, the output of the first C2f module is connected to the input of the third GSConv module, the output of the third GSConv module is connected to the input of the second C2f module, the output of the second C2f module is connected to the input of the first CSDown module, the output of the first CSDown module is connected to the input of the third C2f module, the output of the third C2f module is connected to the input of the second CSDown module, the output of the second CSDown module is connected to the input of the fourth C2f module, the output of the fourth C2f module is connected to the input of the SPPF module, the output of the SPPF module is connected to the input of the SE module, and the output of the SE module is connected to the neck network; The structure of the neck network is as follows: the output of the SE module is connected to the input of the first Upsample module and the input of the C2fCIB module, the output of the first Upsample module is connected to the input of the first Concat module, the output of the first Concat module is connected to the input of the fifth C2f module, the output of the fifth C2f module is connected to the input of the second Upsample module, the output of the second Upsample module is connected to the input of the second Concat module, the output of the second Concat module is connected to the input of the sixth C3f module, the output of the sixth C3f module is connected to the input of the first v10Detect detection head and the input of the ordinary convolution module, the output of the ordinary convolution module is connected to the input of the third Concat module, the output of the third Concat module is connected to the input of the seventh C2f module, the output of the seventh C2f module is connected to the input of the second v10Detect detection head and the input of the third CSDown module, the output of the third CSDown module is connected to the input of the fourth Concat module, the output of the fourth Concat module is connected to the input of the C2fCIB module, and the output of the C2fCIB module is connected to the input of the third v10Detect detection head; The head network includes the first v10Detect detection head, the second v10Detect detection head, and the third v10Detect detection head.

5. The method according to claim 4, characterized in that The C2fCIB modules are as follows: The C2fCIB module includes a first Conv module, the input of the first Conv module is used to input a feature map, the output of the first Conv module is connected to the Spilt module and the input of the second Conv module, the output of the Spilt module is connected to the first CIB module and the input of the second Conv module, the output of the first CIB module is connected to the second CIB module and the input of the second Conv module, the output of the second CIB module is connected to the third CIB module and the input of the second Conv module, the output of the third CIB module is connected to the Concat module and the input of the second Conv module, the output of the Concat module is connected to the input of the second Conv module, and the output of the second Conv module is used to output features.

6. The method according to claim 4, characterized in that The C2f modules are as follows: The C2f module includes the first Conv module, the Spilt module, the Bottleneck module, the Concat module, and the second Conv module; wherein the Bottleneck module includes n serially connected Bottleneck units, and the output of each Bottleneck unit is connected to the input of the second Conv module; The input of the first Conv module is used to input the feature map, the output of the first Conv module is connected to the Spilt module and the input of the second Conv module, the output of the Spilt module is connected to the Bottleneck module and the input of the second Conv module, the output of the Bottleneck module is connected to the input of the Concat module and the input of the second Conv module, and the output of the second Conv module is used to output features.

7. The method according to claim 4, characterized in that The structure of the GSConv module is: A feature map with a channel number of C1 is input, and a normal convolution operation is performed on the feature map to obtain a first feature map with a channel number of C2 / 2. The first feature map with a channel number of C2 / 2 is input to the DW convolution module to obtain a second feature map with a channel number of C2 / 2. The first feature map and the second feature map are input to the splicing module together, and the output of the splicing module is finally combined with the shuffle operation to obtain a feature map with a channel number of C2.

8. The method according to claim 4, characterized in that The CSDown module is as follows: the input feature x_in is input to the CV1 module, the output of the CV1 module is connected to the input of the CV2 module, and the output of the CV2 module obtains the output feature x_out; Among them, the stride of the CV1 module is 1, the convolution kernel is 1×1, the stride of the CV2 module is 2, the convolution kernel is 3×3, and 1 pixel of padding is added to the edge of the feature map.

9. The method according to claim 4, characterized in that The SE modules are as follows: The SE module includes an Inception module, the input of the Inception module is used to input input features, the output of the Inception module is connected to the input of the Global pooling module and the input of the Scale module, the output of the Global pooling module is connected to the input of the first FC module, the output of the first FC module is connected to the input of the ReLU module, the output of the ReLU module is connected to the input of the second FC module, the output of the second FC module is connected to the input of the Sigmoid module, the output of the Sigmoid module is connected to the input of the Scale module, and the final output features of the SE module are obtained from the output of the Scale module; Among them, the number of channels of the output of the Inception module is H*W*C, the number of channels of the output of the Globalpooling module is 1*1*C, the number of channels of the output of the first FC module is 1*1*C / r, the number of channels of the output of the ReLU module is 1*1*C / r, the number of channels of the output of the second FC module is 1*1*C, and the number of channels of the output of the Sigmoid module is H*W*C.

Images