Visual large model PCB defect detection system based on YOLOv10 deep learning network

Abstract

The invention discloses a visual large model PCB defect detection system based on a YOLOv10 deep learning network. The system comprises an image screening module, a defect area positioning module, a defect category identification module, a defect data integration module and a defect evaluation display module. According to the method, the image of the printed circuit board is acquired through an industrial camera, an image quality evaluation standard is set, a high-quality input image is screened, the influence of interference factors is suppressed, image features are captured by a YOLOv10 network trunk extraction layer, a positioning area is accurately screened by applying the intersection and combination ratio of a prediction frame and a real frame, and the offset between the center of a candidate frame and the position of an anchor frame is calculated. According to the method, the positioning precision is strictly controlled, the false detection rate is effectively reduced, defect area images are cut and classified through YOLOv10 classification branches, and detection results are uniformly mapped into standard defect type identifiers according to informatization processing, so that the uniformity and standardization of detection types are realized, and the PCB defect detection efficiency is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of defect detection technology, and in particular to a visual large-model PCB defect detection system based on the YOLOv10 deep learning network. Background Art

[0002] The field of defect detection technology is a core part of quality control and intelligent manufacturing, focusing on the automatic identification and determination of defects in products during production, assembly, packaging, or use using methods such as image processing, machine vision, and deep learning. This field covers multiple areas, including surface flaw recognition, structural integrity inspection, foreign body detection, and dimensional deviation assessment, and is widely used in high-precision industries such as semiconductors, electronics manufacturing, automobiles, and aerospace. In recent years, with the development of neural networks, especially deep learning models such as convolutional neural networks, attention mechanisms, and Transformer architectures, defect detection has gradually transitioned from traditional rule-based algorithms to end-to-end learning-driven methods, improving detection accuracy, adaptability, and speed.

[0003] The large-scale visual model PCB defect detection system is an automated inspection system built on a large-scale visual model. It provides high-precision and robust automatic identification of common defects in the printed circuit board production process, such as broken wires, short circuits, missing pads, foreign matter adhesion, and uneven trace widths. The system aims to improve PCB manufacturing quality control, enabling rapid and accurate defect screening, reducing the burden and error rate of manual inspection, and adapting to process variations across different board types and manufacturing batches, thereby improving overall production efficiency and product consistency.

[0004] Traditional inspection systems do not strictly consider the differences in input image quality, resulting in a large number of low-definition or high-noise images entering the inspection stage, affecting the accuracy and stability of inspection. Traditional systems rely on simple matching of predicted frames and real frames, and do not strictly limit the center point offset, which can easily cause positioning deviations of the inspection target and misjudgment of defect locations. In the defect classification stage, traditional systems do not clearly perform unified mapping of defect types, resulting in inconsistent classification standards for defect types generated by different production batches or different equipment, making it difficult to form standardized management of data. In the defect information recording and processing stage, traditional systems record the number of defects in a single statistical manner, ignoring the statistics of defect area and relative position. They lack multi-dimensional data analysis capabilities and cannot fully reflect the distribution patterns and risk levels of defects on PCB boards, reducing the accuracy of defect assessment and affecting the effectiveness of product quality control. Summary of the Invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and propose a visual large-model PCB defect detection system based on the YOLOv10 deep learning network.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a visual large-model PCB defect detection system based on the YOLOv10 deep learning network, the system comprising: The image screening module obtains bare board samples of printed circuit boards at the pre-mount station, collects printed circuit board images, screens them according to image quality assessment standards, and establishes a YOLOv10 input image benchmark set; The defect area localization module uses the backbone extraction layer of the YOLOv10 detection network to extract image features based on the YOLOv10 input image benchmark set, compares the intersection-over-union ratio of the predicted box and the true box, screens candidate boxes that meet the boundary overlap condition, and generates a PCB image defect localization box set; The defect category recognition module is based on the PCB image defect positioning frame set, cuts the corresponding image area, inputs it into the YOLOv10 classification branch, extracts the confidence of each category in the classification result, identifies the defect type according to the index value corresponding to the maximum score, and obtains the detection defect type label sequence; The defect data integration module combines the area of each positioning frame, the relative coordinates in the image and the defect type identification into a defect information unit based on the detected defect type label sequence, summarizes and records them one by one in a unified structure table, and forms a PCB image defect annotation data table.

[0007] The present invention has improvements in that the YOLOv10 input image benchmark set includes an image clarity evaluation value, a noise level label, an exposure level mark, and an image quality screening identifier; the PCB image defect positioning frame set includes a target area coordinate point set, a bounding box size vector, a positioning offset identifier, and a candidate frame screening flag; the detected defect type label sequence includes a defect category number sequence, a confidence ranking index, a category mapping result, and a classification model output label; and the PCB image defect annotation data table includes an image number index, a defect combination information set, a defect distribution coordinate cluster, and a type frequency statistical item.

[0008] The present invention is improved in that the image screening module includes: The image acquisition submodule obtains bare board samples from the pre-mounting station of the printed circuit board and captures the bare board image using the industrial camera on the automatic optical inspection system. The image is captured and the image size, frame number, and acquisition time are recorded to establish an image acquisition record data set. The image index extraction submodule calls the brightness matrix in each frame of the image based on the image acquisition record data set, calculates the image clarity index, extracts the mean and standard deviation of the grayscale image to calculate the image signal-to-noise ratio, and calculates the mean grayscale value in the pixel grayscale histogram as the average grayscale value to generate a set of image quality indicators; The image quality screening submodule is based on the image quality index set, and compares the image clarity index, signal-to-noise ratio, and average grayscale value with the set image quality screening threshold set. It screens out image frames that do not meet the requirements and retains the qualified image frame numbers to establish a YOLOv10 input image benchmark set.

[0009] The present invention is improved in that the defect area positioning module includes: The feature extraction submodule calls the multi-layer convolution output in the YOLOv10 detection network backbone structure based on the YOLOv10 input image benchmark set, extracts feature maps at multiple resolutions in the image, generates a feature map tensor set using the number of channels and spatial size combination of each feature map, and establishes a multi-scale feature map combination structure value; The candidate box screening submodule obtains the intersection-and-union ratio between the boundary coordinate information of each predicted box and the actual labeled box based on the combined structure value of the multi-scale feature map, calculates the difference between the intersection area and the union area of each predicted box and the corresponding real box, and screens the predicted box set with an intersection-and-union ratio greater than the intersection-and-union threshold to obtain a set of qualified boundary candidate boxes; The target offset calculation submodule calls the qualified boundary candidate frame set, performs difference calculation based on the center coordinates of each frame and the center coordinates of the matching anchor frame, obtains the normalized center offset value of the detection frame, compares each offset value with the set tolerance threshold item by item, screens out the boundary frames whose offset values exceed the tolerance range, and generates a PCB image defect positioning frame set.

[0010] The present invention is improved in that the defect category identification module includes: The image cropping submodule extracts the image region index of each bounding box based on the PCB image defect positioning frame set, performs region cropping on the original image according to the coordinates of the upper left corner and lower right corner of the bounding box, establishes an image slice matrix set for classification input, and generates a defect image region set; The confidence extraction submodule calls the defect image region set and inputs it into the YOLOv10 classification branch. It extracts the confidence vector of the corresponding category in each channel and sorts them according to the confidence score. It calculates and obtains the normalized difference score value of each type of defect in the sample. It locates the candidate label set according to the index corresponding to the maximum value and obtains the category score ranking vector. The category mapping submodule locates the corresponding type identification number in the six-category defect standard label set according to the category score sorting vector, extracts the corresponding number of each image slice and binds it to the image index, establishes a mapping relationship table, and obtains the detection defect type label sequence.

[0011] The present invention is improved in that the defect data integration module includes: The information combination submodule extracts the pixel area value, center point relative coordinates, and defect type code corresponding to each defect positioning frame based on the detected defect type label sequence, combines the three data items into fields, establishes a defect unit set according to the image number sequence, and generates a defect combination information set; The structure recording submodule establishes a two-dimensional structure matrix for each defect unit in the order of the image based on the defect combination information set, marks the matrix fields in the order of area, coordinates, and type label, fills the records into the matrix in sequence, and establishes the image defect structured matrix value; The statistical sorting submodule calls the image defect structured matrix value, calculates the frequency values of the six types of defects in the matrix according to the type label column, arranges them in descending order according to the frequency values, renumbers the sorting results and appends them to the corresponding type label record items, and establishes a PCB image defect annotation data table.

[0012] The present invention is improved in that the system further comprises: The defect assessment and display module extracts the number and image area of each type of defect based on the PCB image defect annotation data table, calculates the defect density, and calculates the risk assessment value of the PCB board in combination with the defect risk coefficient. The calculated value is output to the detection control terminal in combination with the PCB image defect annotation data table to obtain the PCB defect display result; The PCB defect display results include a defect density ratio set, a defect risk weight coefficient, a weighted calculation total value, and an image risk level result.

[0013] The present invention is improved in that the defect assessment display module includes: The defect density extraction submodule extracts the count value and total image area value corresponding to each defect type based on the PCB image defect annotation data table, divides the number of defects of each type by the image area, converts the result into a defect density value per unit area, and establishes a defect density numerical matrix; The risk indicator calculation submodule calls the defect density numerical matrix, combines the risk coefficient set corresponding to the defect type, and multiplies each density value by the corresponding risk coefficient according to the defect type index. It also introduces the center of gravity coefficient and boundary discreteness of each type of defect position to calculate and obtain the PCB defect risk assessment value; The visual result output submodule forms a multi-field display data packet based on the PCB defect risk assessment value and the sorting results of each type of defects in the PCB image defect annotation data table, imports it into the control end interface standard data format specification, and establishes the PCB defect display result.

[0014] Compared with the prior art, the advantages and positive effects of the present invention are: In the present invention, printed circuit board images are collected by an industrial camera, and image quality assessment standards are established to screen high-quality input images and suppress the influence of interference factors. The YOLOv10 network backbone extraction layer is used to capture image features, and the intersection of the predicted frame and the real frame is used to accurately screen the positioning area. The offset between the center of the candidate frame and the anchor frame position is calculated to strictly control the positioning accuracy and effectively reduce the false detection rate. The defect area image is cropped and classified by the YOLOv10 classification branch. The detection results are uniformly mapped to standard defect type identification based on information processing to achieve uniformity and standardization of detection categories. A defect annotation data table is formed by statistical frequency sorting to facilitate refined defect management. By calculating the defect density and risk assessment value, quantitative and structured risk assessment and output are achieved, and the defect information is intuitively displayed on the detection control end, effectively improving the efficiency of PCB defect detection, reducing interference from human factors, and improving the standardization level of production processes and the stability of manufacturing quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present invention will be further described below with reference to the accompanying drawings and examples.

[0016] Figure 1 This is a system flow chart of a visual large-model PCB defect detection system based on the YOLOv10 deep learning network described in the present invention; Figure 2 This is a flowchart of the image screening module of the visual large-model PCB defect detection system based on the YOLOv10 deep learning network described in the present invention; Figure 3 This is a flow chart of the defect area positioning module of the visual large model PCB defect detection system based on the YOLOv10 deep learning network described in the present invention; Figure 4 This is a flow chart of the defect category recognition module of the visual large model PCB defect detection system based on the YOLOv10 deep learning network described in the present invention; Figure 5 This is a flow chart of the defect data integration module of the visual large-model PCB defect detection system based on the YOLOv10 deep learning network described in the present invention; Figure 6 This is a flowchart of the defect assessment and display module of the visual large-model PCB defect detection system based on the YOLOv10 deep learning network described in the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0018] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings and are only for the convenience of describing the present invention and simplifying the description. They do not indicate or imply that the devices or elements referred to must have a specific direction, be constructed and operate in a specific direction, and therefore should not be understood as limiting the present invention. In addition, in the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0019] See also Figure 1 The present invention provides a technical solution: a visual large model PCB defect detection system based on the YOLOv10 deep learning network, the system comprising: The image screening module obtains bare board samples from the pre-mounting station of printed circuit boards (PCBs). The industrial camera on the automated optical inspection system captures PCB images, extracts image clarity, signal-to-noise ratio, and average grayscale value, and screens them based on image quality assessment criteria to establish a YOLOv10 input image benchmark set. Image clarity is measured by performing a Laplacian transform on the image grayscale image and then calculating its variance. A higher variance value indicates sharper image edges. The signal-to-noise ratio is defined as the ratio of image signal intensity to noise intensity and is used to measure the signal quality in an image. The mean grayscale value is the average of the grayscale values of all pixels in the image and is used to assess image exposure. The defect area localization module uses the YOLOv10 input image benchmark set and the backbone extraction layer of the YOLOv10 detection network to extract image features. It compares the intersection-over-union ratio of the predicted box and the true box, selects candidate boxes that meet the boundary overlap condition, calculates the offset between the center coordinates of each candidate box and the anchor box position, and eliminates detection targets with positioning deviations greater than the tolerance threshold to generate a PCB image defect localization box set. The intersection-over-union ratio (IoU) is the ratio of the intersection area of two bounding boxes to the union area, and is a commonly used accuracy evaluation metric for object detection. The center coordinates are the two-dimensional coordinates of the center point of the detection box on the image plane. The anchor box position is a set of preset bounding box shapes that serve as a reference for the detection model to match the actual target box. The defect category recognition module is based on the PCB image defect positioning frame set. It cuts the corresponding image area and inputs it into the YOLOv10 classification branch. It extracts the confidence score of each category in the classification result, identifies the defect type according to the index value corresponding to the maximum score, and uniformly maps it to the defined six defect type identifiers to obtain the detected defect type label sequence. Category confidence indicates the probability that the detection box belongs to a certain category, ranging from 0 to 1; the maximum score index is the position of the maximum value item in the multi-classification output, that is, the corresponding category number; The defect data integration module combines the area of each positioning frame, the relative coordinates in the image, and the defect type identification into a defect information unit based on the defect type label sequence. The unit is summarized and recorded in a unified structure table, and the frequency is counted and sorted according to the defect type to form a PCB image defect annotation data table. The area is the product of the width and height of the detection frame, and the unit is the number of image pixels. The relative coordinates are the center point coordinates, width and height normalized based on the image size, and the value is between 0 and 1. The frequency statistics are the number of occurrences of each type of defect. The defect assessment and display module extracts the number and image area of each type of defect based on the PCB image defect annotation data table, calculates the defect density, and combines it with the defect risk coefficient to calculate the risk assessment value of the PCB board. Combined with the PCB image defect annotation data table, the module outputs the result to the detection control terminal to obtain the PCB defect display result. Defect density is the number of defects per unit area and is often used to describe the concentration of defects. Defect risk factors are risk weights set for different defect types, for example, the short circuit factor is higher than the rat bite factor. The YOLOv10 input image benchmark set includes image clarity assessment values, noise level labels, exposure level marks, and image quality screening flags. The PCB image defect positioning frame set includes the target area coordinate point set, bounding box size vector, positioning offset mark, and candidate frame screening flag. The detection defect type label sequence includes the defect category number sequence, confidence ranking index, category mapping result, and classification model output label. The PCB image defect annotation data table includes the image number index, defect combination information set, defect distribution coordinate cluster, and type frequency statistics. The PCB defect display results include the defect density ratio set, defect risk weight coefficient, weighted calculation total value, and image risk level result.

[0020] See also Figure 2 , the image filtering module includes: The image acquisition submodule obtains bare board samples from the pre-mounting station of the printed circuit board and captures the bare board image using the industrial camera on the automatic optical inspection system. The image is captured and the image size, frame number, and acquisition time are recorded to establish an image acquisition record data set. The image acquisition submodule obtains the bare board sample of the printed circuit board at the pre-mounting station. The loading device needs to complete the conveying of the PCB bare boards piece by piece, and trigger the industrial camera in the automatic optical inspection system in sequence through the image acquisition controller. Each bare board is driven by the motion platform with the help of the servo motor to accurately position it within the focal length range of the industrial camera. The industrial camera takes images when the exposure time is set to 1.2 milliseconds and the resolution is set to 2448×2048 pixels. At the same time, the size of the captured image is recorded as 5008×3288 pixels, and the frame number and timestamp information are written synchronously by the acquisition controller, such as 202505130001 frame, 2025-05-13 10:23:1 5 seconds, the image file, size parameters, frame number and acquisition time are recorded into the image acquisition record data set according to the data structure storage format. In this record format, the image size field represents the total number of pixels in the horizontal and vertical directions of the image in pixels, the frame number field identifies the unique index of the image in the form of year, month, day + serial number, and the acquisition time field uses the standard UTC format to store time information. The system automatically shoots continuously at a speed of 10 frames per second and continuously acquires PCB bare board images with a period of 60 seconds, forming a total of 600 frames of image data files. The mapping relationship between image numbers and image files is established through the database index system. Each acquisition record can be retrieved and verified in the system through the frame number.

[0021] The image index extraction submodule is based on the image acquisition record data set, calls the brightness matrix in each frame of the image, calculates the image clarity index, extracts the mean and standard deviation of the grayscale image to calculate the image signal-to-noise ratio, and calculates the grayscale mean in the pixel grayscale histogram as the average grayscale value to generate a set of image quality indicators; The image index extraction submodule is based on the image frame data stored in the image acquisition record data set, and calls the brightness matrix in each frame of the image, that is, the grayscale matrix data. The matrix is a two-dimensional integer array with an element range of 0 to 255. The submodule first performs a Laplacian convolution operation, performs high-pass filtering on the grayscale difference of the neighborhood of each pixel position, obtains the edge response image, and calculates the variance value of all its pixels. The variance value is used as the clarity index of the frame image. The higher the index value, the richer the edge information and the higher the edge prominence. Then the submodule extracts the grayscale value mean μ and standard deviation σ from the original image grayscale matrix. The signal-to-noise ratio (SNR) is calculated by the ratio of μ to σ, that is, , where μ represents the average value of all pixel values in the image grayscale matrix, σ represents the square root of the mean of the square of the difference between the pixel value and μ, that is, the standard deviation. If the mean value of the grayscale value of a certain frame is μ=130.2 and the standard deviation σ=25.6, then the frame Finally, the submodule counts the pixel gray value histogram, extracts the gray level distribution, and calculates the average gray value of all gray levels (0-255) in a weighted manner. Assuming that the weighted sum of the gray level of the histogram in this frame is 3313200 and the total number of pixels is 5008×3288=16469280, the average gray value is The three indicators constitute the image quality indicator set and are uniformly associated and saved in the image attribute parameter database according to the frame number. The submodule sets the precision of each image indicator to 0.1 bit and uses floating-point data format encoding.

[0022] The image quality screening submodule is based on a set of image quality indicators. It compares the image clarity index, signal-to-noise ratio, and average grayscale value with the set image quality screening threshold set, filters out image frames that do not meet the requirements, retains the qualified image frames, and establishes a YOLOv10 input image benchmark set; The image quality screening submodule uses the aforementioned image quality index set to call the clarity index, signal-to-noise ratio, and average grayscale value, and compares them with the system's preset image quality screening threshold set item by item to determine whether a frame of image meets the standard. If any of the three indicators does not meet the standard, the image frame is determined to be unqualified for the input benchmark. The system default threshold set is: clarity lower limit variance value , signal-to-noise ratio threshold The upper and lower limits of the average gray value threshold are 、 The above thresholds are set based on the actual data distribution during the imaging process of industrial cameras. By sorting the edge distribution variance in 1000 frames of images, the minimum value of the top 80% was selected as the lower limit reference value. After statistical determination, it was determined that the edge blurring phenomenon was obvious when the variance value was lower than 280, and it was finally set to 300; the signal-to-noise ratio threshold Calculate the sample mean using 100 frames of SNR As the benchmark, the lower limit is set to 90%, that is, 0.9×4.73≈4.26, rounded to 4.5; the upper and lower limits of the average grayscale value are set to a safe range of 90 to 210 by analyzing the phenomenon that the grayscale mean of dark or bright images is lower than 85 and higher than 220 respectively. After excluding overexposed and underexposed images, for example, the image of image frame number 202505130045 has a clarity of 342.1, an SNR of 5.086, and an average grayscale value of 201.2. If all three fall within the threshold range, the frame will be retained; if the signal-to-noise ratio of a frame image is 3.2, which is lower than the SNR threshold, or the average grayscale value is 240, which is higher than the upper limit, the image is considered unqualified and is removed from the input candidate set. The screening operation is performed by executing a Boolean judgment expression on the above three indicators The qualified image frame numbers are added to the YOLOv10 input image benchmark set. The system records the screening retention ratio to evaluate the acquisition quality level. After screening, 512 qualified image frames are retained and 88 frames are eliminated from the total number of sampled frames, with a pass rate of 85.3%.

[0023] Table 1 Comparison of image quality screening parameters and sample frame indicators Table 1 lists the metrics for several image frames, which can be compared to determine whether they meet the input benchmark set screening criteria. The results indicate that some images were rejected due to excessive signal-to-noise ratios or grayscale values, ensuring the clarity and quality consistency of the input images for the subsequent object detection model.

[0024] See also Figure 3 , the defect area positioning module includes: The feature extraction submodule uses the YOLOv10 input image benchmark set and calls the multi-layer convolution output in the YOLOv10 detection network backbone structure to extract feature maps at multiple resolutions in the image. It then generates a set of feature map tensors based on the number of channels and spatial size of each feature map, and establishes a multi-scale feature map combination structure value. The feature extraction submodule is based on the YOLOv10 input image benchmark set. First, the pixel matrix is read from each frame of the image and input into the YOLOv10 backbone detection network structure. The backbone of the network is composed of continuously stacked convolution layers, normalization layers and activation function layers. The low-level detail features and high-level semantic features are extracted from different depth positions respectively. The submodule sequentially calls the convolution output results of the 3rd, 6th and 9th layers as feature maps at three resolutions. If the input image size is set to 640×640, the corresponding feature map sizes are 80×80, 4×40 and 80×80 respectively. 0×40 and 20×20, with the number of channels being 64, 128, and 256 respectively. The submodule constructs each feature map as a tensor in the form of a three-dimensional matrix. For example, a feature map of size 80×80 has a tensor dimension of (80, 80, 64). In this dimension, the first two items are spatial dimensions, and the last item is the number of channels. The three sets of feature map tensors are then combined in a spatial pyramid manner to establish a multi-scale feature map combination structure value for subsequent candidate box prediction operations. This structure value is used to maintain feature information at different levels and enhance the ability to detect small targets.

[0025] The candidate box screening submodule obtains the intersection-and-union ratio between the boundary coordinate information of each predicted box and the actual annotated box based on the combined structural value of the multi-scale feature map. It calculates the difference between the intersection area and the union area of each predicted box and the corresponding real box, and screens the predicted box set with an intersection-and-union ratio greater than the intersection-and-union threshold to obtain a set of qualified boundary candidate boxes. The candidate box screening submodule reads the bounding box set in the network prediction output one by one based on the combined structure value of the multi-scale feature map mentioned above. Each predicted box is represented by four boundary point coordinates. Combined with the corresponding real box coordinates in the annotated dataset, the intersection area and union area are calculated respectively. The intersection area is calculated as the area of the overlapping part of the two boxes in the horizontal and vertical coordinate directions. Let the predicted box coordinates be , the real frame coordinates are , then the horizontal width of the intersection is , the vertical height is , the product is the intersection area, the sum of the predicted box area and the true box area minus the intersection area is the union area, and the intersection-union ratio calculation formula is: ; The closer the IoU value is to 1, the more accurate the boundary prediction is. The submodule sets the IoU threshold to , that is, when When , the predicted box is judged as a qualified boundary candidate box. The threshold setting refers to the typical target detection evaluation standard in the COCO dataset and is determined according to the balance point between accuracy and recall. An IoU greater than 0.5 can ensure that most target bodies are covered in the predicted box. For example, the intersection area of a predicted box and the real box is 2400, and the union area is 3900. , is greater than the threshold, it is retained; otherwise, if , then it is directly screened out. This module traverses all the prediction boxes corresponding to each frame image, establishes a set of candidate boxes that meet the intersection-over-union ratio requirements, and stores them in the qualified boundary candidate box set structure.

[0026] The target offset calculation submodule calls the qualified boundary candidate box set and calculates the difference between the center coordinates of each box and the center coordinates of the matching anchor box using the formula: ; The normalized center offset value of the detection frame is obtained by calculation, and each offset value is compared with the set tolerance threshold one by one. The bounding boxes with offset values exceeding the tolerance range are filtered out to generate a set of PCB image defect positioning frames; Among them, Δ represents the normalized center offset value of the detection box, and is the center coordinate of the prediction box, and is the center coordinate of the anchor box, w and h are the width and height of the anchor box respectively; The target offset calculation submodule reads the set of qualified boundary candidate boxes, processes each predicted box in turn, and calculates the offset distance between its center coordinates and the center coordinates of the corresponding anchor box. The center coordinates of the predicted box are defined as: ; The center coordinates of the anchor box are defined as: ; Its anchor box size is: ; Substituting the above values into the offset calculation formula: ; Set the offset tolerance threshold to , which is obtained based on the statistical error between the model output and the true position in multiple sets of images. The analysis of 500 randomly selected matching box samples shows that the center offset rate Δ is still stable when it is in the range of 0.2 to 0.28. Finally, 0.3 is selected as the upper limit threshold. For example, in a certain frame image, the center coordinates of the predicted box are =(230.5,315.4), the center coordinates of the anchor box are =(226.3,309.8), the anchor box width w=40, height h=32, then: ; , so the predicted box is retained; if the offset value of a predicted box is 0.39, which exceeds the tolerance threshold, it is removed from the set. After the submodule completes all offset value comparisons, it retains all candidate bounding boxes that meet the offset constraints and marks them as the final defect localization boxes. The unified output forms the PCB image defect localization box set for subsequent defect classification processing.

[0027] See also Figure 4 ,The defect category identification module includes: The image cropping submodule extracts the image region index of each bounding box based on the PCB image defect positioning frame set, performs region cropping on the original image according to the coordinates of the upper left and lower right corners of the bounding box, establishes an image slice matrix set for classification input, and generates a defect image region set; The image cropping submodule extracts the coordinate information of each bounding box on the original image based on the PCB image defect positioning frame set, and locates the coordinates of the upper left corner of the box. With the lower right corner coordinates , calculate the width and height of the box as , , the submodule calls the original image matrix, performs the image array index operation in the two-dimensional coordinate plane, cuts out a rectangular area to form a slice matrix, the slice matrix size is h×w, and keeps the number of channels of the original image unchanged. For example, if the original image size is 640×640×3, the coordinates of the defect bounding box are the upper left corner (120,150) and the lower right corner (200,230), then the width of the cropped image area is 80 and the height is 80. The dimension of the cropped image slice matrix is 80×80×3. The submodule names each slice according to the original image frame number and the bounding box sequence number, and stores it in the image slice matrix set to form a structured index, such as "Frame045_Box03.jpg", which is used for subsequent classification branch input. All image slices are uniformly adjusted to the uniform specification size of the input network, such as 128×128, and the size standardization operation is completed through the linear interpolation scaling algorithm. Finally, a defect image region set is generated for use by the YOLOv10 classification submodule.

[0028] The confidence extraction submodule calls the defect image region set, inputs it into the YOLOv10 classification branch, extracts the confidence vector of the corresponding category in each channel, and sorts them according to the confidence score using the formula: ; The operation obtains the normalized difference score value of each type of defect in the sample, locates the candidate label set according to the index corresponding to the maximum value, and obtains the category score ranking vector; in, represents the normalized difference score value of the i-th category, is the original confidence value of the j-th image in the i-th category, represents the mean confidence of the i-th class, represents the standard deviation of the i-th category, n is the number of image samples, is the category sensitivity coefficient of category i, δ is the standard deviation shift constant; The confidence extraction submodule calls the above defect image region set and inputs each image into the classification branch of the YOLOv10 detection network. The network output layer contains six output channels, corresponding to six types of defect labels. Each channel outputs the confidence value that the image belongs to the corresponding category, forming a vector group ,in Represents the original confidence value of the j-th image in the i-th category. After obtaining the confidence vectors of all channels, the submodule calculates the mean of each channel vector to obtain , and calculate the standard deviation ,Furthermore, in order to emphasize the sensitivity of each category in the diversity detection task, the system sets the category sensitivity coefficient , this parameter is set according to the category recognition accuracy and target complexity. For example, a higher weight value is set for the easily confused "short circuit" and "excessive tin" categories. =1.35, =1.25, while easy-to-distinguish categories such as "false solder joints" are set with lower sensitivity coefficients, for example =1.10, the system sets the standard deviation offset constant δ=0.05 to prevent the calculation anomaly of the denominator being zero when the standard deviation is zero. The submodule uses the following formula to calculate the normalized difference score value of the i-th category: ; In this formula, the numerator reflects the weighted sum of the degree of outliers in the confidence of category i, and the denominator is used for normalization. If the standard deviation is large, it means that the recognition result is unstable, and the score decreases after normalization; if the outlier amplitude is large but the fluctuation is small, it means that the current category has obvious discrimination ability, and the corresponding score increases. Taking category 2 defects as an example, if there are n=5 image samples and their confidence values are [0.85, 0.88, 0.91, 0.89, 0.92] respectively, then =0.89, =0.0245, set =1.25, and we can get the following formula: ; The system for all categories The scores are sorted in descending order, the category number corresponding to the maximum score is indexed as the candidate label set, and the score sorting vector is recorded , the final output is used for category mapping.

[0029] The category mapping submodule sorts the vector according to the category score, locates the corresponding type identification number in the six-category defect standard label set, extracts the corresponding number of each image slice and binds it to the image index, establishes a mapping relationship table, and obtains the detected defect type label sequence.

[0030] The category mapping submodule reads the aforementioned category score sorting vector, selects the defect category number corresponding to the index position with the maximum value, and searches the system's preset six-category defect label set for a matching table of standard category numbers and Chinese labels, such as "0-false solder joint," "1-missing solder joint," "2-short circuit," "3-excessive tin," "4-insufficient tin," and "5-device offset." The current image slice is bound to this number label and stored in a mapping relationship table along with the original image index. A three-column mapping structure is established: "image number," "candidate number," and "category label." The system uses the frame number and box sequence number in the slice matrix naming rule as the primary key to construct a label sequence structure. For example, the final label of the image "Frame045_Box03.jpg" is "2-short circuit," and its index structure is recorded as "Frame045_Box03, 2, short circuit." The system completes label mapping for all slices and outputs the final detected defect type label sequence for the data integration submodule to call.

[0031] See also Figure 5 , the defect data integration module includes: The information combination submodule extracts the pixel area value, center point relative coordinates, and defect type code corresponding to each defect positioning frame based on the detected defect type label sequence. It then combines the three data items into fields, establishes a defect unit set according to the image number sequence, and generates a defect combination information set. The information combination submodule reads the position information of each defect positioning box on the original image and the corresponding classification results one by one based on the defect type label sequence. First, the area value of each bounding box is calculated. The area calculation method is the product of the width and height of the bounding box, that is, , in pixels. For example, if the upper left corner of a bounding box is (120,180) and the lower right corner is (200,260), then the width is 80, the height is 80, and the area is A=80⋅80=6400 pixels. Then get the coordinates of the center point of the box. Calculated as: ; For example, the center coordinates of the bounding box above are (160, 220). The center coordinates are normalized based on the image size (W, H) = (640, 640). The normalized center coordinates are: ; Then extract the defect type code corresponding to the image slice, such as "2" represents "short circuit", and finally combine the three data fields of area 6400, relative coordinates (0.25, 0.3438), and type code "2" into a group of defect information units, which are expressed as a structure format. , where L is the defect label number. All defect information units are indexed by the image frame number and arranged in order to form a defect combination information set. The data type structure is a mixture of floating point and integer types, corresponding to area (int), relative coordinates (float), and category label (int), respectively. The system unifies the field format and adds structure validity verification to ensure that the range of each field falls within the defined interval to avoid abnormal value disturbance.

[0032] The structure record submodule establishes a two-dimensional structure matrix for each defect unit in the order of the image based on the defect combination information set, annotates the matrix fields in the order of area, coordinates, and type label, fills the records into the matrix in sequence, and establishes the image defect structured matrix value; The structure record submodule reads the above defect combination information set and generates a three-column two-dimensional structure matrix for each information unit. Each row corresponds to a defect instance, and each column of the matrix contains the area, relative coordinates, and category label respectively. The first row of the structured matrix is the header in the format of A, (x, y), L. If an image contains three defect instances, the corresponding matrix dimension is 3×3. Each record item is filled in the order of the image number. For example, if the frame number is 202505130045, the corresponding matrix record is as follows: ; Each row of the matrix record is a structural description item of a single defect. The system stores the structured matrices of all images in series in the image defect structured matrix value set. Each frame of image data has a unique index to facilitate data retrieval and batch analysis. The structure record submodule also sets the matrix field data type and width. The area field is a 32-bit integer, the relative coordinate field is a 2×32-bit floating point type, and the category label field is an 8-bit unsigned integer to ensure structural standardization and facilitate subsequent calls in database systems or AI models.

[0033] The statistical sorting submodule calls the image defect structured matrix value, calculates the frequency values of the six types of defects in the matrix according to the type label column, sorts them in descending order according to the frequency values, renumbers the sorting results and appends them to the corresponding type label record items to establish the PCB image defect annotation data table; The statistical sorting submodule calls the image defect structured matrix value, performs frequency statistics on the "type label" column of all defect records, scans all records in turn and constructs six categories of counters , whenever the type tag value of a record is L=i, the corresponding The value is increased by one, and after the traversal is completed, the frequency sets of six types of defects are formed, for example: ; Sort this frequency set in descending order, and the new order is , corresponding numbers are 3, 1, 5, 0, 2, and 4. The system matches the type label value of each structured matrix record item and adds a sorting number field, that is, the original label value L is appended with its position number in the frequency sorting. For example, the label value "3" corresponds to the maximum frequency and the sorting number is "0", and the label value "4" corresponds to the minimum frequency and the sorting number is "5". The system generates a new structure table containing four columns: "area", "relative coordinates", "label", and "sorting number". It is named PCB image defect annotation data table, which fully records the defect information and its classification priority as the basic data for statistical analysis, risk assessment, and control strategy generation. See also Figure 6 , the defect assessment display module includes: The defect density extraction submodule extracts the count value and total image area value corresponding to each defect type based on the PCB image defect annotation data table. The number of defects of each type is divided by the image area to convert the value into the defect density per unit area and establish a defect density numerical matrix. The defect density extraction submodule is based on the PCB image defect annotation data table. First, the cumulative number of defects of each type and the size information of the corresponding image are read. The total area of the image is calculated by multiplying the length and width of the image by the number of pixels. Assuming the image resolution is W = 640 pixels and H = 640 pixels, the image area is A = 640 × 640 = 409600 pixels. The number of occurrences of each number in the defect label column is counted. The count of "short circuit" type (number 2) is set to 19 times, the count of "false solder joint" type (number 0) is set to 24 times, and the count of "excessive tin" type (number 3) is set to 41 times. The corresponding density values are calculated as follows: ; ; ; The submodule uses the defect type number as the index to create a density value structure array , the unit is "number of defects / pixel", all numerical precisions are retained to level to ensure slight difference comparison under different risk weight conditions, and finally generate a defect density numerical matrix for risk value calculation.

[0034] The risk indicator calculation submodule calls the defect density numerical matrix, combines the risk coefficient set corresponding to the defect type, and multiplies each density value by the corresponding risk coefficient according to the defect type index. It also introduces the center of gravity coefficient and boundary discreteness of each type of defect position, using the formula: ; Calculate and obtain PCB defect risk assessment value; Among them, R represents the PCB defect risk assessment value, is the density value of the j-th type defect, is the risk coefficient of the jth category, is the center of gravity position coefficient of the j-th type defect area, is the boundary dispersion coefficient of the jth type of defect, and m represents the total number of defect types; The risk index calculation submodule calls the aforementioned defect density numerical matrix and introduces the defect type risk coefficient set. The risk coefficients corresponding to the six types of defects are arrays Among them, the "short circuit" setting coefficient is the highest, which is 2.0. Because of its serious impact, it is easy to cause functional interruption. The risk coefficient setting is derived from the process failure rate weight table. The actual impact weight of the "short circuit" category on the PCB life is 0.32. Based on the average impact weight of all defects of 0.16, it is converted into a proportional multiple and rounded to 1 decimal place, resulting in 2.0 as the weight parameter. At the same time, the center of gravity position coefficient of each type of defect is introduced. and boundary dispersion coefficient ,in It is defined as the average value of the Euclidean distance from the center point of all defects to the center point of the image normalized to the interval [0, 1]. Represents the normalized value of the boundary contour length variance of all defect boundary points. For example, for the "excessive tin" defect, =0.42, =0.37, the submodule calculates the normalized dispersion influence value as follows: ; The density value of all six types of defects multiplied by the risk coefficient is added to the normalized discreteness, and the total is accumulated item by item. Finally, the overall risk value of the PCB image is calculated using the following formula: ; For example, bring in some calculation results: ; The sum of all six items gives an overall risk assessment value, for example, R=0.7821, based on which the system can classify risk levels.

[0035] The visual result output submodule forms a multi-field display data package based on the PCB defect risk assessment value and the sorting results of each type of defects in the PCB image defect annotation data table, imports it into the control end interface standard data format specification, and establishes the PCB defect display results.

[0036] The visual result output submodule calls the risk assessment value R and the defect sorting number and original image frame number contained in the image defect annotation data table. The field content contains five data structures: "image number, defect type, risk level, defect density, and sorting number". For example, the image number "202505130045" contains the defect type "short circuit", the sorting number is 2, and the density is , the risk level is set to "medium", the system stratifies R by setting the risk value interval, and the interval is set to "low: ,middle: ,high: "Judge its level, combine the result with the structured image annotation item, encapsulate it into a standard data package, and import it into the JSON format template of the control end interface. Each field is set as a fixed-length character code and integrated into a visual data interface output item that can be parsed by the upper system to complete the generation of PCB defect display results.

[0037] The above are merely preferred embodiments of the present invention and do not limit the present invention in any other form. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A visual large-model PCB defect detection system based on YOLOv10 deep learning network, characterized by: The system comprises: The image screening module obtains bare board samples of printed circuit boards at the pre-mount station, collects printed circuit board images, screens them according to image quality assessment standards, and establishes a YOLOv10 input image benchmark set; The defect area localization module uses the backbone extraction layer of the YOLOv10 detection network to extract image features based on the YOLOv10 input image benchmark set, compares the intersection-over-union ratio of the predicted box and the true box, screens candidate boxes that meet the boundary overlap condition, and generates a PCB image defect localization box set; The defect category recognition module is based on the PCB image defect positioning frame set, cuts the corresponding image area, inputs it into the YOLOv10 classification branch, extracts the confidence of each category in the classification result, identifies the defect type according to the index value corresponding to the maximum score, and obtains the detection defect type label sequence; The defect data integration module combines the area of each positioning frame, the relative coordinates in the image and the defect type identification into a defect information unit based on the detected defect type label sequence, summarizes and records them one by one in a unified structure table, and forms a PCB image defect annotation data table.

2. The visual large model PCB defect detection system based on the YOLOv10 deep learning network according to claim 1 is characterized in that: The YOLOv10 input image benchmark set includes an image clarity assessment value, a noise level label, an exposure level mark, and an image quality screening flag. The PCB image defect positioning frame set includes a target area coordinate point set, a bounding box size vector, a positioning offset identifier, and a candidate frame screening flag. The detected defect type label sequence includes a defect category number sequence, a confidence ranking index, a category mapping result, and a classification model output label. The PCB image defect annotation data table includes an image number index, a defect combination information set, a defect distribution coordinate cluster, and type frequency statistics.

3. The visual large model PCB defect detection system based on the YOLOv10 deep learning network according to claim 1 or 2, characterized in that: The image screening module includes: The image acquisition submodule obtains bare board samples from the pre-mounting station of the printed circuit board and captures the bare board image using the industrial camera on the automatic optical inspection system. The image is captured and the image size, frame number, and acquisition time are recorded to establish an image acquisition record data set. The image index extraction submodule calls the brightness matrix in each frame of the image based on the image acquisition record data set, calculates the image clarity index, extracts the mean and standard deviation of the grayscale image to calculate the image signal-to-noise ratio, and calculates the mean grayscale value in the pixel grayscale histogram as the average grayscale value to generate a set of image quality indicators; The image quality screening submodule is based on the image quality index set, and compares the image clarity index, signal-to-noise ratio, and average grayscale value with the set image quality screening threshold set. It screens out image frames that do not meet the requirements and retains the qualified image frame numbers to establish a YOLOv10 input image benchmark set.

4. The visual large model PCB defect detection system based on the YOLOv10 deep learning network according to claim 1 or 3, characterized in that The defect area positioning module includes: The feature extraction submodule calls the multi-layer convolution output in the YOLOv10 detection network backbone structure based on the YOLOv10 input image benchmark set, extracts feature maps at multiple resolutions in the image, generates a feature map tensor set using the number of channels and spatial size combination of each feature map, and establishes a multi-scale feature map combination structure value; The candidate box screening submodule obtains the intersection-and-union ratio between the boundary coordinate information of each predicted box and the actual labeled box based on the combined structure value of the multi-scale feature map, calculates the difference between the intersection area and the union area of each predicted box and the corresponding real box, and screens the predicted box set with an intersection-and-union ratio greater than the intersection-and-union threshold to obtain a set of qualified boundary candidate boxes; The target offset calculation submodule calls the qualified boundary candidate frame set, performs difference calculation based on the center coordinates of each frame and the center coordinates of the matching anchor frame, obtains the normalized center offset value of the detection frame, compares each offset value with the set tolerance threshold item by item, screens out the boundary frames whose offset values exceed the tolerance range, and generates a PCB image defect positioning frame set.

5. The visual large model PCB defect detection system based on the YOLOv10 deep learning network according to claim 1 or 4, characterized in that: The defect category identification module includes: The image cropping submodule extracts the image region index of each bounding box based on the PCB image defect positioning frame set, performs region cropping on the original image according to the coordinates of the upper left corner and lower right corner of the bounding box, establishes an image slice matrix set for classification input, and generates a defect image region set; The confidence extraction submodule calls the defect image region set and inputs it into the YOLOv10 classification branch. It extracts the confidence vector of the corresponding category in each channel and sorts them according to the confidence score. It calculates and obtains the normalized difference score value of each type of defect in the sample. It locates the candidate label set according to the index corresponding to the maximum value and obtains the category score ranking vector. The category mapping submodule locates the corresponding type identification number in the six-category defect standard label set according to the category score sorting vector, extracts the corresponding number of each image slice and binds it to the image index, establishes a mapping relationship table, and obtains the detection defect type label sequence.

6. The visual large model PCB defect detection system based on the YOLOv10 deep learning network according to claim 1 or 5, characterized in that: The defect data integration module includes: The information combination submodule extracts the pixel area value, center point relative coordinates, and defect type code corresponding to each defect positioning frame based on the detected defect type label sequence, combines the three data items into fields, establishes a defect unit set according to the image number sequence, and generates a defect combination information set; The structure recording submodule establishes a two-dimensional structure matrix for each defect unit in the order of the image based on the defect combination information set, marks the matrix fields in the order of area, coordinates, and type label, fills the records into the matrix in sequence, and establishes the image defect structured matrix value; The statistical sorting submodule calls the image defect structured matrix value, calculates the frequency values of the six types of defects in the matrix according to the type label column, arranges them in descending order according to the frequency values, renumbers the sorting results and appends them to the corresponding type label record items, and establishes a PCB image defect annotation data table.

7. The visual large-model PCB defect detection system based on the YOLOv10 deep learning network according to any one of claims 1 to 6, characterized in that: The system further comprises: The defect assessment and display module extracts the number and image area of each type of defect based on the PCB image defect annotation data table, calculates the defect density, and calculates the risk assessment value of the PCB board in combination with the defect risk coefficient. The calculated value is output to the detection control terminal in combination with the PCB image defect annotation data table to obtain the PCB defect display result; The PCB defect display results include a defect density ratio set, a defect risk weight coefficient, a weighted calculation total value, and an image risk level result.

8. The visual large model PCB defect detection system based on the YOLOv10 deep learning network according to claim 7 is characterized in that: The defect assessment display module includes: The defect density extraction submodule extracts the count value and total image area value corresponding to each defect type based on the PCB image defect annotation data table, divides the number of defects of each type by the image area, converts the result into a defect density value per unit area, and establishes a defect density numerical matrix; The risk indicator calculation submodule calls the defect density numerical matrix, combines the risk coefficient set corresponding to the defect type, and multiplies each density value by the corresponding risk coefficient according to the defect type index. It also introduces the center of gravity coefficient and boundary discreteness of each type of defect position to calculate and obtain the PCB defect risk assessment value; The visual result output submodule forms a multi-field display data packet based on the PCB defect risk assessment value and the sorting results of each type of defects in the PCB image defect annotation data table, imports it into the control end interface standard data format specification, and establishes the PCB defect display result.

9. The visual large-model PCB defect detection system based on the YOLOv10 deep learning network according to claim 5, characterized in that: The category mapping submodule reads the aforementioned category score sorting vector, selects the defect category number corresponding to the maximum index position, searches for a matching table of standard category numbers and Chinese labels in the system's preset six-category defect label set, binds the current image slice to the number label, and stores it together with the original image index in a mapping relationship table. A three-column mapping structure is established for the fields "image number", "candidate number", and "category label". The system uses the frame number and frame sequence number in the slice matrix naming rule as the primary key to construct a label sequence structure. The system completes the label mapping of all slices and outputs the final detection defect type label sequence for call by the data integration submodule.

10. The visual large model PCB defect detection system based on the YOLOv10 deep learning network according to claim 8, characterized in that: The visual result output submodule calls the risk assessment value and the defect sorting number and original image frame number contained in the image defect annotation data table. The field content contains five data structures: "image number, defect type, risk level, defect density, and sorting number". The result is combined with the structured image annotation item, uniformly encapsulated into a standard data packet, and imported into the JSON format template of the control end interface. Each field is set to a fixed-length character code and integrated into a visual data interface output item that can be parsed by the upper system to complete the generation of PCB defect display results.

Images