A large-scale model-based AOI visual inspection optimization method for the electronics industry

Abstract

The present invention provides an optimization method for AOI visual inspection in the electronics industry based on a large model, aiming to improve the flexibility, accuracy and efficiency of automatic optical inspection. The method comprises: collecting an inspection image of the target to be inspected; identifying the edge area of ​​the target object, dynamically adjusting the segmentation parameters according to the size, shape and position characteristics, and dividing the inspection image into multiple image segments; filtering and scoring the segments through a deep convolutional neural network to screen out the segments containing the inspection object; further processing the screened segments to form an array display; combining multiple weight coefficients with the position association between the segments to generate a possible defect value; and finally performing an overall evaluation based on the possible defect value to indicate the defect situation of the inspection object. The present invention significantly improves the detection accuracy and robustness of the system in complex and diverse scenarios by introducing adaptive segmentation, multi-parameter weighting and spatial correlation mechanisms, and is suitable for efficient and accurate automatic optical inspection in the electronics industry.

Description

Technical Field

[0001] The present invention relates to an automatic optical inspection (AOI) technology for the electronics industry, in particular to an AOI visual inspection optimization method based on a large model. Background Art

[0002] In the electronics manufacturing industry, Automatic Optical Inspection (AOI) technology has become a core tool for quality control in the production of high-precision electronic components and circuit boards. AOI systems primarily rely on optical imaging technology and image processing algorithms to quickly and accurately inspect complex targets such as electronic devices and printed circuit boards (PCBs), detecting defects such as poor solder joints, missing components, misaligned components, and cracks. With the increasing miniaturization and high-density packaging of electronic devices, the requirements for the accuracy, efficiency, and intelligence of AOI inspection technology are becoming increasingly demanding. However, traditional AOI systems still face numerous technical bottlenecks when responding to complex and diverse inspection needs.

[0003] Traditional AOI inspection systems primarily rely on rule-based or traditional machine vision-based inspection methods. These methods typically manually define image features during the image preprocessing phase and screen for expected defects and anomalies in the image through algorithms such as feature extraction and edge detection. While this approach meets basic inspection needs to a certain extent, when faced with a wide variety of components and changing inspection requirements, traditional algorithms struggle to adapt to the varying characteristics of diverse inspection objects, often limiting inspection accuracy and flexibility. For electronic components with complex and varied shapes, sizes, colors, and materials, rule-based inspection solutions are increasingly becoming inadequate, resulting in high rates of missed and false detections. Summary of the Invention

[0004] In order to solve the above problems in the prior art, the present invention proposes an AOI visual inspection optimization method for the electronics industry based on a large model, comprising the following steps:

[0005] Step 1: Collect the detection image of the target to be measured;

[0006] Step 2: In the acquired inspection image, the edge area of ​​the target object is identified, and the image segmentation parameters are dynamically adjusted according to the size, shape, and position characteristics of the object. The inspection image is divided into multiple image segments, where each image segment corresponds to a selected section of the inspection image, so that the segmented image segments contain the characteristics of the target object;

[0007] Step 3: Use a large model based on a deep convolutional neural network to filter and score multiple image clips to determine whether each image clip contains part of the detection object;

[0008] Step 4: The image segments that are determined to contain the detection object are input into the neural network for further processing, while the image segments that do not contain the detection object are removed to form a corresponding array display;

[0009] Step 5: Using multiple weight coefficients and combining the positional associations between segments, a possible defect value for each image segment is automatically generated to indicate the possibility that the image segment contains defects;

[0010] Step 6: Generate an overall defect assessment of the inspection image based on the defect probability value of each image segment to indicate the defect status of the inspection object.

[0011] The detection image is collected by a high-resolution imaging device, and the imaging device includes an industrial camera, a microscope camera or a high-definition camera.

[0012] The process of dynamically adjusting the image segmentation parameters includes: determining the edge features of the target object based on the edge detection algorithm, and automatically adjusting the size and segmentation direction of the segmentation window according to the size and shape features to adapt to the specific form of the detected object.

[0013] The process of filtering and scoring the image segments includes: extracting feature maps of the segments using multiple convolutional layers and pooling layers, filtering the segments according to the feature response values ​​of each image segment, and removing segments that do not contain target object features.

[0014] The scoring process for the image segment includes: generating a comprehensive score S based on the feature response values ​​of multiple convolutional layers to reflect the possibility of defects in the image segment. The comprehensive score is calculated as follows:

[0015]

[0016] Among them, L is the total number of convolutional layers, A l is the activation value of the feature map of layer l, w l is the weighting coefficient of each layer.

[0017] In step 5, the defect probability value P of each segment is calculated by combining the weighted parameter and the spatial correlation. The calculation formula of the defect probability value P is as follows:

[0018] P=σ(S ′ )

[0019] Among them, S ′ is the final defect score after spatial correlation correction, σ is a normalization function used to standardize the possible defect value to the range of 0 to 1, and

[0020] The spatial correlation calculation between segments includes identifying adjacent segments and correcting the defect score of each segment by calculating the correlation of the feature responses of adjacent segments. The specific correlation coefficient α k The calculation is as follows:

[0021]

[0022] Among them, S is the score of the current segment, S k is the preliminary score of adjacent segments, and the correlation coefficient α k Used to indicate the spatial diffusion characteristics of defects.

[0023] In step 6, the system generates an overall defect score S based on the aggregated results of the possible defect values. total , the weighted average of the overall ratings is calculated as follows:

[0024]

[0025] Where n is the total number of image clips, P i is the possible defect value of the i-th segment, w i is the weight coefficient of the i-th segment.

[0026] After the overall flaw score is generated, the system strengthens the clips with higher flaw scores. For high-risk clips, the revised score P i ′ It is expressed as follows:

[0027] P′ i =α·P i

[0028] Among them, α is the amplification factor, which is used to increase the score of high-risk fragments to reflect their severity.

[0029] The overall defect assessment result includes the specific location of the defect, a description of the defect possibility, and a description of the severity.

[0030] Beneficial Effects: The present invention provides a large-scale model-based AOI visual inspection optimization method for the electronics industry. By introducing adaptive segmentation, deep convolutional neural networks, and a multi-parameter weighting mechanism, it can accurately identify the inspection object and dynamically adjust the segmentation parameters, thereby improving the flexibility and accuracy of inspection. Through fragment filtering and scoring, spatial correlation calculation, and overall defect assessment, it significantly reduces misjudgments and missed detections, and improves the robustness and inspection efficiency of the system. Especially in the complex and diverse inspection scenarios of the electronics industry, the method of the present invention can fully adapt to a variety of defect types and complex noise environments, achieving efficient and accurate automatic optical inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application, but do not constitute an improper limitation of the present invention. In the drawings:

[0032] Figure 1 A flow chart of the method of the present invention is shown. DETAILED DESCRIPTION

[0033] The following will be combined with the Figure 1 The present invention is described in detail with reference to the accompanying drawings and specific embodiments, wherein the exemplary embodiments and descriptions are only used to explain the present invention but are not intended to limit the present invention.

[0034] This invention provides a large-scale model-based optimization method for AOI visual inspection in the electronics industry to address the current issues of insufficient defect recognition accuracy, low detection flexibility, and poor real-time performance in AOI inspection. By acquiring images of the target to be inspected and applying a deep convolutional neural network based on the large model, combined with adaptive image segmentation and a multi-parameter weighting strategy, the invention performs high-precision, dynamically adjusted defect detection on the target. Specifically, the method includes the following steps:

[0035] Step 1: Collect the detection image of the target

[0036] Before the inspection process begins, high-resolution imaging equipment is used to capture images of the electronic component or circuit board under test. Such imaging equipment, including but not limited to industrial cameras, microscope cameras, and high-definition video cameras, ensures that the fine structure and surface features of the target are captured, resulting in high-quality inspection images. The imaging equipment is preferably mounted on an adjustable stand or robotic arm to allow for flexible adjustment of its angle, distance, and position as needed to optimize the imaging effect.

[0037] During image acquisition, the system automatically adjusts the camera's imaging parameters to suit the material characteristics and surface conditions of the target object. These include, but are not limited to, brightness, contrast, focal length, aperture size, exposure time, and white balance. Specifically, for targets with smooth or highly reflective surfaces, the system can reduce brightness and exposure time to minimize reflective interference. For darker targets or those with rich textures, the system can increase contrast and focal length to enhance image clarity and detail. Furthermore, the system uses autofocus to ensure optimal image clarity for subsequent analysis and processing.

[0038] After image acquisition is complete, the system transmits the image data to the image processing module for subsequent steps such as image segmentation, feature extraction, and defect analysis. Through the above image acquisition steps, the present invention can obtain high-resolution, clear, and detailed inspection images, providing a reliable imaging foundation for subsequent inspection processes.

[0039] Step 2: Identify the edge area of ​​the target object and perform adaptive image segmentation

[0040] In the captured detection image, the present invention first identifies the edge area of ​​the target object in order to accurately locate the area to be detected and ensure that the key features in the detection image are effectively extracted. The edge recognition process of the target object is based on the deep feature extraction capability of the large model. By analyzing the edge and morphological features in the image, the specific position of the target object in the image is determined and its edge contour is clearly extracted. Based on the results of edge recognition, the system will dynamically adjust the image segmentation parameters to adapt to the size, shape and position characteristics of the target object, thereby dividing the detection image into multiple image segments containing specific information.

[0041] The specific implementation steps of adaptive image segmentation include the following:

[0042] The system first performs edge detection on the inspection image to obtain the edge feature map of the inspection object. Edge detection uses image gradient algorithm to ensure the accuracy and clarity of edge recognition.

[0043] In edge detection, the system calculates the gradient value G of each pixel in the horizontal direction (x direction) and the vertical direction (y direction) x and G y , and thus obtain the edge gradient amplitude G of the pixel point:

[0044]

[0045] The edge strength is determined based on the gradient magnitude, thereby generating an edge map.

[0046] The output of edge detection is a binary edge map that can clearly show the outline and morphological features of the target object, providing a reliable basis for subsequent size and shape analysis.

[0047] Based on the edge map, the system analyzes the size and shape of the target object to determine the parameters required for adaptive segmentation. First, the system extracts the contour lines in the edge map using image contour tracking algorithms (such as contour detection based on connected regions). Combined with computer vision-based geometric feature extraction algorithms, the system calculates morphological features such as the object's area, bounding rectangle, and principal orientation.

[0048] Use the connected component algorithm to obtain all closed or open contours in the edge graph and calculate the bounding box of each contour. The bounding box provides the preliminary size of the target object.

[0049] The overall size of the target object is estimated by the width, height, and area of ​​the bounding box. In addition, by calculating the main axis direction of the circumscribed rectangle, the main orientation and shape characteristics of the target object can be further determined, thus assisting segmentation.

[0050] The system also incorporates a large-scale deep convolutional neural network, analyzing the morphological features of the target object through multi-layer convolution to obtain richer structural information. The features extracted by the deep network can identify the detailed features and texture information within the target object, thus supplementing the deficiencies of traditional geometric features and ensuring more accurate segmentation.

[0051] After completing the size and shape analysis, the system automatically adjusts the image segmentation parameters according to the specific characteristics of the target object to achieve adaptive image segmentation. The main technical steps in the adaptive segmentation process are as follows:

[0052] Segmentation parameter settings: Based on the bounding box size and principal axis orientation of the target object, the system automatically sets the size, number, and orientation of the segmented fragments, ensuring that the resulting segments better fit the shape and distribution of the target object. For example, for long, rectangular objects, the system prioritizes segmentation along the principal axis to maximize the inclusion rate of the segmented fragments.

[0053] Multi-scale segmentation: The system applies a multi-scale segmentation strategy when needed, using larger segments for larger objects and smaller segments for smaller ones. This multi-scale segmentation ensures that detailed features of the target object are captured in the image segments, effectively improving detection accuracy.

[0054] Segmentation: The image is segmented using a sliding window algorithm. The step size and size of the sliding window are set to generate segmented segments that match the characteristics of the target object. Each image segment is fed into the subsequent inspection steps as an independent unit for fine-grained defect identification.

[0055] After segmentation, each image segment is independently processed as a region of the target object and proceeds to the subsequent detection steps. The system uses adaptive segmentation to ensure that the image segments accurately contain the target object's characteristic regions, avoiding over- or under-segmentation. This segmentation approach maintains the integrity of the target object information in the segmented segments while reducing the generation of segments containing irrelevant information, thereby improving the accuracy and efficiency of subsequent detection steps.

[0056] Through adaptive segmentation technology, the present invention can flexibly adapt to target objects of varying sizes and shapes, making the image segmentation process highly adaptable. In traditional fixed-window segmentation methods, the variable shape of the target object can easily lead to inaccurate segmentation areas, thereby affecting the reliability of detection. The present invention uses adaptive segmentation to perform targeted segmentation of target objects of varying shapes and sizes, ensuring that the segmented segments more completely contain target information. This adaptability not only improves the accuracy of subsequent detection but also reduces unnecessary computational overhead to a certain extent, providing a reliable imaging foundation for subsequent detection steps.

[0057] Step 3: Filter and score the image clips based on a large model of deep convolutional neural networks

[0058] After image segmentation, the present invention uses a deep convolutional neural network (CNN) within the larger model to process each image segment. Through feature extraction, segment filtering, and defect scoring, the method determines whether each segment contains part of the inspection object and its defect probability. This step specifically involves image segment feature extraction, segment filtering, and defect scoring.

[0059] 1. Image segment feature extraction

[0060] The system inputs each image segment into a pre-trained large model, which consists of multiple convolutional layers, pooling layers, and fully connected layers, and can extract feature information from the segment layer by layer. The feature extraction process is as follows:

[0061] Convolutional layer processing: The image fragment first passes through a series of convolutional layers, using convolution kernels (i.e. filters) to perform convolution operations on the image fragment to extract local features. The convolutional layer calculation method is as follows:

[0062]

[0063] Among them, f i,j is the pixel value of the output feature map after the convolution operation, I i+m,j+n Represents the local pixel value of the image segment, K m,n Represents the weight of the convolution kernel. Each convolution layer can produce multiple feature maps to capture different characteristics of the image segment.

[0064] Activation function: The feature map after each convolution operation will be processed by a nonlinear activation function (such as the ReLU function) to set the negative values ​​in the feature map to zero, thereby improving the nonlinear representation ability of the model:

[0065] f(x)=max(0,x)

[0066] This activation operation helps the model more accurately distinguish the edges and textures of detected objects during feature extraction.

[0067] Pooling layer processing: The pooling layer is connected after the convolution layer to reduce the resolution of the feature map through downsampling operations, reduce the amount of calculation and increase the robustness of the features. The pooling layer usually uses max pooling or average pooling:

[0068]

[0069] Among them, f pool : Pooling layer sampling results;

[0070] The results of the pooling layer are used to generate a simplified feature representation that helps identify global features of image segments.

[0071] Fully connected layer: After multiple layers of convolution and pooling, the feature maps are flattened into vectors and fed into fully connected layers. These layers integrate the feature map information to capture the global characteristics of the image segment. The output vector of the fully connected layer contains high-level features of the image segment, which are used for subsequent filtering and scoring.

[0072] 2. Fragment filtering

[0073] After feature extraction, the system filters the image segments based on the feature maps generated by the large model to reduce the amount of computation. The filtering process aims to identify and remove segments that do not contain the detected object, allowing the system to focus on segments that do contain the detected object for more in-depth analysis.

[0074] Feature response threshold: The system determines whether the image segment contains the detection object by calculating the global response value of the feature map. If the feature response value is lower than the set threshold, it is considered that the segment does not contain the target feature and the system will remove it. The response value is calculated as follows:

[0075]

[0076] Among them, R represents the characteristic response value, f i is the pixel value in the feature map, and N is the total number of pixels in the feature map. The higher the feature response value, the more likely it is to contain the features of the detected object.

[0077] Fragment removal: For image fragments with feature response values ​​lower than the threshold, the system will remove them from the subsequent processing flow, thereby reducing the computational complexity of irrelevant fragments and improving detection efficiency.

[0078] 3. Defect Scoring

[0079] For selected image segments containing the inspection object, the system further calculates a score to determine the potential defect characteristics within the segment. The defect score is based on the feature response values ​​of the large model, calculated by combining the activation results of the feature maps of different convolutional layers, to reflect the likelihood of a segment being defective.

[0080] Feature map activation: In the multi-layer convolution of the model, different convolution layers have different sensitivities to features of different scales and types. The system calculates the probability of defect features in the image segment based on the activation level of the feature map of each layer. Assume that the activation value of the feature map of layer l is A l , the system calculates the comprehensive response value S:

[0081]

[0082] Among them, L is the number of convolution layers, w l is the weighting coefficient of the lth layer, indicating the importance of this layer in the scoring.

[0083] Score calculation: Based on the comprehensive response value, the system normalizes the defect score to ensure the standardization and consistency of the score. The normalized score P can be expressed as:

[0084]

[0085] Where max(S) represents the highest response value among all scored segments. The normalized score ranges from 0 to 1, with higher values ​​indicating a greater likelihood that the segment contains a defect.

[0086] Ultimately, for each image segment containing the inspection object, the system generates a numerical score representing the likelihood of a defect. This score is used in subsequent steps to further assess the defect risk within the segment and generate an overall inspection result. The integrated application of feature extraction, filtering, and scoring in this step effectively improves the accuracy and efficiency of the inspection process, enabling the system to focus on segments containing the inspection object and perform precise defect assessments.

[0087] Step 4: Further process the image segments containing the detected object parts

[0088] For image segments that have been screened to contain the inspection object, the present invention further feeds these segments into the neural network for deeper feature analysis, thereby detecting possible minor flaws or anomalies. This step further refines defect information through multi-layer feature extraction, ensuring that the system captures potential subtle defect characteristics. The system accurately analyzes abnormal areas within the segments based on feature distribution and converts the defect characteristics into an array display, facilitating the subsequent calculation of spatial correlations between segments and the generation of defect assessments.

[0089] 1. Further feature extraction

[0090] Based on the initial filtering and scoring, this step further extracts high-level features from the image clips. The image clips are input into the deep structure of the neural network to extract more detailed feature information. The specific steps are as follows:

[0091] Feature Map Refinement: Image fragments undergo deeper convolution operations within a multi-layer convolutional neural network (CNN), enabling the system to more meticulously analyze local features, particularly edges, textures, and minor defects. After the feature map is generated, the system amplifies and analyzes the responses within each convolutional layer to increase sensitivity to image detail.

[0092] Multi-scale feature analysis: To ensure that minor defects are not overlooked during feature extraction, this invention utilizes a multi-scale feature analysis method to comprehensively detect features of varying sizes. By extracting feature maps at different scales, the system can detect defects of varying sizes. Multi-scale feature response is achieved through a multi-layer convolutional architecture to accommodate the various scales of detail present in the image clip.

[0093] 2. Deep Analysis of Feature Distribution

[0094] After generating a refined feature map, the system conducts an in-depth analysis of the feature distribution within the fragment to accurately identify possible minor defects and anomalies in the fragment. The analysis process includes the following steps:

[0095] Response value clustering: The system clusters the response values ​​in the feature map to distinguish normal areas from possible defect areas. For example, K-means clustering is used to divide the feature response values ​​into several classes, each corresponding to a specific feature region. Regions with higher response values ​​typically correspond to defects or abnormal regions, while regions with lower response values ​​may be normal areas. The clustering algorithm formula is as follows:

[0096]

[0097] Among them, J is the clustering loss function, represents the jth point in the i-th category, μ i Represents the center point of the i-th class. By minimizing J, the system divides the response values ​​in the feature map into several classes to extract defect areas with high response values.

[0098] Defect Area Extraction: By analyzing clustering results, the system identifies highly responsive defect regions within a segment and extracts their characteristic information. After extracting the defect region, the system further calculates the region's characteristic distribution, including its area, shape, location, and its relationship to surrounding normal areas. This process effectively captures minute defects within the segment, supporting defect location and quantification.

[0099] 3. Defect features are converted into array display

[0100] After defect region extraction, the present invention converts the feature maps of these regions into an array display for subsequent spatial correlation calculation and overall defect assessment. The array display provides detailed information about each defect region, including its location in the image segment, response value, and morphological characteristics. This conversion process includes the following steps:

[0101] Feature Map Discretization: This converts the feature map data of the defect area into an array format for subsequent spatial association and computational operations. Specifically, the system discretizes the edges of the defect area and labels the position coordinates of each defect area in the image segment (such as the row and column numbers of the array) to form a two-dimensional array representation of the defect area.

[0102] Array Assignment: In array display, the system assigns the response value of each defect region to the corresponding array element to preserve the intensity information of the defect characteristics. For example, defect regions with high response values ​​are assigned higher values ​​in the array, while normal regions are assigned low or zero values. This array format fully displays the characteristic information of the defect region and provides reliable data support for subsequent spatial correlation calculations between segments.

[0103] Through these further processing steps, the present invention enables in-depth feature analysis of image segments containing the inspection object and clearly presents defect information in an array display format, laying the foundation for subsequent defect assessment and spatial correlation calculations. This processing method ensures that the system can detect subtle flaws or anomalies, significantly improving the accuracy and reliability of inspections.

[0104] Step 5: Generate defect probability value

[0105] After further processing of the image segments, the system automatically generates a defect probability value for each segment using multiple weighting parameters combined with the positional correlation information between the segments. This defect probability value represents the probability that each segment contains a defect. The specific generation method includes weighting the segment features, calculating spatial correlation, and combining the defect probability value.

[0106] 1. Fragment feature weighting

[0107] First, the system weights the features of each image segment based on the distribution of its response values ​​in the feature map to generate a preliminary defect score. Feature weighting is performed by summarizing and weighting the feature response values ​​within each segment to obtain its defect score. The specific steps are as follows:

[0108] Response value summary: The system calculates the characteristic response value of each pixel in the image segment and obtains the weighted average of all response values ​​in the segment. Let the response value in the image segment be R i,j , where i and j represent the position coordinates of the pixel in the segment. The preliminary defect score S of the segment can be expressed as:

[0109]

[0110] Among them, w i,j is a weighting parameter used to adjust the contribution of different pixels to the defect score, where N is the total number of pixels in the segment. Pixels with high response values ​​are given higher weights, making the defect area more prominent in the score.

[0111] Weight selection: weighting parameter w i,j The weighting can be set based on the pixel's position within the segment, the intensity of its response, and the importance of the feature. Pixels with high response values ​​located in the center of the segment are given higher weights to ensure that the main defect features within the segment are prioritized.

[0112] 2. Spatial correlation calculation

[0113] After generating the preliminary defect score, the system further considers the spatial correlation between image segments and modifies the defect score of each segment by calculating the correlation between the response features of adjacent segments. The specific steps of spatial correlation calculation are as follows:

[0114] Adjacent segments identification: The system identifies the adjacent segments of each image segment and defines the segments that share the same boundary or corner points as adjacent segments. Let the current segment be P, and its adjacent segment set be {P k}.

[0115] Correlation calculation: The system calculates the correlation between the feature response of the current segment and its adjacent segments, specifically by comparing the response value S of the current segment and the response value S of the adjacent segment. k Compare and generate the correlation coefficient α k The calculation formula of the correlation coefficient is as follows:

[0116]

[0117] Among them, S k The correlation coefficient α is the initial defect score of the adjacent segments. kIndicates the similarity of feature responses between the current segment and the adjacent segments. A higher value indicates a stronger defect correlation between adjacent segments.

[0118] Spatial diffusion correction: If the defect score of the current segment is high and its correlation coefficient with the adjacent segments exceeds the set threshold α th , the system will increase the scores of adjacent segments to reflect the possible spatial spread of defects. The updated defect score S ′ It can be expressed as:

[0119]

[0120] Among them, β k is an adjustment factor that controls the influence of adjacent segments on the current segment score. Through this correction process, the system can appropriately adjust the segment score based on the spatial distribution of defects to more accurately reflect the diffusion of defects between segments.

[0121] 3. Generate defect probability value

[0122] After completing the segment feature weighting and spatial correlation calculation, the system will combine the weighted feature response value and spatial correlation to calculate the final defect probability value, which indicates the possibility of defects in each image segment. The calculation formula of the defect probability value is as follows:

[0123] p=σ(S ′ )

[0124] Among them, p represents the possible value of defect, S ′ is the final defect score after spatial correlation correction, and σ is a normalization function (such as the Sigmoid function) used to normalize the possible defect value to the range of 0 to 1 to facilitate subsequent defect determination. The normalization function is defined as follows:

[0125]

[0126] Through this normalization, the system can obtain a standardized defect likelihood value for each segment. A higher defect likelihood value indicates a greater probability that the segment contains a defect. This normalized defect likelihood value not only facilitates unified comparison across segments but also avoids misjudgments that might arise from analyzing each segment independently.

[0127] By combining weighted parameters and spatial correlation, the present invention can accurately reflect the defect probability of each segment. Segment feature weighting highlights the location of the main characteristic response, while spatial correlation calculation captures the defect diffusion characteristics between segments, further improving the accuracy of the defect probability value.

[0128] Step 6: Generate an overall defect assessment

[0129] Based on the defect likelihood value of each image segment, the present invention generates an overall defect score for the inspection image, systematically assessing the overall defect status of the inspection object. By aggregating the defect likelihood values ​​of each image segment, an overall defect score is formed, providing a comprehensive description of the target object's defect location, likelihood, and severity. The specific steps involved in generating the score include aggregating segment scores, enhancing high-risk segments, and generating an overall assessment result.

[0130] 1. Clip score aggregation

[0131] First, the system performs a weighted average of the possible defect values ​​of all image segments to generate a preliminary overall defect score to reflect the overall defect status of the inspection object. The specific steps of segment score aggregation are as follows:

[0132] Summary of possible defect values: Assume that the detection image is segmented into n image segments, and the possible defect value of the i-th segment is P i The system calculates the weighted average of the possible defect values ​​of all segments and generates a preliminary overall defect score S total :

[0133]

[0134] Among them, w i is the weight coefficient for the i-th segment, which is used to adjust the segment's contribution to the overall score. The weight coefficient can be set based on factors such as the segment's position in the image and the potential size of the defect, ensuring that segments in key areas have a greater impact on the score.

[0135] Weight setting: In specific implementations, if certain fragments are located in critical areas of the inspection object (such as welding points or high-risk areas), the system can assign these fragments higher weights so that the defects in these important areas are more prominent in the overall defect score.

[0136] 2. High-risk segment enhancement

[0137] Based on the initial overall defect score, the system further enhances the clips with higher scores. The steps for enhancing high-risk clips are as follows:

[0138] Identify high-risk segments: The system sets a high-risk threshold P based on the possible value of defects in each image segment. high If the defect of a certain segment may be worth P i If the value is higher than the threshold, the clip is considered high-risk and the system will mark such clips as defective areas that require special attention.

[0139] Enhanced processing: For high-risk segments, the system amplifies the possible value of the defect to reflect the severity of the defect. For example, the enhanced processing can be achieved by increasing the weight or multiplying the segment score. Assuming that the amplification factor of the high-risk segment is α (α>1), its revised score P′ i It can be expressed as:

[0140] P′ i =α·P i

[0141] Through this magnification operation, high-risk segments will be reflected higher in the overall defect score, effectively indicating the defect risk of the area.

[0142] Overall score adjustment: The system recalculates the overall flaw score after the high-risk segments are enhanced, and reaggregates the flaw potential values ​​of all segments (including the enhanced high-risk segments) to obtain a more indicative overall score S′ total :

[0143]

[0144] Among them, S′ total It represents the overall defect score after enhanced processing, which can more accurately reflect the overall defect situation of the inspection object.

[0145] 3. Generate overall evaluation results

[0146] After obtaining the final overall defect score, the system generates an overall defect assessment result for the inspected object, including a detailed description of the defect location, defect likelihood, and severity. The steps for generating this assessment result are as follows:

[0147] Defect Location Marking: The system marks segments with high defect potential as defective areas and displays their locations in the inspection image output. Marking can include graphic overlays (such as highlighting or a red border) to clearly indicate the specific location of the defective area.

[0148] Defect Likelihood Description: Based on the defect likelihood value of each image segment, the system generates a defect likelihood description for the defect area. If a segment's defect likelihood value exceeds a certain threshold, the system marks it as a "high-likelihood defect area" in the evaluation results. Segments with defect likelihood values ​​in the middle range are marked as "potential defect areas."

[0149] Defect severity assessment: The system scores the overall defect according to S' total Determine the severity of defects in the inspection object. The specific evaluation criteria can be set as a multi-level description, for example, if S′ total If the system is rated as “severely defective”, S′ totalIf the defect rate is between 50% and 90%, it is rated as "moderate defect", and if it is below 50%, it is rated as "minor defect". This classification description helps users quickly understand the overall situation of the defect.

[0150] Through the above steps, the present invention generates a comprehensive defect assessment of the inspection object, providing a detailed report on the defect location, likelihood, and severity. This result provides an important basis for subsequent quality control and defect repair decision-making, and significantly improves the intelligence and accuracy of the inspection system.

[0151] The above description is only a preferred embodiment of the present invention. Therefore, any equivalent changes or modifications made according to the structure, characteristics and principles described in the scope of the patent application of the present invention are included in the scope of the patent application of the present invention.

Claims

1. A large-scale model-based AOI visual inspection optimization method for the electronics industry, characterized by: The following steps are involved: Step 1: Collect the detection image of the target to be measured; Step 2: In the acquired inspection image, the edge area of ​​the target object is identified, and the image segmentation parameters are dynamically adjusted according to the size, shape, and position characteristics of the object. The inspection image is divided into multiple image segments, where each image segment corresponds to a selected section of the inspection image, so that the segmented image segments contain the characteristics of the target object; Step 3: Use a large model based on a deep convolutional neural network to filter and score multiple image clips to determine whether each image clip contains part of the detection object; Step 4: The image segments that are determined to contain the detection object are input into the neural network for further processing, while the image segments that do not contain the detection object are removed to form a corresponding array display; Step 5: Using multiple weighting parameters and the positional associations between segments, the system automatically generates a defect probability value for each image segment to indicate the likelihood that the image segment contains a defect. The weighting parameters are set based on the pixel's position within the segment, the response value intensity, and the feature importance. The system assigns higher weights to high-response pixels located in the center of the segment to ensure that the main defect features within the segment are prioritized. Step 6: Generate an overall defect assessment of the inspection image based on the defect probability value of each image segment to indicate the defect status of the inspection object.

2. The large-scale model-based AOI visual inspection optimization method for the electronics industry according to claim 1, characterized in that: The detection image is collected by a high-resolution imaging device, and the imaging device includes an industrial camera, a microscope camera or a high-definition camera.

3. The large-scale model-based AOI visual inspection optimization method for the electronics industry according to claim 1, characterized in that: The process of dynamically adjusting the image segmentation parameters includes: determining the edge features of the target object based on the edge detection algorithm, and automatically adjusting the size and segmentation direction of the segmentation window according to the size and shape features to adapt to the specific form of the detected object.

4. The large-scale model-based AOI visual inspection optimization method for the electronics industry according to claim 1, characterized in that: The process of filtering and scoring the image segments includes: extracting feature maps of the segments using multiple convolutional layers and pooling layers, filtering the segments according to the feature response values ​​of each image segment, and removing segments that do not contain target object features.

5. The large-scale model-based AOI visual inspection optimization method for the electronics industry according to claim 1, characterized in that: The scoring process for the image segment includes: generating a comprehensive score S based on the feature response values ​​of multiple convolutional layers to reflect the possibility of defects in the image segment. The comprehensive score is calculated as follows: in, is the total number of convolutional layers, For the The activation value of the layer feature map, is the weighting coefficient of each layer.

6. The large model-based AOI visual inspection optimization method for the electronics industry according to claim 5, characterized in that: In step 5, the defect probability value of each segment is calculated by combining the weighted parameter and the spatial correlation. The calculation formula of the defect probability value is as follows: in, is the final defect score after spatial correlation correction, expressed as: in, is the adjustment coefficient used to control the influence of adjacent segments on the current segment score. Correlation coefficient, is the preliminary score of adjacent segments, S is the comprehensive score, is a normalization function used to normalize the possible value of the defect to the range of 0 to 1, and .

7. The large model-based AOI visual inspection optimization method for the electronics industry according to claim 6, characterized in that: The spatial correlation calculation between segments includes identifying adjacent segments and correcting the defect score of each segment by calculating the correlation of the feature responses of adjacent segments. The specific correlation coefficient is calculated as follows: in, Rating for the current clip. is the preliminary score of adjacent segments, the correlation coefficient Used to indicate the spatial diffusion characteristics of defects.

8. The large model-based AOI visual inspection optimization method for the electronics industry according to claim 1, characterized in that: In step 6, the system generates an overall defect score based on the aggregated results of the possible defect values. The weighted average of the overall score is calculated as follows: in, is the total number of image clips, For the The possible value of the defect of each fragment, For the The weight coefficient of each fragment.

9. The large model-based AOI visual inspection optimization method for the electronics industry according to claim 8, characterized in that: After the overall flaw score is generated, the system enhances the clips with higher flaw scores. For high-risk clips, the revised scores are as follows: in, is a magnification factor that increases the score of high-risk segments to reflect their severity.

10. The large model-based AOI visual inspection optimization method for the electronics industry according to claim 8, characterized in that: The overall defect assessment result includes the specific location of the defect, a description of the defect possibility, and a description of the severity.

Images