Electronic industry AOI visual inspection optimization method based on large model

Abstract

The invention provides an electronic industry AOI visual inspection optimization method based on a large model, and aims to improve the flexibility, precision and efficiency of automatic optical inspection. The method comprises the steps of collecting a detection image of a to-be-detected target; identifying an edge area of the target object, dynamically adjusting segmentation parameters according to size, shape and position features, and dividing the detection image into a plurality of image segments; the fragments are filtered and scored through a deep convolutional neural network, and the fragments containing the detection object are screened out; further processing the screened fragments to form array display; generating a flaw possible value by combining the plurality of weight coefficients with the position association between the fragments; and finally, performing overall evaluation based on the defect possible value, and indicating the defect condition of the detected object. According to the method, adaptive segmentation, multi-parameter weighting and spatial correlation mechanisms are introduced, so that the detection precision and robustness of the system in complex and diversified scenes are remarkably improved, and the method is suitable for efficient and accurate automatic optical detection in the electronic industry.

Description

Technical Field

[0001] The invention relates to an automatic optical inspection (AOI) technology for the electronics industry, in particular to an AOI visual inspection optimization method based on a large model. Background Art

[0002] In the electronics manufacturing industry, Automatic Optical Inspection (AOI) technology has become a core tool for quality control of high-precision electronic components and circuit boards. The AOI system mainly relies on optical imaging technology and image processing algorithms to quickly and accurately detect complex targets such as electronic devices and circuit boards (PCBs) to detect various defects such as poor solder joints, missing components, misalignment, cracks, etc. With the increasing miniaturization and high-density packaging of electronic devices, higher and higher requirements are placed on the accuracy, efficiency and intelligence level of AOI detection technology. However, traditional AOI systems still have many technical bottlenecks when dealing with complex and diverse detection needs.

[0003] Traditional AOI inspection systems mainly rely on rule-based or traditional machine vision-based inspection methods. These methods usually manually define image features in the image preprocessing stage, and screen for expected defects and anomalies in the image through algorithms such as feature extraction and edge detection. Although this method meets basic inspection needs to a certain extent, when faced with a wide variety of components and changing inspection requirements, traditional algorithms are difficult to adapt to the feature changes of diverse inspection objects, and inspection accuracy and flexibility are often limited. For electronic components with complex and changeable shapes, sizes, colors and materials, rule-based inspection solutions have gradually shown insufficient response, resulting in high missed detection rates and false detection rates. Summary of the invention

[0004] In order to solve the above problems in the prior art, the present invention proposes an AOI visual inspection optimization method for the electronics industry based on a large model, comprising the following steps:

[0005] Step 1: Collect the detection image of the target to be tested;

[0006] Step 2: In the collected detection image, the edge area of ​​the target object is identified, and the image segmentation parameters are dynamically adjusted according to the size, shape and position characteristics of the object, and the detection image is divided into multiple image segments, where each image segment corresponds to a selected section of the detection image, so that the segmented image segments contain the characteristics of the target object;

[0007] Step 3: Use a large model based on a deep convolutional neural network to filter and score multiple image clips to determine whether each image clip contains part of the detection object;

[0008] Step 4: The image segments determined to contain the detection object are input into the neural network for further processing, and the image segments that do not contain the detection object are removed to form a corresponding array display;

[0009] Step 5: Using multiple weight coefficients and combining the position associations between the segments, the possible defect value of each image segment is automatically generated to indicate the possibility that the image segment contains defects;

[0010] Step 6: Generate an overall defect assessment of the inspection image based on the defect probability value of each image segment to indicate the defect status of the inspection object.

[0011] The detection image is collected by a high-resolution imaging device, and the imaging device includes an industrial camera, a microscope camera or a high-definition video camera.

[0012] The process of dynamically adjusting the segmentation parameters of the image includes: determining the edge features of the target object based on an edge detection algorithm, and automatically adjusting the size and segmentation direction of the segmentation window according to the size and shape features to adapt to the specific form of the detected object.

[0013] The process of filtering and scoring the image segments includes: extracting feature maps of the segments using multiple convolutional layers and pooling layers, filtering the segments according to the feature response values ​​of each image segment, and removing segments that do not contain target object features.

[0014] The scoring process for the image segment includes: generating a comprehensive score S based on the feature response values ​​of multiple convolutional layers to reflect the possibility of defects in the image segment. The comprehensive score is calculated as follows:

[0015]

[0016] Among them, L is the total number of convolutional layers, A l is the activation value of the feature map of the lth layer, w l is the weighting coefficient of each layer.

[0017] In step 5, the defect probability value P of each segment is calculated by combining the weighted parameter and the spatial correlation. The calculation formula of the defect probability value P is as follows:

[0018] P = σ(S ′ )

[0019] Among them, S ′ is the final defect score after spatial correlation correction, σ is a normalization function used to standardize the possible defect value to the range of 0 to 1, and

[0020] The spatial correlation calculation between segments includes identifying adjacent segments and correcting the defect score of each segment by calculating the correlation of the characteristic responses of adjacent segments. The specific correlation coefficient α k The calculation is as follows:

[0021]

[0022] Among them, S is the score of the current segment, S k is the preliminary score of adjacent fragments, and the correlation coefficient α k Used to indicate the spatial diffusion characteristics of defects.

[0023] In step 6, the system generates an overall defect score S based on the aggregated results of the possible defect values. total , the weighted average of the overall rating is calculated as follows:

[0024]

[0025] Where n is the total number of video clips, P i is the possible defect value of the i-th fragment, w i is the weight coefficient of the ith fragment.

[0026] After the overall defect score is generated, the system strengthens the clips with higher defect scores. For high-risk clips, the revised score P i ′ It is expressed as follows:

[0027] P′ i =α·P i

[0028] Among them, α is the amplification factor, which is used to increase the score of high-risk clips to reflect their severity.

[0029] The overall defect assessment result includes the specific location of the defect, a description of the defect possibility and a description of the severity.

[0030] Beneficial effects: The present invention provides an optimization method for AOI visual inspection in the electronics industry based on a large model. By introducing adaptive segmentation, deep convolutional neural networks and multi-parameter weighting mechanisms, it can accurately identify the inspection object and dynamically adjust the segmentation parameters to improve the flexibility and accuracy of the inspection. Through fragment filtering and scoring, spatial correlation calculation and overall defect evaluation, it significantly reduces misjudgments and missed judgments, and improves the robustness and inspection efficiency of the system. Especially in complex and diverse inspection scenarios in the electronics industry, the method of the present invention can fully adapt to a variety of defect types and complex noise environments, and realize efficient and accurate automatic optical inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present application, but do not constitute an improper limitation of the present invention. In the drawings:

[0032] Figure 1 A flow chart of the method of the present invention is shown. DETAILED DESCRIPTION

[0033] The following will be combined with the attached Figure 1 The present invention is described in detail with reference to the accompanying drawings and specific embodiments, wherein the exemplary embodiments and descriptions are only used to explain the present invention but are not intended to limit the present invention.

[0034] The present invention provides an optimization method for AOI visual inspection in the electronics industry based on a large model to solve the problems of insufficient defect recognition accuracy, low detection flexibility and poor real-time performance in existing AOI inspections. The present invention collects images of the target to be inspected, uses a deep convolutional neural network based on a large model, and combines adaptive image segmentation and multi-parameter weighting strategies to perform high-precision, dynamically adjusted defect detection on the inspection target. Specifically, the method of the present invention includes the following steps:

[0035] Step 1: Collect the detection image of the target to be tested

[0036] Before the detection process begins, a high-resolution imaging device is used to collect the detection image of the electronic component or circuit board to be tested. The imaging device includes but is not limited to an industrial camera, a microscope camera, a high-definition camera, etc., to ensure that the fine structure and surface features of the target to be tested can be captured to obtain high-quality detection images. The imaging device is preferably arranged on an adjustable bracket or a mechanical arm so that its angle, distance and position can be flexibly adjusted when necessary to optimize the imaging effect.

[0037] During the image acquisition process, the system automatically adjusts the imaging parameters of the camera to adapt to the material characteristics and surface conditions of different target objects. These imaging parameters include but are not limited to brightness, contrast, focal length, aperture size, exposure time and white balance. Specifically, for targets with smooth or highly reflective surfaces, the system can reduce brightness and exposure time to reduce reflective interference; for darker targets or targets with rich texture details, the system can appropriately increase contrast and focal length to improve image clarity and detail performance. In addition, the system can also use the autofocus function to ensure that the target has optimal clarity in the image for subsequent analysis and processing.

[0038] After the image acquisition is completed, the system transmits the image data to the image processing module for subsequent steps such as image segmentation, feature extraction and defect analysis. Through the above image acquisition steps, the present invention can obtain high-resolution, clear and detailed inspection images, providing a reliable image basis for subsequent inspection processes.

[0039] Step 2: Identify the edge area of ​​the target object and perform adaptive image segmentation

[0040] In the collected detection image, the present invention first identifies the edge area of ​​the target object in order to accurately locate the area to be detected and ensure that the key features in the detection image are effectively extracted. The edge recognition process of the target object is based on the deep feature extraction capability of the large model. By analyzing the edge and morphological features in the image, the specific position of the target object in the image is determined, and its edge contour is clearly extracted. Based on the results of edge recognition, the system will dynamically adjust the image segmentation parameters to adapt to the size, shape and position characteristics of the target object, thereby dividing the detection image into multiple image segments containing specific information.

[0041] The specific implementation steps of adaptive image segmentation include the following:

[0042] The system first performs edge detection on the detection image to obtain the edge feature map of the detection object. Edge detection uses image gradient algorithm to ensure the accuracy and clarity of edge recognition.

[0043] In edge detection, the system calculates the gradient value G of each pixel in the image in the horizontal direction (x direction) and the vertical direction (y direction). x and G y , and thus obtain the edge gradient amplitude G of the pixel point:

[0044]

[0045] The strength of the edge is determined based on the magnitude of the gradient, thereby generating an edge map.

[0046] The output of edge detection is a binary edge map that can clearly show the contour and morphological features of the target object, providing a reliable basis for subsequent size and shape analysis.

[0047] The system analyzes the size and shape of the target object based on the edge map in order to determine the parameters required for adaptive segmentation. First, the system extracts the contour lines in the edge map through an image contour tracking algorithm (such as contour detection based on connected regions), and combines the geometric feature extraction algorithm of computer vision to calculate the morphological features of the object, such as the area, circumscribed rectangle, and main direction.

[0048] Use the connected component algorithm to obtain all closed or open contours in the edge graph and calculate the bounding box of each contour. The bounding box provides the preliminary size of the target object.

[0049] The overall size of the target object can be estimated by the width, height and area of ​​the bounding box. In addition, by calculating the main axis direction of the circumscribed rectangle, the main direction and shape characteristics of the target object can be further determined to assist segmentation.

[0050] The system also combines a large model of deep convolutional neural network to analyze the morphological features of the target object through multi-layer convolution to obtain richer structural information. The features extracted by the deep network can identify the detailed features and texture information inside the target object, thereby supplementing the deficiencies of traditional geometric features and ensuring a more accurate segmentation effect.

[0051] After completing the size and shape analysis, the system automatically adjusts the image segmentation parameters according to the specific characteristics of the target object to achieve adaptive image segmentation. The main technical steps in the adaptive segmentation process are as follows:

[0052] Segmentation parameter setting: Based on the bounding box size and main axis direction of the target object, the system automatically sets the size, number and direction of the segmented fragments so that the segmented fragments can better fit the shape and distribution of the target object. For example, for a long strip of target object, the system will prioritize segmentation along the main axis direction to increase the inclusion rate of the segmented fragments.

[0053] Multi-scale segmentation: The system can apply a multi-scale segmentation strategy when necessary, that is, using larger segmentation fragments for larger objects and smaller segmentation fragments for smaller target objects. This multi-scale segmentation ensures that the detailed features of the target object are included in the image fragment, effectively improving the accuracy of detection.

[0054] Segmentation implementation: Use the sliding window algorithm to segment the image, set the step size and size of the sliding window, and generate segmented segments that meet the characteristics of the target object. Each image segment will be sent to the subsequent detection step as an independent unit for fine-grained defect identification.

[0055] After segmentation, each image segment enters the subsequent detection step independently as a regional unit of the detection object. The system uses adaptive segmentation to ensure that the image segment accurately contains the characteristic area of ​​the target object, avoiding over-segmentation or under-segmentation. This segmentation method ensures the integrity of the target object information in the segmented segment while reducing the generation of segmented segments containing irrelevant information, thereby improving the accuracy and efficiency of the subsequent detection steps.

[0056] Through adaptive segmentation technology, the present invention can flexibly adapt to target objects of different sizes and shapes, making the image segmentation process highly adaptable. In the traditional fixed window segmentation method, the variable shape of the target object can easily lead to inaccurate segmentation areas, thereby affecting the reliability of detection. The present invention uses adaptive segmentation to perform targeted segmentation of target objects of different shapes and sizes to ensure that the segmented fragments can more completely contain target information. This adaptability not only improves the accuracy of subsequent detection, but also reduces unnecessary computational overhead to a certain extent, providing a reliable image basis for subsequent detection steps.

[0057] Step 3: Filter and score the image clips based on a large model of a deep convolutional neural network

[0058] After the image segmentation is completed, the present invention uses the deep convolutional neural network (CNN) in the large model to process each image segment, and through feature extraction, segment screening and defect scoring, it determines whether each segment contains part of the detection object and its defect probability. This step is specifically divided into image segment feature extraction, segment filtering and defect scoring.

[0059] 1. Image segment feature extraction

[0060] The system inputs each image segment into a pre-trained large model, which consists of multiple convolutional layers, pooling layers, and fully connected layers, and can extract feature information from the segment layer by layer. The feature extraction process is as follows:

[0061] Convolutional layer processing: The image fragment first passes through a series of convolutional layers, using convolution kernels (i.e. filters) to perform convolution operations on the image fragment to extract local features. The convolutional layer calculation method is as follows:

[0062]

[0063] Among them, f i,j is the pixel value of the output feature map after the convolution operation, I i+m,j+n Represents the local pixel value of the image segment, K m,n Represents the weight of the convolution kernel. Each convolution layer can produce multiple feature maps to capture different features of the image fragment.

[0064] Activation function: The feature map after each convolution operation will be processed by a nonlinear activation function (such as the ReLU function) to set the negative values ​​in the feature map to zero, thereby improving the nonlinear representation ability of the model:

[0065] f(x)=max(0,x)

[0066] This activation operation helps the model to more accurately distinguish the edges and textures of detected objects during feature extraction.

[0067] Pooling layer processing: The pooling layer is connected after the convolution layer to reduce the resolution of the feature map through downsampling operations, reduce the amount of calculation and increase the robustness of the features. The pooling layer usually uses maximum pooling (Max Pooling) or average pooling (Average Pooling):

[0068]

[0069] Among them, f pool : Pooling layer sampling results;

[0070] The results of the pooling layer are used to generate a simplified feature representation that helps identify global features of image segments.

[0071] Fully connected layer: After multiple layers of convolution and pooling, the feature map is flattened into a vector and input to the fully connected layer. The fully connected layer integrates the feature map information to capture the global characteristics of the image segment. The output vector of the fully connected layer contains the high-level features of the image segment for subsequent filtering and scoring.

[0072] 2. Fragment filtering

[0073] After feature extraction, the system filters the image segments based on the feature map generated by the large model to reduce the amount of calculation. The filtering process aims to identify segments that do not contain the detected object and remove them so that the system can focus on the segments that contain the detected object for more in-depth analysis.

[0074] Feature response threshold: The system determines whether the image segment contains the detection object by calculating the global response value of the feature map. If the feature response value is lower than the set threshold, it is considered that the segment does not contain the target feature and the system will remove it. The response value is calculated as follows:

[0075]

[0076] Among them, R represents the characteristic response value, f i is the pixel value in the feature map, and N is the total number of pixels in the feature map. The higher the feature response value, the more likely it is to contain the features of the detected object.

[0077] Fragment elimination: For image fragments whose feature response values ​​are lower than the threshold, the system will eliminate them from the subsequent processing flow, thereby reducing the amount of calculation for irrelevant fragments and improving detection efficiency.

[0078] 3. Defect scoring

[0079] For the screened image segments containing the inspection object, the system further calculates the score to determine the possible defect features in the segment. The defect score is based on the feature response value of the large model, which is calculated by the activation results of the feature maps of different convolutional layers to reflect the defect possibility of the segment.

[0080] Feature map activation: In the multi-layer convolution of the model, different convolution layers have different sensitivities to features of different scales and types. The system calculates the possibility of defect features in the image segment based on the activation level of the feature map of each layer. Assume that the activation value of the feature map of layer l is A l , the system calculates the comprehensive response value S:

[0081]

[0082] Among them, L is the number of convolution layers, w l is the weight coefficient of the lth layer, indicating the importance of this layer in the scoring.

[0083] Score calculation: Based on the comprehensive response value, the system normalizes the defect score to ensure the standardization and consistency of the score. The normalized score P can be expressed as:

[0084]

[0085] Among them, max(S) represents the highest response value among all the scoring segments. The normalized score ranges from 0 to 1, and the higher the value, the greater the possibility that the segment contains defects.

[0086] Finally, for each image segment containing the inspection object, the system generates a numerical score representing the probability of defects. This score is used in subsequent steps to further evaluate the defect risk in the segment and generate the overall inspection result. In this step, the comprehensive application of feature extraction, filtering and scoring effectively improves the accuracy and efficiency of the inspection process, allowing the system to focus on the segments containing the inspection object and perform accurate defect assessment.

[0087] Step 4: Further process the image segments containing the detected object parts

[0088] For the image segments that have been screened out to contain the parts of the inspection object, the present invention continues to input these segments into the neural network for deeper feature analysis, thereby detecting possible minor flaws or anomalies. This step further refines the defect information through multi-layer feature extraction to ensure that the system can capture potential subtle defect features. The system will accurately analyze the abnormal areas in the segments based on the feature distribution, and convert the defect features into an array display in the form of an array, so that the spatial correlation between the segments can be calculated and the defect assessment can be generated in the subsequent steps.

[0089] 1. Further feature extraction

[0090] Based on the preliminary filtering and scoring, this step further extracts high-level features from the image fragments. The image fragments are input into the deep structure of the neural network to extract more detailed feature information. The specific steps are as follows:

[0091] Refinement of feature maps: Image fragments undergo deeper convolution operations in multi-layer convolutional neural networks (CNNs), allowing the system to analyze local features more carefully, especially edge features, texture features, and tiny defects. After the feature map is generated, the system will amplify and analyze the response in each convolution layer to increase sensitivity to image details.

[0092] Multi-scale feature analysis: To ensure that minor defects are not ignored during feature extraction, the present invention uses a multi-scale feature analysis method to perform comprehensive detection of features of different sizes. By extracting feature maps at different scales, the system can detect defects of different sizes. Multi-scale feature response can be achieved by combining multiple layers of convolutional structures to adapt to the detailed features of different scales that may exist in the image clips.

[0093] 2. Deep Analysis of Feature Distribution

[0094] After generating the refined feature map, the system conducts an in-depth analysis of the feature distribution within the fragment to accurately identify the tiny defects and anomalies that may exist in the fragment. The analysis process includes the following steps:

[0095] Response value clustering: The system clusters the response values ​​in the feature map to distinguish normal areas from possible defective areas. For example, K-means clustering is used to divide the feature response values ​​into several classes, each class corresponding to a specific feature region. Higher response value areas usually correspond to defects or abnormal areas, while low response value areas may be normal parts. The clustering algorithm formula is as follows:

[0096]

[0097] Among them, J is the clustering loss function, represents the jth point in the i-th category, μ i Represents the center point of the i-th class. By minimizing J, the system divides the response values ​​in the feature map into several classes to extract defective areas with high response values.

[0098] Defective area extraction: By analyzing the clustering results, the system can identify the high-response defect area in the segment and extract the characteristic information of the area. After the defective area is extracted, the system further calculates the characteristic distribution of the area, including area, shape, position and edge relationship with the surrounding normal area. This process can effectively capture the tiny defects in the segment and provide support for the location and quantification of defects.

[0099] 3. Defect features are converted into array display

[0100] After the defect area extraction is completed, the present invention converts the feature maps of these areas into array displays for subsequent spatial correlation calculation and overall defect assessment. The array display provides detailed information for each defect area, including its location in the image segment, response value and morphological characteristics. This conversion process includes the following steps:

[0101] Feature map discretization: Convert the feature map data of the defect area into an array form for subsequent spatial association and calculation operations. Specifically, the system discretizes the edges of the defect area and marks the position coordinates of each defect area in the image segment (such as the row and column numbers of the array) to form a two-dimensional array representation of the defect area.

[0102] Array assignment: In the array display, the system assigns the response value of each defect area to the corresponding array unit to retain the intensity information of the defect feature. For example, defect areas with high response values ​​are assigned higher values ​​in the array, while normal areas are assigned low values ​​or zero values. This array form can fully display the characteristic information of the defect area and provide reliable data support for the subsequent spatial correlation calculation between fragments.

[0103] Through the above further processing steps, the present invention can perform in-depth feature analysis on the image fragments containing the inspection object, and clearly present the defect information in the form of array display, laying the foundation for subsequent defect evaluation and spatial correlation calculation. This processing method ensures that the system can detect subtle flaws or anomalies, significantly improving the accuracy and reliability of detection.

[0104] Step 5: Generate defect probability value

[0105] After further processing of the image segments, the system automatically generates a defect probability value for each image segment using multiple weighted parameters combined with the positional association information between the segments. The defect probability value represents the probability that each segment contains a defect. The specific generation method includes segment feature weighting, spatial association calculation, and comprehensive calculation of the defect probability value.

[0106] 1. Fragment feature weighting

[0107] First, the system weights the features of each segment based on the distribution of the response values ​​of the image segments in the feature map to generate a preliminary defect score. Feature weighting is performed by summarizing and weighting the feature response values ​​in each segment to obtain its defect score. The specific steps are as follows:

[0108] Response value summary: The system calculates the characteristic response value of each pixel in the image segment and obtains the weighted average of all response values ​​in the segment. Let the response value in the image segment be R i,j , where i and j represent the position coordinates of the pixel in the segment. The preliminary defect score S of the segment can be expressed as:

[0109]

[0110] Among them, w i,j is a weighting parameter used to adjust the contribution of different pixels to the defect score, and N is the total number of pixels in the segment. Pixels with high response values ​​will be given higher weights so that the defect area is more prominently reflected in the score.

[0111] Weight selection: weighting parameter w i,j It can be set based on the pixel's position within the fragment, the intensity of the response value, and the importance of the feature. For pixels with high response values ​​located in the center of the fragment, the system assigns a higher weight to ensure that the main defect features in the fragment are given priority.

[0112] 2. Spatial correlation calculation

[0113] After generating the preliminary defect score, the system further considers the spatial correlation between image segments and corrects the defect score of each segment by calculating the correlation between the response features of adjacent segments. The specific steps of spatial correlation calculation are as follows:

[0114] Adjacent segment identification: The system identifies the adjacent segments of each image segment and defines the segments that share boundaries or corner points with it as adjacent segments. Let the current segment be P, and its adjacent segment set be {P k}.

[0115] Correlation calculation: The system calculates the correlation between the feature response of the current segment and its adjacent segments, specifically by comparing the response value S of the current segment and the response value S of the adjacent segment. k Compare and generate the correlation coefficient α k The calculation formula of correlation coefficient is as follows:

[0116]

[0117] Among them, S k The correlation coefficient α is the preliminary defect score of adjacent segments. kIt indicates the similarity of the feature response between the current fragment and the adjacent fragment. The higher the value, the stronger the defect correlation between the adjacent fragments.

[0118] Spatial diffusion correction: If the defect score of the current segment is high and its correlation coefficient with the adjacent segment exceeds the set threshold α th , the system will increase the scores of adjacent segments to reflect the possible spatial spread of defects. The updated defect score S ′ It can be expressed as:

[0119]

[0120] Among them, β k is an adjustment coefficient used to control the influence of adjacent segments on the current segment score. Through this correction process, the system can appropriately adjust the segment score according to the spatial distribution of defects to more accurately reflect the defect diffusion between segments.

[0121] 3. Generate possible defect value

[0122] After completing the segment feature weighting and spatial correlation calculation, the system will comprehensively calculate the weighted feature response value and spatial correlation into the final defect possibility value, which indicates the possibility of defects in each image segment. The calculation formula of the defect possibility value is as follows:

[0123] p=σ(S ′ )

[0124] Among them, p represents the possible value of defect, S ′ is the final defect score after spatial correlation correction, and σ is a normalization function (such as Sigmoid function), which is used to normalize the possible defect value to the range of 0 to 1 to facilitate subsequent defect determination. The normalization function is defined as follows:

[0125]

[0126] Through the above normalization, the system can obtain the standardized defect probability value of each fragment. The higher the defect probability value, the greater the probability that the fragment contains defects. The normalized defect probability value not only facilitates unified comparison across fragments, but also avoids misjudgment that may occur due to independent analysis of fragments.

[0127] By combining weighted parameters and spatial correlation, the present invention can accurately reflect the defect probability of each segment. The segment feature weighting highlights the main feature response position, and the spatial correlation calculation captures the defect diffusion characteristics between segments, further improving the accuracy of the possible defect value.

[0128] Step 6: Generate an overall defect assessment

[0129] Based on the possible defect value of each image segment, the present invention generates an overall defect score for the inspection image to systematically evaluate the overall defect situation of the inspection object. By aggregating the possible defect values ​​of each image segment, an overall defect score is formed, thereby providing a comprehensive description of the defect location, possibility and severity of the target object. The specific generation steps include segment score aggregation, high-risk segment enhancement and generation of overall evaluation results.

[0130] 1. Clip score aggregation

[0131] First, the system performs a weighted average of the possible defect values ​​of all image segments to generate a preliminary overall defect score to reflect the overall defect status of the inspection object. The specific steps of segment score aggregation are as follows:

[0132] Summary of possible defect values: Assume that the detection image is segmented into n image segments, and the possible defect value of the i-th segment is P i The system calculates the weighted average of the possible defect values ​​of all segments and generates a preliminary overall defect score S total :

[0133]

[0134] Among them, w i is the weight coefficient of the ith segment, which is used to adjust the contribution of the segment in the overall score. The weight coefficient can be set based on factors such as the location of the segment in the image and the size of the possible defect value, to ensure that segments in key areas have a greater impact in the score.

[0135] Weight setting: In specific implementations, if certain segments are located in critical areas of the inspection object (such as welding points or high-risk areas), the system can assign higher weights to these segments so that the defects in these important areas are more prominent in the overall defect score.

[0136] 2. High-risk segment enhancement

[0137] Based on the preliminary overall defect score, the system further enhances the clips with higher scores. The steps for enhancing high-risk clips are as follows:

[0138] Identify high-risk clips: The system sets a high-risk threshold P based on the possible value of defects in each image clip. high If the defect of a certain segment may be worth P i If the value is higher than the threshold, the clip is considered as a high-risk clip. The system will mark such clips as defective areas that need special attention.

[0139] Enhanced processing: For high-risk segments, the system amplifies the possible value of the defect to reflect the severity of the defect. For example, enhanced processing can be achieved by increasing the weight or multiplying the segment score. Assuming that the amplification factor of the high-risk segment is α (α>1), its revised score P′ i It can be expressed as:

[0140] P′ i =α·P i

[0141] Through this magnification operation, high-risk segments will be reflected higher in the overall defect score, effectively indicating the defect risk in that area.

[0142] Overall score adjustment: The system recalculates the overall defect score after the high-risk segments are enhanced, and re-aggregates the defect potential values ​​of all segments (including the enhanced high-risk segments) to obtain a more indicative overall score S′ total :

[0143]

[0144] Among them, S′ total It represents the overall defect score after enhanced processing, which can more accurately reflect the overall defect situation of the inspection object.

[0145] 3. Generate overall evaluation results

[0146] After obtaining the final overall defect score, the system generates an overall defect assessment result for the inspection object, including a detailed description of the defect location, defect probability and severity. The steps to generate this assessment result are as follows:

[0147] Defect location marking: The system marks the fragments with high defect potential values ​​as defect areas and displays their locations in the output results of the inspection image. The marking method can use graphic overlay (such as highlighting or red border) to clearly mark the specific location of the defect area.

[0148] Defect Possibility Description: Based on the defect probability value of each image segment, the system generates a defect area probability description. If the defect probability value of a segment is higher than a certain threshold, the system will mark it as a "high-probability defect area" in the evaluation results. For segments with defect probability values ​​in the middle range, the system marks them as "potential defect areas."

[0149] Defect severity assessment: The system scores the overall defect according to S' total Determine the severity of the defects of the inspection object. The specific evaluation criteria can be set as a multi-level description. For example, if S′ total If S′ exceeds 90%, the system is rated as “severely defective”. totalBetween 50% and 90%, it is rated as "moderate defect", and below 50% is "minor defect". This classification description helps users quickly understand the overall situation of the defect.

[0150] Through the above steps, the present invention generates a comprehensive defect assessment of the inspection object and provides a detailed report on the defect location, probability and severity. This result provides an important basis for subsequent quality control and defect repair decisions, and significantly improves the intelligence and accuracy of the inspection system.

[0151] The above description is only a preferred embodiment of the present invention, so all equivalent changes or modifications made according to the structure, characteristics and principles described in the scope of the patent application of the present invention are included in the scope of the patent application of the present invention.

Claims

1. An AOI visual inspection optimization method for the electronics industry based on a large model, characterized by: The following steps are involved: Step 1: Collect the detection image of the target to be tested; Step 2: In the collected detection image, the edge area of ​​the target object is identified, and the image segmentation parameters are dynamically adjusted according to the size, shape and position characteristics of the object, and the detection image is divided into multiple image segments, where each image segment corresponds to a selected section of the detection image, so that the segmented image segments contain the characteristics of the target object; Step 3: Use a large model based on a deep convolutional neural network to filter and score multiple image clips to determine whether each image clip contains part of the detection object; Step 4: The image segments determined to contain the detection object are input into the neural network for further processing, and the image segments that do not contain the detection object are removed to form a corresponding array display; Step 5: Using multiple weight coefficients and combining the position associations between the segments, the possible defect value of each image segment is automatically generated to indicate the possibility that the image segment contains defects; Step 6: Generate an overall defect assessment of the inspection image based on the defect probability value of each image segment to indicate the defect status of the inspection object.

2. The method for optimizing AOI visual inspection in the electronics industry based on a large model as claimed in claim 1, characterized in that: The detection image is collected by a high-resolution imaging device, and the imaging device includes an industrial camera, a microscope camera or a high-definition video camera.

3. The method for optimizing AOI visual inspection in the electronics industry based on a large model as claimed in claim 1, characterized in that: The process of dynamically adjusting the segmentation parameters of the image includes: determining the edge features of the target object based on an edge detection algorithm, and automatically adjusting the size and segmentation direction of the segmentation window according to the size and shape features to adapt to the specific form of the detected object.

4. The method for optimizing AOI visual inspection in the electronics industry based on a large model as claimed in claim 1, characterized in that: The process of filtering and scoring the image segments includes: extracting feature maps of the segments using multiple convolutional layers and pooling layers, filtering the segments according to the feature response values ​​of each image segment, and removing segments that do not contain target object features.

5. The method for optimizing AOI visual inspection in the electronics industry based on a large model as claimed in claim 1, characterized in that: The scoring process for the image segment includes: generating a comprehensive score S based on the feature response values ​​of multiple convolutional layers to reflect the possibility of defects in the image segment. The comprehensive score is calculated as follows: Among them, L is the total number of convolutional layers, A l is the activation value of the feature map of layer l, w l is the weight coefficient of each layer.

6. The method for optimizing AOI visual inspection in the electronics industry based on a large model as claimed in claim 1, characterized in that: In step 5, the defect probability value P of each segment is calculated by combining the weighted parameter and the spatial correlation. The calculation formula of the defect probability value P is as follows: P = σ(S′) Where S′ is the final defect score after spatial correlation correction, σ is a normalization function used to normalize the possible defect value to the range of 0 to 1, and 7. The method for optimizing AOI visual inspection in the electronics industry based on a large model as claimed in claim 1, characterized in that: The spatial correlation calculation between segments includes identifying adjacent segments and correcting the defect score of each segment by calculating the correlation of the characteristic responses of adjacent segments. The specific correlation coefficient α k The calculation is as follows: Among them, S is the score of the current segment, S k is the preliminary score of adjacent fragments, and the correlation coefficient α k Used to indicate the spatial diffusion characteristics of defects.

8. The method for optimizing AOI visual inspection in the electronics industry based on a large model as claimed in claim 1, characterized in that: In step 6, the system generates an overall defect score S based on the aggregated results of the possible defect values. total , the weighted average of the overall rating is calculated as follows: Where n is the total number of video clips, P i is the possible defect value of the i-th fragment, w i is the weight coefficient of the ith fragment.

9. The method for optimizing AOI visual inspection in the electronics industry based on a large model as claimed in claim 8, characterized in that: After the overall defect score is generated, the system strengthens the clips with higher defect scores. For high-risk clips, the revised score P′ i It is expressed as follows: P′ i =α·P i Among them, α is the amplification factor, which is used to increase the score of high-risk clips to reflect their severity.

10. The method for optimizing AOI visual inspection in the electronics industry based on a large model as claimed in claim 8, characterized in that: The overall defect assessment result includes the specific location of the defect, a description of the defect possibility and a description of the severity.

Images