Ceramic packaging substrate detection method and system based on visual inspection and meta-learning

Abstract

The present invention relates to the technical field of defect recognition, and discloses a ceramic packaging substrate detection method and system based on visual inspection and meta-learning. The method comprises acquiring a visual image of a substrate to be inspected, extracting shape and size features and surface texture features, and constructing a feature database; judging the shape defects of the substrate by a constructed packaging substrate shape classification model, and identifying the type of surface defects according to texture features; the system comprises a feature acquisition module, a shape defect detection module, and a surface defect detection module; the present invention improves detection accuracy and efficiency, and is suitable for automated quality control.

Description

Technical Field

[0001] The present invention relates to the field of defect recognition technology, and more specifically, to a ceramic package substrate detection method and system based on visual detection and meta-learning. Background Art

[0002] With the rapid development of the electronic information industry, integrated circuit packaging technology is also constantly innovating. As a key component of integrated circuit packaging, the quality of ceramic packaging substrates directly affects the performance and reliability of packaged products. Therefore, it is very important to accurately and efficiently detect defects on ceramic packaging substrates. At present, there are some technical solutions for packaging substrate defect detection, but there are still some shortcomings.

[0003] A Chinese patent with the authorization announcement number CN103543168B discloses an X-ray detection method and system for multi-layer package substrate defects. The method obtains a tomographic image of a multi-layer package substrate through X-ray 3D imaging, detects physical defects of through holes and interconnections between substrates, and determines the defect mechanism through an expert system. However, the method mainly detects internal structural defects of the substrate, and has limited detection capabilities for surface defects. In addition, the cost of X-ray detection equipment is relatively high, which is not conducive to large-scale application.

[0004] A Chinese patent application with publication number CN117474928A discloses a method for detecting surface defects of ceramic package substrates based on a meta-learning model. The method uses a meta-learning paradigm to extract features and detect defects in ceramic package substrate images. However, the method fails to fully utilize the shape and size characteristics of the ceramic package substrate, and has insufficient detection capabilities for shape defects. In addition, the method does not explain in detail how to construct a meta-learning model, and there may be problems such as difficulty in training and insufficient generalization capabilities in practical applications.

[0005] A Chinese patent with the authorization announcement number CN112819845B discloses a method, medium and device for detecting contour, line width and line spacing defects of flexible packaging substrates. The method extracts contours through improved guided filtering and adaptive threshold segmentation algorithm, and detects line width and line spacing defects by fitting contour curves. However, the method is mainly aimed at flexible substrates and has limited applicability to rigid ceramic packaging substrates. In addition, the method fails to effectively utilize the texture features of the surface of the ceramic packaging substrate and has insufficient ability to identify surface defects.

[0006] In summary, the existing technology still has shortcomings in the detection of ceramic package substrate defects. How to simultaneously utilize the shape and size characteristics and surface texture characteristics of the ceramic package substrate to construct a high-precision and high-efficiency detection method to determine shape defects and identify surface defect types is a technical problem that needs to be solved in this field. Summary of the invention

[0007] In order to overcome the above-mentioned defects of the prior art, the present invention provides a ceramic package substrate detection method and system based on visual inspection and meta-learning.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] Ceramic package substrate inspection method based on visual inspection and meta-learning, including:

[0010] Acquire a visual image of the ceramic package substrate to be inspected, extract shape and size features and surface texture features of the visual image, and build a shape and size feature database and a surface texture feature database;

[0011] Constructing a packaging substrate shape classification model, and judging whether the ceramic packaging substrate has shape defects according to the shape and size characteristics of the ceramic packaging substrate and the packaging substrate shape classification model;

[0012] The surface defect type of the ceramic package substrate to be inspected is identified according to the surface texture characteristics of the ceramic package substrate.

[0013] Further, the obtaining of visual images of the ceramic package substrate to be inspected includes: obtaining visual images of N batches of ceramic package substrates to be inspected under different viewing angles and lighting conditions; the viewing angles at least include a front viewing angle, a back viewing angle, and a side viewing angle; each viewing angle includes at least three lighting modes of annular, coaxial, and oblique illumination; N is an integer greater than 0;

[0014] The constructing of the shape and size feature database and the surface texture feature database comprises:

[0015] Label the defect type of each sample in the visual image, associate the shape and size features with the corresponding defect type labels and store them to build a shape and size feature database; associate the surface texture features with the corresponding defect type labels and store them to build a surface texture feature database.

[0016] Furthermore, the extracting of shape and size features of the visual image includes:

[0017] Extract the contour edge of the ceramic package substrate from the visual image and obtain the contour edge point set , where n is the number of contour points, is the coordinate of the i-th contour point;

[0018] Performing polygon fitting on the extracted contour edge to obtain shape and size characteristics of the ceramic package substrate, wherein the shape and size characteristics include polygon vertex coordinates, side length, angle, fillet radius and edge straightness of the ceramic package substrate;

[0019] The polygon fitting of the extracted contour edge comprises:

[0020] Assume a polygon to fit the contour edge point set, The coordinates of the vertices of the polygon are , for The coordinates of the jth vertex of the polygon; establish the polygon fitting error function E, and solve , find a set of vertex coordinates that minimizes the polygon fitting error, .

[0021] Furthermore, judging whether the ceramic package substrate has a shape defect according to the shape and size characteristics of the ceramic package substrate and the package substrate shape classification model includes:

[0022] The extracted shape and size features are input into a pre-trained package substrate shape classification model to identify the shape category of the ceramic package substrate;

[0023] According to the identified shape category, a corresponding standard size range is obtained from a pre-built shape standard database;

[0024] The actual size of the ceramic package substrate represented by the extracted shape and size features is compared with the standard size range of the ceramic package substrate to determine whether the ceramic package substrate has shape defects.

[0025] Furthermore, the package substrate shape classification model adopts a convolutional neural network based on deep learning, and the package substrate shape classification model includes a convolutional layer, a pooling layer, a fully connected layer and a Softmax layer;

[0026] The step of inputting the extracted shape and size features into a pre-trained packaging substrate shape classification model to identify the shape category of the ceramic packaging substrate includes:

[0027] The shape size features including the coordinates of the polygon vertices, side length, angle, fillet radius and edge straightness constitute a one-dimensional shape feature vector of fixed length as the input of the package substrate shape classification model; the one-dimensional shape feature vector passes through the convolution layer to generate a shape feature map; the shape feature map enters the pooling layer, and the pooling layer downsamples the shape feature map to obtain a reduced-size shape feature dimensionality reduction map; the fully connected layer flattens the shape feature dimensionality reduction map output by the pooling layer into a one-dimensional shape mapping vector, and extracts high-level semantic features of the one-dimensional shape mapping vector; the Softmax layer obtains probability values ​​representing different shape categories based on the high-level semantic features of the one-dimensional shape mapping vector extracted by the fully connected layer, and the category with the largest probability value is the determined ceramic package substrate shape category recognition result.

[0028] Furthermore, comparing the actual size of the ceramic package substrate represented by the extracted shape and size features with the standard size range of the ceramic package substrate to determine whether the ceramic package substrate has a shape defect includes:

[0029] Calculating a size difference metric between an actual size of a ceramic package substrate and a standard size, wherein the size difference metric includes a vertex coordinate difference, a side length difference, an angle difference, a fillet radius difference, and an edge straightness difference;

[0030] The comprehensive shape difference index is calculated based on vertex coordinate differences, side length differences, angle differences, corner radius differences, and edge straightness differences;

[0031] If the comprehensive shape difference index exceeds a preset shape difference threshold, it is determined that the ceramic package substrate has a shape defect; otherwise, it is determined that the shape of the ceramic package substrate is qualified.

[0032] Further, the identifying the surface defect type of the ceramic package substrate to be inspected according to the surface texture feature of the ceramic package substrate includes:

[0033] Based on the surface texture features, searching for a sample most similar to the substrate to be tested in a surface texture feature database;

[0034] Analyze the defect types of the most similar samples to obtain preliminary defect diagnosis results of the substrate to be tested;

[0035] The meta-learning method is used to perform a second round of inspection on the substrate to be tested to obtain the final surface defect type determination result.

[0036] Furthermore, based on the surface texture feature, searching the surface texture feature database for a sample most similar to the substrate to be tested comprises:

[0037] Calculate the similarity between the surface texture feature of the substrate to be tested and the feature of each historical sample in the surface texture feature database to obtain a similarity measurement value;

[0038] According to the similarity metric, the historical samples in the surface texture feature database are sorted, and the Top-N1 samples with the highest similarity are selected as the most similar samples;

[0039] The defect type labels corresponding to the Top-N1 most similar samples are obtained as candidates for the defect type of the substrate to be tested.

[0040] Furthermore, the analyzing the defect type of the most similar sample to obtain a preliminary defect diagnosis result of the substrate to be tested includes:

[0041] Perform statistical analysis on the defect type labels of the top-N1 most similar samples and calculate the frequency of each defect type.

[0042] Determine whether the frequency of each defect type exceeds the preset frequency threshold , the statistics exceed the frequency threshold The number of defect types n2; if n2=1, the defect type is used as the preliminary diagnosis result of the substrate to be tested, and the substrate to be tested is marked as a regular sample; if n2=0, the substrate to be tested is determined to contain unknown defect types, and the substrate to be tested is marked as a difficult sample; if n2≥2, the frequency difference ratio between the highest frequency defect type and other high frequency defect types is calculated. ;

[0043] If the frequency difference ratio Greater than the preset frequency difference threshold , the defect type with the highest frequency is taken as the preliminary diagnosis result of the substrate to be tested, and the substrate to be tested is marked as a regular sample; otherwise, it is determined that the substrate to be tested contains an unknown defect type, and the substrate to be tested is marked as a difficult sample.

[0044] Furthermore, the second round of detection of the substrate to be tested by using the meta-learning method includes:

[0045] The surface texture features of the ceramic package substrate to be inspected, including conventional samples and difficult samples, are input into the meta-learning model to calculate the confidence scores of the samples for each defect type;

[0046] For difficult samples, the defect type with the highest confidence score is selected as the final defect diagnosis result of the difficult sample;

[0047] For conventional samples, the defect type with the highest confidence score is selected as the second defect diagnosis result of the conventional samples;

[0048] Compare the preliminary defect diagnosis result and the second defect diagnosis result of the conventional sample. If the two are consistent, the result is used as the final defect diagnosis result of the conventional sample. If the two are inconsistent, the preliminary defect diagnosis result and the second defect diagnosis result are integrated to form the final defect diagnosis result of the conventional sample.

[0049] A ceramic package substrate detection system based on visual inspection and meta-learning, which is used to implement the above-mentioned ceramic package substrate detection method based on visual inspection and meta-learning, and the system includes:

[0050] Feature acquisition module: used to acquire the visual image of the ceramic package substrate to be inspected, extract the shape and size features and surface texture features of the visual image, and build a shape and size feature database and a surface texture feature database;

[0051] Shape defect detection module: used to build a packaging substrate shape classification model, and determine whether the ceramic packaging substrate has shape defects based on the shape and size characteristics of the ceramic packaging substrate and the packaging substrate shape classification model;

[0052] Surface defect detection module: used to identify the surface defect type of the ceramic package substrate to be detected according to the surface texture characteristics of the ceramic package substrate.

[0053] An electronic device comprises a memory, a central processing unit, and a computer program stored in the memory and executable on the central processing unit. When the central processing unit executes the computer program, the above-mentioned ceramic package substrate detection method based on visual inspection and meta-learning is implemented.

[0054] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the above-mentioned ceramic package substrate detection method based on visual inspection and meta-learning.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] The present invention provides high precision and high robustness in shape classification through the combination of polygon fitting and convolutional neural network (CNN), ensuring accurate recognition of shape defects; utilizes meta-learning method to perform secondary detection on surface texture features, thereby improving the recognition accuracy of complex defect types; combines similarity calculation with meta-learning to improve adaptability to unknown defects; adopts deep learning model for shape classification and defect recognition, combined with the construction and management of feature database, realizes intelligent detection process and reduces manual intervention; reduces the demand and error rate of manual detection, improves detection speed and the degree of automation of production process, thereby reducing overall production cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 is a method flow chart of a ceramic package substrate detection method based on visual inspection and meta-learning in the present invention;

[0059] Figure 2 A flow chart of a method for extracting shape, size and surface texture features of a ceramic package substrate in a ceramic package substrate detection method based on visual inspection and meta-learning of the present invention;

[0060] Figure 3 A flow chart of a method for identifying shape categories of ceramic package substrates in a ceramic package substrate detection method based on visual detection and meta-learning of the present invention;

[0061] Figure 4 A flow chart of a method for determining shape defects of a ceramic package substrate in a ceramic package substrate detection method based on visual inspection and meta-learning of the present invention;

[0062] Figure 5 A flow chart of a method for obtaining preliminary defect diagnosis results of a substrate to be tested in a ceramic package substrate detection method based on visual detection and meta-learning of the present invention;

[0063] Figure 6 A flow chart of a method for obtaining a final surface defect type determination result in a ceramic package substrate detection method based on visual inspection and meta-learning of the present invention;

[0064] Figure 7 It is a functional module diagram of the ceramic package substrate detection system based on visual inspection and meta-learning in the present invention;

[0065] Figure 8 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application;

[0066] Fig. 9 A schematic diagram of the computer-readable storage medium structure provided for one embodiment of the present application. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Example 1

[0069] See also Figure 1 As shown, this embodiment provides a ceramic package substrate detection method based on visual inspection and meta-learning, including:

[0070] Step S1000, obtaining a visual image of a ceramic package substrate to be inspected, extracting shape and size features and surface texture features of the visual image, and constructing a shape and size feature database and a surface texture feature database;

[0071] Further, step S1000 includes:

[0072] Step S1100, obtaining visual images of N batches of ceramic package substrates to be inspected under different viewing angles and lighting conditions; the viewing angles at least include a front viewing angle, a back viewing angle, and a side viewing angle; each viewing angle includes at least three lighting modes: annular, coaxial, and oblique; N is an integer greater than 0;

[0073] Specifically, an industrial camera is used to build a multi-viewing angle and multi-light source image acquisition system, and the ceramic packaging substrate is photographed in all directions to obtain visual images at different angles and lighting conditions; by adjusting the camera position and angle, images from at least three perspectives, namely, front, back, and side, are obtained, and each perspective includes images under three lighting modes: annular, coaxial, and oblique; multiple batches of ceramic packaging substrates are collected to obtain a sufficient number of visual image data sets with comprehensive scenes; step S1100 uses an industrial camera to build an image acquisition system, which can achieve stable and high-quality visual image acquisition. Compared with ordinary cameras, industrial cameras have higher resolution, faster frame rate, and wider dynamic range, and can capture fine textures and defect details on the surface of ceramic substrates; in order to obtain a full picture of the ceramic substrate, it is necessary to shoot from multiple perspectives such as the front, back, and side; the front image is mainly used to detect the key dimensions of the substrate; the back image can check the back defects of the substrate; the side image helps to analyze the thickness and edge contour of the substrate; by adjusting the camera position and angle, images of required perspectives can be flexibly acquired.

[0074] In order to deal with interference factors such as reflections, shadows, and textures on the surface of ceramic substrates, images under multiple lighting modes need to be collected at each viewing angle. Annular lighting is conducive to highlighting edge and contour features; coaxial lighting can reduce shadows and provide uniform lighting; oblique lighting can enhance surface texture; and through the fusion of different lighting images, the morphological characteristics of the substrate can be fully reflected. The system can adapt to different viewing angles and lighting conditions, ensuring that high-quality visual data can be obtained in various environments, enhancing the versatility and adaptability of the detection system.

[0075] Step S1100 repeatedly collects multiple batches of ceramic substrates to obtain a large-scale visual image data set. Batch collection can improve the representativeness and diversity of the data and cover various defect situations in the production process. The larger the amount of data, the better the performance of the subsequent training detection model. At the same time, the acquisition process needs to fully simulate the actual application scenario, such as simulating different defect types, different imaging conditions, etc., so as to obtain a comprehensive and practical data set. In summary, step S1100 can obtain high-quality, multi-view, and multi-light visual images of ceramic substrates by carefully designing the image acquisition scheme. These image data provide a data basis for subsequent feature extraction, defect detection, process optimization and other links, and are the primary prerequisite for realizing intelligent detection. Through batch acquisition and scene simulation, the large-scale data set obtained can fully train the detection model to ensure the robustness and generalization of its performance. High-quality visual images are the cornerstone of accurate analysis of ceramic substrate defects and the basis for ensuring the effectiveness of the detection scheme.

[0076] Step S1200, preprocessing the acquired visual image;

[0077] Specifically, the visual image is distorted and corrected to eliminate the image distortion caused by lens distortion and restore the true shape of the substrate; the visual image is denoised using a median filter algorithm to smooth the image noise while maintaining edge details; the image contrast is enhanced using a histogram equalization algorithm to highlight textures and defect areas; the multi-view and multi-illumination images are geometrically corrected using an image registration method to ensure spatial alignment between images; step S1200 performs a series of preprocessing operations on the acquired original visual image to improve image quality and prepare for feature extraction; first, distortion correction is required to eliminate geometric distortions such as barrel distortion and pincushion distortion caused by industrial camera lenses; by establishing a lens distortion model, estimating distortion parameters, and then resampling and correcting the image, the true shape of the substrate can be restored and the accuracy of dimensional measurement can be improved. Secondly, an image denoising method is used to smooth image noise. In the process of ceramic substrate image acquisition, Gaussian noise, salt and pepper noise, etc. are inevitably introduced, which will interfere with the detection of defects; median filtering is a classic denoising algorithm. It can effectively suppress impulse noise by replacing the central pixel with the median of the pixel value in the sliding window; compared with mean filtering, median filtering has a better effect on maintaining edge details. On the basis of denoising, the contrast of the image is enhanced by histogram equalization; the grayscale distribution of the ceramic substrate image may be relatively concentrated, resulting in unclear texture and defects; histogram equalization enhances the contrast of the image by stretching the grayscale distribution, highlighting the texture and defect area, and creating conditions for subsequent feature extraction. Finally, due to the geometric misalignment between images acquired under multiple viewing angles and illuminations, image registration methods need to be used for correction and alignment; through feature point matching, the geometric transformation relationship between images of different viewing angles and illuminations is estimated, and the image is spatially registered by rotating, translating, scaling and other transformations, providing a basis for subsequent multi-view and multi-illumination feature fusion.

[0078] In summary, step S1200 uses distortion correction, image denoising, contrast enhancement, image registration and other means to perform standardized preprocessing on the visual image. On the one hand, the imaging error is corrected to restore the true morphology of the substrate, and on the other hand, the image noise is smoothed and the texture details are enhanced. The preprocessed image is corrected in geometry and grayscale, highlighting the effective visual information, and has good conditions for feature extraction. Although preprocessing is a behind-the-scenes work, it is crucial for subsequent defect detection and is a key link to ensure that the detection results are accurate and reliable. Only high-quality visual images can extract effective shape and texture features and train a detection model with excellent performance. Image preprocessing is the basic work of intelligent detection and needs to be highly valued. Automatically acquire visual images under multi-viewing angles and multi-lighting conditions, combined with image preprocessing techniques (such as distortion correction, denoising, and contrast enhancement), to achieve efficient automated data processing.

[0079] Step S1300, extracting shape and size features and surface texture features of the ceramic package substrate from the visual image;

[0080] Furthermore, if Figure 2 As shown, step S1300 includes:

[0081] Step S1310, extracting the contour edge of the ceramic package substrate from the visual image to obtain a contour edge point set , where n is the number of contour points, is the coordinate of the i-th contour point;

[0082] Step S1320, performing polygon fitting on the extracted contour edge to obtain shape and size characteristics of the ceramic package substrate, wherein the shape and size characteristics include polygon vertex coordinates, side length, angle, fillet radius and edge straightness of the ceramic package substrate;

[0083] Step S1330, extracting surface texture features of the ceramic package substrate from the visual image.

[0084] Specifically, step S1300 extracts shape and size features and surface texture features from the preprocessed visual image. The shape and size features characterize the geometric shape of the ceramic substrate, and the surface texture features describe the spatial distribution law of the surface texture of the substrate. The two types of features reflect the defect properties of the substrate from different angles. In order to obtain the shape and size features, the Canny operator is first used to detect the edge of the image. The Canny operator accurately extracts the target contour from the image through Gaussian smoothing, gradient calculation, non-maximum suppression, double threshold detection and other steps. Compared with other operators, the Canny operator has better signal-to-noise ratio and positioning accuracy. On this basis, the extracted contour edge is polygonally fitted to obtain key parameters that characterize the shape of the substrate, such as the length and width of the minimum circumscribed rectangle, the fillet radius at the corner, the straightness of the edge, etc. These parameters can be used to subsequently judge the size specifications and shape defects of the substrate.

[0085] When extracting surface texture features, GLCM is first used to describe the statistical characteristics of local textures. GLCM calculates the co-occurrence probability of image grayscale in different directions and step sizes to obtain texture parameters reflecting properties such as contrast, correlation, energy, and entropy. These parameters can quantify the roughness, consistency, and repeatability of textures. In order to further describe the variation of textures at different scales and directions, Gabor wavelets are introduced to perform multi-scale and multi-directional texture extraction on images. Gabor wavelets can be compared to the receptive field of the human eye, and use filter groups of multiple scales and directions to extract texture features. The scale reflects the coarseness of the texture, and the direction reflects the anisotropy of the texture. Gabor features can more comprehensively describe the spatial properties of the texture.

[0086] In summary, step S1300 uses edge detection, polygon fitting, GLCM, Gabor transform and other methods to extract shape and texture features from the ceramic substrate image. The Canny operator can accurately outline the substrate contour, polygon fitting can quantify the shape attributes of the contour, GLCM describes the statistical characteristics of the texture, and Gabor transform further describes the multi-scale and anisotropic nature of the texture. Feature extraction is the key to defect detection. Good features can reveal the essential differences of defects and highlight effective discriminant information. Only by selecting appropriate feature descriptions can the foundation for subsequent intelligent discrimination be laid. By deeply analyzing the imaging laws of substrate defects and specifically designing geometric and texture features, the discriminant information of defects can be retained to the maximum extent, thereby improving the accuracy and robustness of the detection method.

[0087] The method for performing polygon fitting on the extracted contour edge comprises:

[0088] Assume a Polygon To fit the contour edge point set, The coordinates of the vertices of the polygon are , for The coordinates of the jth vertex of the polygon; establish the polygon fitting error function E, and solve , find a set of vertex coordinates that minimizes the polygon fitting error;

[0089] The polygon fitting error function is:

[0090]

[0091] in:

[0092] :It is The weight of each contour point can be set according to the point's position on the contour, curvature and other characteristics. The larger the weight, the greater the influence of the point on the fitting result.

[0093] :Indicates taking the Contour points to The minimum weighted distance between the vertices of the polygon;

[0094] : is the parameter of Minkowski distance, is the Euclidean distance, is the Manhattan distance, is the Chebyshev distance; The larger the value, the less sensitive it is to abnormal points. When the edge of the substrate is straight and smooth, a larger r value can be used. When the edge is uneven and has sharp changes, a smaller r value can be used to improve the ability to fit the shape details.

[0095] :Indicates the i-th contour point and The angle of the jth vertex of the polygon;

[0096] :It is about the angle A function to adjust the influence of the contour point to the vertex direction, for example, ,in is a non-negative parameter, so the distance consistent with the vertex direction will get a smaller cost; the selection of parameter k can be combined with prior information such as the aspect ratio and inclination of the substrate design. For example, for a rectangular substrate, a larger k value can be selected to make the fitting result closer to a rectangle;

[0097] : is the regularization parameter, About The regularization term of the vertex coordinates of the polygon is used to control the complexity of the shape, such as taking It represents the sum of the squares of the lengths of each side, which will tend to obtain a polygon with shorter sides and more regular shape. When the substrate shape is simple and regular, a larger λ can be selected to obtain a more regular fitting result. When the shape is irregular, λ can be reduced to better fit the contour.

[0098] Minkowski distance is a generalized form of distance metric that can describe the distance between two points in different dimensional spaces. It is a generalization of Euclidean distance and Manhattan distance. Using the polygon fitting error function, an optimal p-gon can be obtained to approximate the extracted contour edge. The vertex coordinates of the p-gon are The geometric dimension parameters of the substrate are directly given, and the length of each side can be directly calculated, and the aspect ratio, area, etc. can be obtained accordingly. The slope and angle of the side reflect the straightness characteristics of the edge. Therefore, this formula provides a mathematical model for the contour fitting of step S1320. By minimizing the polygon error function, a robust fitting result can be obtained in the presence of noise and outliers. The formula introduces mechanisms such as weight, Minkowski distance, and angle function, which can be flexibly adjusted according to the specific characteristics of the ceramic substrate.

[0099] When the contour noise increases, the r value can be increased to reduce the influence of outliers; when the substrate shape is more regular, the λ and k values ​​can be increased to obtain a simpler fitting result; conversely, when the shape is more complex and changeable, the λ and k values ​​can be reduced to better fit the contour details.

[0100] In general, the polygon fitting formula makes full use of the extracted contour information and can obtain accurate and robust shape parameter estimation. The various parameters of the formula can be flexibly set according to the prior knowledge of the substrate and production requirements, making the fitting results more in line with practical problems. This provides a reliable shape expression for subsequent dimensional measurement and defect detection, which is of great significance to improving the accuracy and efficiency of ceramic substrate detection.

[0101] Step S1400, annotating the defect type of each sample in the visual image, associating and storing the shape and size features with the corresponding defect type labels, and constructing a shape and size feature database; associating and storing the surface texture features with the corresponding defect type labels, and constructing a surface texture feature database;

[0102] Step S1400 can significantly improve the accuracy of detection by systematically associating features and defect types, so that subsequent classification models can more accurately identify and classify defects; rich feature and label data enhance the generalization ability of the model, making it more stable in a variety of detection scenarios; in addition, this storage method of associated information promotes defect analysis and prediction, provides data support for subsequent improvements and optimizations, and further improves production efficiency; fast and accurate detection and classification of defects helps reduce the time and cost of manual inspection, thereby improving the overall efficiency of the production process. In summary, step S1400 not only improves the performance of the detection system by building a feature database, but also provides strong support for comprehensive defect management and prevention.

[0103] Step S2000, constructing a packaging substrate shape classification model, and judging whether the ceramic packaging substrate has a shape defect according to the shape and size characteristics of the ceramic packaging substrate and the packaging substrate shape classification model;

[0104] Furthermore, step S2000 includes:

[0105] Step S2100, inputting the extracted shape and size features into a pre-trained package substrate shape classification model to identify the shape category of the ceramic package substrate; the package substrate shape classification model adopts a convolutional neural network (CNN) based on deep learning, and the package substrate shape classification model includes a convolutional layer, a pooling layer, a fully connected layer and a Softmax layer;

[0106] Furthermore, if Figure 3 As shown, step S2100 includes:

[0107] Step S2110, forming a one-dimensional shape feature vector of fixed length from shape dimension features including polygon vertex coordinates, side length, angle, fillet radius and edge straightness as input to a package substrate shape classification model;

[0108] Step S2120, the one-dimensional shape feature vector passes through a convolution layer to generate a shape feature map;

[0109] Step S2130, the shape feature map enters the pooling layer, and the pooling layer downsamples the shape feature map to obtain a reduced-size shape feature dimension reduction map;

[0110] Step S2140: The fully connected layer flattens the shape feature dimensionality reduction map output by the pooling layer into a one-dimensional shape mapping vector, and extracts high-level semantic features of the one-dimensional shape mapping vector;

[0111] Step S2150, the Softmax layer obtains probability values ​​representing different shape categories based on the high-level semantic features of the one-dimensional shape mapping vector extracted by the fully connected layer, and the category with the largest probability value is the determined ceramic package substrate shape category recognition result.

[0112] Specifically, a convolutional neural network (CNN) based on deep learning is used as a shape classification model. CNN can effectively identify complex shape patterns through feature extraction by convolution layer, dimension reduction by pooling layer, and classification by fully connected layer. First, the shape size features such as polygon vertex coordinates, side length, angle, etc. obtained in step S1320 are formed into a one-dimensional vector of fixed length as the input of the CNN model. The input features pass through the first convolution layer, and convolve with the input through multiple convolution kernels to extract local feature patterns and generate multiple feature maps. Then, the feature maps are downsampled through the pooling layer to reduce the size of the feature maps. Then, several convolution layers and pooling layers are used alternately to gradually extract higher-level and more abstract shape feature representations. After the last pooling layer, multiple two-dimensional feature maps are flattened into one-dimensional vectors and input into the fully connected layer. The fully connected layer transforms and extracts semantics of features through matrix multiplication and nonlinear activation. Finally, the output of the fully connected layer is converted into a normalized vector representing the probability of different shape categories through the Softmax layer, and the one with the largest probability is taken as the final shape recognition result.

[0113] In order to train the package substrate shape classification model, it is necessary to further expand more shape feature samples of ceramic package substrates based on the shape size feature database constructed in step S1400, and manually annotate them according to predefined shape categories to create a training data set; randomly divide the sample set into a training set and a validation set, and input them into the CNN model for training; use optimization algorithms such as stochastic gradient descent to adjust the parameters of each layer of CNN by minimizing the cross entropy loss function on the training set; at the same time, evaluate the model performance on the validation set to avoid overfitting; and finally obtain a shape classification CNN model with superior performance. Step S2100 uses deep learning technology to automatically learn to extract hierarchical representations of shape features from a large number of samples, and establish an end-to-end mapping from shape size features to geometric shape categories, which can quickly and accurately determine the shape attribution of the substrate, provide a reference for subsequent size inspection, and help narrow the comparison range and improve detection efficiency. Compared with traditional shape recognition methods, the CNN-based classification method fully exploits the high-dimensional semantic information of shape features, and has the advantages of strong feature extraction capability, high classification accuracy, and good generalization performance. It can cope with the realistic challenges of diverse shapes of ceramic substrates and large differences between batches, thereby improving the robustness and adaptability of shape defect detection.

[0114] Step S2200, according to the identified shape category, obtaining the corresponding standard size range from a pre-built shape standard database;

[0115] Specifically, the shape design parameters of ceramic packaging substrates of different batches and models may be different, so it is necessary to pre-establish a shape standard database to store standard size information of various types of substrates; the database uses the substrate model as an index to record key size parameters such as design length, width, fillet radius, inclination angle, and the allowable tolerance range of various common shapes; for the substrate shape category identified in step S2100, the corresponding size parameters are queried in the shape standard database to obtain the standard values ​​and upper and lower limits of each indicator for subsequent size comparison; through systematic management of shape standard data, detection criteria can be quickly obtained, reducing the time for manual review of design drawings and improving detection efficiency; at the same time, the database supports flexible expansion and can be updated in time with changes in production processes, product upgrades, etc., providing a reliable decision-making basis for shape defect detection; step S2200 makes full use of existing shape design knowledge and constructs a knowledge base so that detection standards can be quickly reused, ensuring the standardization and consistency of the detection process.

[0116] Step S2300, comparing the actual size of the ceramic package substrate represented by the shape and size features extracted in step S1300 with the standard size range obtained in step S2100, to determine whether the ceramic package substrate has a shape defect;

[0117] Furthermore, if Figure 4 As shown, step S2300 includes:

[0118] Step S2310, calculating a size difference metric between an actual size and a standard size of the ceramic package substrate, wherein the size difference metric includes a vertex coordinate difference, a side length difference, an angle difference, a fillet radius difference, and an edge straightness difference;

[0119] Step S2320, calculating a comprehensive shape difference index according to vertex coordinate differences, side length differences, angle differences, fillet radius differences, and edge straightness differences;

[0120] Step S2330, if the comprehensive shape difference index exceeds a preset shape difference threshold, it is determined that the ceramic package substrate has a shape defect; otherwise, it is determined that the shape of the ceramic package substrate is qualified;

[0121] Specifically, step S2310 introduces multiple size difference measurement indicators, comprehensively considers multiple shape features such as vertex position, side length, angle, fillet, straightness, etc., and comprehensively quantifies the deviation between the actual shape and the standard; based on the above-mentioned multiple shape features, a comprehensive shape difference index is formed, so that the shape defect directly corresponds to a single quantitative index D, which is convenient for subsequent setting of thresholds and automatic discrimination; by assigning a difference metric (comprehensive shape difference index), the deviation quantification can be closer to reality and more targeted; the comprehensive shape difference index is compared with the shape difference threshold to realize automatic defect judgment, get rid of the subjectivity of manual visual inspection, and improve detection efficiency and consistency; the selection of the shape difference threshold needs to be based on the statistical analysis of a large number of samples, taking into account both the recognition rate and the misjudgment rate, which requires good experimental design and data accumulation; this step realizes the binary classification of defects (qualified / defective) by comparing with the threshold, providing a decision-making basis for subsequent defect disposal.

[0122] The calculation of the comprehensive shape difference index based on vertex coordinate difference, side length difference, angle difference, fillet radius difference and edge straightness difference includes:

[0123]

[0124] in:

[0125] : Comprehensive shape difference index, the value range is , The larger the value, the greater the difference between the actual shape and the standard shape. The smaller it is, the closer the two are.

[0126] : size difference metric, representing the difference in vertex coordinates , side length difference , Angle Difference , Corner radius difference and edge straightness difference ;

[0127] : Size difference measure The scaling factor is used to adjust the dimensions and value ranges of different types of differences so that they are The contribution of is basically balanced and can be set based on historical data or expert experience;

[0128] : Size difference measure The nonlinear index reflects right The nonlinear effect of hour, The impact is amplified when hour, The impact of is narrowed down and can be selected according to the importance and sensitivity of different types of differences;

[0129] : Size difference measure The weight coefficient reflects the relative contribution of different types of differences to the overall difference. , which can be determined comprehensively based on factors such as product functions, process requirements, and defect modes.

[0130] When a size difference measure When it increases, the corresponding factor It will also increase, leading to The overall value of increases, indicating that the overall shape deviation increases; on the contrary, when When decreasing, The value of will also decrease; when all When they are close to 0, It will also converge to 0, indicating that the actual shape is highly consistent with the standard shape; by flexibly setting The value of It can more sensitively capture key types of shape deviations and suppress the interference of minor differences, so as to more accurately and comprehensively evaluate the overall shape quality; The value can quickly screen and locate shape defects and provide a quantitative basis for process optimization;

[0131] ΔV reflects the deviation of the overall size, ΔL and ΔA correspond to the errors of length and angle size respectively, ΔR measures the difference of rounded corner features, and ΔS focuses on the shape of edge contour. This formula adopts the form of product-power function to integrate the five size difference metrics into a comprehensive index, which not only considers the independent influence of different types of differences, but also takes into account the interaction between them. Can fully reflect shape deviation; introduce scaling factor and nonlinear index , giving the formula more flexibility, the numerical range and nonlinearity of different difference quantities can be adjusted according to actual conditions, which can amplify the impact of key differences and reduce the interference of minor differences; the contribution of each difference is integrated in the form of weighted geometric mean, and the weight It can reflect the relative importance of different types of differences. The geometric mean can effectively smooth the impact of differences of different magnitudes and avoid individual maximum or minimum values ​​dominating the results. Shift to ( , making it numerically more intuitive, easier to compare and interpret, The closer it is to 0, the smaller the shape deviation is. The larger it is, the more significant the shape deviation is.

[0132] Step S3000 , identifying the surface defect type of the ceramic package substrate to be inspected according to the surface texture feature of the ceramic package substrate.

[0133] Furthermore, step S3000 includes:

[0134] Step S3100, based on the surface texture feature, searching the surface texture feature database for the sample most similar to the substrate to be tested;

[0135] Further, step S3100 includes:

[0136] Step S3110, calculating the similarity between the surface texture feature of the substrate to be tested and the feature of each historical sample in the surface texture feature database to obtain a similarity measurement value;

[0137] Step S3120, sorting the historical samples in the surface texture feature database according to the similarity measurement value, selecting the top-N1 samples with the highest similarity as the most similar samples, where N1 is an integer greater than 0;

[0138] Step S3130 , obtaining defect type labels corresponding to the Top-N1 most similar samples as candidates for the defect type of the substrate to be tested.

[0139] Specifically, in step S3100, similarity retrieval is performed in historical samples using surface texture features, and the reference sample closest to the defect mode of the substrate to be tested can be quickly locked. The defect types corresponding to these similar samples provide a priori reference for determining the defect type of the substrate to be tested. The core of similarity retrieval is the measurement of feature similarity. Common feature distance metrics include: Euclidean distance, which is the square root of the sum of the squares of the differences in each dimension of the feature vector, which intuitively reflects the absolute distance between two samples in the feature space; Mahalanobis distance, which takes into account the correlation between feature components and reduces the influence of the dimension differences of different dimensions; cosine similarity, which calculates the cosine value of the angle between feature vectors, measures the degree of proximity in the vector direction, and is insensitive to absolute value differences. Choosing a suitable distance metric can more accurately measure the similarity between samples. After calculating the similarity of all historical samples and the sample to be tested, the top-N1 most similar samples are sorted and selected. The selection of the N1 value needs to weigh the quantity and quality of the reference samples. If the N1 value is too large, irrelevant interference samples may be introduced; if the N1 value is too small, it may lack representativeness. Generally, 10 to 20 samples with the smallest distance or the largest similarity can be selected as references. Obtaining the defect type labels corresponding to these samples can provide multiple reference opinions for defect determination of the substrate to be tested.

[0140] Through the similarity retrieval in step S3100, a group of samples closest to the defect pattern of the substrate to be tested can be quickly found in the massive historical data. This reference mechanism based on sample similarity can significantly improve the efficiency and accuracy of defect judgment. On the one hand, through feature similarity matching, the exhaustive search of the entire database is avoided, narrowing the scope of judgment; on the other hand, the defect type represented by the most similar sample has a strong guiding role for the substrate to be tested and can be used as a powerful basis for judgment. At the same time, feature similarity plays a key role in the retrieval process. Using a variety of commonly used distance measurement methods, the similarity of samples can be comprehensively evaluated from different angles, especially the measurement methods such as Mahalanobis distance and cosine similarity that take into account the feature distribution structure, which can overcome the limitations of absolute distance and improve the robustness of similarity judgment. In short, step S3100 makes full use of the empirical knowledge of historical detection data, and provides a reliable priori reference for defect judgment through the similarity reasoning mechanism, which is a key step to improve detection accuracy and efficiency.

[0141] Step S3200, analyzing the defect type of the most similar sample to obtain a preliminary defect diagnosis result of the substrate to be tested;

[0142] Furthermore, if Figure 5 As shown, step S3200 includes:

[0143] Step S3210, performing statistical analysis on the defect type labels of the Top-N1 most similar samples, and calculating the frequency of occurrence of each defect type;

[0144] Step S3220: determine whether the frequency of occurrence of each defect type exceeds a preset frequency threshold. , the statistics exceed the frequency threshold The number of defect types n2; if n2=1, the defect type is used as the preliminary diagnosis result of the substrate to be tested, and the substrate to be tested is marked as a regular sample; if n2=0, the substrate to be tested is determined to contain unknown defect types, and the substrate to be tested is marked as a difficult sample; if n2≥2, the frequency difference ratio between the highest frequency defect type and other high frequency defect types is calculated. ;

[0145] Step S3230: If the frequency difference ratio Greater than the preset frequency difference threshold , the defect type with the highest frequency is taken as the preliminary diagnosis result of the substrate to be tested, and the substrate to be tested is marked as a regular sample; otherwise, it is determined that the substrate to be tested contains an unknown defect type, and the substrate to be tested is marked as a difficult sample.

[0146] Specifically, step S3200 can preliminarily determine the defect type of the substrate to be tested by statistically analyzing the defect types of the most similar samples. Since the most similar samples have a high degree of feature similarity with the substrate to be tested, their defect patterns are usually relatively close. Therefore, analyzing the defect type distribution of these samples has important guiding significance for inferring the defect type of the substrate to be tested. First, perform frequency statistics on the defect type labels of the Top-N1 samples to find out the one or several defect types that appear the most frequently. The type with the highest frequency often represents the most likely diagnosis result. Then, set a frequency threshold , if the highest frequency exceeds the frequency threshold, it is considered that the defect type of the substrate to be tested has been found. The setting of the threshold needs to take into account the accuracy and coverage of the judgment. The higher the frequency threshold, the greater the confidence of the judgment, but the recognition coverage may be reduced; the lower the threshold, the more defect types can be identified, but the accuracy of the judgment may decrease. It is necessary to select a suitable compromise point according to the actual application requirements. When the highest frequency does not reach the threshold, or there are multiple inconsistent types with close frequencies, it means that no convincing judgment can be found in the known defect types, and the substrate to be tested may contain unknown new defects, which requires further in-depth detection using intelligent diagnosis technology. This majority voting mechanism can quickly diagnose most common and typical defects, control the confidence of the judgment through the frequency threshold, and leave difficult and rare defects to subsequent more sophisticated detection, achieving a good balance between accuracy and efficiency.

[0147] Frequency Difference Ratio It reflects the relative difference in the number of occurrences between the highest frequency type and other high frequency types. The value of can be 10%-20%. The smaller the difference ratio, the closer the frequencies of these types are, and the more difficult it is to distinguish the dominant category. For example, the highest frequency type appears 10 times and the other type appears 9 times, then their frequency difference ratio is (10-9) / 10=10%, which is lower than the threshold of 20%. It can be considered that the two types are comparable and have similar frequencies. The introduction of the frequency difference ratio can quantitatively evaluate the similarity of different defect types in detection frequency.

[0148] It is worth noting that when calculating the frequency difference ratio, only high-frequency defect types are considered, generally the top three to five types that occur the most frequently. Low-frequency types appear less frequently and are not representative enough, so including them in the analysis may introduce noise interference. Focusing on analyzing high-frequency types can more accurately determine the frequency differences and distribution characteristics between the main defect types. In addition, in actual applications, if the frequency of a certain defect type is significantly higher than that of other types and exceeds the preset absolute threshold or relative threshold, it can also be directly determined as a dominant type without calculating the frequency difference ratio. In this case, the defect type has shown a clear dominant position, and it can be basically determined that the sample belongs to this type of defect.

[0149] In general, step S3200 makes a preliminary diagnosis of the substrate to be tested based on the defect type distribution of the most similar samples, making full use of the empirical knowledge of existing defects, which can significantly improve the detection efficiency and accuracy of common defects and is an effective initial screening mechanism. At the same time, it screens samples suspected of new defects and identifies special cases that are difficult to classify, providing clues for the subsequent introduction of intelligent diagnosis technology, so that the detection process can focus on key points and break through difficulties, and overall balance speed and accuracy.

[0150] For example, assume that there are the following defect type label statistics of the top-N1 most similar samples:

[0151] Defect type A: 6 times;

[0152] Defect type B: 3 times;

[0153] Defect type C: 1 time;

[0154] Step S3210: Count the occurrence frequency of each defect type:

[0155] A: 6 times;

[0156] B: 3 times;

[0157] C: 1 time;

[0158] Step S3220: Determine whether the frequency of each defect type exceeds a preset frequency threshold (assuming the threshold is 4):

[0159] A: 6 times (exceeding the threshold);

[0160] B: 3 times (not exceeding the threshold);

[0161] C: 1 time (not exceeding the threshold);

[0162] Count the number of defect types n2 that exceed the frequency threshold:

[0163] n2=1 (type A);

[0164] Since n2=1, defect type A is used as the preliminary diagnosis result of the substrate to be tested.

[0165] Step S3230: This step does not need to be executed because n2=1, and the conclusion is reached directly.

[0166] In this example, defect type A is used as the preliminary diagnosis result of the substrate to be tested because its frequency exceeds the preset threshold and it is the only defect type that meets the conditions.

[0167] Step S3300, using a meta-learning method to perform a second round of testing on the ceramic package substrate to be tested, including conventional samples and difficult samples, to obtain a final surface defect type determination result;

[0168] Furthermore, if Figure 6 As shown, step S3300 includes:

[0169] Step S3310, inputting the surface texture features of the ceramic package substrate to be inspected including conventional samples and difficult samples into the meta-learning model, and calculating the confidence scores of the samples for each defect type;

[0170] Step S3320: For the difficult sample, select the defect type with the highest confidence score as the final defect diagnosis result of the difficult sample;

[0171] Step S3330: for the regular sample, select the defect type with the highest confidence score as the second defect diagnosis result of the regular sample;

[0172] Step S3340, compare the preliminary defect diagnosis result and the second defect diagnosis result of the conventional sample. If the two are consistent, the result is used as the final defect diagnosis result of the conventional sample; if the two are inconsistent, the preliminary defect diagnosis result and the second defect diagnosis result are integrated to form the final defect diagnosis result of the conventional sample.

[0173] Specifically, the meta-learning model is pre-trained on defect data sets in multiple material fields and has the ability to classify defects across materials, processes, and types. For difficult samples that are difficult to identify in the initial screening, meta-learning technology is introduced for in-depth diagnosis. The meta-learning model is an intelligent diagnostic technology based on small sample learning, which can quickly adapt to new defect types through a small number of labeled samples. Unlike traditional machine learning that directly learns the mapping relationship between "defect-type", the meta-learning model learns a set of general "learning strategies" that can extract common knowledge from past learning experience and then apply it to small sample learning of new defects. This "learning how to learn" approach enables the model to have a certain generalization ability. When faced with unfamiliar defects brought by new processes and new materials, it can quickly understand the distribution law of its characteristics and make accurate classifications. The surface texture features of difficult samples are input into the trained meta-learning model, and the model can give confidence scores for each learned defect type and select the highest score as the diagnosis result. This flexible judgment method based on confidence can adapt to the open changes of defect categories.

[0174] To further improve the reliability of meta-learning diagnosis, step S3300 compares its results with the initial screening results. If the two are consistent, the final judgment can be more confident; if the two are inconsistent, a weighted average strategy is adopted for fusion, and the results with high confidence will be given a greater weight, and the results with low confidence will have a smaller weight. The dynamic adjustment of the weight can be determined based on factors such as the typicality of the defect mode and the familiarity of the material process. This soft weighting method can not only give full play to the advantages of the two judgments, but also suppress the influence of local misjudgment.

[0175] In summary, step S3300 uses meta-learning technology to diagnose difficult defects, giving full play to the rapid adaptability of small sample learning and meeting the challenge of determining new defects. By comparing and integrating with the initial screening results, the robustness of the diagnosis is further improved. The meta-learning method has a certain ability to transfer fields, which expands the scope of application of the detection system from a single material and a single process to multiple materials and multiple processes, making it closer to actual application needs.

[0176] It should be noted that meta-learning technology has demonstrated unique advantages in the discovery and identification of new defects. Traditional supervised learning models can only identify a limited number of labeled defect types, and are often helpless in the face of newly emerging defects. Meta-learning models can quickly learn the key features of new defects from a small number of labeled samples, and can draw inferences from one example like a human, breaking through the limitations of the inherent knowledge base. This enables the detection system to have the ability to continuously evolve and learn, and can autonomously learn new defect patterns as production practice advances, forming a virtuous cycle of positive feedback.

[0177] For example, suppose a new type of ceramic packaging substrate uses an innovative sintering process, which produces some defect types that have never been seen before. The traditional detection model cannot accurately identify these defects due to the lack of training samples. The meta-learning model has learned a set of methods to quickly extract feature patterns from unfamiliar data through pre-training. Engineers only need to extract a small number of new defect samples from the production line, label their defect types, and input them into the meta-learning model for learning. The model can grasp the key features of these new defects in a short time for subsequent defect judgment. The entire process does not require the collection of a large number of samples of the defect, saving time and cost.

[0178] This small sample learning paradigm enables the knowledge base of the detection system to break through the barriers of fixation and possess the ability to grow autonomously. This is of great significance for dealing with actual scenarios where new materials, new processes, and new defects are constantly emerging. It enables the detection solution to truly keep pace with the times and always stand at the forefront of technology, providing a powerful quality monitoring method to ensure the innovation of ceramic substrate products.

[0179] Step S4000, outputting a test result report of the ceramic package substrate.

[0180] Specifically, the inspection result report of the ceramic package substrate includes the shape defect detection results, surface defect type determination results, and corresponding defect descriptions and improvement suggestions of the ceramic package substrate. Through step S4000, the shape defect detection results and surface defect type diagnosis results obtained in the previous steps are summarized and output to form a complete ceramic package substrate inspection report. The report not only lists the detected defects, but also gives corresponding defect descriptions and improvement suggestions, providing a basis for enterprises to fully grasp the product quality status. Feedback of the inspection result report to the production management department in a timely manner can achieve closed-loop management of quality issues, help enterprises optimize production processes, improve product quality, and thus improve market competitiveness.

[0181] In summary, the ceramic package substrate detection method based on visual inspection and meta-learning provided in this embodiment acquires image data from multiple perspectives and multiple illuminations, extracts shape and texture features, and constructs a feature database; detects shape defects through shape classification and size comparison, diagnoses surface defects through similarity retrieval and meta-learning, and outputs a complete inspection report. This method integrates machine vision and artificial intelligence technology to achieve comprehensive intelligent detection of ceramic package substrate defects, providing new ideas and methods for improving the yield of substrate products.

[0182] Example 2

[0183] This embodiment provides a ceramic package substrate detection system based on visual inspection and meta-learning on the basis of embodiment 1. Figure 7 As shown, including:

[0184] Feature acquisition module: used to acquire the visual image of the ceramic package substrate to be inspected, extract the shape and size features and surface texture features of the visual image, and build a shape and size feature database and a surface texture feature database;

[0185] Shape defect detection module: used to build a packaging substrate shape classification model, and determine whether the ceramic packaging substrate has shape defects based on the shape and size characteristics of the ceramic packaging substrate and the packaging substrate shape classification model;

[0186] Surface defect detection module: used to identify the surface defect type of the ceramic package substrate to be detected according to the surface texture characteristics of the ceramic package substrate.

[0187] The feature acquisition module includes an image acquisition unit, an image preprocessing unit, a feature extraction unit and a database construction unit;

[0188] The image acquisition unit is used to acquire visual images of N batches of ceramic package substrates to be inspected under different viewing angles and lighting conditions; the viewing angles at least include a front viewing angle, a back viewing angle, and a side viewing angle; each viewing angle includes at least three lighting modes: annular, coaxial, and oblique; N is an integer greater than 0;

[0189] The image preprocessing unit is used to preprocess the acquired visual image;

[0190] The feature extraction unit is used to extract shape and size features and surface texture features of the ceramic package substrate from the visual image;

[0191] The database construction unit is used to annotate the defect type of each sample in the visual image, associate and store the shape and size features with the corresponding defect type labels, and construct a shape and size feature database; associate and store the surface texture features with the corresponding defect type labels, and construct a surface texture feature database.

[0192] In the feature extraction unit, the step of extracting the shape, size and surface texture features of the ceramic package substrate from the visual image includes:

[0193] Extract the contour edge of the ceramic package substrate from the visual image and obtain the contour edge point set , where n is the number of contour points, is the coordinate of the i-th contour point;

[0194] Performing polygon fitting on the extracted contour edge to obtain shape and size characteristics of the ceramic package substrate, wherein the shape and size characteristics include polygon vertex coordinates, side length, angle, fillet radius and edge straightness of the ceramic package substrate;

[0195] Extract surface texture features of ceramic package substrates from visual images.

[0196] The shape defect detection module includes a shape category recognition unit, a standard size acquisition unit and a shape defect determination unit;

[0197] The shape category recognition unit is used to input the extracted shape size features into a pre-trained packaging substrate shape classification model to identify the shape category of the ceramic packaging substrate;

[0198] The package substrate shape classification model adopts a convolutional neural network (CNN) based on deep learning, and the package substrate shape classification model includes a convolution layer, a pooling layer, a fully connected layer and a Softmax layer;

[0199] In the shape category recognition unit, the extracted shape size features are input into a pre-trained package substrate shape classification model to recognize the shape category of the ceramic package substrate, including:

[0200] The shape dimension features including the polygon vertex coordinates, side length, angle, fillet radius and edge straightness are used to form a one-dimensional shape feature vector of fixed length as the input of the package substrate shape classification model;

[0201] The one-dimensional shape feature vector passes through the convolution layer to generate a shape feature map;

[0202] The shape feature map enters the pooling layer, which downsamples the shape feature map to obtain a reduced-size shape feature dimension reduction map;

[0203] The fully connected layer flattens the shape feature dimensionality reduction map output by the pooling layer into a one-dimensional shape mapping vector and extracts high-level semantic features of the one-dimensional shape mapping vector;

[0204] The Softmax layer obtains the probability values ​​representing different shape categories based on the high-level semantic features of the one-dimensional shape mapping vector extracted by the fully connected layer. The category with the largest probability value is the determined ceramic package substrate shape category recognition result.

[0205] The standard size acquisition unit is used to acquire the corresponding standard size range from a pre-built shape standard database according to the identified shape category;

[0206] The shape defect determination unit is used to compare the actual size of the ceramic package substrate represented by the shape and size characteristics with the standard size range of the ceramic package substrate to determine whether the ceramic package substrate has a shape defect;

[0207] In the shape defect determination unit, comparing the actual size of the ceramic package substrate represented by the shape size feature with the standard size range of the ceramic package substrate to determine whether the ceramic package substrate has a shape defect includes:

[0208] Calculating a size difference metric between an actual size of a ceramic package substrate and a standard size, wherein the size difference metric includes a vertex coordinate difference, a side length difference, an angle difference, a fillet radius difference, and an edge straightness difference;

[0209] The comprehensive shape difference index is calculated based on vertex coordinate differences, side length differences, angle differences, corner radius differences, and edge straightness differences;

[0210] If the comprehensive shape difference index exceeds a preset shape difference threshold, it is determined that the ceramic package substrate has a shape defect; otherwise, it is determined that the shape of the ceramic package substrate is qualified.

[0211] The surface defect detection module includes a similar sample retrieval unit, a preliminary diagnosis unit and a final diagnosis unit;

[0212] The similar sample retrieval unit is used to retrieve the sample most similar to the substrate to be tested in the surface texture feature database based on the surface texture feature;

[0213] In the similar sample retrieval unit, the step of retrieving the sample most similar to the substrate to be tested in the surface texture feature database based on the surface texture feature comprises:

[0214] Calculate the similarity between the surface texture feature of the substrate to be tested and the feature of each historical sample in the surface texture feature database to obtain a similarity measurement value;

[0215] According to the similarity metric, the historical samples in the surface texture feature database are sorted, and the Top-N1 samples with the highest similarity are selected as the most similar samples;

[0216] The defect type labels corresponding to the Top-N1 most similar samples are obtained as candidates for the defect type of the substrate to be tested.

[0217] The preliminary diagnosis unit is used to analyze the defect type of the most similar sample to obtain a preliminary defect diagnosis result of the substrate to be tested;

[0218] The preliminary diagnosis unit analyzes the defect type of the most similar sample to obtain the preliminary defect diagnosis result of the substrate to be tested, including:

[0219] Perform statistical analysis on the defect type labels of the top-N1 most similar samples and calculate the frequency of each defect type.

[0220] Determine whether the frequency of each defect type exceeds the preset frequency threshold , the statistics exceed the frequency threshold The number of defect types n2; if n2=1, the defect type is used as the preliminary diagnosis result of the substrate to be tested, and the substrate to be tested is marked as a regular sample; if n2=0, the substrate to be tested is determined to contain unknown defect types, and the substrate to be tested is marked as a difficult sample; if n2≥2, the frequency difference ratio between the highest frequency defect type and other high frequency defect types is calculated. ;

[0221] If the frequency difference ratio Greater than the preset frequency difference threshold , the defect type with the highest frequency is taken as the preliminary diagnosis result of the substrate to be tested, and the substrate to be tested is marked as a regular sample; otherwise, it is determined that the substrate to be tested contains an unknown defect type, and the substrate to be tested is marked as a difficult sample.

[0222] The final diagnosis unit is used to perform a second round of detection on the to-be-detected ceramic package substrates including conventional samples and difficult samples using a meta-learning method to obtain a final surface defect type determination result;

[0223] In the final diagnosis unit, the second round of testing of the to-be-tested ceramic package substrates including conventional samples and difficult samples using the meta-learning method includes:

[0224] The surface texture features of the ceramic package substrate to be inspected, including conventional samples and difficult samples, are input into the meta-learning model to calculate the confidence scores of the samples for each defect type;

[0225] For difficult samples, the defect type with the highest confidence score is selected as the final defect diagnosis result of the difficult sample;

[0226] For conventional samples, the defect type with the highest confidence score is selected as the second defect diagnosis result of the conventional samples;

[0227] Compare the preliminary defect diagnosis result and the second defect diagnosis result of the conventional sample. If the two are consistent, the result is used as the final defect diagnosis result of the conventional sample. If the two are inconsistent, the preliminary defect diagnosis result and the second defect diagnosis result are integrated to form the final defect diagnosis result of the conventional sample.

[0228] The ceramic package substrate inspection system based on visual inspection and meta-learning also includes a report output module, which is used to output an inspection result report of the ceramic package substrate.

[0229] Example 3

[0230] This embodiment discloses an electronic device 500, such as Figure 8As shown, the electronic device 500 may include one or more processors and one or more memories. The memories store computer readable codes, which, when executed by the one or more processors, may execute the ceramic package substrate detection method based on visual inspection and meta-learning as described above.

[0231] The method or system according to the embodiment of the present application can also be used by Figure 8 The electronic device architecture shown in FIG. Figure 8 As shown, the electronic device 500 may include a bus 501, one or more CPUs 502, a read-only memory (ROM) 503, a random access memory (RAM) 504, a communication port 505 connected to a network, an input / output component 506, a hard disk 507, etc. The storage device in the electronic device 500, such as the ROM 503 or the hard disk 507, may store the ceramic package substrate detection method based on visual inspection and meta-learning provided in the present application. The ceramic package substrate detection method based on visual inspection and meta-learning may, for example, include: obtaining a visual image of the ceramic package substrate to be detected, extracting the shape and size features and surface texture features of the visual image, and constructing a shape and size feature database and a surface texture feature database; constructing a package substrate shape classification model, and judging whether the ceramic package substrate has a shape defect according to the shape and size features of the ceramic package substrate and the package substrate shape classification model; identifying the type of surface defect of the ceramic package substrate to be detected according to the surface texture features of the ceramic package substrate.

[0232] Furthermore, the electronic device 500 may also include a user interface 508. Figure 8 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 8 One or more components of an electronic device are shown.

[0233] Example 4

[0234] This embodiment discloses a computer-readable storage medium 600, such as Fig. 9 As shown, computer-readable instructions are stored on the computer-readable storage medium 600. When the computer-readable instructions are executed by the processor, the ceramic package substrate detection method based on visual inspection and meta-learning according to the embodiment of the present application described with reference to the above figures can be executed. The storage medium 600 includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache). Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0235] In addition, according to the implementation of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the present application provides a non-transitory machine-readable storage medium, which stores machine-readable instructions, and the machine-readable instructions can be executed by a processor to execute instructions corresponding to the method steps provided in the present application, such as: obtaining a visual image of a ceramic package substrate to be inspected, extracting shape and size features and surface texture features of the visual image, and constructing a shape and size feature database and a surface texture feature database; constructing a package substrate shape classification model, and judging whether the ceramic package substrate has a shape defect according to the shape and size features of the ceramic package substrate and the package substrate shape classification model; identifying the type of surface defects of the ceramic package substrate to be inspected according to the surface texture features of the ceramic package substrate. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed.

[0236] The methods, systems, and devices of the present application may be implemented in many ways. For example, the methods, systems, and devices of the present application may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present application may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers a recording medium storing a program for executing the method according to the present application.

[0237] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0238] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A ceramic package substrate detection method based on visual inspection and meta-learning, characterized in that: The method comprises: Acquire a visual image of the ceramic package substrate to be inspected, extract shape and size features and surface texture features of the visual image, and build a shape and size feature database and a surface texture feature database; Constructing a packaging substrate shape classification model, and judging whether the ceramic packaging substrate has shape defects according to the shape and size characteristics of the ceramic packaging substrate and the packaging substrate shape classification model; the packaging substrate shape classification model adopts a convolutional neural network based on deep learning; Identify the surface defect type of the ceramic package substrate to be inspected according to the surface texture characteristics of the ceramic package substrate; The determining whether the ceramic package substrate has a shape defect comprises: The extracted shape and size features are input into a pre-trained packaging substrate shape classification model to identify the shape category of the ceramic packaging substrate; according to the identified shape category, the corresponding standard size range is obtained from a pre-built shape standard database; the actual size of the ceramic packaging substrate represented by the extracted shape and size features is compared with the standard size range of the ceramic packaging substrate to determine whether the ceramic packaging substrate has shape defects; The comparing the actual size of the ceramic package substrate represented by the extracted shape and size features with the standard size range of the ceramic package substrate to determine whether the ceramic package substrate has a shape defect includes: Calculate the size difference metric between the actual size and the standard size of the ceramic package substrate, the size difference metric including vertex coordinate difference, side length difference, angle difference, fillet radius difference and edge straightness difference; calculate a comprehensive shape difference index based on the vertex coordinate difference, side length difference, angle difference, fillet radius difference and edge straightness difference; if the comprehensive shape difference index exceeds a preset shape difference threshold, determine that the ceramic package substrate has a shape defect; otherwise, determine that the shape of the ceramic package substrate is qualified.

2. The method for detecting ceramic package substrates based on visual inspection and meta-learning according to claim 1, characterized in that: The step of acquiring visual images of the ceramic package substrate to be inspected comprises: acquiring visual images of N batches of the ceramic package substrate to be inspected under different viewing angles and lighting conditions; the viewing angles at least include a front viewing angle, a back viewing angle, and a side viewing angle; each viewing angle at least includes three lighting modes: annular, coaxial, and oblique; and N is an integer greater than 0; The constructing of the shape and size feature database and the surface texture feature database comprises: Label the defect type of each sample in the visual image, associate the shape and size features with the corresponding defect type labels and store them to build a shape and size feature database; associate the surface texture features with the corresponding defect type labels and store them to build a surface texture feature database.

3. The method for detecting ceramic package substrates based on visual inspection and meta-learning according to claim 1, characterized in that: The extracting of shape and size features of the visual image comprises: Extract the contour edge of the ceramic package substrate from the visual image and obtain the contour edge point set , where n is the number of contour points, is the coordinate of the i-th contour point; Performing polygon fitting on the extracted contour edge to obtain shape and size characteristics of the ceramic package substrate, wherein the shape and size characteristics include polygon vertex coordinates, side length, angle, fillet radius and edge straightness of the ceramic package substrate; The polygon fitting of the extracted contour edge comprises: Assume a polygon to fit the contour edge point set, The coordinates of the vertices of the polygon are , for The coordinates of the jth vertex of the polygon; establish the polygon fitting error function E, and solve , find a set of vertex coordinates that minimizes the polygon fitting error, .

4. The method for detecting ceramic package substrates based on visual inspection and meta-learning according to claim 1, characterized in that: The package substrate shape classification model includes a convolution layer, a pooling layer, a fully connected layer and a Softmax layer; The step of inputting the extracted shape and size features into a pre-trained package substrate shape classification model to identify the shape category of the ceramic package substrate includes: The shape size features including the coordinates of the polygon vertices, side length, angle, fillet radius and edge straightness constitute a one-dimensional shape feature vector of fixed length as the input of the package substrate shape classification model; the one-dimensional shape feature vector passes through the convolution layer to generate a shape feature map; the shape feature map enters the pooling layer, and the pooling layer downsamples the shape feature map to obtain a reduced-size shape feature dimensionality reduction map; the fully connected layer flattens the shape feature dimensionality reduction map output by the pooling layer into a one-dimensional shape mapping vector, and extracts high-level semantic features of the one-dimensional shape mapping vector; the Softmax layer obtains probability values ​​representing different shape categories based on the high-level semantic features of the one-dimensional shape mapping vector extracted by the fully connected layer, and the category with the largest probability value is the determined ceramic package substrate shape category recognition result.

5. The method for detecting ceramic package substrates based on visual inspection and meta-learning according to claim 1, characterized in that: The method of identifying the surface defect type of the ceramic package substrate to be inspected according to the surface texture feature of the ceramic package substrate comprises: Based on the surface texture features, searching for a sample most similar to the substrate to be tested in a surface texture feature database; Analyze the defect types of the most similar samples to obtain preliminary defect diagnosis results of the substrate to be tested; The meta-learning method is used to perform a second round of inspection on the substrate to be tested to obtain the final surface defect type determination result.

6. The method for detecting ceramic package substrates based on visual inspection and meta-learning according to claim 5, characterized in that: The method of searching the surface texture feature database for the sample most similar to the substrate to be tested based on the surface texture feature comprises: Calculate the similarity between the surface texture feature of the substrate to be tested and the feature of each historical sample in the surface texture feature database to obtain a similarity measurement value; According to the similarity metric, the historical samples in the surface texture feature database are sorted, and the Top-N1 samples with the highest similarity are selected as the most similar samples; The defect type labels corresponding to the Top-N1 most similar samples are obtained as candidates for the defect type of the substrate to be tested.

7. The method for detecting ceramic package substrates based on visual inspection and meta-learning according to claim 6, characterized in that: The step of analyzing the defect type of the most similar sample to obtain a preliminary defect diagnosis result of the substrate to be tested includes: Perform statistical analysis on the defect type labels of the top-N1 most similar samples and calculate the frequency of each defect type. Determine whether the frequency of each defect type exceeds the preset frequency threshold , the statistics exceed the frequency threshold The number of defect types n2; if n2=1, the defect type is used as the preliminary diagnosis result of the substrate to be tested, and the substrate to be tested is marked as a regular sample; if n2=0, the substrate to be tested is determined to contain unknown defect types, and the substrate to be tested is marked as a difficult sample; if n2≥2, the frequency difference ratio between the highest frequency defect type and other high frequency defect types is calculated. ; If the frequency difference ratio Greater than the preset frequency difference threshold , the defect type with the highest frequency is taken as the preliminary diagnosis result of the substrate to be tested, and the substrate to be tested is marked as a regular sample; otherwise, it is determined that the substrate to be tested contains an unknown defect type, and the substrate to be tested is marked as a difficult sample.

8. The method for detecting ceramic package substrates based on visual inspection and meta-learning according to claim 7, characterized in that: The second round of detection of the substrate to be tested by using the meta-learning method includes: The surface texture features of the ceramic package substrate to be inspected, including conventional samples and difficult samples, are input into the meta-learning model to calculate the confidence scores of the samples for each defect type; For difficult samples, the defect type with the highest confidence score is selected as the final defect diagnosis result of the difficult sample; For conventional samples, the defect type with the highest confidence score is selected as the second defect diagnosis result of the conventional samples; Compare the preliminary defect diagnosis result and the second defect diagnosis result of the conventional sample. If the two are consistent, the result is used as the final defect diagnosis result of the conventional sample. If the two are inconsistent, the preliminary defect diagnosis result and the second defect diagnosis result are integrated to form the final defect diagnosis result of the conventional sample.

9. A ceramic package substrate detection system based on visual inspection and meta-learning, which is used to implement the ceramic package substrate detection method based on visual inspection and meta-learning as described in any one of claims 1 to 8, characterized in that: The system comprises: Feature acquisition module: used to acquire the visual image of the ceramic package substrate to be inspected, extract the shape and size features and surface texture features of the visual image, and build a shape and size feature database and a surface texture feature database; Shape defect detection module: used to build a packaging substrate shape classification model, and determine whether the ceramic packaging substrate has shape defects based on the shape and size characteristics of the ceramic packaging substrate and the packaging substrate shape classification model; Surface defect detection module: used to identify the surface defect type of the ceramic package substrate to be detected according to the surface texture characteristics of the ceramic package substrate.

Images