Method for automatically detecting quality of conductive ink screen printing based on big data model

Abstract

The invention discloses an automatic conductive ink screen printing quality detection method based on a big data model. The method comprises the following steps: acquiring hyperspectral image data, three-dimensional shape data and real-time impedance distribution data of a conductive ink screen printing layer; performing local outlier factor optimization processing on the hyperspectral image data, and extracting a pixel-level spectral feature vector; performing vertical weight function Gaussian regression filtering on the three-dimensional shape data to generate a topological smooth curved surface coordinate set; optimizing the real-time impedance distribution data based on the conductivity distribution matrix, and outputting a noise reduction conductivity matrix; constructing an association graph and a node feature tensor based on the detection key points, and generating silk-screen multi-modal data; through modal interaction, features are extracted in stages, and cross-modal joint features are extracted through multi-stage joint clustering optimization; and a task space separation head module is input, and a quality evaluation result is output through classification and regression branches. According to the method, multi-modal data deep fusion, noise robustness improvement, cross-modal feature accurate extraction and efficient real-time detection are realized.

Description

Technical Field

[0001] The present invention relates to the field of intelligent detection technology, and in particular to an automatic detection method for conductive ink screen printing quality based on a big data model. Background Art

[0002] Conductive ink screen printing is a key process in the manufacture of electronic devices, and its quality directly affects circuit conductivity and product reliability. At present, the detection of screen printing defects mainly relies on a single detection method (such as optical imaging or electrical performance testing), which has the following problems: Single detection dimension: Traditional methods find it difficult to simultaneously capture multi-physics coupling defects such as material composition (such as oxidation), geometric deformation (such as warping) and electrical properties (such as conductivity mutation).

[0003] Severe noise interference: Hyperspectral images are easily affected by ambient light and sensor noise, three-dimensional morphology data is inaccurately filtered due to surface roughness, and impedance measurement data is easily affected by contact resistance.

[0004] Insufficient feature fusion: Existing technologies have limited ability to model the association of multimodal data and cannot effectively utilize cross-modal complementary information (such as the coordinated changes in spectral reflectance and conductivity).

[0005] Lack of real-time performance: Traditional algorithms have low computational efficiency in complex scenarios and are unable to meet the rapid inspection requirements of production lines.

[0006] Although some existing technologies involve multimodal detection, the following defects are common: defect detection is based on hyperspectral data, but the three-dimensional morphology and impedance data are not integrated, resulting in a high rate of missed detection of geometric deformation and conductive defects; convolutional networks are used to process conductivity distribution, but data noise is not optimized, and there is a lack of cross-modal feature alignment mechanism, resulting in a high rate of misjudgment of cold solder joints and oxidized areas. Summary of the invention

[0007] In order to overcome the shortcomings of the prior art, the present invention provides an automatic detection method for conductive ink silk screen printing quality based on a big data model, which realizes deep fusion of multimodal data, improvement of noise robustness, precise extraction of cross-modal features and efficient real-time detection.

[0008] To solve the above problems, the technical solution adopted by the present invention is as follows: A method for automatically detecting conductive ink screen printing quality based on a big data model comprises the following steps: Obtain hyperspectral image data, three-dimensional morphology data and real-time impedance distribution data of the conductive ink screen printing layer; The local outlier factor optimization processing is performed on the hyperspectral image data to extract the pixel-level spectral feature vector; The three-dimensional morphological data is filtered by vertical weight function Gaussian regression to generate a topological smooth surface coordinate set; Optimize the real-time impedance distribution data based on the conductivity distribution matrix and output the noise-reduced conductivity matrix; Construct association graphs and node feature tensors based on detected key points to generate silk screen multimodal data; Extract cross-modal joint features through modal interaction, phased feature extraction, and multi-level joint clustering optimization; The input task space separation head module outputs the quality assessment results including defect type and location coordinates through classification and regression branches.

[0009] As a preferred embodiment of the present invention, the local outlier factor optimization process includes: Set the optimization window and calculate the pixel kurtosis in the window through the fourth-order and second-order cumulants of the hyperspectral image data; Get each pixel in the window The distance to other pixels, take the Small distance as pixel Distance, and construct pixels Distance neighborhood; Get each pixel To neighboring cells The reachable distance and generate the local reachable density; According to the pixel Distance neighborhood and local reachability density are calculated for each pixel. The local outlier factor of Determine the window detection threshold based on the number of window pixels; When the pixel kurtosis is greater than the window detection threshold, the pixels whose local outlier factors exceed the preset value are removed and then the next optimization window is processed; otherwise, the next optimization window is directly processed; Traverse all windows to complete data optimization.

[0010] As a preferred implementation manner of the present invention, the vertical weight function Gaussian regression filtering includes: Set the initial values ​​of the vertical weight functions of each axis to be the same and set the residual convergence accuracy threshold; right The axis is subjected to Gaussian regression filtering to generate the initial three-dimensional reference surface, and the residual function value is obtained through the three-dimensional Euclidean distance; The iteration is terminated when the differences in the residual function values ​​of each axis are all less than the residual convergence accuracy threshold; otherwise, the vertical weight function is dynamically corrected based on the axial residual distribution and surface gradient; The weight function matrix is ​​updated to generate a new three-dimensional filtering reference surface, and a topological smooth surface coordinate set is output when the termination condition is met.

[0011] As a preferred embodiment of the present invention, optimizing real-time impedance distribution data includes: Extract the difference between the boundary voltage and the reference voltage as an input vector; Initial guessing stage: The conductivity distribution is reconstructed through two-level fully connected layers, the activation function uses PReLU and the initial conductivity distribution matrix is ​​generated through matrix multiplication; Noise reduction optimization stage: the initial conductivity distribution matrix is ​​input into the convolutional network of the encoder-decoder structure. The encoder uses the PReLU activation function and the decoder uses the Leaky ReLU activation function. The network parameters are trained end-to-end and the denoised conductivity matrix is ​​output.

[0012] As a preferred embodiment of the present invention, generating silk screen multimodal data includes: Extracting a set of geometric key points containing the principal curvature of the surface from a topologically smooth surface coordinate set; Extracting a spectral keypoint set containing spectral gradient from pixel-level spectral feature vector; Extracting conductivity mutation key point set from the denoised conductivity matrix; The key point set is fused to generate a multi-dimensional feature vector set, and a key point association graph is constructed. The edge weight is calculated based on the cross-modal similarity; The geometric structure is enforced by intra-modal label constraints and the label alignment regularization term is calculated; Complementary alignment is achieved through cross-modal projection loss, and the complementary alignment regularization term is calculated; The regularization term is combined with the task loss optimization to output silkscreen multimodal data consisting of the association graph structure and node feature tensors.

[0013] As a preferred embodiment of the present invention, extracting cross-modal joint features includes: Generate cross-modal fusion input based on two-to-two modal interactions based on detected key points; An improved ResNet50 three-stream network is used to extract multi-scale fusion features in stages; The defect-sensitive features are enhanced through the local-global feature association graph and multi-level joint clustering optimization is performed to output cross-modal joint features.

[0014] As a preferred embodiment of the present invention, the two-to-two modal interactions include: hyperspectral and three-dimensional morphology interaction, three-dimensional morphology and impedance interaction, splicing interaction features and original data to generate enhanced input; The staged feature extraction includes: extracting spectral-spatial features, geometric topological features and conductivity gradient features in an independent parameter stage, and generating cross-modal shared features in a shared parameter stage.

[0015] As a preferred embodiment of the present invention, the multi-level joint clustering optimization includes: Channel clustering: dynamically group feature channels and minimize the second-order statistics within the group; Intra-class-modal clustering: triplet loss is used to constrain the cross-modal distance of similar samples; Inter-class separation: enlarging the difference of heterogeneous features through maximum margin loss; Construct the overall optimization objective function to optimize the network parameters, as shown in Formula 23: (twenty three); In the formula, is the overall optimization goal, , , is the weight coefficient, Remove redundancy from channel-level features, is the intra-class-modal feature alignment, For inter-class feature separation.

[0016] As a preferred embodiment of the present invention, outputting the quality assessment result includes: The cross-modal joint features are input into the task space separation head module and processed hierarchically according to the downsampling ratio of the feature map; The classification branch uses focal loss to optimize the output anchor box category probability; The regression branch uses smooth L1 loss to predict the box offset; Dynamically adjust the anchor box matching strategy based on cross-modal feature similarity; Non-maximum suppression is performed on candidate boxes whose classification confidence is higher than the threshold, multi-scale prediction results are fused, and structured quality assessment results are output.

[0017] As a preferred embodiment of the present invention, the classification process comprises: Large-scale feature maps through dense anchor boxes Predict small target defects, where is the width and height of the small target anchor box, is the number of small target anchor boxes; Small-sized feature maps are obtained through sparse anchor boxes Predict large target defects, where is the width and height of the large target anchor box, is the number of large target anchor boxes.

[0018] Compared with the prior art, the present invention has the following beneficial effects: Detection accuracy is significantly improved: Through local outlier factor optimization (LOF) and Gaussian regression filtering, the noise suppression rate of hyperspectral images is greatly improved, and the smoothness error of three-dimensional morphology surfaces is significantly reduced. The cross-modal regularization term significantly reduces the alignment error of similar defect features and greatly improves the separation of heterogeneous features.

[0019] Enhanced multi-modal collaborative analysis capabilities: The key point correlation diagram combines spectral gradient, surface curvature and conductivity mutation characteristics, and has a high detection rate for complex defects (such as oxidation-related deformation). The three-stream deep network achieves multi-scale feature fusion through the independent-shared parameter stage, and accurately identifies tiny defects.

[0020] Real-time and robustness optimization: The cascade processing architecture of the task space separation head module (TSDHead) greatly shortens the single-board detection time and supports full-speed operation of the production line. The dynamic anchor frame matching strategy combines geometric overlap (IoU) and cross-modal semantic similarity to significantly reduce the mismatch rate.

[0021] High industrial adaptability: The hardware execution unit (hyperspectral imager, 3D laser scanner, etc.) is directly compatible with existing detection equipment, without the need to modify the production line. The end-to-end training model supports online incremental learning and can adapt to different conductive ink formulas and process parameters.

[0022] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a step diagram of the method for automatically detecting the quality of conductive ink screen printing of the present invention; Figure 2 It is a logic block diagram of the vertical weight function Gaussian regression filtering of the present invention. DETAILED DESCRIPTION

[0024] See also Figure 1 The conductive ink screen printing quality automatic detection method based on the big data model provided by the present invention comprises the following steps: Step S1: Acquire hyperspectral image data, three-dimensional morphology data and real-time impedance distribution data of the conductive ink screen printing layer; Step S2: Optimizing the hyperspectral image data based on the local outlier factor to extract the pixel-level spectral feature vector; Step S3: applying vertical weight function Gaussian regression filtering to the three-dimensional morphology data to generate a topological smooth surface coordinate set; Step S4: optimizing the model of the real-time impedance distribution data based on the conductivity distribution matrix, and outputting a noise-reduced conductivity matrix; Step S5: extract several key detection points, construct a correlation graph and node feature tensor for multi-dimensional correlation to form silk screen multimodal data; Step S6: extracting cross-modal joint features from silk screen multimodal data through modal interaction, phased feature extraction, and multi-level joint clustering optimization; Step S7: Input the cross-modal joint features into the task space separation head module, and output the quality assessment results including defect type and location coordinates through the classification branch, regression branch and matching strategy.

[0025] In the above step S2, when performing the optimization process based on the local outlier factor, it includes: The optimization window is set, and the pixel kurtosis in the window is obtained by the fourth-order cumulant and second-order cumulant of the hyperspectral image data, as shown in Formula 1, Formula 2, and Formula 3, respectively: (1); (2); (3); In the formula, is the pixel kurtosis, is the hyperspectral image data, is the fourth-order cumulant of the hyperspectral image data, is the second-order cumulant of the hyperspectral image data, The hyperspectral image data Dimensional data, is the dimension of the hyperspectral image data, is the mean value of the hyperspectral image data; Get each pixel in the window The distance to other pixels is determined by Small distance, recorded as pixel distance, and get each pixel Pixel Distance neighborhood, as shown in Formula 4: (4); In the formula, For pixels Distance neighborhood, For each pixel Pixel Randomly select a pixel in the distance neighborhood To each pixel The distance For pixels distance; Get each pixel To Pixel The reachable distance of each pixel is further obtained The local reachable density is shown in Formula 5 and Formula 6 respectively: (5); (6); In the formula, For each pixel To Pixel The reachable distance, For each pixel The local reachable density of For pixels The number of pixels contained in the distance neighborhood; According to each pixel Pixel Distance neighborhood and each pixel To Pixel The local reachable density of each pixel is obtained The local outlier factor of is as shown in Formula 7: (7); In the formula, is the local outlier factor, is the number of pixels in the window; The window detection threshold is determined based on the number of pixels in the window, as shown in Formula 8: (8); In the formula, is the window detection threshold, is the determining factor, is the number of pixels in the window, is a preset constant; Determine whether the pixel kurtosis is greater than the window detection threshold; If yes, the pixels whose local outlier factors in the optimization window are greater than the preset value are removed, and the detection of the next optimization window is performed; if no, the detection of the next optimization window is directly performed; This process continues until all optimized windows are detected and the optimized hyperspectral image data is obtained.

[0026] See also Figure 2 In the above step S3, when the vertical weight function Gaussian regression filtering is adopted, it includes: The vertical weight function in the Gaussian regression weight function of each coordinate axis direction of the three-dimensional topography data is set to the same initial value, and the residual convergence accuracy threshold of each axis is set respectively; For 3D shape data Gaussian regression filtering is performed in the coordinate axis direction to generate an initial three-dimensional reference surface, and the residual function value of each three-dimensional data point is obtained. The residual function value is calculated using the three-dimensional Euclidean distance formula; Compare the difference between the residual function values ​​of the current iteration cycle and the previous cycle, and terminate the iteration when the differences in all coordinate axes are less than the residual convergence accuracy threshold of the corresponding axis; If the iteration continues, the vertical weight function of the corresponding direction is corrected according to the residual distribution characteristics of each axis, where The weight adjustment amount in the axial direction is dynamically compensated based on the gradient of surface height change; The modified three-dimensional weight function matrix is ​​used to re-obtain Gaussian regression weights and generate a new three-dimensional filtering reference surface; When the iteration termination condition is met, the topological smooth surface coordinate set is output; Among them, the Gaussian regression weight function is shown in Formula 9: (9); In the formula, is the Gaussian regression vertical weight function, is the upper limit of integration, indicating the signal processing range; is the original signal at position The observed value at For the Position after iteration The filtering result at is the horizontal weight function, used for spatial weighting; is a vertical weight function, which adjusts the weight through the residual to suppress outliers. renew; is the integral variable, which means relative to The offset of is the number of iterations.

[0027] In the above step S4, when the model is optimized, it includes: Obtain boundary voltage measurement data from real-time impedance distribution data, and extract the difference between the boundary voltage and a reference voltage as an input vector; Initial guessing stage: The input vector is passed through a two-level fully connected layer to reconstruct the conductivity distribution, where the fully connected layer activation function uses a parameterized rectified linear unit and generates an initial conductivity distribution matrix through matrix multiplication; Noise reduction optimization stage: the initial conductivity distribution matrix is ​​input into the convolutional neural network for feature learning and noise suppression. The network structure includes: encoder module and decoder module; The network parameters are optimized through end-to-end training, and the noise-reduced conductivity matrix is ​​finally output; Among them, the parameterized corrected linear unit is shown in formula 10: (10); In the formula, is a parameterized rectified linear unit, is the input value of the neuron, is a learnable negative slope parameter that is automatically optimized by back-propagation during model training; The initial conductivity distribution matrix is ​​shown in formula 11: (11); In the formula, is the initial conductivity distribution matrix, with dimension ; is the input vector, which is composed of the difference between the real-time measured boundary voltage and the reference voltage, and its dimension is , is the number of voltage measurement points; is the weight matrix of the first fully connected layer, with dimension , is the number of neurons in the hidden layer; is the bias term of the first fully connected layer, with dimension ; is the weight matrix of the second fully connected layer, with dimension , is the number of pixels in the conductivity distribution matrix; is the bias term of the first fully connected layer, with dimension ; The encoder module uses three layers of two-dimensional convolution kernels to extract spatial features. Each layer of convolution is followed by a PReLU activation function and batch normalization, and the size of the feature map decreases layer by layer. The decoder module uses 3 layers of deconvolution kernels for image reconstruction, and each layer of deconvolution is followed by a Leaky ReLU activation function and batch normalization.

[0028] In the above step S5, when performing multi-dimensional association to form silk screen multimodal data, it includes: Extracting a set of geometric key points containing the principal curvature of the surface from a topologically smooth surface coordinate set; Extracting a spectral key point set containing spectral gradients in the pixel-level spectral feature vector; Extracting conductivity mutation key point set in the denoised conductivity matrix; Different key point sets are combined into multidimensional feature vector sets, and key point association graphs are constructed , is a multidimensional feature vector set, is an edge set, where the edge set The weight of is calculated by the cross-modal similarity, as shown in Formula 12: (12); In the formula, Edge Set Midpoint and nodes The connection weights between is an exponential function used to map distance to similarity. is the square of the Euclidean distance between node feature vectors, For Node The multidimensional feature vector of For Node The multidimensional feature vector of is the bandwidth parameter of the Gaussian kernel function, For Node and nodes The label consistency indicator function of For the same mode, the geometric structure is strengthened by label constraints, and the intra-modal label alignment regularization term is obtained, as shown in Formula 13: (13); In the formula, is the intra-modality label alignment regularization term, is a set of nodes in the same mode, is the similarity matrix element based on category labels, and are nodes in the same mode, For Node and The connection weight between them; For different modalities, complementary alignment is achieved through cross-modal projection loss, and the cross-modal complementary alignment regularization term is obtained, as shown in Formula 14: (14); In the formula, is the cross-modal complementary alignment regularization term, , The modal and modal A collection of nodes (such as is the hyperspectral key point, is the impedance key point); For modal The projection matrix, For modal The projection matrix, For Node The multidimensional feature vector of For Node The multidimensional feature vector of and Each belongs to the mode and modal A collection of nodes; The regularization terms are combined to obtain the total regularization term, and the total regularization term is trained jointly with the task loss. The final output is the association graph Silk screen multimodal data composed of topological structure and node feature tensor.

[0029] In the above step S6, when extracting cross-modal joint features, it includes: The silk-screen multimodal data is subjected to two-to-two modal interactions based on the detection key points to generate cross-modal fusion input; A three-stream network is built based on the improved ResNet50, which extracts features in stages and outputs multi-scale fused feature tensors; The defect-sensitive features are enhanced by constructing a local-global feature association graph, and multi-level joint clustering optimization is performed to output cross-modal joint features; Among them, when performing two-to-two modal interactions, it includes: interaction between hyperspectral and three-dimensional morphology, interaction between three-dimensional morphology and impedance, and splicing the interaction features with the original data to form enhanced input; The hyperspectral spectrum interacts with the three-dimensional morphology as shown in formula 15: (15); In the formula, For hyperspectral and three-dimensional morphology Two-way interactive features; is the hyperspectral feature tensor, is the coordinate of the three-dimensional morphology point cloud; It is a cross-modal attention mechanism, with hyperspectral features as queries and morphological coordinates as keys; It is a cross-modal attention mechanism, with morphological coordinates as queries and hyperspectral features as keys; The three-dimensional topography interacts with the impedance as shown in Equation 16: (16); In the formula, Three-dimensional shape and impedance distribution The gradient weighted interaction features of is the shape-impedance cross-modal attention, is the denoised conductivity matrix, For Hadamard, is the spatial norm of the conductivity gradient; The interactive features are concatenated with the original data to form enhanced input, as shown in Formula 17: (17); In the formula, , , is the enhanced input data, The impedance-morphology interaction features (such as conductivity anomalies in concave areas of the morphology) are integrated into the hyperspectral data. The high-spectral-impedance features are integrated into the 3D topography data. Fusion of topographic-hyperspectral features for impedance distribution data (e.g. correction of conductive properties in highly variable regions); When extracting features in stages, it includes: independent parameter stage Stage0~Stage2 layers, shared parameter stage (Stage3~Stage5 layers), where the convolution kernel parameters of Stage0~Stage2 layers are completely independent and not shared, and the number of channels of Stage3~Stage5 layers increases step by step (for example, Stage3 layer has 512 channels and Stage5 layer has 2048 channels); Extraction of spectral-spatial features at the independent parameter stage , geometric topological features And the conductivity gradient characteristics ; , , It is a three-stream ResNet50 network with independent parameters, which processes the hyperspectral spectrum separately. , three-dimensional morphology , Impedance Distribution , To encode local spectral variations (such as material composition differences) in hyperspectral images, Characterize the geometric topology of 3D surfaces (e.g. solder joint height distribution, warpage area), Capture the spatial gradient characteristics of conductivity (such as sudden changes at line breaks); in, The level of the independent parameter stage.

[0030] The shared parameter stage generates cross-modal shared features, as shown in Formula 18: (18); In the formula, is a ResNet50 network with parameter sharing, is the level of the shared parameter stage, , , is the trimodal feature output from the independent parameter stage, For splicing operation; To share features across modalities; When outputting the multi-scale fusion feature tensor, it includes: (19); In the formula, is the multi-scale fusion feature tensor, is the channel attention weighted fusion, For multi-scale feature fusion, the three-modal weighted features of the independent parameter stage (Stage0~2) are added step by step; To fuse shared features, add the multi-level features of the shared parameter stage (Stage3~5); Multi-level joint clustering optimization, including: channel clustering, intra-class-modal clustering, inter-class separation, and construction of the overall optimization objective function to optimize network parameters; Channel clustering: Dynamically group feature channels to minimize the second-order statistics within the group, as shown in Formula 20: (20); In the formula, Remove redundancy from channel-level features, is the number of channel groups, For the The set of channels for the group, For the The number of channels in the group, For Channel The covariance matrix of For Channel The trace of the covariance matrix of ; Intra-class-modal clustering: triplet loss is used to constrain the cross-modal distance of similar samples, as shown in Formula 21: (twenty one); In the formula, is the intra-class-modal feature alignment, is a triple, is the anchor point sample, For Positive samples of the same type and modality, For Positive samples of the same type but different modalities; , , is the feature vector of the sample; is the interval parameter, which controls the minimum distance difference between intra-class and cross-modal samples; Inter-class separation: The difference between heterogeneous features is enlarged through the maximum margin loss, as shown in Formula 22: (twenty two); In the formula, For inter-class feature separation, is the feature similarity calculation, is the scaling factor, It is the Softplus function; The overall optimization objective function is shown in Formula 23: (twenty three); In the formula, is the overall optimization goal, , , is the weight coefficient.

[0031] In the above step S7, when outputting the quality assessment result including the defect type and location coordinates, it includes: The cross-modal joint features are input into the task space separation head module and processed hierarchically according to the downsampling ratio of the feature map; The defect category probability distribution is output for each anchor box using focal loss optimization through the classification branch, as shown in Formula 24: (twenty four); In the formula, is the probability distribution of defect categories, The model predicts The probability that an anchor box belongs to the real defect category, is the category balance factor, is the sum of the number of anchor boxes of all scales, An exponential factor to adjust the weight of difficult samples; The bounding box offset is predicted by the regression branch using a smooth L1 loss, as shown in Formula 25: (25); In the formula, Predict bounding box offsets for the model, is the actual bounding box offset, Predict bounding box offsets for models on different regression targets, To traverse the four regression targets (center points ,width ,high ), is the smooth L1 function; The anchor box-truth matching strategy is dynamically adjusted based on the cross-modal feature similarity, as shown in Formula 26: (26); In the formula, is the intersection-over-union ratio between the anchor box and the real box, is the balance coefficient between geometry and semantics, is the cosine similarity of cross-modal features, is the average vector of cross-modal joint features in the anchor box region, is the cross-modal joint feature average vector of the real defect area, It is the anchor box-truth matching strategy; Non-maximum suppression is performed on candidate boxes whose classification confidence is higher than the threshold, and the multi-scale prediction results are fused to output the structured quality assessment results, as shown in Formula 27: (27); In the formula, It is a set of quality assessment results, including all detected defect information; is the defect category label, indicating the type of defect detected; is the bounding box coordinate, indicating the position of the defect in the image, is the coordinate of the upper left corner of the bounding box, is the coordinate of the lower right corner of the bounding box; Among them, when the hierarchical processing is performed according to the downsampling ratio of the feature map, it includes: The large-scale feature map (1 / 4 original image resolution) predicts small target defects through dense anchor boxes, and the anchor box size set is ,in, is the width and height of the small target anchor box, is the number of small target anchor boxes; Small-sized feature maps (1 / 16 original image resolution) are used to predict large target defects through sparse anchor boxes. The anchor box size set is ,in, is the width and height of the large target anchor box, is the number of large target anchor boxes. The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. A method for automatically detecting the quality of conductive ink screen printing based on a big data model, characterized in that: The following steps are involved: Obtain hyperspectral image data, three-dimensional morphology data and real-time impedance distribution data of the conductive ink screen printing layer; The local outlier factor optimization processing is performed on the hyperspectral image data to extract the pixel-level spectral feature vector; The three-dimensional morphological data is filtered by vertical weight function Gaussian regression to generate a topological smooth surface coordinate set; Optimize the real-time impedance distribution data based on the conductivity distribution matrix and output the noise-reduced conductivity matrix; Construct association graphs and node feature tensors based on detected key points to generate silk screen multimodal data; Extract cross-modal joint features through modal interaction, phased feature extraction, and multi-level joint clustering optimization; The input task space separation head module outputs the quality assessment results including defect type and location coordinates through classification and regression branches.

2. The method for automatically detecting conductive ink screen printing quality based on a big data model according to claim 1, characterized in that: The local outlier factor optimization process includes: Set the optimization window and calculate the pixel kurtosis in the window through the fourth-order and second-order cumulants of the hyperspectral image data; Get each pixel in the window The distance to other pixels is Small distance as pixel Distance, and construct pixels Distance neighborhood; Get each pixel To neighboring cells The reachable distance and generate the local reachable density; According to the pixel Distance neighborhood and local reachability density are calculated for each pixel. The local outlier factor of Determine the window detection threshold based on the number of window pixels; When the pixel kurtosis is greater than the window detection threshold, the pixels whose local outlier factors exceed the preset value are removed and then the next optimization window is processed; otherwise, the next optimization window is directly processed; Traverse all windows to complete data optimization.

3. The method for automatically detecting conductive ink screen printing quality based on a big data model according to claim 1, characterized in that: The vertical weight function Gaussian regression filtering includes: Set the initial values ​​of the vertical weight functions of each axis to be the same and set the residual convergence accuracy threshold; right The axis is subjected to Gaussian regression filtering to generate the initial three-dimensional reference surface, and the residual function value is obtained through the three-dimensional Euclidean distance; The iteration is terminated when the differences in the residual function values ​​of each axis are all less than the residual convergence accuracy threshold; otherwise, the vertical weight function is dynamically corrected based on the axial residual distribution and surface gradient; The weight function matrix is ​​updated to generate a new three-dimensional filtering reference surface, and a topological smooth surface coordinate set is output when the termination condition is met.

4. The method for automatically detecting conductive ink screen printing quality based on a big data model according to claim 1, characterized in that: Optimized real-time impedance distribution data, including: Extract the difference between the boundary voltage and the reference voltage as an input vector; Initial guessing stage: The conductivity distribution is reconstructed through two-level fully connected layers, the activation function uses PReLU and the initial conductivity distribution matrix is ​​generated through matrix multiplication; Noise reduction optimization stage: the initial conductivity distribution matrix is ​​input into the convolutional network of the encoder-decoder structure. The encoder uses the PReLU activation function and the decoder uses the Leaky ReLU activation function. The network parameters are trained end-to-end and the denoised conductivity matrix is ​​output.

5. The method for automatically detecting conductive ink screen printing quality based on a big data model according to claim 1, characterized in that: Generate silk screen multimodal data, including: Extracting a set of geometric key points containing the principal curvature of the surface from a topologically smooth surface coordinate set; Extracting a spectral keypoint set containing spectral gradient from pixel-level spectral feature vector; Extracting conductivity mutation key point set from the denoised conductivity matrix; The key point set is fused to generate a multi-dimensional feature vector set, and a key point association graph is constructed. The edge weight is calculated based on the cross-modal similarity; The geometric structure is enforced by intra-modal label constraints and the label alignment regularization term is calculated; Complementary alignment is achieved through cross-modal projection loss, and the complementary alignment regularization term is calculated; The regularization term is combined with the task loss optimization to output silkscreen multimodal data consisting of the association graph structure and node feature tensors.

6. The method for automatically detecting conductive ink screen printing quality based on a big data model according to claim 5, characterized in that: Extract cross-modal joint features, including: Generate cross-modal fusion input based on two-to-two modal interactions based on detected key points; An improved ResNet50 three-stream network is used to extract multi-scale fusion features in stages; The defect-sensitive features are enhanced through the local-global feature association graph and multi-level joint clustering optimization is performed to output cross-modal joint features.

7. The method for automatically detecting conductive ink screen printing quality based on a big data model according to claim 6, characterized in that: The two-to-two modal interactions include: hyperspectral and three-dimensional morphology interaction, three-dimensional morphology and impedance interaction, and splicing interaction features and original data to generate enhanced input; The staged feature extraction includes: extracting spectral-spatial features, geometric topological features and conductivity gradient features in an independent parameter stage, and generating cross-modal shared features in a shared parameter stage.

8. The method for automatically detecting conductive ink screen printing quality based on a big data model according to claim 6, characterized in that: The multi-level joint clustering optimization comprises: Channel clustering: dynamically group feature channels and minimize the second-order statistics within the group; Intra-class-modal clustering: triplet loss is used to constrain the cross-modal distance of similar samples; Inter-class separation: enlarging the difference of heterogeneous features through maximum margin loss; Construct the overall optimization objective function to optimize the network parameters, as shown in Formula 23: （23）； In the formula, is the overall optimization goal, , , is the weight coefficient, Remove redundancy from channel-level features, is the intra-class-modal feature alignment, For inter-class feature separation.

9. The method for automatically detecting conductive ink screen printing quality based on a big data model according to claim 6 or 8, characterized in that: Output quality assessment results, including: The cross-modal joint features are input into the task space separation head module and processed hierarchically according to the downsampling ratio of the feature map; The classification branch uses focal loss to optimize the output anchor box category probability; The regression branch uses smooth L1 loss to predict the box offset; Dynamically adjust the anchor box matching strategy based on cross-modal feature similarity; Non-maximum suppression is performed on candidate boxes whose classification confidence is higher than the threshold, multi-scale prediction results are fused, and structured quality assessment results are output.

10. The method for automatically detecting conductive ink screen printing quality based on a big data model according to claim 9, characterized in that: When grading, it includes: Large-scale feature maps through dense anchor boxes Predict small target defects, where is the width and height of the small target anchor box, is the number of small target anchor boxes; Small-sized feature maps are obtained through sparse anchor boxes Predict large target defects, where is the width and height of the large target anchor box, is the number of large target anchor boxes.

Images