A Method and System for 3D Defect Detection of Ceramic Packaging Substrates Based on Auxiliary Information Flow

Abstract

This invention provides a method and system for 3D defect detection of ceramic packaging substrates based on auxiliary information flow, relating to the field of 3D industrial product surface defect detection technology. The method includes: acquiring 3D point cloud data of the ceramic packaging substrate to be inspected, and extracting voxel features of all non-empty voxels in the ceramic packaging substrate by dividing it into voxel meshes; constructing and training a 3D defect detection model based on auxiliary information flow; the 3D defect detection model based on auxiliary information flow includes: a backbone network, an auxiliary voxel information flow module, and a heat map perception regression module; inputting the extracted voxel features of all non-empty voxels into the trained 3D defect detection model based on auxiliary information flow for defect detection, and obtaining the defect detection result. This invention does not rely on manually set anchor frames, solving the problems of low detection accuracy and low detection rate of ceramic packaging substrate defects caused by the small size, low resolution, limited information, and multiple defect distributions of defective objects.

Description

Technical Field

[0001] This invention relates to the field of 3D industrial product surface defect detection technology, and in particular to a method and system for 3D defect detection of ceramic packaging substrates based on auxiliary information flow. Background Technology

[0002] During the manufacturing process of integrated circuit ceramic packaging substrates, micron-scale defects such as cracks, pinholes, bubbles, scratches, and stains may occur on their surfaces, causing immeasurable damage to the application scenarios of the ceramic packaging substrates. 3D defect detection is crucial in defect-centric industrial quality inspection, typically requiring high precision and high detection rates.

[0003] Chinese patent application number 202411875326.9, entitled "A 3D Point Cloud Target Recognition Method Based on Bidirectional Long Short-Term Memory Networks," provides a 3D target detection method for general scenarios. This method first preprocesses the raw point cloud data to save computational resources and memory usage; then, it extracts forward and backward sequence information to obtain the long-range dependencies of points; finally, it aggregates these local and global features so that the entire point cloud can be effectively utilized for final recognition. However, when processing 3D anomaly detection tasks, the implicit abnormal foreground information conflicts with the inherent characteristics of general point clouds. Furthermore, as an unordered set of points, forcibly establishing forward and backward sequence relationships in a point cloud can disrupt its spatial topology, potentially leading to sensitivity to transformations such as rotation and translation.

[0004] Chinese patent application number 202411845977.3, entitled "Real-time Semantic Segmentation of Point Cloud for Autonomous Driving Obstacle Detection Method and System," provides a 3D obstacle detection method and system for autonomous driving scenarios. This method uses a deep learning architecture to obtain the segmentation results of a segmentation model. Different segmented images obtained from the segmentation results are defined as regions, and labeled objects identified within these regions are considered parts. When pixel labels in two regions are the same but belong to different parts and the distance is less than a distance threshold, they are considered conflicting. If the confidence score of the current region is higher than the confidence threshold of the current label, the labels of the pixel images segmented from that region are retained and merged; otherwise, the label is deleted from that region. Simultaneously, a contour extraction algorithm is used to identify the boundaries of objects, obtaining an optimized image with updated labels. However, this method performs poorly on point cloud projection images when dealing with abnormal targets with large scale variations and uneven aspect ratios. The confidence score is easily affected by environmental interference, potentially triggering a chain reaction of incorrect label deletion.

[0005] Chinese patent application number 202410556437.7, entitled "A Method and Related Equipment for Detecting Defects in Sheet Metal Based on 2D and 3D Images," provides a sheet metal defect detection system that combines 2D images and 3D point clouds. The system first uses a 2D camera to acquire image information of abnormal areas. Then, the image information is input into an improved YOLOv7 model for surface defect type detection. Finally, point cloud information is acquired using a 3D camera, and abnormal point cloud information is determined and recorded as abnormal coordinates based on the variance of the point cloud coordinates along the Z-axis. However, this system uses serial processing instead of feature-level fusion for 2D defect detection and 3D coordinate analysis, using Z-axis variance to determine abnormalities, ignoring the positional information of the 3D point cloud itself and the semantic information contained in the 3D features. Summary of the Invention

[0006] To address the shortcomings of the existing technologies, this invention proposes a 3D defect detection method and system for ceramic packaging substrates based on auxiliary information flow. This method utilizes voxel-enhanced sparse feature extraction and an anchor-frame-free 3D target regression mechanism. The aim is to solve the problems of low detection accuracy and low detection rate of ceramic packaging substrate defects caused by the small size, low resolution, limited information, and multiple defect distributions of defective objects, without relying on manually pre-set anchor frames of size and scale.

[0007] On the one hand, this invention proposes a 3D defect detection method for ceramic packaging substrates based on auxiliary information flow, which includes the following process:

[0008] Acquire 3D point cloud data of the ceramic packaging substrate to be tested, and extract voxel features of all non-empty voxels in the ceramic packaging substrate by dividing voxel mesh.

[0009] A 3D defect detection model based on auxiliary information flow is constructed and trained to obtain a trained 3D defect detection model based on auxiliary information flow.

[0010] The 3D defect detection model based on auxiliary information flow is as follows: A backbone network consisting of several layers of 3D sparse convolutions is used to encode the input voxel features. The first layer of 3D sparse convolutions is used to extract sparse features. Starting from the second layer, each layer of 3D sparse convolutions downsamples the output features of the previous layer. After downsampling at each layer, the auxiliary voxel information flow module abstracts, extracts, aligns, and senses the spatial dimensions of the downsampled features to generate aligned features. The aligned features are then concatenated and fused with the downsampled features to obtain dense features and their position coordinates. These dense features are used as the output features of the current 3D sparse convolution. Finally, a heatmap-aware regression module fuses and decodes the output features of the backbone network to obtain the defect detection result.

[0011] The extracted voxel features of all non-empty voxels are input into the trained 3D defect detection model based on auxiliary information flow for defect detection, and the defect detection results are obtained.

[0012] Furthermore, the method for extracting voxel features of all non-empty voxels in the ceramic packaging substrate by dividing a voxel grid is as follows:

[0013] The acquired 3D point cloud data is divided into spatial resolutions of... The voxel grid; in which The length of each voxel grid; Width of each voxel grid; The height of each voxel grid;

[0014] In all the voxel grids that have been divided, for each non-empty voxel, obtain the point-by-point features of each point within that non-empty voxel;

[0015] The point-by-point features mentioned therein include: 3D coordinates and intensity values;

[0016] The set of point-by-point features of all points within the non-empty voxel is taken as the voxel feature of the non-empty voxel.

[0017] Furthermore, the backbone network consists of several layers of three-dimensional sparse convolutions; wherein the first layer of three-dimensional sparse convolution takes voxel features as input to obtain the first layer of sparse features; starting from the second layer, each layer of three-dimensional sparse convolution takes the output features of the previous layer as input for downsampling, and uses an auxiliary voxel information flow module to enhance and fuse the obtained downsampled features and the input features of the layer to obtain the output features of the layer; thus, the output features of the last layer of three-dimensional sparse convolution are obtained and used as the output features of the backbone network;

[0018] The heatmap perception regression module uses sub-manifold sparse convolutional layers to fuse and decode the output features of the backbone network, generating heatmaps and 3D regression boxes, and then obtaining prediction boxes through threshold filtering.

[0019] Furthermore, the specific content of the auxiliary voxel information flow module is as follows:

[0020] Use the output features of the current 3D sparse convolutional layer in the backbone network as the current input features;

[0021] The current input features are abstracted and extracted using auxiliary sparse convolutional blocks to obtain global abstract features;

[0022] By using a channel perceptron to perform spatial dimension alignment and channel perception on global abstract features, alignment-perceptual features are obtained.

[0023] The system concatenates and aligns the perceived features with the current input features, and then uses sub-manifold sparse convolutional blocks to process the concatenated features, thereby obtaining dense features and their location coordinates.

[0024] Furthermore, the auxiliary sparse convolution block comprises, in series, one sparse convolutional layer and two sub-manifold sparse convolutional layers.

[0025] Furthermore, the submanifold sparse convolutional block comprises, in series, a normalization layer and a nonlinear activation layer.

[0026] Furthermore, the training method for the 3D defect detection model based on auxiliary information flow is as follows:

[0027] A defect sample set is constructed by acquiring several sets of 3D point cloud data of ceramic packaging substrates with labeled defects, and then randomly divided into a training set and a validation set according to a preset ratio.

[0028] Set the batch size, initial learning rate, training period, learning rate decay strategy, and decay weights;

[0029] The training set is input into the 3D defect detection model based on auxiliary information flow and trained using the Adam optimizer;

[0030] During each training round, the total loss of the 3D defect detection model is calculated based on the predicted bounding box output by the 3D defect detection model and the labeled ground truth.

[0031] The total loss is the sum of the target localization loss under smoothing loss supervision and the heatmap-aware regression loss;

[0032] By minimizing the total loss of the 3D defect detection model, the model parameters are iteratively optimized until the set training cycle is completed.

[0033] The trained 3D defect detection model based on auxiliary information flow is validated using a validation set to obtain a well-trained 3D defect detection model based on auxiliary information flow.

[0034] Furthermore, the defect detection results include: all predicted bounding boxes output by the 3D defect detection model based on auxiliary information flow and the confidence score corresponding to each predicted bounding box; wherein the confidence score is obtained from the heatmap generated by the 3D defect detection model based on auxiliary information flow.

[0035] On the other hand, this invention proposes a 3D defect detection system for ceramic packaging substrates based on auxiliary information flow, the system comprising:

[0036] The data acquisition module is used to acquire 3D point cloud data of the ceramic packaging substrate to be tested;

[0037] The voxelization module is used to divide the 3D point cloud data of the ceramic packaging substrate to be inspected into voxel meshes and extract the voxel features of all non-empty voxels in the ceramic packaging substrate from the division results.

[0038] The detection model training module is used to construct and train a 3D defect detection model based on auxiliary information flow to obtain a trained 3D defect detection model based on auxiliary information flow.

[0039] The defect detection module is used to input the voxel features of all extracted non-empty voxels into the trained 3D defect detection model based on auxiliary information flow for defect detection and to obtain the defect detection results.

[0040] Thirdly, the present invention proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the method for 3D defect detection of ceramic packaging substrates based on auxiliary information flow.

[0041] The beneficial effects of adopting the above technical solution are as follows:

[0042] 3D defect detectors typically rely on handcrafted proxies, such as anchor boxes or center points. Therefore, sparse voxel features need to be densified and processed by a dense prediction head, inevitably requiring additional computation. The method of this invention divides the point cloud into a voxel mesh and inputs it into the network. It expands the feature volume through auxiliary sparse convolution, enhancing the sparse backbone and aggregating the global information flow of dense features. Furthermore, the method utilizes a heatmap-aware regression module to predict voxel scores and generates 3D regression boxes based on a set threshold. This regression module does not rely on manually pre-defined anchor boxes of specific sizes and scales. Attached Figure Description

[0043] Figure 1 This is a flowchart of the 3D defect detection method for ceramic packaging substrates based on auxiliary information flow in this embodiment;

[0044] Figure 2 This is a schematic diagram of the 3D defect detection method for ceramic packaging substrates based on auxiliary information flow in this embodiment;

[0045] Figure 3 This is a structural diagram of the ceramic packaging substrate 3D defect detection system based on auxiliary information flow in this embodiment; Detailed Implementation

[0046] To facilitate understanding of this application, specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments. The following embodiments are illustrative of the invention but are not intended to limit its scope. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this application.

[0047] Example 1:

[0048] 3D defect detection of ceramic packaging substrates is a highly challenging task due to the small size, low resolution, and limited information of the defect targets. This embodiment presents a 3D defect detection method for ceramic packaging substrates based on auxiliary information flow, such as... Figure 1 As shown, the method includes the following steps:

[0049] The 3D point cloud data of the ceramic packaging substrate to be tested is acquired, and the voxel features of all non-empty voxels in the ceramic packaging substrate are extracted by dividing the voxel mesh.

[0050] The method for extracting voxel features of all non-empty voxels in the ceramic packaging substrate by dividing a voxel grid is as follows:

[0051] The acquired 3D point cloud data is divided into spatial resolutions of... The voxel grid; in which The length of each voxel grid; Width of each voxel grid; The height of each voxel grid.

[0052] In all the voxel grids that are divided, for each non-empty voxel, obtain the point-by-point features of each point within that non-empty voxel.

[0053] The point-by-point features include: 3D coordinates and intensity values.

[0054] The set of point-by-point features of all points within the non-empty voxel is taken as the voxel feature of the non-empty voxel.

[0055] A 3D defect detection model based on auxiliary information flow is constructed and trained to obtain a trained 3D defect detection model based on auxiliary information flow.

[0056] The 3D defect detection model based on auxiliary information flow includes: a backbone network, an auxiliary voxel information flow module, and a heat map perception regression module.

[0057] The backbone network consists of several layers of three-dimensional sparse convolutions. The first layer of three-dimensional sparse convolution takes voxel features as input to obtain the first layer of sparse features. Starting from the second layer, each layer of three-dimensional sparse convolution takes the output features of the previous layer as input for downsampling, and uses an auxiliary voxel information flow module to enhance and fuse the obtained downsampled features and the input features of the layer to obtain the output features of the layer. This leads to the output features of the last layer of the backbone network.

[0058] In this embodiment, point cloud Divided into 3D voxel mesh .like Figure 2 As shown, using a series of Sparse convolution progressively converts 3D voxels into downsampled sizes of: The multi-layer output features are represented by each output feature, which encodes the aggregated features of the point cloud within the sparse space.

[0059] The auxiliary voxel information flow module is used to abstract and extract the input downsampled features, align the spatial dimensions, and perceive the channels to generate aligned features; the aligned features are then concatenated and fused with the input features corresponding to the downsampled features to obtain dense features and their position coordinates, and the resulting dense features are used as the output features of this layer.

[0060] In this embodiment, the purpose of the auxiliary voxel information flow is to aggregate global information from dense features. Existing multi-scale structures employ complex and varied fusion strategies, performing multiple transformations on feature channels. While aggregating effective semantic information, they combine background features with interference information. Inspired by VoxelNeXt, auxiliary voxel features effectively promote information integration. Therefore, this module introduces auxiliary sparse convolution to transform sparse volumes into dense features, enhancing the features of the sparse backbone network. This transformation directly manifests as an expansion of the receptive field.

[0061] The specific content of the auxiliary voxel information flow module is as follows:

[0062] Use the output features of the current 3D sparse convolutional layer in the backbone network as the current input features.

[0063] In this embodiment, as Figure 2 As shown, the output features of each layer of the 3D sparse convolution in the backbone network are used as the input features of the auxiliary voxel information flow module.

[0064] The current input features are abstracted and extracted using an auxiliary sparse convolutional block to obtain global abstract features; wherein the auxiliary sparse convolutional block includes, in sequence, a sparse convolutional layer and two sub-manifold sparse convolutional layers.

[0065] In this embodiment, auxiliary sparse convolutions are introduced into the backbone network, and a hierarchical feature concatenation strategy is used to aggregate deep cues and abstractly extract global information. For example... Figure 2 As shown, especially with the use of auxiliary sparse convolution Extracting features from backbone network from fine-grained global information. The deeper clues. Among them These are layers 1, 2, 3, and 4 of the backbone network, respectively. The output features of sparse convolution. The abstract extraction process can be represented as:

[0066] ;

[0067] in This represents an auxiliary sparse convolution block, which consists of a sparse convolution layer with a stride of 2 and two sub-manifold sparse convolution layers with a stride of 1 connected in series. Represents global abstract features; This represents the input features.

[0068] By using a channel perceptron to perform spatial dimension alignment and channel perception on global abstract features, aligned perceptual features are obtained.

[0069] In this embodiment, the purpose of setting up the channel perceptron is to expand the voxel scale and extract channel attention of global abstract features while matching and aligning spatial dimensions. This spatial dimension alignment and fusion preserves important spatial structure and feature information. The processing of the channel perceptron can be represented as follows:

[0070] ;

[0071] in Representing alignment-aware features Position coordinates; Representing global abstract features Position coordinates; The coordinates of any point in the sparse space representing the global abstract features; Representing sparse space coordinates Axis coordinates; Representing sparse space coordinates Axis coordinates; Representing sparse space coordinates Axis coordinates.

[0072] The system concatenates and aligns the perceived features with the current input features, and then uses sub-manifold sparse convolutional blocks to process the concatenated features, thereby obtaining dense features and their location coordinates.

[0073] The submanifold sparse convolutional block consists of a normalization layer and a nonlinear activation layer.

[0074] In this embodiment, auxiliary voxel features are fused to integrate depth cues and pixel-level contextual information. Furthermore, sub-manifold sparse convolutional layers are added. To maintain density and the number of effective points, the final dense feature is obtained. and its position coordinates , is represented as:

[0075] ;

[0076] ;

[0077] in This represents a sparse convolutional block of a submanifold, containing a normalized layer and a nonlinear activation layer. Connectivity layers representing features; This indicates that the voxels are placed at the same 3D position coordinates; Indicates and The set of position coordinates at the same 3D location, and having , express Position coordinates; express Any coordinate in the array; express The coordinate values; express The coordinate values.

[0078] In this embodiment, by... and Dense features are obtained by concatenating features. This enhances the density of voxel features. Spatial dimension alignment and fusion preserve important spatial structure and feature information.

[0079] In this embodiment, in industrial production scenarios, the appearance features of anomalies and defects, such as the scale and size of target objects, exhibit high uncertainty and diversity, making accurate prediction difficult in advance. Therefore, this embodiment abandons the mainstream detection framework structure, which relies on dense anchor boxes for regression. The output of the backbone network is converted into two-dimensional spatial features, specifically through high-dimensional compression. The resulting dense features are input into the heatmap-aware regression module, which applies the voxel scores predicted by the heatmap to the 3D regression bounding boxes. This regression module does not rely on manually pre-setting anchor boxes of specific sizes and scales.

[0080] The heatmap perception regression module uses a sub-manifold sparse convolutional layer to fuse and decode the output features of the last layer of the backbone network, generating a heatmap and a 3D regression box, and then obtains the prediction box through threshold filtering.

[0081] In this embodiment, the output characteristics of the backbone network are utilized. Generate heatmap and 3D regression box Specifically, it can be summarized as follows:

[0082] ;

[0083] ;

[0084] in Indicates the size of the input feature; This represents the seven dimensions of the 3D regression bounding box. The coordinates of the center point of the regression box are represented in three dimensions; This indicates the scale information of the regression box. Indicates the orientation angle of the regression box.

[0085] Prediction box From heat map Regression frame and the set threshold Generate, represented as:

[0086] ;

[0087] in Representing the three-dimensional regression box The first in A three-dimensional regression box; Representing a heatmap The first in Each heatmap value.

[0088] The training method for the 3D defect detection model based on auxiliary information flow is as follows:

[0089] A defect sample set is constructed by acquiring several sets of 3D point cloud data of ceramic packaging substrates with labeled defects, and then randomly divided into a training set and a validation set according to a preset ratio.

[0090] In this embodiment, training is performed on a ceramic packaging substrate dataset. During data segmentation, following the design strategy of existing defect detection datasets, the defect sample set is randomly divided into a training set and a validation set with a ratio of 3:1. The characteristics of defects in each sample are analyzed and counted, including scale, size, and number of points, to achieve a balanced distribution of samples.

[0091] Configure batch size, initial learning rate, training period, learning rate decay strategy, and decay weights.

[0092] The training set is input into the 3D defect detection model based on auxiliary information flow and trained using the Adam optimizer.

[0093] During each training round, the total loss of the 3D defect detection model is calculated based on the predicted bounding boxes output by the model and the actual labels.

[0094] The total loss is the sum of the target localization loss and the heatmap-aware regression loss under smoothing loss supervision.

[0095] ;

[0096] in This represents the total loss of the 3D defect detection model; This represents the target localization loss under smoothing loss supervision; This represents the heatmap-aware regression loss; and Regression heatmaps representing the actual and predicted values, respectively.

[0097] By minimizing the total loss of the 3D defect detection model, the model parameters are iteratively optimized until the set training cycle is completed.

[0098] The trained 3D defect detection model based on auxiliary information flow is validated using a validation set to obtain a well-trained 3D defect detection model based on auxiliary information flow.

[0099] In this embodiment, the 3D defect detection model based on auxiliary information flow is trained from scratch in an end-to-end manner using the ADAM optimizer. The entire network is trained on a single RTX 3090 GPU with a batch size of 8, a learning rate of 3e-3, 500 epochs without pre-training, and takes approximately 10 hours. It decays to 1e-4 in cosine annealing. The decay weights are 0.01.

[0100] The extracted voxel features of all non-empty voxels are input into the trained 3D defect detection model based on auxiliary information flow for defect detection, and the defect detection results are obtained.

[0101] The defect detection results include: all predicted bounding boxes output by the 3D defect detection model based on auxiliary information flow and the confidence score corresponding to each predicted bounding box; wherein the confidence score is obtained from the heatmap generated by the 3D defect detection model based on auxiliary information flow.

[0102] The 3D defect detection method for integrated circuit ceramic packaging substrates based on auxiliary information flow described in this invention has achieved a 3D accuracy of 42.40% and a BEV accuracy of 41.13% in multiple experiments with test data of various challenging ceramic packaging substrate sample types under complex scenarios. The auxiliary sparse convolution proposed in this invention effectively expands the receptive field of sparse convolution, and the proposed no-NMS mechanism exhibits good generalization and robustness in defect samples with large variations in appearance size, unbalanced aspect ratios, and micrometer-scale defects. In the production process of products with high precision requirements, this invention can essentially capture the dynamic changes in the product manufacturing process and the subtle defects that may be caused by different environmental factors.

[0103] Example 2:

[0104] This embodiment presents a 3D defect detection system for ceramic packaging substrates based on auxiliary information flow, such as... Figure 3 As shown, the system includes:

[0105] The data acquisition module is used to acquire 3D point cloud data of the ceramic packaging substrate to be tested;

[0106] The voxelization module is used to divide the 3D point cloud data of the ceramic packaging substrate to be inspected into voxel meshes and extract the voxel features of all non-empty voxels in the ceramic packaging substrate from the division results.

[0107] The detection model training module is used to construct and train a 3D defect detection model based on auxiliary information flow to obtain a trained 3D defect detection model based on auxiliary information flow.

[0108] The defect detection module is used to input the voxel features of all extracted non-empty voxels into the trained 3D defect detection model based on auxiliary information flow for defect detection and to obtain the defect detection results.

[0109] Example 3:

[0110] This embodiment proposes an electronic device, including: one or more processors, and a memory, wherein the memory is used to store instructions, and when the instructions are executed by the one or more processors, the one or more processors execute the aforementioned method for 3D defect detection of ceramic packaging substrates based on auxiliary information flow.

[0111] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the 3D defect detection method for ceramic packaging substrates based on auxiliary information flow as described in the embodiments. It is understood that the electronic device may also include input / output (I / O) interfaces and communication components.

[0112] The processor is used to execute all or part of the steps in the 3D defect detection method for ceramic packaging substrates based on auxiliary information flow as described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in the electronic device, as well as application-related data.

[0113] The processor can be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the 3D defect detection method for ceramic packaging substrates based on auxiliary information flow described in the above embodiments.

[0114] Example 4:

[0115] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0116] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the 3D defect detection method for ceramic packaging substrates based on auxiliary information flow described in the various embodiments of this application.

[0117] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disk, optical disk, server, APP (Application) application store, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the aforementioned 3D defect detection method for ceramic packaging substrates based on auxiliary information flow.

[0118] Example 5:

[0119] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned 3D defect detection method for ceramic packaging substrates based on auxiliary information flow.

[0120] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0121] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0122] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of this disclosure and its equivalents, then the intent of this disclosure also includes these modifications and variations.

Claims

1. A method for ceramic package substrate 3D defect detection based on auxiliary information flow, characterized in that, The method comprises the following processes: 3D point cloud data of a ceramic packaging substrate to be detected is acquired, and voxel features of all non-empty voxels in the ceramic packaging substrate are extracted by dividing a voxel grid; A 3D defect detection model based on auxiliary information flow is constructed and trained to obtain a trained 3D defect detection model based on auxiliary information flow; The 3D defect detection model based on auxiliary information flow is: a backbone network composed of a plurality of layers of three-dimensional sparse convolution is used to perform feature coding on the input voxel features; wherein the first layer of three-dimensional sparse convolution is used to extract sparse features; starting from the second layer of three-dimensional sparse convolution, each layer of three-dimensional sparse convolution performs down-sampling on the output features of the previous layer of three-dimensional sparse convolution; and after each layer of down-sampling, an auxiliary voxel information flow module is used to abstractly extract, spatially align and channel perceive the obtained down-sampled features to generate aligned perceived features; then the aligned perceived features and the down-sampled features are feature spliced and fused to obtain dense features and the position coordinates of the dense features, and the obtained dense features are taken as the output features of the current three-dimensional sparse convolution; a heat map perception regression module is used to fuse and decode the output features of the backbone network to obtain a defect detection result; All non-empty voxel features extracted are input into the trained 3D defect detection model based on auxiliary information flow for defect detection to obtain a defect detection result. 2.The ceramic package substrate 3D defect detection method based on auxiliary information flow according to claim 1, wherein, The method for extracting voxel features of all non-empty voxels in the ceramic packaging substrate by dividing a voxel grid is: The acquired 3D point cloud data is divided into voxel grids with spatial resolution of , wherein is the length of each voxel grid; is the width of each voxel grid; is the height of each voxel grid; In all divided voxel grids, for each non-empty voxel, the point-by-point features of each point in the non-empty voxel are acquired; The point-by-point features include 3D coordinates and intensity values; The set of point-by-point features of all points in the non-empty voxel is taken as the voxel features of the non-empty voxel.

3. The ceramic packaging substrate 3D defect detection method based on auxiliary information flow according to claim 2, wherein The backbone network is composed of a plurality of layers of three-dimensional sparse convolution; The first layer of three-dimensional sparse convolution takes the voxel features as input to obtain the first layer of sparse features; starting from the second layer, each layer of three-dimensional sparse convolution takes the output features of the previous layer as input to perform down-sampling, and an auxiliary voxel information flow module is used to enhance and fuse the obtained down-sampled features and the input features of the layer to obtain the output features of the layer; and the output features of the last layer of three-dimensional sparse convolution are taken as the output features of the backbone network; The heat map perception regression module uses a sub-manifold sparse convolution layer to fuse and decode the output features of the backbone network to generate a heat map and a three-dimensional regression box, and then a prediction box is obtained through threshold screening.

4. The method of claim 3, wherein the method further comprises: The specific content of the auxiliary voxel information flow module is: The features output by the current three-dimensional sparse convolution layer in the backbone network are taken as the current input features; An auxiliary sparse convolution block is used to abstractly extract the current input features to obtain global abstract features; A channel perceiver is used to spatially align and channel perceive the global abstract features to obtain aligned perceived features; The spliced alignment perception feature is aligned with the current input feature, and a sub-manifold sparse convolution block is used to process the spliced feature to obtain a dense feature and a position coordinate of the dense feature.

5. The method of claim 4, wherein the method further comprises: The auxiliary sparse convolution block comprises, in sequence: one sparse convolution layer and two sub-manifold sparse convolution layers.

6. The method of claim 4, wherein the method further comprises: The sub-manifold sparse convolution block comprises, in sequence: a normalization layer and a nonlinear activation layer.

7. The method of claim 4, wherein the method further comprises: The training method of the 3D defect detection model based on auxiliary information flow comprises: A plurality of groups of ceramic packaging substrate 3D point cloud data with labeled defects are obtained to construct a defect sample set, and the defect sample set is randomly divided into a training set and a verification set according to a preset proportion; A batch size, an initial learning rate, a training period, a learning rate decay strategy, and a decay weight are set; The training set is input into the 3D defect detection model based on auxiliary information flow, and the 3D defect detection model is trained using an Adam optimizer; In each training process, the total loss of the 3D defect detection model is calculated according to the prediction box output by the 3D defect detection model and the labeled real label. The total loss is the sum of the target positioning loss and the heat map perception regression loss under the supervision of the smooth loss. The model parameters are iteratively optimized by minimizing the total loss of the 3D defect detection model until the set training period is completed. The trained 3D defect detection model based on auxiliary information flow is verified using the verification set to obtain the trained 3D defect detection model based on auxiliary information flow.

8. The method of claim 7, wherein the method further comprises: The defect detection result comprises all prediction boxes output by the 3D defect detection model based on auxiliary information flow and a confidence score corresponding to each prediction box, wherein the confidence score is obtained from a heat map generated by the 3D defect detection model based on auxiliary information flow.

9. A ceramic package substrate 3D defect detection system based on auxiliary information flow for implementing the ceramic package substrate 3D defect detection method based on auxiliary information flow according to any one of claims 1-8, characterized in that, The system comprises: A data acquisition module for acquiring 3D point cloud data of a ceramic packaging substrate to be detected; A voxelization module for voxel grid division of the 3D point cloud data of the ceramic packaging substrate to be detected, and extracting voxel features of all non-empty voxels in the ceramic packaging substrate from the division result; A detection model training module for constructing and training a 3D defect detection model based on auxiliary information flow to obtain a trained 3D defect detection model based on auxiliary information flow; A defect detection module for inputting the extracted voxel features of all non-empty voxels into the trained 3D defect detection model based on auxiliary information flow for defect detection to obtain a defect detection result.

10. A computer program product, characterised in that, A computer program or instructions for implementing the ceramic packaging substrate 3D defect detection method based on auxiliary information flow of any one of claims 1-8 when executed by a processor.

Images