Edge computing-based quality inspection and analysis methods for electronic components, servers, and media

Abstract

This application discloses an edge computing-based method, server, and medium for quality inspection and analysis of electronic components. Based on an edge computing architecture, this application constructs a set of phantom appearance representations constrained by topological completeness rules, calculates the differences between the actual appearance representation and the phantom representation at the levels of feature distribution and topological manifold, generates a strangeness quantification map, and finally achieves accurate defect location and classification of electronic components by matching with an abnormal mode definition library. This method improves the edge side's real-time ability to judge the compliance of complex component design specifications, enabling the edge side to have production line health status awareness and knowledge base adaptive evolution capabilities.

Description

Technical Field

[0001] This application relates to the field of computer data analysis technology, and in particular to a method, server and medium for quality inspection and analysis of electronic components based on edge computing. Background Technology

[0002] With the deepening advancement of industrial internet and intelligent manufacturing technologies, electronic component production lines are continuously evolving towards higher speeds, greater precision, and automation. Printed circuit board assemblies, chip packages, and various connector components, as core building blocks of electronic products, directly determine the performance and lifespan of end products through their appearance quality and electrical connection reliability. With the widespread application of surface mount technology and micro-assembly processes, component pin pitches are continuously shrinking, pad sizes are entering the micrometer scale, and wiring density and layer counts are significantly increasing. These trends place unprecedentedly stringent demands on quality inspection during the production process.

[0003] Existing inspection solutions mostly employ a centralized cloud processing model. High-resolution images are acquired at the production line, uploaded to the cloud for inference, and then the results are sent back. The latency introduced by the round-trip transmission of image data in this architecture restricts the real-time response capability of high-speed production lines. The bandwidth pressure caused by multiple production lines operating concurrently can easily lead to queue backlogs, and fluctuations in cloud resource scheduling further affect the stability of the inspection rhythm. Other existing visual inspection models focus on extracting semantic features of appearance but do not incorporate the pin connection relationships and wiring geometric constraints defined in the design drawings into the judgment criteria, making it difficult to accurately distinguish between allowable tolerance deformation and substantial structural damage. Therefore, there is an urgent need for a high-precision quality inspection method that achieves millisecond-level response at the edge and integrates topological completeness constraints. Summary of the Invention

[0004] This application provides an edge computing-based method, server, and medium for quality inspection and analysis of electronic components, which solves the problem of insufficient real-time performance caused by data transmission delay and bandwidth limitations in traditional centralized cloud-based quality inspection solutions. It also overcomes the limitations of manual visual inspection or fixed-rule machine vision solutions in identifying defects in the topological structure of complex components.

[0005] This application provides, in one aspect, an edge computing-based method for quality inspection and analysis of electronic components, applied to an electronic component quality inspection and analysis server. The method includes:

[0006] The edge computing nodes deployed on the production line receive the design drawing definition data stream and surface continuous image stream corresponding to the electronic components to be inspected.

[0007] The lightweight backbone network pre-installed on the edge computing node is invoked to perform frame-by-frame shallow appearance representation extraction processing on the continuous surface image stream, resulting in an initial appearance representation set recording the pixel-level response distribution of the component surface. The topological completeness constraint rule parsing processing of the design drawing definition data stream is performed using a sparse autoencoder-based residual semantic parser, resulting in an edge-side logic verification map recording the physical connection relationship constraints and geometric tolerance range of the components.

[0008] Using a preset adversarial feature illusion construction strategy and the edge-side logic verification map as a guiding condition, the initial appearance representation set is subjected to reverse perturbation iterative correction processing to generate an illusion appearance representation set that is constrained by the topological completeness constraint rules in the feature space distribution.

[0009] Calculate the non-consistent property of the feature distribution of the initial appearance representation set and the phantom appearance representation set in the preset feature dimension space, and calculate the topological manifold connectivity difference between the virtual geometric topology of the components represented by the phantom appearance representation set and the reference geometric topology of the components defined by the edge-side logic verification map.

[0010] Based on the non-self-consistent properties of the feature distribution and the differences in connectivity of the topological manifold, an unfamiliar quantitative map describing the degree of deviation of each spatial location region from the design specifications in the continuous surface image stream is generated and input into a preset abnormal mode definition library for matching to determine the defect location descriptor and defect type descriptor of the electronic component to be inspected.

[0011] One embodiment of this application provides an electronic component quality inspection and analysis server, including:

[0012] A processor; a storage device having a computer program stored thereon; a network interface for providing network communication functions; when the computer program is executed by the processor, the processor enables the processor to implement any of the edge computing-based electronic component quality inspection and analysis methods described above.

[0013] One embodiment of this application provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements the steps of the edge computing-based electronic component quality inspection and analysis method.

[0014] Therefore, the embodiments of this application have the following beneficial effects: Based on the edge computing architecture, by constructing a set of phantom appearance representations constrained by topological completeness rules, and calculating the differences between the actual appearance representation and the phantom representation at the two levels of feature distribution and topological manifold, a strange quantitative map is generated. Finally, by matching with the abnormal mode definition library, the precise defect location and classification of electronic components are achieved. This method improves the edge side's real-time discrimination capability for the design specification compliance of complex components, enabling the edge side to have production line health status perception and knowledge base adaptive evolution capability. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating an edge computing-based quality inspection and analysis method for electronic components, provided as an embodiment of this application.

[0017] Figure 2 This is a schematic diagram of the basic structure of an electronic component quality inspection and analysis server provided in an embodiment of this application.

[0018] Figure 3 This is a functional block diagram of an electronic component quality inspection and analysis device provided in an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0020] Please see Figure 1 , Figure 1 This is a flowchart of an edge computing-based electronic component quality inspection and analysis method provided in an embodiment of this application. The method can be executed by an electronic component quality inspection and analysis server, or by both the electronic component quality inspection and analysis server and the server. The method may include steps 110-150.

[0021] The embodiments of this application aim to solve the problem of insufficient real-time performance caused by data transmission delay and bandwidth limitations in traditional centralized cloud-based quality inspection solutions, while overcoming the limitations of manual visual inspection or fixed-rule machine vision solutions in identifying defects in the topological structure of complex components.

[0022] In this embodiment, the edge computing node is a computing unit with independent data processing and storage capabilities. It establishes a communication connection with the image acquisition device and design drawing data source on the production line side through a physical interface. The electronic components to be inspected refer to various printed circuit board assemblies, chip packages, or connector components that sequentially pass through the image acquisition station on the production line. The design drawing definition data stream contains structured descriptive information such as the pin network connection relationships, wiring layer routing specifications, and pad geometry parameters established during the design phase of the components. The continuous surface image stream is a sequence of two-dimensional digital image matrices obtained by the image acquisition device through continuous imaging of the electronic components to be inspected.

[0023] Step 110: Receive the design drawing definition data stream and surface continuous image stream corresponding to the electronic components to be inspected through the edge computing node deployed on the production line side.

[0024] In this embodiment, the edge computing node is provided with a dual-channel data receiving interface, wherein the first data channel is used to access the design drawing definition data stream from the production line design data server. The design drawing definition data stream is organized in accordance with a preset electronic design exchange format and includes a netlist file description segment, a routing constraint rule segment, and a pad size parameter segment.

[0025] In the netlist file description section, text blocks starting with the keyword "NET" sequentially list the set of pin node identifiers belonging to the same electrical network. Each pin node identifier is composed of a component reference number and a pin number. In the routing constraint rules section, the sequence of vertex coordinates of the closed polygon defined by the keyword "ROUTE_KEEPIN" defines the boundary of the allowed routing area, and the numerical parameters defined by the keyword "TRACE_WIDTH" specify the minimum width constraint for signal traces. In the pad size parameters section, the pad object defined by the keyword "PAD_STACK" includes the pad center point coordinates, pad shape type identifier, and pad width and height dimensions.

[0026] The second data channel is used to receive a continuous surface image stream from an industrial camera array. This continuous surface image stream is obtained by a high-resolution industrial camera continuously acquiring images of components passing through the detection area at a preset frame rate. Each frame is a two-dimensional digital image matrix with three color channels: red, green, and blue. The number of rows in the matrix corresponds to the number of photosensitive units in the vertical direction of the image sensor, and the number of columns corresponds to the number of photosensitive units in the horizontal direction. The value of each matrix element is the quantized brightness response value of the corresponding photosensitive unit.

[0027] While receiving the data stream, the edge computing node performs initial time window alignment between the design drawing definition data stream and the continuous surface image stream based on timestamp labels. Specifically, the timing synchronization module inside the edge computing node parses the component batch identifier and pipeline cycle number from the header of the design drawing definition data stream, and parses the image acquisition timestamp from the frame header of the continuous surface image stream. By looking up a pre-configured batch and cycle mapping table, it establishes a corresponding association between the design drawing definition data stream and the continuous surface image stream belonging to the same component on the time axis. The edge computing node has a data buffer queue inside to smooth out data arrival rate jitter that may be introduced by production line cycle fluctuations. The buffer queue adopts a first-in-first-out circular buffer structure.

[0028] Step 120: Call the lightweight backbone network pre-placed on the edge computing node to perform frame-by-frame shallow appearance representation extraction processing on the continuous surface image stream, and obtain an initial appearance representation set that records the pixel-level response distribution of the component surface. Use a residual semantic parser based on sparse autoencoder to perform topological completeness constraint rule parsing processing on the design drawing definition data stream, and obtain an edge-side logic verification map that records the physical connection relationship constraints and geometric tolerance range of the components.

[0029] In this embodiment, the lightweight backbone network is a feature extraction network structure that has undergone pre-compression and parameter quantization. Its number of network layers is significantly reduced compared to a general backbone network, and the number of convolutional kernels has been pruned to accommodate the limited computing resources and storage capacity of edge computing nodes. The original version of the lightweight backbone network can use a residual network structure pre-trained on a general image recognition task. Then, a channel pruning algorithm is used to filter out convolutional kernels in each convolutional layer whose average response activation value is lower than the pruning threshold. Parameter quantization is then used to map floating-point weights to low-bit-width integer representations. Finally, the pruned and quantized network weight file is burned into the non-volatile memory of the edge computing node.

[0030] The lightweight backbone network retains only the first few convolutional blocks used to extract shallow visual primitives such as textures, edges, and corners, without including deep semantic abstraction layers. The edge-side logic verification graph explicitly transforms the physical connection rules and manufacturing tolerance information implicit in the design graph into a structured data set that can be constrained in the feature space. This graph is used to guide the subsequent adversarial feature modification process.

[0031] Step 121: Perform inter-frame redundancy information deduplication and key image frame unit filtering on the continuous surface image stream to obtain the target image sequence set.

[0032] In a continuous image stream, adjacent image frames typically contain a large amount of repetitive visual information due to the relatively slow movement speed of components and stable lighting conditions. This step employs an inter-frame deduplication strategy based on perceptual hash difference.

[0033] Specifically, for the first and second consecutive images arriving in a continuous image stream, the two images are first converted from the red-green-blue color space to a luminance-chrominance separation space, and the luminance component two-dimensional digital image matrix is ​​extracted. The luminance component two-dimensional digital image matrix is ​​then scaled, reducing its row and column numbers to a preset ratio of its original size, resulting in a first scaled luminance matrix and a second scaled luminance matrix. Discrete cosine transform is then performed on the first and second scaled luminance matrices respectively, transforming the spatial domain luminance distribution to the frequency domain, and extracting several coefficients from the low-frequency region of the frequency domain coefficient matrix to construct a first perceptual hash vector and a second perceptual hash vector.

[0034] The system sequentially compares the magnitudes of the positional coefficients of the first and second perceptual hash vectors, counts the number of positions where the coefficients are inconsistent, and compares this number with a preset inter-frame variation threshold. If the number of inconsistent positions is lower than the inter-frame variation threshold, the second frame image is determined to not introduce significant new visual content relative to the first frame image, is marked as a redundant frame, and is discarded. If the number of inconsistent positions is higher than or equal to the inter-frame variation threshold, the second frame image is determined to be a key image frame unit and is retained in the target image sequence set.

[0035] Then, using the second frame as the new reference frame, the comparison process is repeated with the third frame until all image frames in the continuous image stream on the surface have been traversed. The target image sequence set is composed of several key image frame units selected in the order of their original timestamps. Each key image frame unit is a two-dimensional digital image matrix with three color channels: red, green, and blue.

[0036] Step 122: Each key image frame unit in the target image sequence set is sequentially input into the shallow convolution stacking module of the lightweight backbone network. The fixed-size convolution kernel group in the shallow convolution stacking module extracts the spatial variation pattern of pixel values ​​in different local neighborhoods of the key image frame unit to generate a primary pixel response map set.

[0037] The shallow convolutional stacking module of the lightweight backbone network consists of several convolutional layers, activation layers, and pooling layers arranged consecutively. In this embodiment, the shallow convolutional stacking module includes a first convolutional layer and a second convolutional layer. The input of the first convolutional layer is a three-channel two-dimensional digital image matrix of a key image frame unit, where the number of rows is denoted as the height dimension and the number of columns is denoted as the width dimension. The first convolutional layer is configured with a preset number of convolutional kernels, each of which is a weight matrix of size 3×3 pixels. Each convolutional kernel spans three color channels in the depth direction, therefore each convolutional kernel contains 3 sets of 3x3 weight parameters.

[0038] The convolution operation of the first convolutional layer uses a sliding window approach. The sliding window moves row by row in the height direction with a stride of one pixel, and column by column in the width direction with a stride of one pixel. At each sliding window position, the pixel values ​​within the window's coverage area are multiplied element-wise with the weight parameters of the corresponding convolutional kernel position, and all products are summed to obtain the convolutional response value of that window position on that convolutional kernel. After traversing all sliding window positions, each convolutional kernel generates a two-dimensional convolutional response map with the number of rows equal to the height dimension minus 2 and the number of columns equal to the width dimension minus 2. The convolutional response maps generated by all the convolutional kernels configured in the first convolutional layer are stacked along the depth direction to form the response output tensor of the first convolutional layer, whose dimensions are the number of convolutional kernels in the first convolutional layer, the height dimension minus 2, and the width dimension minus 2.

[0039] The output tensor of the first convolutional layer undergoes a non-linear mapping using a linearly rectified activation function. This function performs an element-wise operation on each element of the input tensor; if the element value is greater than 0, the original value is output; otherwise, 0 is output. After this activation function, the dimension of the output tensor remains consistent with the input tensor.

[0040] The activated tensor enters the max pooling layer. The max pooling layer uses a 2×2 pixel pooling window, sliding non-overlappingly with a stride of two pixels in both the height and width directions. Within the 2x2 region covered by each pooling window, the maximum value among four elements is selected as the output value for that window position. After max pooling, the height and width dimensions of the output tensor are reduced to half their original values, while the depth dimension remains unchanged.

[0041] The output of the max-pooling layer serves as the input to the second convolutional layer. The second convolutional layer has a larger number of convolutional kernels than the first convolutional layer, each kernel also measuring 3×3 pixels and spanning all channels of the first convolutional layer's output in the depth direction. The convolution and linear rectified activation operations of the second convolutional layer are the same as those of the first convolutional layer, ultimately outputting a multi-channel set of primary pixel response maps. Each primary pixel response map in this set corresponds to a spatial variation pattern extracted by a specific convolutional kernel, such as a horizontal brightness jump response map, a vertical brightness jump response map, a 45-degree texture response map, or a corner response map with a specific curvature.

[0042] Step 123: Perform cross-channel response intensity normalization processing on each primary pixel response map in the primary pixel response map set, map the pixel response amplitude between different channels to a unified response value range, obtain a standardized primary pixel response map set, and use the standardized primary pixel response map set as the initial appearance representation set.

[0043] Because the weight magnitude distributions of different convolutional kernels differ, and the statistical frequencies of the visual patterns responded to by different channels in the image vary, the dynamic range of the response values ​​in the primary pixel response maps of different channels is inconsistent. This step uses a local response normalization method to normalize the response intensity across channels. For the response value at each spatial location and in each channel within the primary pixel response map set, the neighborhood channel range participating in the normalization calculation is first determined. This neighborhood channel range includes the current channel and a predetermined number of adjacent channels before and after it. For channels located at the edge of the channel dimension, the neighborhood range is truncated at the boundary.

[0044] The sum of the squares of the response values ​​of all channels within the neighborhood of the given spatial location is calculated, and a small constant is added to the sum to prevent the denominator from being zero, which is then used as the normalized denominator. The original response value of the current channel is divided by the normalized denominator, and the quotient is used as the normalized response value of that spatial location in that channel. After traversing all spatial locations and all channels, a set of normalized primary pixel response maps is formed. Each normalized primary pixel response map in the set has similar statistical characteristics of response amplitude, eliminating scale differences between channels while preserving the relative response strength comparison between different channels at the same spatial location. At this point, the set of normalized primary pixel response maps is the initial appearance representation set recording the pixel-level response distribution of the component surface. This set is a three-dimensional tensor structure, whose depth dimension is equal to the number of convolutional kernels in the second convolutional layer, and whose height and width dimensions are determined by the max pooling operation.

[0045] Step 124: Perform semantic hierarchical segmentation on the design drawing definition data flow to separate the component pin connection relationship description layer, wiring routing constraint description layer, and pad geometry definition description layer contained in the design drawing definition data flow.

[0046] The design graph definition data stream is stored in a structured text format, containing various types of definition entries. This step uses keyword matching and a syntax parser to scan and semantically classify the design graph definition data stream line by line. The edge computing node has a pre-built syntax parser, constructed based on extended Backus paradigm rules, capable of identifying the boundaries of statement blocks guided by different keywords in the design graph definition data stream.

[0047] The scanner starts reading the data stream character by character from the beginning of the data stream defined in the design drawing. When it detects the keyword "NET", the parser enters the netlist parsing state and continues reading the subsequent text until it encounters the next keyword or the end of the statement. All text read during this period is then categorized into the component pin connection description layer. The component pin connection description layer records the connection topology between nodes in each electrical network. Its data structure is an associative container with electrical network identifiers as keys and a list of pin node identifiers as values.

[0048] When the scanner recognizes the keywords "ROUTE_KEEPIN" or "TRACE_WIDTH", the syntax parser enters the routing constraint parsing state, continuously reading subsequent text until it encounters the next keyword or statement end marker, at which point the read content is incorporated into the routing constraint description layer. The routing constraint description layer records the path coordinate sequence of the signal traces and constraint parameters such as minimum trace width and minimum trace spacing. The trace path is represented by an ordered sequence of two-dimensional coordinate points, with each point containing both an x-coordinate and a y-coordinate value.

[0049] When the scanner detects the keywords "PAD_STACK" or "PAD_GEOM", the parser enters the pad geometry parsing state, continuously reading subsequent text until it encounters the next keyword or statement end marker. The read content is then categorized into the pad geometry definition description layer. This layer records the center coordinates, shape type identifier, width, and height dimensions of each pad. The three description layer data are stored in three independent data structures for subsequent parser branches to process.

[0050] Step 125: The component pin connection relationship description layer, wiring direction constraint description layer and pad geometry definition description layer are parsed by a residual semantic parser based on sparse autoencoder to generate an edge-side logic verification map.

[0051] The residual semantic parser based on sparse autoencoders comprises two parallel parsing branches, one for handling connectivity semantics and the other for handling geometric constraint semantics. The first and second parsing branches share the input layer but are independent in their internal parameter structures and output formats.

[0052] Step 1251: Input the component pin connection relationship description layer to the first parsing branch of the residual semantic parser based on sparse autoencoder, extract the set of topological connection path vectors between pin nodes through the sparse constraint encoding layer in the first parsing branch, and convert the set of topological connection path vectors into a matrix description structure representing the physical connection relationship constraints of the connection on / off state between pins.

[0053] The sparse constraint coding layer in the first parsing branch consists of an encoder and a sparse constraint layer. The encoder receives netlist information from the component pin connection description layer and first assigns an initial embedding vector to each unique pin node identifier appearing in the netlist. The initial embedding vector is a multi-dimensional floating-point vector, the dimension of which is determined by a preset topology embedding dimension parameter. The initial values ​​of each element in the vector are randomly sampled from a normal distribution with a mean of 0 and a preset standard deviation as the normal distribution parameter.

[0054] The encoder constructs adjacency relationships between pin nodes based on the connectivity defined in the netlist. For each pin node contained within an electrical network in the netlist, an undirected edge is established between any two different pin nodes within that electrical network. The encoder performs multi-round graph neural network message-passing iterations. In each iteration, for each pin node in the graph, the encoder aggregates the current embedding vectors of all its neighboring nodes by summing the embedding vectors of the neighboring nodes element-wise. The aggregated neighbor vectors are concatenated with the current node's own embedding vector, and the concatenated vector is input to a fully connected layer. The fully connected layer contains a weight matrix and a bias vector, and outputs the updated node embedding vector after matrix multiplication and bias addition. The activation function of the fully connected layer uses a leaky linear rectified function, which assigns a small non-zero slope to negative inputs instead of directly setting it to zero.

[0055] After a preset number of iterations of message passing, the encoder outputs the final embedding vectors for each pin node. The sparse constraint layer applies L1 norm regularization constraints to the final embedding vectors. Specifically, it adds a penalty term proportional to the sum of the absolute values ​​of the elements in the embedding vector to the loss function and adjusts the network weights through backpropagation, causing most dimensions of the embedding vectors to approach zero, retaining only a few dimensions with significant response values ​​to extract the essential topological features of the connectivity relationships. The encoder, trained with sparse constraints, outputs embedding vectors that form a set of topological connection path vectors. Each topological connection path vector in the set represents the embedding position of a pin node in the topological space.

[0056] Perform an inner product operation on the topological connection path vectors corresponding to any two pin nodes, i.e., multiply element-wise and then sum them. Input the inner product result into a preset sign function. If the inner product result is greater than a preset connectivity threshold, output a first logical value indicating that there is a pre-existing electrical connection between the pin pairs; if the inner product result is less than or equal to the connectivity threshold, output a second logical value indicating that there is no pre-existing electrical connection. Arrange the binary indicators of all pin pairs according to their pin indices to form a matrix description structure of physical connection constraints. This matrix description structure is a symmetric binary square matrix, the order of which is equal to the total number of pin nodes. The value of the element in the i-th row and j-th column of the square matrix is ​​a first logical value indicating that there is a pre-existing electrical connection between the i-th pin and the j-th pin, and a second logical value indicating that there is no pre-existing electrical connection.

[0057] Step 1252: The routing constraint description layer and the pad geometry definition description layer are jointly input into the second parsing branch of the residual semantic parser based on sparse autoencoder. The geometric gap margin between the ideal routing trajectory defined in the routing constraint description layer and the pad boundary defined in the pad geometry definition description layer is calculated through the residual mapping layer in the second parsing branch. A distribution descriptor describing the range of geometric tolerance deviations that allow for appearance deformation in different regions is generated.

[0058] The residual mapping layer in the second analytical branch consists of a geometric interval calculation unit and a residual encoder. The geometric interval calculation unit processes each ideal wiring trajectory segment in the wiring routing constraint description layer. For an ideal wiring trajectory represented by an ordered sequence of coordinate points, the geometric interval calculation unit performs linear interpolation sampling between every two adjacent coordinate points of the segment. The interpolation density is controlled by a preset parameter for the number of sampling points per unit length, generating a discrete sampling point sequence. Each discrete sampling point has an abscissa and a ordinate value.

[0059] For each discrete sampling point, the geometric interval calculation unit traverses all pad objects defined in the pad geometry definition description layer. For each pad object, the corresponding point-to-boundary distance calculation method is used based on its shape type identifier. If the shape type identifier indicates that the pad is rectangular, the minimum perpendicular distance from the discrete sampling point to the lines containing the four sides of the rectangle is calculated. This minimum perpendicular distance is then combined with the determination result of whether the discrete sampling point falls inside the rectangle. If the discrete sampling point is inside the rectangle, the distance is negative; if it is outside, it is positive, resulting in a directed distance. If the shape type identifier indicates that the pad is circular, the Euclidean distance from the discrete sampling point to the center of the circle is calculated, minus the radius of the circle, to obtain the directed distance. The minimum directed distance from the discrete sampling point to all pad boundaries is taken as the original geometric interval at that sampling point.

[0060] The original geometric interval is compared with a preset manufacturing process capability upper limit, which is calculated by multiplying the standard deviation of process deviations from historical production line statistics by a preset process capability coefficient. If the original geometric interval is greater than the manufacturing process capability upper limit, the upper limit is used as the geometric interval margin at that sampling point; if the original geometric interval is less than or equal to the manufacturing process capability upper limit, it is used as the geometric interval margin. The geometric interval margin reflects the safe redundancy space between the wiring trace and the pad at that spatial location to accommodate appearance deformation.

[0061] The residual encoder receives the geometric interval margin values ​​and their corresponding spatial coordinates of all discrete sampling points. Through several fully connected layers and an upsampling layer, it extends these geometric interval margin values ​​to a continuous two-dimensional space covering the entire component surface. The first fully connected layer of the residual encoder maps the two-dimensional spatial coordinates to a high-dimensional feature space, and the second fully connected layer maps the high-dimensional features back to scalar interval margin prediction values. The upsampling layer uses bilinear interpolation to interpolate the interval margin estimates at any continuous coordinate position based on the interval margin values ​​of the discrete sampling points. The residual encoder traverses a pre-defined uniform grid on the component surface, outputting the geometric interval margin estimate at each grid point. Finally, it generates a distribution descriptor for the geometric tolerance range. This distribution descriptor is a two-dimensional matrix with the same spatial dimensions as the initial appearance representation set, where each element represents the maximum allowable appearance deformation offset at the corresponding spatial position.

[0062] Step 1253: Spatial registration is performed on the matrix description structure and the distribution descriptor to establish a mapping index relationship between the position of each pin node in the matrix description structure and the corresponding spatial region in the distribution descriptor.

[0063] In the matrix description structure, each pin node corresponds to the center coordinates of the pads defined in the pad geometry definition description layer. This step establishes a mapping table from the pin node identifier to the horizontal and vertical coordinates of the pad center by reading the center coordinates of each pad object in the pad geometry definition description layer. Subsequently, based on the two-dimensional matrix indexing rules of the distribution descriptor, the origin position, grid spacing, and total number of rows and columns of the grid dot matrix in the distribution descriptor are determined. For each pin node, the horizontal coordinate of its pad center is subtracted from the horizontal coordinate of the origin, and then divided by the grid spacing to obtain a floating-point column index value, which is rounded down and used as the column index in the distribution descriptor; the vertical coordinate of its pad center is subtracted from the vertical coordinate of the origin, and then divided by the grid spacing to obtain a floating-point row index value, which is rounded down and used as the row index in the distribution descriptor.

[0064] The row and column indices of the distribution descriptors corresponding to the i-th and j-th pins in the matrix description structure are combined to establish a mapping index relationship from the pin pair identifier to the spatial grid position range of the distribution descriptor. This mapping index relationship is stored in the form of key-value pairs, where the key is the pin index pair identifier and the value is the grid coordinate range consisting of the start row index, end row index, start column index, and end column index. This range usually covers a rectangular neighborhood area centered on the pin pad and determined by a preset search radius.

[0065] Step 1254 involves fusing the matrix description structure and distribution descriptor after establishing the mapping index relationship to generate an edge logic verification map containing the component physical connection relationship constraint field and the geometric tolerance range field.

[0066] The data structure fusion process uses the matrix description structure as the value domain of the physical connection constraint field, the distribution descriptor as the value domain of the geometric tolerance range field, and the mapping index relationship established in step 1253 as the pointer field of the two fields, and encapsulates them together into a unified edge-side logical verification graph data structure.

[0067] The edge-side logic verification map is represented in memory as a composite structure containing three members: the first member is a pointer to a binary matrix, used to store physical connection constraints; the second member is a pointer to a floating-point two-dimensional matrix, used to store the geometric tolerance range; and the third member is a pointer to a mapping table, used to store the correspondence between pin pair identifiers and grid coordinate ranges.

[0068] The edge-side logic verification graph is read and parsed by the adversarial feature illusion strategy in subsequent steps to apply topological completeness constraint rules.

[0069] Step 130: Using a preset adversarial feature illusion strategy and the edge-side logic verification graph as a guiding condition, perform reverse perturbation iterative correction processing on the initial appearance representation set to generate an illusion appearance representation set that is constrained by topological completeness rules in the feature space distribution.

[0070] This step aims to generate a fictitious but conforming appearance representation to the component design specifications, corresponding to the characteristic response distribution that a component without manufacturing defects should exhibit under ideal conditions. By subjecting the actual extracted initial appearance representation set to constrained perturbation correction towards this fictitious representation, the degree of appearance deviation caused by defects can be amplified, thereby enhancing the detection sensitivity of defect signals in subsequent steps. The core of the adversarial feature phantom construction strategy lies in the fact that the perturbation is not arbitrarily applied, but rather a directional iterative search under the joint constraints of the connection topology invariance constraints and geometric deformation boundary constraints defined by the edge-side logic verification map.

[0071] Step 131: Parse the matrix description structure of the physical connection relationship constraints corresponding to the physical connection relationship constraints of the components from the edge-side logic verification map, and expand the matrix description structure into a set of constraint vector fields acting on the specified dimension of the feature space.

[0072] In the matrix description structure, the binary square matrix is ​​transformed into a set of orthogonal basis vectors through singular value decomposition (SVD). SVD decomposes the binary square matrix into a product of a left singular vector matrix, a singular value diagonal matrix, and a right singular vector matrix. Since the binary square matrix is ​​symmetric, the left and right singular vector matrices are transposes of each other. Right singular vectors corresponding to singular values ​​greater than a preset singular value threshold are selected from the singular value diagonal matrix to form a set of topology pattern vectors. The dimension of each topology pattern vector is equal to the total number of pin nodes, and the element values ​​in the vector represent the participation level of each pin node in that topology pattern.

[0073] The characteristic subspace directions spanned by elements in the topology pattern vector that have a pre-existing electrical connection between corresponding pin pairs are marked as permissible directions. Specifically, for pin pairs with the first logic value in the matrix description structure, the projection direction of the corresponding topology pattern vector onto the dimension of that pin pair in the characteristic space is marked as a differentiable direction. The characteristic subspace directions corresponding to elements in the topology pattern vector that do not have a pre-existing electrical connection between corresponding pin pairs are marked as prohibited directions, i.e., the projection directions are marked as zero-space directions. All differentiable directions and zero-space directions are combined according to their corresponding spatial positions to form a constraint vector field.

[0074] A local projection operator is defined at each location in the feature space of the constraint vector field. This local projection operator is implemented in the form of a projection matrix. The projection matrix is ​​constructed as follows: take all differentiable direction unit vectors, calculate the outer product of each differentiable direction unit vector with its own transpose, and sum all the outer product results element by element to obtain the projection matrix. This projection matrix can decompose any input feature vector into components parallel to the differentiable direction and components perpendicular to the differentiable direction. The component parallel to the differentiable direction is the product of the projection matrix and the input feature vector, and the component perpendicular to the differentiable direction is the input feature vector minus the parallel component.

[0075] Step 132: Parse the distribution descriptor corresponding to the geometric tolerance range from the edge-side logic verification graph, and quantize the distribution descriptor into a perturbation amplitude upper limit mask acting on each feature response point in the initial appearance representation set.

[0076] Each element value in the distribution descriptor represents the maximum permissible deformation offset at the corresponding spatial location. This step directly uses the maximum permissible deformation offset value in the distribution descriptor as the maximum change that the feature response point at the corresponding spatial location can be modified. Since the distribution descriptor is a two-dimensional matrix, while the initial appearance representation set is a three-dimensional tensor with a depth dimension, the two-dimensional matrix needs to be copied and expanded along the depth dimension to ensure that its depth dimension matches the number of channels in the initial appearance representation set.

[0077] The expanded perturbation amplitude upper bound mask is a three-dimensional tensor with the exact same dimensions as the initial appearance representation set. Its height and width dimensions are identical to the distribution descriptor, and its depth dimension equals the number of channels in the initial appearance representation set. The element values ​​at position coordinates x, y, and channel c in the perturbation amplitude upper bound mask are equal to the element values ​​at position coordinates x, y in the distribution descriptor. The perturbation amplitude upper bound mask stipulates that, during subsequent iterative corrections, the absolute value of the change in the eigenvalue at each feature response point must not exceed the limit value at the corresponding position in the mask.

[0078] Step 133: Construct the initial adversarial perturbation field, which is a random noise distribution with the same dimension and spatial size as the initial appearance representation set.

[0079] Each element value in the initial adversarial perturbation field is independently sampled from a truncated normal distribution with a mean of 0 and a variance of a preset small perturbation amplitude. The truncation boundary is set to a range of ±3 times the variance to ensure that the initial perturbation does not introduce extreme deviations to the characteristic response. During sampling, a random number conforming to a standard normal distribution is first generated, and multiplied by the preset small perturbation amplitude to obtain an untruncated perturbation value. If the absolute value of the untruncated perturbation value is greater than 3 times the small perturbation amplitude, it is set to a value with the same sign as the untruncated perturbation value and the same 3 times the small perturbation amplitude; otherwise, the original value is retained. The generated random perturbation values ​​are then filled sequentially according to the row, column, and channel coordinates of the three-dimensional tensor to form a complete initial adversarial perturbation field.

[0080] Step 134: The initial adversarial perturbation field and the initial appearance representation set are superimposed for the first time to generate the candidate perturbation appearance representation set for the first round of iteration.

[0081] The initial overlay process involves adding the element values ​​at the corresponding spatial locations. Specifically, for a location specified by height coordinate y, width coordinate x, and channel coordinate c, the response value of the candidate perturbation appearance representation set at that location is equal to the arithmetic sum of the response value of the initial appearance representation set at that location and the perturbation value of the initial adversarial perturbation field at that location. This addition operation is performed iteratively across all height coordinates, width coordinates, and channel coordinates.

[0082] Step 135: Input the candidate perturbation appearance representation set into the preset feature distribution distance metric function, and calculate the geodesic distance increment between the candidate perturbation appearance representation set and the initial appearance representation set on the preset high-dimensional feature manifold.

[0083] The feature distribution distance metric function employs a discrete approximation method based on bulldozer distance. First, both the candidate perturbation appearance representation set and the initial appearance representation set are normalized to a probability mass distribution per unit total mass. The normalization process involves: calculating the sum of the absolute values ​​of all elements in the candidate perturbation appearance representation set to obtain the candidate total mass; calculating the sum of the absolute values ​​of all elements in the initial appearance representation set to obtain the initial total mass. Dividing each element value in the candidate perturbation appearance representation set by the candidate total mass yields the candidate probability mass distribution; dividing each element value in the initial appearance representation set by the initial total mass yields the initial probability mass distribution.

[0084] The candidate probability mass distribution and the initial probability mass distribution are considered as two multidimensional discrete probability distributions. Calculating the bulldozer distance between them requires solving an optimal transport planning problem, namely, finding the transport scheme that minimizes the total transport cost under the constraints that the mass transported from each location in the initial probability mass distribution to each location in the candidate probability mass distribution is non-negative, and the total outbound volume from each location equals its initial probability mass, and the total inbound volume equals its candidate probability mass. The total transport cost is obtained by multiplying the transport mass on each transport path by the weighted sum of the Euclidean distances in the feature space between the two endpoints of that path. The Euclidean distance in the feature space is calculated as follows: for the i-th feature point in the initial probability mass distribution and the j-th feature point in the candidate probability mass distribution, calculate the sum of the squares of the elements of the difference between their eigenvectors, and then take the square root.

[0085] Solving the optimal transport planning problem can employ the network simplex algorithm or the entropy-regularized iterative scaling algorithm. The geodesic distance increment is the difference between the minimum total transport cost calculated in the current iteration step and the minimum total transport cost in the previous iteration. For the first iteration, the minimum total transport cost in the previous iteration can be considered infinite or the ratio of the increment to the initial state can be directly calculated.

[0086] Step 136: Perform projection correction processing on the candidate perturbation appearance representation set based on the constraint vector field, and compress the feature components in the candidate perturbation appearance representation set that conflict with the prohibited overstepping direction indicated by the constraint vector field.

[0087] For the feature vector at each spatial location in the candidate perturbation appearance representation set, projection correction is performed using the local projection operator defined in step 131. The feature vector arranged along the channel dimension at the current spatial location is treated as a column vector, and matrix multiplication is performed with the projection matrix. The product is the projection component of the original feature vector in the allowed direction of change; this projection component is the constrained projected feature vector. The difference between the original feature vector and the projection component is the compressed prohibited direction component.

[0088] After this operation, the feature distribution variation represented in the candidate perturbation appearance representation set is confined to a subspace permitted by the connection topology rules, avoiding spurious feature patterns that violate the electrical connection constraints between pins due to random perturbations. The projection correction process is performed independently for each spatial location, and the processed data is called the constrained projection perturbation appearance representation set.

[0089] Step 137: Perform amplitude clipping processing on the set of perturbation appearance representations after constraint projection based on the upper limit mask of perturbation amplitude, and forcibly truncate the feature response points in the set of perturbation appearance representations after constraint projection whose change exceeds the maximum change specified by the upper limit mask of perturbation amplitude to the range of the maximum change.

[0090] For each feature response point in the constrained projection perturbation appearance representation set, calculate the difference between the response value of that point and the response value of the corresponding point in the initial appearance representation set. Compare the absolute value of this difference with the limit value at the corresponding position in the perturbation amplitude upper limit mask. If the absolute difference is less than or equal to the limit value, retain the response value; if the absolute difference is greater than the limit value, modify the response value to the response value of the corresponding point in the initial appearance representation set plus or minus the limit value, with the sign of the addition or subtraction consistent with the original difference. Amplitude clipping is performed independently for each feature response point, resulting in an amplitude-constrained perturbation appearance representation set.

[0091] Step 138: The set of appearance representations of amplitude-limited perturbations is used as the input of the perturbation field for the new round of iterations, and the initial superposition processing, feature distribution distance metric function calculation, projection correction processing and amplitude clipping processing are repeated until the geodesic distance increment converges to the preset stable interval.

[0092] This step iteratively optimizes the adversarial perturbation field. At the beginning of each iteration, the amplitude-constrained perturbation appearance representation set of the current round replaces the initial adversarial perturbation field in step 134, and is superimposed on the initial appearance representation set to generate a new batch of candidate perturbation appearance representation sets. These sets then sequentially undergo the feature distribution distance metric function calculation in step 135, the projection correction processing in step 136, and the amplitude clipping processing in step 137. The iteration termination condition is set when the absolute value of the geodesic distance increment between two consecutive iterations is less than a preset convergence threshold. The preset convergence threshold is dynamically determined based on the total quality and feature dimension of the initial appearance representation set. Once the convergence condition is met, the iteration loop terminates.

[0093] Step 139: The set of amplitude-constrained perturbation appearance representations that have reached the convergence condition is determined as the set of phantom structure appearance representations.

[0094] Each feature response point in the phantom appearance representation set has been corrected by projection of the constraint vector field, satisfying the connection topology invariance specified by the physical connection constraints of the components. Simultaneously, its change relative to the initial appearance representation set is within the appearance deformation boundary conditions specified by the geometric tolerance range. This phantom appearance representation set represents the projection of the ideal component appearance in the feature space, which is closest to the actual observed appearance and fully conforms to the design specification definition. The phantom appearance representation set and the initial appearance representation set are completely identical in data structure and dimension, both being three-dimensional tensor structures.

[0095] Step 140: Calculate the non-consistent property of the feature distribution of the initial appearance representation set and the phantom appearance representation set in the preset feature dimension space, and calculate the topological manifold connectivity difference between the virtual geometric topology of the components represented by the phantom appearance representation set and the reference geometric topology of the components defined by the edge-side logic verification map.

[0096] This step quantitatively assesses the degree of inconsistency between the initial appearance representation and the phantom appearance representation at the feature distribution and topological structure levels, generating unfamiliar quantifiable criteria that reflect the degree to which the component under quality inspection deviates from the design specifications. The non-self-consistent property of feature distribution measures the degree of deviation from the pixel-level response difference of the appearance representation, while the difference in topological manifold connectivity measures the degree of deviation from the geometric consistency of the component's physical connection structure.

[0097] Step 141: Obtain the initial feature vector at each spatial location in the initial appearance representation set, and obtain the phantom feature vector at the corresponding spatial location in the phantom appearance representation set, and construct a feature vector pairing sequence indexed by spatial location.

[0098] The initial appearance representation set and the phantom appearance representation set have the same height, width, and depth dimensions. Therefore, at the same height coordinate (y) and width coordinate (x), an initial feature vector consisting of all elements in the depth dimension can be extracted from the initial appearance representation set, and a phantom feature vector consisting of all elements in the depth dimension can be extracted from the phantom appearance representation set. The lengths of both the initial and phantom feature vectors are equal to the number of channels in the initial appearance representation set. The initial and phantom feature vectors corresponding to the same height and width coordinates are combined into vector pairs and arranged in a row-first, column-second scanning order to form a feature vector pairing sequence. The length of the sequence is equal to the height dimension multiplied by the width dimension.

[0099] Step 142: For each pair of feature vectors in the feature vector pairing sequence, calculate the cosine dissimilarity measure between the initial feature vector and the phantom feature vector in the vector space.

[0100] Let the initial feature vector be vector Vinit, and its k-th element be Vinit. k The phantom feature vector is a vector Vhall, whose k-th element is Vhall. k The value of k ranges from 1 to the number of channels. First, calculate the inner product (Inner) of vectors Vinit and Vhall. The calculation process is as follows: for k, iterate from 1 to the number of channels, accumulating the values ​​of Vinit. k Multiply by Vhall k The product result. Calculate the L2 norm NormInit of the vector Vinit. The calculation process of NormInit is as follows: for k, iterate from 1 to the channel number, accumulating Vinit... k The squared value of Vhall is calculated, and the square root of the accumulated result is taken. The L2 norm NormHall of the vector Vhall is calculated. The NormHall calculation process is as follows: for k, iterate from 1 to the channel number, and accumulate Vhall. kThe squares of the sums are taken, and the square root of the sum is taken. The inner product Inner is divided by the product of NormInit and NormHall to obtain the cosine similarity CosSim. The difference between the constant 1 and the cosine similarity CosSim is taken as the cosine dissimilarity measure CosDis. The value of CosDis ranges from 0 to 2. When the two vectors are in the same direction, CosDis is 0, and when the directions are completely opposite, CosDis is 2. The larger the value, the more significant the difference in direction between the two vectors.

[0101] Step 143: For each pair of feature vectors in the feature vector pairing sequence, calculate the Euclidean magnitude difference metric between the initial feature vector and the phantom feature vector in the vector space.

[0102] Calculate the L2 norm NormInit for vector Vinit and the L2 norm NormHall for vector Vhall, following the same procedure as in step 142. Calculate the difference between NormInit and NormHall, and take the absolute value of this difference as the Euclidean modulus difference metric LenDis. LenDis reflects the difference in overall response intensity between the initial appearance representation and the phantom appearance representation at the same spatial location; a larger value indicates a more drastic change in response intensity.

[0103] Step 144: The cosine dissimilarity metric and the Euclidean modulus difference metric at the same spatial location are fused to generate the local feature non-consistent response value at that spatial location.

[0104] The fusion operation employs a weighted geometric average. Specifically, the weighted cosine anisotropy is obtained by multiplying the cosine dissimilarity metric CosDis with the first weighting factor Wcos; the weighted modulus difference is obtained by multiplying the Euclidean modulus difference metric LenDis with the second weighting factor Wlen. The square root of the product of the weighted cosine anisotropy and the weighted modulus difference is used as the local feature non-self-consistent response value NonSelfResp. The first weighting factor Wcos and the second weighting factor Wlen are pre-calibrated based on the relative sensitivity of the feature channels containing the initial feature vectors to directional and intensity differences; typically, the sum of Wcos and Wlen is a constant. The local feature non-self-consistent response values ​​at all spatial locations are organized into a two-dimensional feature distribution non-self-consistent attribute map according to the original height and width coordinates. The number of rows in this map equals the height dimension, and the number of columns equals the width dimension.

[0105] Step 145: Extract the contour response channels that record the edge contour information of components from the phantom appearance representation set, perform edge connectivity tracing processing on the contour response channels, and generate a phantom appearance edge topology skeleton descriptor composed of continuous edge segments.

[0106] During the training phase of the lightweight backbone network, an edge-aware regularization term can be added to the loss function to guide specific channels to selectively respond to image gradient directions, so that the response maps of some channels correspond to the physical edge contours of components. In this step, within the phantom appearance representation set, the Laplacian operator response value of the two-dimensional response map is calculated for each channel. The Laplacian operator response is calculated as follows: for each internal pixel position in the response map, the sum of its four neighboring pixel values ​​(top, bottom, left, and right) minus four times the center pixel value is taken as the Laplacian response at that position. The variance of the Laplacian response value for each channel is calculated, and the K channels with the largest variances are selected as contour response channels, where K is a preset positive integer.

[0107] The response maps of the selected K contour response channels are subjected to an element-wise maximum maximization operation. Specifically, at each height and width coordinate, the response values ​​of the K channels at that location are compared, and the maximum value is selected as the pixel value of the fused composite edge response map at that location. A non-maximum suppression algorithm is applied to the composite edge response map: for each pixel location in the composite edge response map, the pixel value at that location is compared with the pixel values ​​of its two adjacent locations along the gradient direction. If the pixel value at that location is not a local maximum, it is set to 0; otherwise, the original value is retained. The gradient direction is quantized into one of four directions based on the arctangent of the ratio of the Sobel gradient response in the height and width directions at that location.

[0108] A dual-threshold hysteresis connection algorithm is applied to the composite edge response map after non-maximum suppression. Two thresholds are set: a high threshold and a low threshold, with the high threshold being greater than the low threshold. All pixel positions in the composite edge response map are traversed. Positions with pixel values ​​greater than the high threshold are marked as strong edge points; positions with pixel values ​​between the low and high thresholds are marked as weak edge points; and positions with pixel values ​​below the low threshold are directly suppressed to 0. Subsequently, for each weak edge point, it is checked whether a strong edge point exists within its eight-neighborhood. If so, the weak edge point is upgraded to a strong edge point and retained; otherwise, it is suppressed to 0.

[0109] For the retained strong edge points, connected component extraction is performed. An eight-neighborhood-based seed filling algorithm is used to aggregate spatially adjacent strong edge points into the same edge segment, and a unique segment identifier is assigned to each edge segment. Each edge segment is represented by an ordered sequence of pixel coordinates, the order of which is determined by tracing the connection path from one end of the segment along its eight neighbors to the other end. All edge segments together constitute the topological skeleton descriptor of the phantom appearance edge.

[0110] Step 146: Extract the matrix description structure recorded in the edge-side logic verification graph and convert the matrix description structure into a reference topology connection graph composed of nodes and connecting edges.

[0111] Pin pairs corresponding to elements in the matrix description structure that take the first logical value are designated as node pairs with connected edges. For each node pair with connected edges, the center coordinates of the pads corresponding to the i-th and j-th pins are read from the pad geometry definition description layer. On the two-dimensional plane, a straight line segment is drawn with the center coordinates of the i-th pin's pad as the starting point and the center coordinates of the j-th pin's pad as the ending point, representing the geometric path of the connected edge. A node identifier is assigned to each node, which is the pin node identifier; an edge identifier is assigned to each connected edge, which is formed by concatenating the start node identifier and the end node identifier, and is associated with the geometric path representation of the connected edge. All nodes and connected edges constitute the reference topology connection graph.

[0112] Step 147: Project the phantom appearance edge topology skeleton descriptor onto the geometric reference plane where the reference topology connection diagram is located, and calculate the overlap length ratio and deviation angle difference between each edge line segment in the phantom appearance edge topology skeleton descriptor and the corresponding connection edge in the reference topology connection diagram.

[0113] The projection process is achieved by transforming the pixel coordinates in the topological skeleton descriptor of the illusory appearance edge to the design drawing coordinate system according to the calibration parameters of the image acquisition device. The calibration parameters of the image acquisition device include the scaling factor between the pixel size and the actual physical size, the offset of the image coordinate system origin in the design drawing coordinate system, and possible lens distortion coefficients. The pixel coordinate sequence of the edge line segments is converted into a physical coordinate sequence in the design drawing coordinate system using these calibration parameters.

[0114] For each transformed phantom edge segment, the nearest connecting edge in the reference topology connection graph is selected as the corresponding connecting edge. The spatial Euclidean distance is calculated as follows: calculate the coordinates of the midpoint of the phantom edge segment, calculate the shortest distance from that midpoint to the geometric path of each connecting edge in the reference topology connection graph, and select the connecting edge with the smallest shortest distance as the corresponding connecting edge. The curve length of the overlapping portion between the phantom edge segment and the corresponding connecting edge is calculated. The overlapping portion is determined as follows: for each sampling point on the phantom edge segment, calculate its perpendicular distance to the geometric path of the corresponding connecting edge. If the perpendicular distance is less than a preset overlap threshold, the sampling point is considered to belong to the overlapping portion. The curve lengths of the overlapping portions are summed and divided by the total length of the phantom edge segments to obtain the overlap ratio RatioOverlap.

[0115] Simultaneously, the principal direction angles of the phantom edge segments and the corresponding connecting edges are calculated. The principal direction angles of the phantom edge segments are obtained by performing principal component analysis on their pixel coordinate sequences, taking the angle value of the first principal component direction. The principal direction angle of the corresponding connecting edge is determined by the direction of the line connecting its start and end coordinates. The absolute value of the difference between the two direction angles is taken as the deviation angle difference AngDiff. If the difference is greater than half of the straight angle, it is corrected by subtracting the difference from the straight angle, so that the value of AngDiff ranges from 0 to half of the straight angle.

[0116] Step 148: Construct a topological connectivity difference metric function based on the overlap length ratio and the difference in deviation angle, and use this function to calculate the local topological manifold connectivity difference value.

[0117] The topological connectivity difference function, TopoDiff, is designed as the product of a monotonically decreasing function of the overlap length ratio RatioOverlap and a monotonically increasing function of the deviation angle difference AngDiff. The monotonically decreasing function is the negative form of the natural exponential function exp, i.e., the result of multiplying RatioOverlap by a negative adjustment factor α with the natural constant e as the base. The monotonically increasing function is a linear function, i.e., constant 1 + adjustment factor β × AngDiff. The adjustment factor α controls the steepness of the influence of the overlap length ratio on the difference measure value, and the adjustment factor β controls the contribution weight of the deviation angle difference to the difference measure value. The local topological connectivity difference value, LocalTopoDiff, is equal to the product of the natural exponential function value and the linear function value.

[0118] The LocalTopoDiff of each phantom edge segment is calculated by traversing all phantom edge segments. For blank areas not covered by any phantom edge segments, the LocalTopoDiff value is obtained by inverse distance weighted interpolation of the LocalTopoDiff values ​​of neighboring edge segments. The interpolation process is as follows: for each blank grid point, a predetermined number of nearest edge segments are found, the Euclidean distance from the grid point to each edge segment is calculated, and the reciprocal of the distance is used as the weight to perform a weighted average of the LocalTopoDiff values ​​of each edge segment. The interpolation results are extended to the entire spatial region of the phantom appearance representation set to generate a topological manifold connectivity difference distribution map covering the entire space. This distribution map is a two-dimensional matrix with the number of rows and columns equal to the height and width dimensions of the phantom appearance representation set, respectively.

[0119] Step 150: Generate an unfamiliar quantitative map describing the degree of deviation of each spatial location region from the design specifications in the continuous image stream of the surface based on the non-consistent property of feature distribution and the difference in topological manifold connectivity. Then, input it into the preset abnormal mode definition library for matching to determine the defect location descriptor and defect type descriptor of the electronic component to be inspected.

[0120] This step integrates the discrepancies between the appearance and topology levels into a unified unfamiliarity metric, and then compares it with known anomaly patterns to locate and classify defects. The value of each grid point in the unfamiliarity metric map comprehensively reflects the degree of deviation of that spatial location from the design specifications in both appearance pixel response and physical topology connectivity dimensions.

[0121] Step 151: Spatially align and overlay the non-self-consistent attribute map of feature distribution with the topological manifold connectivity difference distribution map, so that each grid point in the non-self-consistent attribute map of feature distribution has the same spatial coordinate reference as the corresponding grid point in the topological manifold connectivity difference distribution map.

[0122] Since both the feature distribution non-consistent attribute map and the topological manifold connectivity difference distribution map are derived from the same original image coordinate system, they have the same number of rows and columns, and the physical spatial locations corresponding to each row and column index are consistent, thus possessing natural spatial alignment properties. This step achieves alignment and overlay by constructing a blank joint data raster structure. The joint data raster has three bands: the first band stores height coordinate information, the second band stores width coordinate information, and the third band stores joint feature data. For each raster point in the joint data raster, its height and width coordinates are filled into the first and second bands, respectively. The local feature non-consistent response value at the corresponding position in the feature distribution non-consistent attribute map and the local topological manifold connectivity difference value at the corresponding position in the topological manifold connectivity difference distribution map are used as a pair of correlated values ​​and filled into the third band.

[0123] Step 152: For each aligned grid point, extract the local feature non-self-consistent response value NonSelfResp of the grid point in the feature distribution non-self-consistent attribute map, and extract the local topological manifold connectivity difference value LocalTopoDiff of the grid point in the topological manifold connectivity difference distribution map.

[0124] Step 153: Based on the preset unfamiliarity aggregation logic, the local feature non-consistent response value and the local topological manifold connectivity difference value are jointly mapped to generate the joint unfamiliarity response value at the grid point.

[0125] The pre-defined unfamiliarity aggregation logic employs a nonlinear fusion function with adjustable weights, constructed based on a multivariate adaptive regression spline method. During construction, NonSelfResp and LocalTopoDiff are used as two-dimensional input vectors, and the desired joint unfamiliarity response value is used as the output scalar. Complex mapping relationships are approximated by recursively dividing the input space into regions and fitting linear basis functions within each region. Specifically, for any given pair of NonSelfResp and LocalTopoDiff values, it is first determined which leaf node region of the multivariate adaptive regression spline model it falls into. The leaf node region is defined by a series of binary decision conditions regarding the values ​​of NonSelfResp or LocalTopoDiff. Then, the linear combination coefficients corresponding to that leaf node region are extracted, including constant terms, coefficients of NonSelfResp, coefficients of LocalTopoDiff, and coefficients of any possible nonlinear basis function terms. Substituting NonSelfResp and LocalTopoDiff into this linear combination expression, the joint unfamiliarity response value NoveltyVal is calculated. When both NonSelfResp and LocalTopoDiff are in the low value range, the output value increases slowly to suppress false alarms caused by noise; when at least one input component enters the high value range, the output value rises rapidly to enhance the response sensitivity to significant anomalies.

[0126] The algorithm iterates through all grid points covered by the continuous image stream on the surface, performs the joint mapping operation described above on each grid point, and arranges the calculated NoveltyVal according to the height and width coordinates of that grid point to generate a strange metric map covering the complete appearance area of ​​the electronic component to be inspected. The strange metric map is a two-dimensional floating-point matrix, where the number of rows equals the height dimension and the number of columns equals the width dimension.

[0127] Step 154: Perform spatial connectivity analysis on the unfamiliarity quantification graph, aggregate adjacent grid points in the unfamiliarity quantification graph whose joint unfamiliarity response values ​​exceed the preset response threshold into unfamiliarity abnormal connectivity regions, and assign an abnormal connectivity region identifier to each unfamiliarity abnormal connectivity region.

[0128] Spatial connectivity analysis employs a seed-fill-based region growing algorithm. The preset response threshold, ThreshNovelty, is determined based on the statistical distribution of the unfamiliarity metric map of historical normal samples; for example, a high percentile of the historical normal sample unfamiliarity value distribution can be used as the threshold. The region growing algorithm traverses each raster point in the unfamiliarity metric map in a row-first, column-second scanning order. When a raster point with a joint unfamiliarity response value greater than ThreshNovelty is encountered for the first time and is not marked as visited, it is used as a seed point and added to a newly created connectivity queue. A unique anomalous connectivity identifier is assigned to this new connectivity queue; the identifier can be an incrementing integer.

[0129] Take a grid point from the connected component queue and examine each of its eight neighboring grid points. If the joint unfamiliarity response value of the neighboring grid points is greater than ThreshNovelty and they are not marked as visited, add the neighboring grid point to the connected component queue and mark it as visited. Repeat this process until the connected component queue is empty, at which point the extraction of an unfamiliarity-anomalous connected component is complete. Continue scanning subsequent grid points until the entire unfamiliarity metric graph has been traversed, completing the extraction and labeling of all unfamiliarity-anomalous connected components.

[0130] Step 155: For each unfamiliarity-abnormal connected domain, extract the joint unfamiliarity response distribution pattern within the unfamiliarity-abnormal connected domain. The joint unfamiliarity response distribution pattern includes the spatial gradient change trend and peak clustering pattern of the joint unfamiliarity response values ​​within the unfamiliarity-abnormal connected domain.

[0131] The spatial gradient trend is obtained by calculating the Sobel gradients of the joint unfamiliarity response values ​​at each grid point within the connected domain in the height and width directions. For a grid point within the connected domain, the Sobel gradient in the height direction is the difference between the joint unfamiliarity response values ​​of the grid points above and below that point; the Sobel gradient in the width direction is the difference between the joint unfamiliarity response values ​​of the grid points to the right and to the left of that point. The gradient direction angle is calculated based on the ratio of the height gradient to the width gradient, and the gradient direction angle ranges from 0 to 2π radians. The gradient direction angles of all grid points within the connected domain are statistically analyzed to construct a gradient direction histogram, with each interval of the histogram covering a preset angle range. After normalization, the gradient direction histogram forms a gradient direction distribution vector, which describes the dominant directional trend of unfamiliarity changes within the connected domain.

[0132] Peak clustering patterns are obtained by locating the set of local maxima of the joint unfamiliarity response values ​​within a connected component. For a grid point within a connected component, if its joint unfamiliarity response value is greater than the joint unfamiliarity response values ​​of all grid points in its eight neighborhoods, then that grid point is marked as a local maximum. The spatial coordinates of all local maxima are extracted, and the convex hull of these coordinates is calculated. The convex hull is the smallest convex polygon containing all local maxima. The geometric properties of the convex hull are extracted as peak clustering pattern descriptors, including the area, perimeter, aspect ratio of the smallest bounding rectangle, and the central moment invariant calculated based on the vertex coordinates of the convex hull. These descriptors together constitute the joint unfamiliarity response distribution pattern of this unfamiliarity-anomaly connected component.

[0133] Step 156: Combine the joint unfamiliarity response distribution pattern with the reference abnormal mode distribution pattern stored in the preset abnormal mode definition library to determine the defect location descriptor and defect type descriptor of the electronic component to be inspected.

[0134] The pre-defined abnormal mode definition library is a structured database containing multiple records, each corresponding to a known component defect type. Each record contains the following fields: defect type identifier, defect semantic descriptor, and reference abnormal mode distribution pattern. The reference abnormal mode distribution pattern is obtained by statistically learning the typical gradient direction distribution vector and peak aggregation morphology descriptor of the defect type from a large number of labeled defect type samples using the same data structure and extraction method as in step 155.

[0135] Step 1561: Perform similarity matching between the joint unfamiliarity response distribution pattern and the reference abnormality mode distribution patterns stored in the preset abnormality mode definition library. When the similarity metric between the joint unfamiliarity response distribution pattern and one of the reference abnormality mode distribution patterns in the preset abnormality mode definition library reaches the preset matching acceptance condition, determine the defect semantic descriptor corresponding to the reference abnormality mode distribution pattern as the defect type descriptor corresponding to the unfamiliarity abnormal connected domain.

[0136] The similarity matching process employs a multi-scale structural similarity measurement strategy. First, the gradient direction distribution vector to be matched is compared with the gradient direction distribution vector of each reference record in the anomalous modality definition library, and the Bach distance between them is calculated. The Bach distance is calculated by multiplying the square roots of the two gradient direction distribution vectors interval by interval, summing the results, taking the natural logarithm of the sum, and then taking the negative value. A smaller Bach distance indicates a greater similarity between the two distributions.

[0137] Secondly, the geometric attributes in the peak aggregation morphology descriptor to be matched are compared with the corresponding geometric attributes in the reference record. The absolute value of the relative deviation of each attribute value is calculated, and the absolute value of the relative deviation is multiplied by the preset weight factor of the corresponding attribute and then summed to obtain the comprehensive score of geometric attribute deviation.

[0138] The weighted sum of the Bhattacharyya distance and geometric attribute deviation scores yields the total similarity metric for the reference record. A smaller total similarity metric indicates a greater similarity between the pattern to be matched and the reference pattern. The record with the smallest total similarity metric is selected from all reference records, and it is determined whether this minimum total similarity metric is less than a preset matching acceptance threshold. If it is less, the match is successful, and the defect semantic descriptor stored in the matched reference record is assigned to the current unfamiliarity anomaly connected domain as a defect type descriptor. Examples of defect semantic descriptors include "pad peeling," "bridging short circuit," "scratching damage," "pin cold solder joint," "solder mask defect," or "character printing offset." If the minimum total similarity metric is greater than or equal to the matching acceptance threshold, the defect type descriptor of the current unfamiliarity anomaly connected domain is marked as an unknown anomaly.

[0139] Step 1562: Extract the spatial coverage coordinate range of the unfamiliarity anomaly connected component in the unfamiliarity quantization map, calculate the minimum bounding rectangle vertex coordinate sequence of the spatial coverage coordinate range, and use the minimum bounding rectangle vertex coordinate sequence as the defect location descriptor corresponding to the unfamiliarity anomaly connected component.

[0140] Traverse the height and width coordinates of all grid points within the unfamiliarity anomaly connected domain, recording the minimum height coordinate Ymin, maximum height coordinate Ymax, minimum width coordinate Xmin, and maximum width coordinate Xmax. These four extreme values ​​determine the coordinates of the four vertices of the minimum bounding rectangle: the top-left vertex coordinates Xmin and Ymin, the top-right vertex coordinates Xmax and Ymin, the bottom-left vertex coordinates Xmin and Ymax, and the bottom-right vertex coordinates Xmax and Ymax. Arrange these four vertex coordinates in clockwise or counterclockwise order to form a vertex coordinate sequence. This vertex coordinate sequence is the defect location descriptor, used to identify the spatial region where the defect is located on the component surface. Simultaneously, the coordinate values ​​in the defect location descriptor can be converted into actual physical size coordinates according to the calibration parameters of the image acquisition device, for use by production line maintenance personnel or subsequent automated repair equipment for positioning.

[0141] Step 1563: Based on the abnormal connected component identifiers, defect type descriptors, and defect location descriptors corresponding to all unfamiliar abnormal connected components, generate a comprehensive defect judgment result for the electronic components to be inspected.

[0142] The comprehensive defect assessment results are output in the form of a structured data document. The root node of the document contains an array of defects, with each element corresponding to a detected unfamiliarity anomaly connected component. Each array element contains four subfields: anomaly connected component identifier, defect type descriptor, defect location descriptor, and average unfamiliarity response. The average unfamiliarity response value is the arithmetic mean of the joint unfamiliarity responses of all grid points within that connected component. In addition, the document includes summary information such as the total number of defects, the count for each defect type, and the overall quality assessment conclusion. The overall quality assessment conclusion is generated based on the defect type and quantity, combined with preset acceptable quality standard rules, such as "qualified," "pending review," or "unqualified."

[0143] After determining the defect location descriptor and defect type descriptor, this application embodiment further provides a mechanism for continuous monitoring of production line health status and dynamic updating of the anomaly knowledge base, so as to make full use of the quality inspection result data continuously generated by edge computing nodes and realize closed-loop feedback optimization of the production process.

[0144] In an optional embodiment, the method further includes steps 210-230.

[0145] Step 210: Obtain the subsequent unfamiliarity quantification map generated after the electronic component with the same design drawing definition data flow as the electronic component to be inspected passes through the edge computing node for quality inspection. Align the unfamiliarity quantification map with the subsequent unfamiliarity quantification map in the time series dimension to generate a time-series unfamiliarity evolution sequence describing the evolution of the joint unfamiliarity response value at the same spatial coordinate position over time.

[0146] The edge computing node executes steps 110 to 150 for each component continuously passing through the pipeline, generating a corresponding unfamiliar metric map. Internally, the edge computing node maintains a time-series database to store each unfamiliar metric map and its associated metadata in the order of component passage. The metadata includes timestamps, component serial numbers, production line numbers, etc.

[0147] This step retrieves several consecutive unfamiliarity metric maps with the same design drawing definition data flow identifier and sorted by passage time from the time series database, forming a map time series. For each unfamiliarity metric map in the map time series, its height and width dimensions are the same as the initial unfamiliarity metric map. Using the same height and width coordinates, the joint unfamiliarity response value NoveltyVal at coordinate positions y and x in the t-th map is extracted sequentially. t Find the NoveltyVal corresponding to all t values ​​from 1 to the total number of graphs T. tArranged in ascending order of t, they form the temporal unfamiliarity evolution sequence at spatial coordinate positions y and x. The temporal unfamiliarity evolution sequence is a one-dimensional time series data of length T.

[0148] Step 220: Perform trend mode decoupling processing on the time-series unfamiliarity evolution sequence to separate the slow drift trend component reflecting system performance degradation and the instantaneous jump trend component reflecting sudden process anomalies contained in the time-series unfamiliarity evolution sequence.

[0149] The trend mode decoupling process employs a seasonality and trend decomposition algorithm based on local weighted regression. The algorithm's inputs are the time-series unfamiliarity evolution sequence and a user-preset periodic window parameter (Period). First, the algorithm calculates the locally weighted regression smoothing value of the time-series unfamiliarity evolution sequence at each time point. For time point t, a subsequence is taken within half the time points of each preceding and following period. Each time point i in the subsequence is assigned a weight, calculated using a cubic weighting function based on the time distance between time point i and time point t; the closer the distance, the greater the weight. A local straight line is fitted using weighted least squares, and the value of this line at time point t represents the trend term at that point. t .

[0150] Subtracting the trend term from the original temporal unfamiliarity evolution sequence yields the detrended sequence. Averaging the values ​​corresponding to all period positions in the detrended sequence—for example, averaging the values ​​at positions where t equals 1, 1+Period, 1+2Period, etc.—results in the season term. t Subtracting the trend term and then the season term from the original time-series unfamiliarity evolution sequence yields the residual term. t .

[0151] The trend term, or slow drift trend component, characterizes the system performance degradation caused by slowly changing factors such as equipment wear, environmental temperature and humidity variations, and batch differences in raw materials. In the residual term, pulse components exceeding the preset statistical fluctuation range are detected using statistical process control methods. Specifically, the mean moving range of the residual term is calculated, and the mean moving range is multiplied by an unbiased constant related to the sample size to obtain the estimated standard deviation. For each time point in the residual term, if its absolute value exceeds the range of the estimated standard deviation multiplied by the control limit coefficient, this residual value is extracted as the instantaneous jump trend component, or Jump. t It represents sudden changes such as abnormal incoming batches, sudden equipment failures, and operational errors.

[0152] Step 230: Based on the weight configuration relationship between the slow drift trend component and the instantaneous jump trend component in the preset early warning logic, the slow drift trend component and the instantaneous jump trend component are aggregated to generate a production line health status descriptor pointing to the upstream process of the edge computing node on the production line side.

[0153] The pre-defined early warning logic is stored in the configuration file of the edge computing node in the form of a rule table. Each rule in the rule table defines a combination of conditions and a corresponding early warning level. The combination of conditions consists of logical judgment clauses about the slow drift trend component and the instantaneous jump trend component, connected by "AND" and "OR" logical connectors. For example, the condition combination of one rule could be "The Trend value at M consecutive time points exceeds the early warning baseline Baseline1", with a corresponding early warning level of "Attention"; the condition combination of another rule could be "The Jump value at a single time point exceeds the tolerance limit", with a corresponding early warning level of "Warning".

[0154] The situation aggregation process is executed by a rule engine. The rule engine iterates through the rules in the rule table in priority order, extracting the slow drift trend component (Trend) and the instantaneous jump trend component (Jump) corresponding to the current temporal unfamiliarity evolution sequence, and evaluating whether the condition combination of each rule is true. Once the condition combination of any rule is true, the rule engine stops iterating and returns the corresponding warning level descriptor, such as "Normal," "Attention," "Warning," or "Abnormal." The rule engine also records the specific condition combination that triggers the warning and encapsulates the warning level descriptor and the trigger condition descriptor together into a production line health situation descriptor. The production line health situation descriptor is pushed to the production line manufacturing execution system or Andon dashboard through the communication interface of the edge computing node, so that production managers can monitor the health status of upstream processes in a timely manner.

[0155] In an optional embodiment, the method further includes steps 310-330.

[0156] Step 310: Obtain the defect type descriptor corresponding to each unfamiliarity abnormal connected domain in the unfamiliarity quantification graph, and map the defect type descriptor to the corresponding defect semantic superordinate category node according to the node membership path of the defect type descriptor in the preset defect semantic hierarchy tree to perform semantic hierarchy up-dimensionality processing, and obtain the defect semantic superordinate category identifier.

[0157] The pre-defined defect semantic hierarchy tree is a tree-shaped knowledge structure. Its root node represents the most abstract defect category concept, the leaf nodes represent specific defect type descriptors, and the intermediate nodes represent higher-level category descriptors of different granularities. This hierarchy tree is pre-constructed by domain experts based on the similarity of defect causes, manifestations, and repair methods, and stored in an extensible markup language format.

[0158] This step searches for a leaf node that perfectly matches each defect type descriptor determined in step 1561 within a preset defect semantic hierarchy tree. Starting from the matched leaf node, it traces upwards along the parent node pointer for several levels, the number of levels being controlled by a preset semantic dimensionality upgrade parameter. Upon reaching the target level, the node name of the current node is read and used as the higher-level semantic category identifier for the defect. For example, the defect type descriptors "bridging short circuit" and "solder residue" may trace back to the same parent node, whose node name is "redundant defect," so the higher-level semantic category identifier for the defect is "redundant defect." This semantic hierarchy upgrade process categorizes specific defects into a broader defect family, facilitating subsequent knowledge generalization and transfer at higher abstraction levels.

[0159] Step 320: Extract the set of reference unfamiliarity distribution patterns corresponding to all leaf nodes under the superior category node of the defect semantic hierarchy tree, and configure the joint unfamiliarity response distribution pattern associated with the defect type descriptor in the unfamiliarity quantification graph as the query sample condition for the current pattern reweighted adaptation processing.

[0160] Using the defect semantic superordinate category node identified in step 310 as the root, traverse its subtree structure in the preset defect semantic hierarchy tree and collect the node names of all leaf nodes under the subtree. Based on these leaf node names, retrieve the corresponding records in the abnormal modality definition library, extract the reference abnormal modality distribution pattern in each record, and merge all the extracted reference abnormal modality distribution patterns into a reference unfamiliarity distribution pattern set. Each element in this set contains a gradient direction distribution vector and a set of peak clustering morphology descriptors.

[0161] The joint unfamiliarity response distribution pattern extracted in step 155 from the current unfamiliarity anomaly connected domain, including its gradient direction distribution vector and peak clustering morphology descriptor, is used as the query sample condition for pattern reweighting adaptation processing.

[0162] Step 330: Using the feature distribution region occupied by the query sample conditions as the adjustment reference benchmark, perform pattern reweighting adaptation processing on the reference unfamiliarity distribution pattern set. In the preset abnormal modality definition library, the prototype representation boundary corresponding to the defect semantic superior category node is extended to cover the feature distribution region occupied by the query sample conditions, so that the extended prototype representation boundary simultaneously encloses the original feature distribution region corresponding to the reference unfamiliarity distribution pattern set and the newly added feature distribution region corresponding to the query sample conditions.

[0163] Pattern reweighting adaptation is achieved by adjusting the cluster center and cluster radius of the higher-level category node in the abnormal modality definition library. First, the interval-wise average of all gradient direction distribution vectors in the reference unfamiliarity distribution pattern set is extracted to form the original cluster center vector, CenterOld. The gradient direction distribution vector in the query sample conditions is denoted as VectorQuery. The updated cluster center vector, CenterNew, is calculated as follows: multiply the original cluster center vector CenterOld by the original number of samples, add the query vector VectorQuery, and then divide by the original number of samples plus 1. The original number of samples is the number of elements in the reference unfamiliarity distribution pattern set.

[0164] Calculate the Bartholomew's distance DistQuery from the query vector VectorQuery to the updated cluster center vector CenterNew. Multiply DistQuery by the preset expansion coefficient ExpandCoeff to obtain the expansion increment. Extract the original cluster radius RadiusOld recorded in the abnormal modality definition library for this parent category node. The original cluster radius is the maximum value of the Bartholomew's distances from all reference patterns under this node to the original cluster center. Compare RadiusOld with the expansion increment, and take the larger of the two as the updated cluster radius RadiusNew.

[0165] In the abnormal modality definition library, CenterNew replaces the original cluster center vector, and RadiusNew replaces the original cluster radius, completing the expansion and update of the prototype representation boundary. The expanded prototype representation boundary covers the original feature distribution area corresponding to the reference unfamiliarity distribution pattern set and the newly added feature distribution area corresponding to the query sample conditions. This process enables the abnormal modality definition library to dynamically learn and cover newly emerging abnormal manifestations under the same superordinate category as actual production line detection data accumulates. In the subsequent similarity matching processing in step 1561, unfamiliarity abnormal connected components similar to the newly added forms can be correctly classified into the superordinate category with higher confidence, thereby improving the robustness and recall of unfamiliarity pattern matching.

[0166] In summary, this application embodiment is based on an edge computing architecture. By constructing a set of phantom appearance representations constrained by topological completeness rules, and calculating the differences between the actual appearance representation and the phantom representation at the levels of feature distribution and topological manifold, a strangeness quantification map is generated. Finally, by matching with an abnormal mode definition library, the precise defect location and classification of electronic components are achieved. This method improves the edge side's real-time discrimination capability for the design specification compliance of complex components, enabling the edge side to have production line health status awareness and knowledge base adaptive evolution capability.

[0167] Please see Figure 2The figure is a schematic diagram of the basic structure of an electronic component quality inspection and analysis server 200 provided in an embodiment of this application. The electronic component quality inspection and analysis server 200 includes: a processor 201; a storage device 202 on which a computer program 2020 is stored; and a network interface 203 for providing network communication functions. When the computer program 2020 is executed by the processor 201, the processor 201 implements any of the edge computing-based electronic component quality inspection and analysis methods described above.

[0168] Please see Figure 3 This application provides a functional block diagram of an electronic component quality inspection and analysis device, which includes:

[0169] The quality inspection information acquisition module is used to receive the design drawing definition data stream and surface continuous image stream corresponding to the electronic components to be inspected through the edge computing node deployed on the production line side;

[0170] The quality inspection information analysis module is used to call the lightweight backbone network pre-installed on the edge computing node to perform frame-by-frame shallow appearance characterization extraction processing on the continuous surface image stream, to obtain an initial appearance characterization set that records the pixel-level response distribution of the component surface, and to use a residual semantic parser based on sparse autoencoder to perform topological completeness constraint rule parsing processing on the design drawing definition data stream, to obtain an edge-side logic verification map that records the physical connection relationship constraints and geometric tolerance range of the components.

[0171] The perturbation iteration correction module is used to perform reverse perturbation iteration correction processing on the initial appearance representation set using a preset adversarial feature illusion construction strategy and the edge-side logic verification map as a guiding condition, to generate an illusion appearance representation set that is constrained by the topological completeness constraint rules in the feature space distribution.

[0172] The representation set calculation module is used to calculate the non-consistent property of the feature distribution of the initial appearance representation set and the phantom appearance representation set in a preset feature dimension space, and to calculate the topological manifold connectivity difference between the virtual geometric topology of the components represented by the phantom appearance representation set and the reference geometric topology of the components defined by the edge-side logic verification map.

[0173] An anomaly defect matching module is used to generate an unfamiliar quantitative map describing the degree of deviation of each spatial location region from the design specifications in the continuous surface image stream based on the non-consistent property of the feature distribution and the difference in connectivity of the topological manifold, and input it into a preset abnormal mode definition library for matching, so as to determine the defect location descriptor and defect type descriptor of the electronic component to be inspected.

[0174] Based on the above, a readable storage medium is provided, on which a program or instructions are stored, and when the program or instructions are executed by a processor, the steps of the above method are implemented.

[0175] Furthermore, it should be noted that this application also provides a computer program product, which may include a computer program that can be stored in a computer-readable storage medium. The processor of the electronic component quality inspection and analysis server reads the computer program from the computer-readable storage medium, and the processor can execute the computer program, causing the electronic component quality inspection and analysis server to perform the aforementioned... Figure 1 The methods described in the corresponding embodiments are already known, and therefore will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer program product embodiments related to this application, please refer to the description of the method embodiments of this application.

[0176] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

Claims

1. A quality inspection and analysis method for electronic components based on edge computing, characterized in that, The method includes: The edge computing nodes deployed on the production line receive the design drawing definition data stream and surface continuous image stream corresponding to the electronic components to be inspected. The lightweight backbone network pre-installed on the edge computing node is invoked to perform frame-by-frame shallow appearance representation extraction processing on the continuous surface image stream, resulting in an initial appearance representation set recording the pixel-level response distribution of the component surface. The topological completeness constraint rule parsing processing of the design drawing definition data stream is performed using a sparse autoencoder-based residual semantic parser, resulting in an edge-side logic verification map recording the physical connection relationship constraints and geometric tolerance range of the components. Using a preset adversarial feature illusion construction strategy and the edge-side logic verification map as a guiding condition, the initial appearance representation set is subjected to reverse perturbation iterative correction processing to generate an illusion appearance representation set that is constrained by the topological completeness constraint rules in the feature space distribution. Calculate the non-consistent property of the feature distribution of the initial appearance representation set and the phantom appearance representation set in the preset feature dimension space, and calculate the topological manifold connectivity difference between the virtual geometric topology of the components represented by the phantom appearance representation set and the reference geometric topology of the components defined by the edge-side logic verification map. Based on the non-self-consistent properties of the feature distribution and the differences in connectivity of the topological manifold, an unfamiliar quantitative map describing the degree of deviation of each spatial location region from the design specifications in the continuous surface image stream is generated and input into a preset abnormal mode definition library for matching to determine the defect location descriptor and defect type descriptor of the electronic component to be inspected.

2. The method according to claim 1, characterized in that, The process involves calling a lightweight backbone network pre-installed on the edge computing node to perform frame-by-frame shallow appearance representation extraction on the continuous surface image stream, obtaining an initial appearance representation set recording the pixel-level response distribution of the component surface. Then, a residual semantic parser based on sparse autoencoder is used to perform topological completeness constraint rule parsing on the design drawing definition data stream, resulting in an edge-side logic verification map recording the physical connection constraints and geometric tolerance ranges of the components, including: The continuous surface image stream is subjected to inter-frame redundancy information deduplication and key image frame unit filtering to obtain a set of target image sequences; Each key image frame unit in the target image sequence set is sequentially input into the shallow convolution stacking module of the lightweight backbone network. The fixed-size convolution kernel group in the shallow convolution stacking module extracts the spatial variation pattern of pixel values ​​in different local neighborhoods of the key image frame unit, and generates a primary pixel response map set. Perform cross-channel response intensity normalization processing on each primary pixel response map in the primary pixel response map set, and map the pixel response amplitude between different channels to a unified response value range to obtain a standardized primary pixel response map set, and use the standardized primary pixel response map set as the initial appearance characterization set. The design drawing definition data stream is subjected to semantic hierarchical segmentation processing to separate the component pin connection relationship description layer, wiring direction constraint description layer and pad geometry definition description layer contained in the design drawing definition data stream; The residual semantic parser based on sparse autoencoder parses the component pin connection description layer, the wiring direction constraint description layer, and the pad geometry definition description layer to generate the edge-side logic verification map.

3. The method according to claim 2, characterized in that, The step of parsing the component pin connection description layer, the wiring direction constraint description layer, and the pad geometry definition description layer using the sparse autoencoder-based residual semantic parser to generate the edge-side logic verification map includes: The component pin connection relationship description layer is input to the first parsing branch of the residual semantic parser based on sparse autoencoder. The set of topological connection path vectors between pin nodes is extracted through the sparse constraint encoding layer in the first parsing branch, and the set of topological connection path vectors is converted into a matrix description structure representing the physical connection relationship constraints of the connection on / off state between pins. The routing constraint description layer and the pad geometry definition description layer are jointly input into the second parsing branch of the residual semantic parser based on sparse autoencoder. The geometric gap margin between the ideal routing trajectory defined in the routing constraint description layer and the pad boundary defined in the pad geometry definition description layer is calculated through the residual mapping layer in the second parsing branch, and a distribution descriptor describing the range of geometric tolerance deviations describing the allowable appearance deformation range of different regions is generated. Spatial registration is performed on the matrix description structure and the distribution descriptor to establish a mapping index relationship between the position of each pin node in the matrix description structure and the corresponding spatial region in the distribution descriptor. The matrix description structure and the distribution descriptor after establishing the mapping index relationship are fused to generate the edge-side logic verification map containing the component physical connection relationship constraint field and the geometric tolerance range field.

4. The method according to any one of claims 1-3, characterized in that, The method of using a preset adversarial feature illusion construction strategy, guided by the edge-side logic verification map, to perform reverse perturbation iterative correction processing on the initial appearance representation set, generates an illusion appearance representation set that is constrained by the topological completeness constraint rules in the feature space distribution, including: The matrix description structure of the physical connection relationship constraint corresponding to the physical connection relationship constraint of the component is parsed from the edge-side logic verification map, and the matrix description structure is expanded into a set of constraint vector fields acting on a set dimension of the feature space. The constraint vector fields are used to indicate the directions that are prohibited from being exceeded and the directions that are allowed to change when modifying the feature response in the feature space. The distribution descriptor corresponding to the geometric tolerance deviation range is parsed from the edge-side logic verification map. The distribution descriptor is quantized into a perturbation amplitude upper limit mask acting on each feature response point in the initial appearance representation set. The perturbation amplitude upper limit mask specifies the maximum amount of change that each feature response point is allowed to be modified during the iterative correction process. An initial adversarial perturbation field is constructed, wherein the initial adversarial perturbation field is a random noise distribution with the same dimension and spatial size as the initial appearance representation set, and each element value in the random noise distribution is uniformly sampled within a preset small perturbation interval; The initial adversarial perturbation field and the initial appearance representation set are superimposed for the first time to generate the candidate perturbation appearance representation set for the first round of iteration. The candidate perturbation appearance representation set is input into a preset feature distribution distance metric function to calculate the geodesic distance increment between the candidate perturbation appearance representation set and the initial appearance representation set on a preset high-dimensional feature manifold. The geodesic distance increment is used to measure the degree of feature offset of the candidate perturbation appearance representation set relative to the initial appearance representation set. The candidate disturbance appearance representation set is projected and corrected according to the constraint vector field. The feature components in the candidate disturbance appearance representation set that conflict with the prohibited crossing direction indicated by the constraint vector field are compressed, and the compressed feature components are redistributed to the allowed change direction indicated by the constraint vector field to obtain the constraint-projected disturbance appearance representation set. The amplitude clipping process is performed on the constraint-projected disturbance appearance representation set according to the disturbance amplitude upper limit mask. The feature response points in the constraint-projected disturbance appearance representation set whose change exceeds the maximum change specified by the disturbance amplitude upper limit mask are forcibly truncated to the range of the maximum change, thus obtaining the amplitude-limited disturbance appearance representation set. The set of appearance representations of amplitude-limited perturbations is used as the input of the perturbation field for a new round of iterations, and the initial superposition process, feature distribution distance metric function calculation, projection correction process, and amplitude clipping process are repeatedly executed until the geodesic distance increment converges to the preset stable interval. The set of amplitude-limited perturbation appearance representations that has reached the convergence condition is defined as the phantom appearance representation set. Each feature response point in the phantom appearance representation set satisfies the connection topology invariance specified by the component physical connection relationship constraint and the appearance deformation boundary condition specified by the geometric tolerance range.

5. The method according to claim 1, characterized in that, The calculation of the non-consistent property of the feature distribution of the initial appearance representation set and the phantom appearance representation set in the preset feature dimension space includes: Obtain the initial feature vector at each spatial location in the initial appearance representation set, and obtain the phantom feature vector at the corresponding spatial location in the phantom appearance representation set, and construct a feature vector pairing sequence indexed by spatial location; For each pair of feature vectors in the feature vector pairing sequence, the cosine dissimilarity metric of the initial feature vector and the phantom feature vector in the vector space is calculated. The cosine dissimilarity metric is obtained by removing the inner product of the initial feature vector and the phantom feature vector from the normalization constant, and is used to characterize the degree of divergence between the two vectors in the direction. For each pair of feature vectors in the feature vector pairing sequence, calculate the Euclidean magnitude difference metric between the initial feature vector and the phantom feature vector in the vector space. The Euclidean magnitude difference metric is used to characterize the degree of attenuation or expansion of the response intensity of the two vectors. The cosine dissimilarity metric and the Euclidean modulus difference metric at the same spatial location are fused to generate a local feature non-consistent response value at that spatial location. All local feature non-consistent response values ​​at all spatial locations are then organized into a feature distribution non-consistent attribute map according to the original spatial arrangement order.

6. The method according to claim 1 or 5, characterized in that, The calculation of the topological manifold connectivity difference between the virtual geometric topology of the components represented by the phantom appearance representation set and the reference geometric topology of the components defined by the edge-side logic verification map includes: Extract the contour response channels that record the edge contour information of components from the set of phantom appearance representations, perform edge connectivity tracing processing on the contour response channels, and generate a phantom appearance edge topology skeleton descriptor composed of continuous edge line segments. Extract the matrix description structure recorded in the edge-side logic verification map, and convert the matrix description structure into a reference topology connection diagram composed of nodes and connecting edges. The nodes in the reference topology connection diagram correspond to the positions of component pins, and the connecting edges correspond to the proper electrical connection paths between pins. Project the phantom appearance edge topology skeleton descriptor onto the geometric reference plane where the reference topology connection diagram is located, and calculate the overlap length ratio and deviation angle difference between each edge line segment in the phantom appearance edge topology skeleton descriptor and the corresponding connection edge in the reference topology connection diagram. A topological connectivity difference metric function is constructed based on the overlap length ratio and the deviation angle difference. The output value of the topological connectivity difference metric function increases as the overlap length ratio decreases and as the deviation angle difference increases. The local topological manifold connectivity difference value between all edge segments in the topological skeleton descriptor of the phantom appearance and the corresponding connecting edges in the reference topological connection graph is calculated by traversing and calculating the topological manifold connectivity difference value between them and the corresponding connecting edges in the reference topological connection graph. The local topological manifold connectivity difference value is interpolated and extended to the entire spatial region of the entire set of phantom appearance representations according to the spatial location of its corresponding edge segment, generating a topological manifold connectivity difference distribution map covering the entire space.

7. The method according to claim 1, characterized in that, The method generates an unfamiliar quantitative map describing the degree of deviation from design specifications for each spatial location region in the continuous surface image stream based on the non-consistent properties of the feature distribution and the differences in topological manifold connectivity. This map is then input into a preset abnormal mode definition library for matching to determine the defect location descriptor and defect type descriptor of the electronic component to be inspected, including: The non-self-consistent attribute map of feature distribution and the topological manifold connectivity difference distribution map are spatially aligned and superimposed so that each grid point in the non-self-consistent attribute map of feature distribution has the same spatial coordinate reference as the corresponding grid point in the topological manifold connectivity difference distribution map. For each aligned grid point, extract the local feature non-consistent response value of the grid point in the feature distribution non-consistent attribute map, and extract the local topological manifold connectivity difference value of the grid point in the topological manifold connectivity difference distribution map. Based on the preset unfamiliarity aggregation logic, the local feature non-consistent response value and the local topological manifold connectivity difference value are jointly mapped to generate the joint unfamiliarity response value at the grid point. The joint unfamiliarity response value reflects both the degree of offset at the appearance representation level and the degree of variation at the topological structure level. Traverse all grid points covered by the continuous surface image stream, arrange the joint unfamiliarity response values ​​at each grid point according to their original spatial positions, and generate an unfamiliarity quantification map covering the complete appearance area of ​​the electronic component to be inspected. Spatial connectivity analysis is performed on the unfamiliarity quantification map to aggregate adjacent grid points in the unfamiliarity quantification map whose joint unfamiliarity response values ​​exceed a preset response threshold into unfamiliarity anomalous connectivity regions, and an anomalous connectivity region identifier is assigned to each unfamiliarity anomalous connectivity region. For each unfamiliarity-abnormal connected domain, the joint unfamiliarity response distribution pattern within the unfamiliarity-abnormal connected domain is extracted. The joint unfamiliarity response distribution pattern includes the spatial gradient change trend and peak clustering pattern of the joint unfamiliarity response values ​​within the unfamiliarity-abnormal connected domain. By combining the joint unfamiliarity response distribution pattern and the reference abnormal mode distribution pattern stored in the preset abnormal mode definition library, the defect location descriptor and defect type descriptor of the electronic component to be inspected are determined.

8. The method according to claim 7, characterized in that, The method of combining the joint unfamiliarity response distribution pattern and the reference abnormal mode distribution pattern stored in the preset abnormal mode definition library to determine the defect location descriptor and defect type descriptor of the electronic component to be inspected includes: The joint unfamiliarity response distribution pattern is matched with the reference abnormality mode distribution patterns stored in the preset abnormality mode definition library. When the similarity metric between the joint unfamiliarity response distribution pattern and one of the reference abnormality mode distribution patterns in the preset abnormality mode definition library reaches the preset matching acceptance condition, the defect semantic descriptor corresponding to the reference abnormality mode distribution pattern is determined as the defect type descriptor corresponding to the unfamiliarity abnormal connected domain. Extract the spatial coverage coordinate range of the unfamiliarity-abnormal connected component in the unfamiliarity quantification map, calculate the minimum bounding rectangle vertex coordinate sequence of the spatial coverage coordinate range, and use the minimum bounding rectangle vertex coordinate sequence as the defect location descriptor corresponding to the unfamiliarity-abnormal connected component; Based on the abnormal connected component identifiers, defect type descriptors, and defect location descriptors corresponding to all unfamiliar abnormal connected components, a comprehensive defect judgment result is generated for the electronic component to be inspected.

9. A quality inspection and analysis server for electronic components, characterized in that, include: A processor; a storage device having a computer program stored thereon; a network interface for providing network communication functions; when the computer program is executed by the processor, the processor enables the processor to implement the edge computing-based electronic component quality inspection and analysis method as described in any one of claims 1-8.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the edge computing-based electronic component quality inspection and analysis method as described in any one of claims 1-8.

Images