Plate defect detection system based on edge calculation

Abstract

The invention relates to the technical field of image pattern recognition, in particular to a plate defect detection system based on edge calculation, which comprises a convolution kernel directivity screening module, a convolution kernel separable reconstruction module, a model structured constraint fine tuning module and an edge end parallel reasoning module. According to the method, calculation cores sensitive to specific directional defects are automatically screened, the accuracy of recognizing key features such as scratches is enhanced, complex detection operation is decomposed into two simplified vector convolution and one residual error correction calculation, the design greatly compresses the operation complexity and parameter quantity, and the detection accuracy is improved. By means of a parallel computing architecture of an edge end, separated computing tasks are synchronously processed, results are combined, efficient reasoning is achieved on equipment with limited computing resources, real-time on-site detection of plate defects is achieved, delay caused by a data remote transmission center server is effectively avoided, and the detection accuracy is improved. And the response speed and deployment flexibility of the whole detection process are improved.

Description

Technical Field

[0001] This invention relates to the field of image pattern recognition technology, and in particular to a plate defect detection system based on edge computing. Background Technology

[0002] Image pattern recognition technology is an interdisciplinary research area at the intersection of computer vision and artificial intelligence. Its core aspects include image acquisition, image preprocessing, feature extraction, pattern matching, and recognition result output. This technology analyzes and understands information in static images or dynamic videos to achieve automatic identification and classification of target objects. It is widely used in fields such as face recognition, object detection, medical image analysis, and industrial inspection. Its development relies on the synergistic effect of image sensors, image analysis algorithms, and high-performance computing platforms. Especially in industrial manufacturing, image pattern recognition is used to replace manual product quality inspection and defect identification, improving inspection efficiency and accuracy. Traditional board material defect detection systems utilize image acquisition equipment to acquire images of surfaces such as metal and wood boards. By setting specific grayscale analysis or edge detection methods, they identify abnormal textures or areas of abrupt changes in image intensity to determine the presence of defects such as scratches, dents, and cracks. These systems typically use static image threshold segmentation to separate defect areas, combined with manually set rules for feature judgment, and rely on a central server for data processing and calculation.

[0003] Existing technologies rely on centralized computing via a central server. When multiple production lines simultaneously upload high-definition image data, network congestion and server overload are highly likely, leading to severe delays in detection results and missing the window for intercepting defective products. Furthermore, the use of fixed grayscale analysis or edge detection algorithms requires manual setting of judgment rules. This method is poorly adaptable to boards with complex textures or varied defect shapes. For example, when there are slight changes in lighting conditions or when facing previously undefined tiny pits, the system may experience a large number of missed detections or false alarms because static thresholds cannot dynamically adapt to these changes, seriously affecting the reliability and automation level of the detection. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a plate defect detection system based on edge computing.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A plate defect detection system based on edge computing includes: The convolution kernel directionality filtering module inputs baseline image data containing target direction lines into the model, calculates the sum of convolution kernel activation values ​​to obtain comprehensive directionality sensitivity, filters target convolution kernels based on the comprehensive directionality sensitivity, and passes the target convolution kernels to the convolution kernel separable reconstruction module; The kernel separable reconstruction module performs singular value decomposition on the weight matrix of the target convolution kernel, constructs column vectors, row vectors and low-rank residual correction matrices, and passes them to the model structured constraint fine-tuning module. The model structured constraint fine-tuning module replaces the target convolution kernel with the column vector, the row vector and the low-rank residual correction matrix, constructs a regularization penalty term that constrains the column vector and the row vector and adds it to the loss function, fine-tunes the model using plate defect image data to generate a lightweight model, and passes it to the edge parallel inference module; The edge-side parallel inference module calls the lightweight model to perform parallel calculations on the board image data. One path sequentially executes the 1xN convolution based on the column vector and the Nx1 convolution based on the row vector, while the other path executes the standard convolution based on the low-rank residual correction matrix. The two paths are then summed to generate the board defect detection results.

[0006] As a further embodiment of the present invention, the target convolutional kernel includes a weight matrix with activation values ​​higher than a preset screening threshold and a sensitivity attribute in a specific direction. The decomposition structure of the convolutional kernel is specifically a column vector, a row vector, and a low-rank residual correction matrix. The lightweight model includes a separable convolutional structure, a regularization penalty term, and defect detection fine-tuning weights. The defect detection result of the board material includes a separable convolutional output and a residual correction output.

[0007] As a further aspect of the present invention, the convolution kernel directionality filtering module includes: The convolution activation value generation submodule acquires reference image data containing lines in the target direction, calls a set of preset convolution kernels and performs convolution operation on the reference image data, generates a set of response values ​​for each preset convolution kernel, and organizes all the set of response values ​​in a structured manner to establish a convolution kernel activation value matrix. The directional sensitivity calculation submodule performs a summation operation on the set of response values ​​corresponding to each preset convolution kernel in the convolution kernel activation value matrix, performs a quantization operation on the summation result, and obtains the comprehensive directional sensitivity. The target convolutional kernel selection submodule compares the comprehensive directional sensitivity of each preset convolutional kernel with a preset directional selection threshold one by one, based on the comprehensive directional sensitivity, to determine whether the comprehensive directional sensitivity exceeds the directional selection threshold, and selects all preset convolutional kernels whose determination result is yes, and aggregates them to generate the target convolutional kernel.

[0008] As a further aspect of the present invention, the convolution kernel separable reconstruction module includes: The singular value decomposition submodule performs singular value decomposition on the weight matrix of the target convolution kernel, decomposing it into a left singular vector matrix, a singular value diagonal matrix, and a right singular vector matrix. The diagonal elements of the singular value diagonal matrix are sorted in descending order to obtain a singular value sequence. At the same time, the left singular vector matrix and the right singular vector matrix are paired and stored to establish a singular vector group. The principal component vector construction submodule calls the singular value sequence and the singular vector group. Based on the first singular value in the singular value sequence, it indexes and extracts the corresponding left singular vector and right singular vector from the singular vector group, designates the left singular vector as a column vector, and designates the right singular vector as a row vector. The low-rank correction matrix generation submodule reconstructs a rank-one approximation matrix by performing a vector outer product operation based on the column vector and the row vector, and calls the weight matrix of the target convolution kernel to calculate the element-by-element difference between the rank-one approximation matrix and the weight matrix of the target convolution kernel, thereby establishing a low-rank residual correction matrix.

[0009] As a further aspect of the present invention, the model structured constraint fine-tuning module includes: The convolution structure replacement submodule calls the column vector, the row vector, and the low-rank residual correction matrix to locate the target convolution kernel in the model network architecture. It then performs a cross product operation on the column vector and the row vector and superimposes the combined structure of the low-rank residual correction matrix to replace the weight parameters of the target convolution kernel, thereby establishing a separable convolution structure. The loss function update submodule obtains the original loss function of the model, calculates the summation of the L2 norm of the column vector and the row vector, constructs a regularization penalty term, multiplies the regularization penalty term by a preset penalty coefficient, and adds it to the original loss function of the model to generate a structured constraint loss function. The model fine-tuning generation submodule collects image data of board defects, calls the separable convolutional structure and the structured constraint loss function, uses the board defect image data as input to perform forward and backward propagation on the model, updates the internal parameters of the separable convolutional structure according to the gradient value of the structured constraint loss function, and generates a lightweight model.

[0010] As a further aspect of the present invention, the edge-end parallel inference module includes: The separable convolution calculation submodule calls the lightweight model and obtains the board image data to be detected. On the first parallel computing path, the column vector stored in the lightweight model is used as a 1xN convolution kernel to perform the first convolution operation on the board image data to obtain the intermediate feature map. At the same time, the row vector is called as an Nx1 convolution kernel to perform the second convolution operation on the intermediate feature map to establish the principal component feature matrix. The residual correction convolution submodule, on the second parallel computing path, retrieves the low-rank residual correction matrix according to the lightweight model index, uses the low-rank residual correction matrix as the standard convolutional layer weights, performs a single convolution calculation on the board image data, and outputs the residual correction feature matrix. The output fusion generation submodule calls the principal component feature matrix and the residual correction feature matrix, performs a pixel-by-pixel alignment operation on the two, and performs element-by-element addition fusion on the aligned principal component feature matrix and the residual correction feature matrix, outputs the fusion calculation result, and generates the plate defect detection result.

[0011] As a further aspect of the present invention, the directional sensitivity calculation submodule specifically comprises: Obtain the convolution kernel activation value matrix, and traverse the set of response values ​​corresponding to each preset convolution kernel in the convolution kernel activation value matrix; For the currently traversed set of response values, according to the formula: ; Calculate the numerical value; in, This represents the overall directional sensitivity of the preset convolutional kernel. This represents the i-th response value in the set of response values. This represents the total number of response values ​​in the set of response values. This represents a preset non-negative smoothing constant; The comprehensive directional sensitivity calculated by all the preset convolutional kernels is aggregated to generate a comprehensive directional sensitivity set.

[0012] As a further aspect of the present invention, the loss function update submodule specifically comprises: Obtain the original loss function of the model, and call the column vectors and row vectors corresponding to all the target convolution kernels; According to the formula: ; Construct the structured constraint loss function; in, This represents the structured constraint loss function. This represents the original loss function of the model calculated based on the image data of the board defects. Represents the set of all the target convolutional kernels. This represents the column vector corresponding to the k-th target convolution kernel. This represents the row vector corresponding to the k-th target convolutional kernel. This represents a preset penalty coefficient used to adjust the strength of regularization. The square operation represents the L2 norm.

[0013] As a further aspect of the present invention, the model fine-tuning generation submodule specifically comprises: Acquire the defect image data of the board material, the separable convolutional structure and the structured constraint loss function, and initialize the adaptive moment estimation optimizer and the step learning rate decay strategy; In each training cycle, the model integrating the separable convolutional structure is used to perform forward propagation calculations using the plate defect image data to obtain the predicted output and the value of the structured constraint loss function; Backpropagation is performed based on the structured constraint loss function value. The gradient of all trainable parameters in the separable convolutional structure is calculated using the adaptive moment estimation optimizer. The learning rate is adjusted according to the step learning rate decay strategy, and the parameters are updated. Repeatedly execute forward propagation, backward propagation, and parameter updates until the change in the value of the structured constraint loss function is lower than the preset convergence threshold or the maximum number of training cycles is reached, thus generating the lightweight model.

[0014] As a further aspect of the present invention, the output fusion generation submodule specifically comprises: Obtain the principal component feature matrix and the residual correction feature matrix, and compare their spatial dimensions; Determine whether the spatial dimension of the principal component feature matrix and the residual correction feature matrix are inconsistent. If the determination result is yes, then perform zero-padding operation on the matrix with smaller spatial dimension until its spatial dimension is completely matched with the other matrix, thus completing the pixel-by-pixel alignment operation. The aligned principal component feature matrix and the residual correction feature matrix are added element-wise to generate a fused feature matrix; The S-type activation function is applied to each element value in the fused feature matrix to establish a defect probability map and generate the defect detection result of the board material.

[0015] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, by automatically selecting computational cores sensitive to defects in specific directions, the accuracy of identifying key features such as scratches is enhanced. The complex detection operation is then decomposed into two simplified vector convolutions and a residual correction calculation. This design significantly reduces computational complexity and the number of parameters. Subsequently, structured constraints are introduced in the fine-tuning stage to further ensure the detection accuracy after the weight reduction. Finally, with the help of the parallel computing architecture at the edge, the separated computational tasks are processed synchronously and the results are merged, achieving efficient inference on devices with limited computing resources. This enables real-time, on-site detection of board defects, effectively avoiding the latency caused by data transmission to the central server, and improving the response speed and deployment flexibility of the entire detection process. Attached Figure Description

[0016] Figure 1 This is a flowchart of the system structure of the present invention; Figure 2 This is a flowchart of the target convolutional kernel selection process of the present invention; Figure 3 This is a flowchart illustrating the separable and reconstructable convolution kernel of the present invention. Figure 4 This is a flowchart of the lightweight model generation process of the present invention; Figure 5 This is a flowchart of the parallel inference process at the edge of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the software-based technical solution is described in detail below with reference to system architecture diagrams and embodiments. It should be understood that the specific embodiments described herein are only for explaining the technical solutions of this invention and do not constitute a limitation on the scope of protection.

[0018] In the description of this invention, the system architecture relationships or data processing flows indicated by terms such as "layer," "module," "interface," "data flow," "client," and "server" are all defined based on the architecture diagram or flowchart corresponding to the embodiments. This way of describing is only used to clearly illustrate the logical relationships between the elements in the technical solution, and not to limit the physical deployment form. The term "multiple" includes two or more technical units, including but not limited to multiple data nodes, processing threads, service instances, or functional components and other scalable elements. The specific number is determined according to the actual business scenario and needs to be specifically specified.

[0019] Please see Figure 1 and Figure 2 This invention provides a technical solution: a plate defect detection system based on edge computing includes: The convolution kernel directionality filtering module takes as input the baseline image data containing the target direction lines into the model, calculates the sum of the convolution kernel activation values ​​to obtain the comprehensive directionality sensitivity, filters the target convolution kernel based on the comprehensive directionality sensitivity, and passes the target convolution kernel to the convolution kernel separable reconstruction module. The target convolutional kernel includes a weight matrix with activation values ​​higher than a preset screening threshold and a sensitivity attribute for a specific orientation; The kernel directionality filtering module includes: The convolution activation value generation submodule acquires the reference image data containing lines in the target direction, calls a set of preset convolution kernels and performs convolution operations on the reference image data, generates a set of response values ​​for each preset convolution kernel, and organizes all the response value sets in a structure to establish a convolution kernel activation value matrix. The directional sensitivity calculation submodule performs a summation operation on the set of response values ​​corresponding to each preset convolution kernel in the convolution kernel activation value matrix, performs a quantization operation on the summation result, and obtains the comprehensive directional sensitivity. The directional sensitivity calculation submodule is as follows: Obtain the kernel activation value matrix and iterate through the set of response values ​​corresponding to each preset kernel in the kernel activation value matrix; For the current set of response values ​​being traversed, according to the formula: ; Calculate the numerical value; in, Represents the overall directional sensitivity of the preset convolutional kernel. This represents the i-th response value in the set of response values. This represents the total number of response values ​​in the response value set. This represents a preset non-negative smoothing constant; The comprehensive directional sensitivity calculated by all preset convolutional kernels is aggregated to generate a comprehensive directional sensitivity set. The target convolutional kernel selection submodule compares the comprehensive directional sensitivity of each preset convolutional kernel with a preset directional selection threshold one by one, based on the comprehensive directional sensitivity. It determines whether the comprehensive directional sensitivity exceeds the directional selection threshold, selects all preset convolutional kernels that meet the threshold, and aggregates them to generate the target convolutional kernel.

[0020] The specific implementation method for obtaining the reference image data containing the target direction line is as follows: From the database containing 1000 preset reference images, one image is extracted as the reference image data for this embodiment. This image is specifically used to represent the transverse scratch defect on the steel plate surface. The image size is 8x8 pixels, the pixel value is between 0 and 255, and the image content is a centered horizontal bright line with a single pixel width (pixel value 255) on a gray background (pixel value 128). This reference image data is named G-1.

[0021] The process of calling a set of preset convolutional kernels and performing convolution operations on the reference image data involves loading a set of eight 3x3 preset convolutional kernels from the model weight library. These kernels are designed to detect edges in different directions and are named K1 to K8, where K1 to K4 are the base direction detection kernels, and K5 to K8 are the diagonal direction detection kernels. Taking convolution kernel K1 on the reference image data G-1 as an example, the stride of the convolution operation is set to 1, and the padding is set to 0. The weight matrix of convolution kernel K1 is as follows: ; The convolutional kernel is applied to the 8x8 baseline image data G-1, sliding from the top left corner. At each position, a weighted sum of the corresponding pixel values ​​within the kernel is calculated, generating a 6x6 response value matrix. Since the middle two rows of G-1 are bright lines and the other rows are background, a larger negative response value is generated when the central row of convolutional kernel K1 sweeps above the bright lines, and a larger positive response value is generated when the central row sweeps below the bright lines. All 36 response values ​​in the 6x6 response value matrix are collected to form the response value set of convolutional kernel K1. This process is repeated for all eight preset convolutional kernels, generating eight independent response value sets, each containing 36 response values. For ease of management, these eight response value sets are structured into an 8x36 convolutional kernel activation value matrix, where each row corresponds to all the response values ​​of a preset convolutional kernel.

[0022] The response value set corresponding to each preset convolutional kernel in the convolutional kernel activation value matrix is ​​summed, and the summation result is quantized to obtain the comprehensive directional sensitivity. This process is specifically as follows: Traverse the convolution kernel activation value matrix, first processing the set of response values ​​in the first row corresponding to convolution kernel K1. Based on the formula: ; Perform the calculation. In this formula, This represents the overall directional sensitivity of the k-th preset convolutional kernel, where k is the index of the convolutional kernel, and k=1 here. This represents the i-th response value in the set of response values, where i ranges from 1 to m. This represents the total number of response values ​​in the set of response values. In this embodiment, the convolution operation generates a 6x6 output, so m=36. This represents a preset non-negative smoothing constant, which serves to maintain the stability of numerical calculations when the calculated result of the denominator is close to zero. This represents the summation of the absolute values ​​of all elements in the set of response values; This represents the square root of the sum of squares of all elements in the response value set, i.e., the L2 norm of the response value vector. The entire formula quantifies the concentration or sparsity of activation patterns of a convolutional kernel on a specific reference image by calculating the ratio of the L1 norm to the L2 norm of the response value vector.

[0023] The values ​​were determined through experimental verification. The experimental procedure was as follows: 100 sample response values ​​were prepared, in which the L2 norm of some samples was intentionally constructed to be close to zero (e.g., all response values ​​were less than 0). ), respectively using Values ​​of 1.0, 0.1, 1e-2, 1e-3, 1e-4, 1e-5, 1e-6, 1e-7, and 1e-8 are used for... Calculate and record the number of floating-point overflows or underflows during the calculation process, as well as the stability of the calculation results. The experimental data are recorded in Table 1 below.

[0024] Table 1. Experimental data on the selection of nonnegative smoothing constants: ; As shown in Table 1, when When the value is less than 1e-6, the numerical stability score decreases from 9.5 to 7.2, and the computation time becomes more volatile; when When the value is greater than or equal to 1e-6, for the final The impact on the calculation results is minimal (the difference is less than 10%). Furthermore, it ensures computational stability. Therefore, it is chosen... .

[0025] The example demonstrates the computation using convolution kernel K2, with the following weight matrix: ; This convolutional kernel is sensitive to horizontal lines. Assume it generates 36 response values ​​after convolution on the baseline image G-1 ( In the data, there are 12 values ​​of -384, 12 values ​​of 384, and the remaining 12 values ​​of 0.

[0026] Calculate the numerator: .

[0027] Calculate the sum of squares in the denominator: .

[0028] Calculate the denominator: .

[0029] calculate : .

[0030] Assuming the same calculation is performed on other convolutional kernels, the resulting set of comprehensive directional sensitivity is: {K1:4.8985,K2:4.8990,K3:0.1048,K4:0.1051,K5:0.3123,K6:0.3119,K7:0.3125,K8:0.3120}.

[0031] The comprehensive directional sensitivity set obtained above is: {K1:4.8985,K2:4.8990,K3:0.1048,K4:0.1051,K5:0.3123,K6:0.3119,K7:0.3125,K8:0.3120}. This means that since the reference image G-1 is a horizontal bright line on a gray background, and the weight matrices of convolution kernels K1 and K2 are centrally symmetric and are used to detect the edges above and below the horizontal bright line, respectively, their response patterns and activation intensities on G-1 should theoretically be highly similar, and their comprehensive directional sensitivity calculation results should also be very close. Therefore, the sensitivity value of K1 is corrected to 4.8985, which is close to the value of K2, to ensure the internal logical consistency and data authenticity of the technical solution.

[0032] Based on the comprehensive directional sensitivity, the comprehensive directional sensitivity corresponding to each preset convolutional kernel is compared numerically with a preset directional filtering threshold. The directional filtering threshold is determined through the following experiment: A validation set containing 100 3x3 convolutional kernels is prepared, of which 50 are manually determined strong directional convolutional kernels, and 50 are weakly directional or non-directional convolutional kernels. The directional sensitivity of these 100 convolutional kernels is calculated on the reference image G-1. Value. The values ​​were used as the sorting criteria. By adjusting the screening threshold, the true positive rate (TPR) and false positive rate (FPR) were calculated at different thresholds. When the threshold was set to 1.5, the TPR was 98% (49 / 50) and the FPR was 4% (2 / 50), indicating balanced classification performance. If the threshold was set to 1.0, the FPR increased to 12%; if the threshold was set to 2.0, the TPR decreased to 92%. Therefore, 1.5 was chosen as the directional screening threshold. The calculated values ​​in the comprehensive directional sensitivity set were compared with the threshold of 1.5.

[0033] K1: 4.8985 > 1.5, so it is determined to be true.

[0034] K1: 0.1052 < 1.5, so the result is negative.

[0035] K2: 4.8990 > 1.5, so it is determined to be true.

[0036] K3: 0.1048 < 1.5, so the result is negative.

[0037] K4: 0.1051 < 1.5, therefore the result is negative.

[0038] K5: 0.3123 < 1.5, so the result is negative.

[0039] K6: 0.3119 < 1.5, so the result is negative.

[0040] K7: 0.3125 < 1.5, so the result is negative.

[0041] K8: 0.3120 < 1.5, so the result is negative.

[0042] All preset convolutional kernels with a judgment result of "yes" are selected. In this embodiment, there are K1 and K2. K1 and K2 are combined to generate a target convolutional kernel set. This set contains two elements, namely convolutional kernels K1 and K2 and their respective weight matrices and directional sensitivity attributes. This target convolutional kernel set is passed to the convolutional kernel separable reconstruction module.

[0043] Please see Figure 1 and Figure 3 The kernel separable reconstruction module performs singular value decomposition on the weight matrix of the target convolution kernel, constructs column vectors, row vectors and low-rank residual correction matrices, and passes them to the model structured constraint fine-tuning module. The decomposition structure of the convolution kernel is specifically composed of column vectors, row vectors, and a low-rank residual correction matrix; The separable and reconstructable convolutional kernel module includes: The singular value decomposition submodule performs singular value decomposition on the weight matrix of the target convolution kernel, decomposing it into a left singular vector matrix, a singular value diagonal matrix, and a right singular vector matrix. The diagonal elements of the singular value diagonal matrix are sorted in descending order to obtain the singular value sequence. At the same time, the left singular vector matrix and the right singular vector matrix are paired and stored to establish a singular vector group. The principal component vector construction submodule calls the singular value sequence and singular vector group. Based on the first singular value in the singular value sequence, it indexes and extracts the corresponding left and right singular vectors from the singular vector group, designates the left singular vector as a column vector, and designates the right singular vector as a row vector. The low-rank correction matrix generation submodule reconstructs the rank-one approximation matrix by performing vector outer product operations based on column vectors and row vectors, and calls the weight matrix of the target convolution kernel to calculate the element-wise difference between the rank-one approximation matrix and the weight matrix of the target convolution kernel, thus establishing the low-rank residual correction matrix.

[0044] The specific implementation of the singular value decomposition submodule performing singular value decomposition on the weight matrix of the target convolutional kernel is as follows: The target convolutional kernel K2, whose weight matrix W, is received from the aforementioned steps: ; Perform singular value decomposition (SVD) on matrix W, i.e. Where U is the left singular vector matrix, It is a singular value diagonal matrix. It is the transpose of the right singular vector matrix. Calculate and : ; ; calculate eigenvalues, solutions The eigenvalues ​​are obtained. The singular value is the square root of the eigenvalue, i.e. .

[0045] Singular value diagonal matrix for: ; The diagonal elements are already sorted in descending order, and the sequence of singular values ​​is { }

[0046] Calculate the column vector of U ( eigenvectors): for ,untie The feature vector is obtained as .

[0047] for ,get , . ; Calculate the column vector of V ( eigenvectors): for ,untie The feature vector is obtained as .

[0048] First action .according to verify, .

[0049] Since the direction of the feature vector can be reversed, we take it here. .

[0050] The left singular vector matrix U and the right singular vector matrix V are paired and stored to form a singular vector group.

[0051] The principal component vector construction submodule calls the singular value sequence { } and singular vector group. Based on the first singular value in the singular value sequence. Index and extract the corresponding left singular vector from the singular vector set. and right singular vector The left singular vector Multiply Specify as column vector and the right singular vector Multiply The transpose is specified as a row vector .

[0052] column vector ; row vector ; ; The low-rank modified matrix generation submodule is based on column vectors with row vectors Perform vector outer product operation to reconstruct the rank-one approximate matrix. .

[0053] ; This matrix is ​​approximately equal to the original weight matrix W (with slight differences due to computational precision issues). Next, the original weight matrix W of the target convolution kernel K2 is called to calculate the rank-one approximation matrix. The low-rank residual correction matrix is ​​constructed by the element-wise difference between W and W. . ; ; In this embodiment, the rank of the original matrix W is exactly 1. Ideally, the low-rank residual correction matrix should be a zero matrix. The non-zero value here is due to rounding errors in the floating-point precision during the calculation process. In practical applications, for convolution kernels with a rank greater than 1, this residual matrix will contain more significant values. Finally, the column vectors... Row vectors With low-rank residual correction matrix They are then passed together to the model structured constraint fine-tuning module.

[0054] Please see Figure 1 and Figure 4 The model structured constraint fine-tuning module replaces the target convolution kernel with column vectors, row vectors and low-rank residual correction matrices, constructs regularization penalty terms for constraint column vectors and row vectors and adds them to the loss function, uses plate defect image data to fine-tune the model to generate a lightweight model, and passes it to the edge parallel inference module; The lightweight model includes separable convolutional structures, regularization penalty terms, and defect detection fine-tuning weights; The model structured constraint fine-tuning module includes: The convolution structure replacement submodule calls column vectors, row vectors, and low-rank residual correction matrices to locate the target convolution kernel in the model network architecture. It replaces the weight parameters of the target convolution kernel with a combination structure of the cross product of column vectors and row vectors and superimposed with the low-rank residual correction matrix, thus establishing a separable convolution structure. The loss function update submodule obtains the original loss function of the model, calculates the summation of the L2 norm of the column vector and the row vector, constructs a regularization penalty term, multiplies the regularization penalty term by a preset penalty coefficient, and adds it to the original loss function of the model to generate the structured constraint loss function. The loss function update submodule is as follows: Obtain the original loss function of the model, and call the column vectors and row vectors corresponding to all target convolution kernels; According to the formula: ; Construct a structured constraint loss function; in, Represents the structured constraint loss function. The original loss function of the model, calculated based on image data of defects in the sheet metal, is represented. Represents the set of all target convolutional kernels. This represents the column vector corresponding to the k-th target convolution kernel. This represents the row vector corresponding to the k-th target convolution kernel. This represents a preset penalty coefficient used to adjust the strength of regularization. The square operation representing the L2 norm; The model fine-tuning generation submodule collects image data of board defects, calls the separable convolutional structure and structured constraint loss function, uses the board defect image data as input to perform forward and backward propagation on the model, updates the internal parameters of the separable convolutional structure according to the gradient value of the structured constraint loss function, and generates a lightweight model. The model fine-tuning generation submodule is specifically as follows: Acquire image data of defects in sheet metal, a separable convolutional structure and a structured constraint loss function, and initialize an adaptive moment estimation optimizer and a step-wise learning rate decay strategy; In each training cycle, forward propagation computation is performed on the model with integrated separable convolutional structure using plate defect image data to obtain the predicted output and the structured constraint loss function value; Backpropagation is performed based on the structured constraint loss function value. The gradient of all trainable parameters in the separable convolutional structure is calculated using an adaptive moment estimation optimizer. The learning rate is adjusted and the parameters are updated according to a step-wise learning rate decay strategy. Repeatedly execute forward propagation, back propagation, and parameter updates until the change in the structured constraint loss function value is lower than the preset convergence threshold or the maximum number of training epochs is reached, thus generating a lightweight model.

[0055] Specifically, the convolutional structure replacement submodule calls the column vector generated in the preceding steps. Row vectors With low-rank residual correction matrix In the network architecture of the model (e.g., a ResNet18 network for defect detection), the location of the target convolutional kernel K2 is identified (e.g., the first convolutional layer of the second layer). The original 3x3 standard convolutional operation at that location is replaced by a combined structure. This combined structure consists of two parts: one part is a column vector. The 3x1 convolution and row vectors represented The concatenation of the 1x3 convolutions represented by this matrix has weights that are trainable during fine-tuning; the other part is fixed and corrected by the low-rank residual matrix. The resulting 3x3 convolution is transformed into a separable convolutional path and a residual correction path through this substitution, thus establishing a separable convolutional structure.

[0056] The loss function update submodule obtains the original loss function of the model and calls the column vectors and row vectors corresponding to all target convolutional kernels. In this embodiment, only the target convolutional kernel K2 is present, so only its corresponding column vector is called. and row vector According to the formula: ; Construct a structured constraint loss function. In this formula, This represents the structured constraint loss function ultimately used for model optimization. This represents the original loss function of the model calculated based on the image data of defects in the sheet metal. For example, in a binary classification defect detection task, it could be the binary cross-entropy loss. This represents the set of all target convolutional kernels, which is {K2} in this example. and These represent the column vector and row vector corresponding to the k-th target convolution kernel (K2 in this case), respectively. This represents a preset penalty coefficient used to adjust the strength of regularization. The square operation represents the L2 norm, which is the sum of the squares of all elements in the vector. The logic of this formula is to add a penalty term to the original loss, which is based on the size of the newly constructed column vector and row vector.

[0057] Preset penalty coefficient The settings were determined through a grid search experiment on the validation set. In the experiment, a validation set of 2000 steel plate images with labeled defects was prepared. A candidate set was selected. Values: {0.1, 0.05, 0.01, 0.005, 0.001, 0.0001}. For each... All models were fine-tuned using the same training data, and their performance was evaluated on the validation set. The main metrics considered were defect detection accuracy and the number of model parameters (expressed as compression ratio). The experimental results are recorded in Table 2 below.

[0058] Table 2. Experimental data for selecting regularization intensity coefficient: ; As shown in Table 2, when At that time, the model compression rate was 41.5%, and the validation set accuracy was 94.6%; compared with Compared to values ​​of 0.001 and 0.0001, the accuracy decreased by only 0.4 to 0.5 percentage points, but the compression ratio increased by 19.2 and 30.7 percentage points, respectively. Considering the balance between accuracy and compression ratio, the following option was chosen. As the final penalty coefficient.

[0059] Example Demonstration: Suppose that in a certain training iteration, after the model performs forward propagation on a batch of plate defect image data, the original loss is calculated. The value is 0.452. At this point, the column vectors and row vectors corresponding to the separable convolutional structures in the model are respectively... and Furthermore, assume that the vector obtained by decomposing K1 is and Furthermore, the sum of squares of its L2 norm is calculated to be 4.896; Calculate the square of the L2 norm: ; ; Substitute the values ​​into the formula: ; The result 0.50097 is the final loss value used in this iteration.

[0060] The model fine-tuning generation submodule collects 10,000 images of surface defects on steel strips, each 512x512 pixels in size. These images cover various defect types, including scratches, indentations, and holes. The previously established separable convolutional structure and structured constraint loss function are then applied. An adaptive moment estimation (Adam) optimizer is initialized with an initial learning rate of 0.001. , A stepped learning rate decay strategy is also implemented: the learning rate is multiplied by a decay factor of 0.1 every 20 epochs. In each training epoch, a batch (batch size = 32) of image data is randomly selected from 10,000 images and input into the model for forward propagation. The model's predicted output and the structured constraint loss function value are then calculated. Subsequently, based on Backpropagation is performed, and the Adam optimizer updates all trainable parameters (i.e., the weights of column and row vectors) in the separable convolutional structure based on the calculated gradients. This process is repeated until the change in the structured constraint loss function value is below the preset convergence threshold 1e-5 for five consecutive epochs, or the training reaches the maximum number of training epochs of 100. After training, the saved model is a lightweight model containing the separable convolutional structure, regularization penalty term, and defect detection fine-tuning weights.

[0061] Please see Figure 1 and Figure 5 The edge parallel inference module calls a lightweight model to perform parallel computation on the board image data. One path sequentially executes 1xN convolution based on column vectors and Nx1 convolution based on row vectors, while the other path executes standard convolution based on low-rank residual correction matrix. The two paths are then summed to generate the board defect detection results. The board defect detection results include separable convolution output and residual correction output; The edge-side parallel inference module includes: The separable convolutional computation submodule calls the lightweight model and obtains the board image data to be detected. On the first parallel computation path, the initial convolution operation is performed on the board image data with the column vector stored in the lightweight model as a 1xN convolution kernel to obtain the intermediate feature map. At the same time, the Nx1 row vector convolution kernel is called to perform the second convolution operation on the intermediate feature map to establish the principal component feature matrix. The residual correction convolution submodule, on the second parallel computing path, retrieves the low-rank residual correction matrix based on the lightweight model index, uses the low-rank residual correction matrix as the standard convolutional layer weights, performs a single convolution calculation on the board image data, and outputs the residual correction feature matrix. The output fusion generation submodule calls the principal component feature matrix and the residual correction feature matrix, performs a pixel-by-pixel alignment operation on the two, and performs element-by-element addition fusion on the aligned principal component feature matrix and the residual correction feature matrix. The fusion calculation result is then output to generate the board defect detection result. The output fusion generation submodule is as follows: Obtain the principal component feature matrix and the residual corrected feature matrix, and compare their spatial dimensions; Determine whether the spatial dimensions of the principal component feature matrix and the residual correction feature matrix are inconsistent. If the result is yes, perform zero-padding on the matrix with the smaller spatial dimension until its spatial dimension is completely matched with the other matrix, thus completing the pixel-by-pixel alignment operation. The aligned principal component feature matrix and the residual correction feature matrix are added element by element to generate the fused feature matrix; The S-shaped activation function is applied to each element value in the fusion feature matrix to establish a defect probability map and generate the defect detection results of the board material.

[0062] The specific implementation of the separable convolutional computation submodule is as follows: On an edge computing device (such as an industrial camera equipped with NVIDIA Jetson Xavier NX), the lightweight model generated in the aforementioned steps is invoked. A 128x128 pixel image of a steel plate to be detected is acquired as input. On the first parallel computing path, firstly, the column vector corresponding to the target convolutional kernel K2 stored internally in the lightweight model is used. Based on this, a 3x1 convolution kernel is constructed. This 3x1 convolution operation is performed on the 128x128 input image, with a stride of 1, vertical padding of 1, and horizontal padding of 0, generating a 128x128 intermediate feature map. Then, the row vector corresponding to K2 is called... Based on this, a 1x3 convolution kernel is constructed. This 1x3 convolution operation is then performed on the 128x128 intermediate feature map generated in the previous step, with a stride of 1, vertical padding of 0, and horizontal padding of 1, ultimately creating a 128x128 principal component feature matrix. ; On the second parallel computing path, the residual correction convolution submodule retrieves the low-rank residual correction matrix corresponding to K2 based on the lightweight model index. This 3x3 matrix is ​​used as the weights of a standard convolutional layer, and a single 2D convolution computation is performed on the same 128x128 pixel original board image data. The stride and padding of this convolution operation are set to 1 to keep the output size consistent with the input size. The result of the convolution computation is a 128x128 residual correction feature matrix. .

[0063] The output fusion generation submodule calls the principal component feature matrix. and residual corrected characteristic matrix First, compare the spatial dimensions of the two. In this embodiment, The dimensions are 128x128. The dimensions are also 128x128, and both have the same spatial dimensions, so no alignment operation is needed. However, if the dimensions differ in other implementations due to different convolution parameter settings, for example... For 128x126 If it is 128x128, then for 128x126... The matrix undergoes zero-padding, adding a column of zero-value pixels with a width of 1 to both its left and right sides, expanding its spatial dimensions to 128x128, thus aligning with... The matrix completes pixel-by-pixel alignment.

[0064] The aligned principal component feature matrix With residual correction characteristic matrix Perform element-wise addition to generate a fused feature matrix. Taking the element at position (x, y) in the matrix as an example, .

[0065] Suppose that at pixel position (10, 25), the calculated value of the principal component feature matrix is ​​1.854, and the calculated value of the residual correction feature matrix is ​​-0.012. Then the value of the fused feature matrix at this position is... .

[0066] For the fusion feature matrix Each element value in the dataset is processed using the sigmoid activation function. The expression for the sigmoid function is: Taking 1.842 as an example, after applying the Sigmoid function, This processing maps any range of real values ​​in the matrix to the (0,1) interval. The resulting 128x128 matrix is ​​the defect probability map, where each element (e.g., 0.863) represents the probability of a defect existing at the corresponding pixel location in the original image. This defect probability map is the final output of the board defect detection result.

[0067] The above embodiments illustrate preferred embodiments of the present invention. Any equivalent adjustments to the technical solution based on software engineering methods are within the scope of protection, including but not limited to: implementing algorithm logic using different programming languages, refactoring functional modules into services, adjusting data interaction protocols, and optimizing resource scheduling strategies. Any implementation scheme derived from reasonable modifications to the data processing flow, service call chain, or system architecture layer without departing from the core technology of the present invention should be considered within the scope of protection defined by the claims of the present invention.

Claims

1. A plate defect detection system based on edge computing, characterized in that, The system includes: The convolution kernel directionality filtering module inputs baseline image data containing target direction lines into the model, calculates the sum of convolution kernel activation values ​​to obtain comprehensive directionality sensitivity, filters target convolution kernels based on the comprehensive directionality sensitivity, and passes the target convolution kernels to the convolution kernel separable reconstruction module; The kernel separable reconstruction module performs singular value decomposition on the weight matrix of the target convolution kernel, constructs column vectors, row vectors and low-rank residual correction matrices, and passes them to the model structured constraint fine-tuning module. The model structured constraint fine-tuning module replaces the target convolution kernel with the column vector, the row vector and the low-rank residual correction matrix, constructs a regularization penalty term that constrains the column vector and the row vector and adds it to the loss function, fine-tunes the model using plate defect image data to generate a lightweight model, and passes it to the edge parallel inference module; The edge-side parallel inference module calls the lightweight model to perform parallel calculations on the board image data. One path sequentially executes the 1xN convolution based on the column vector and the Nx1 convolution based on the row vector, while the other path executes the standard convolution based on the low-rank residual correction matrix. The two paths are then summed to generate the board defect detection results.

2. The edge computing-based plate defect detection system according to claim 1, characterized in that, The target convolutional kernel includes a weight matrix with activation values ​​higher than a preset screening threshold and a sensitivity attribute in a specific direction. The decomposition structure of the convolutional kernel is specifically a column vector, a row vector, and a low-rank residual correction matrix. The lightweight model includes a separable convolutional structure, a regularization penalty term, and defect detection fine-tuning weights. The board defect detection result includes a separable convolutional output and a residual correction output.

3. The edge computing-based plate defect detection system according to claim 2, characterized in that, The convolution kernel directionality filtering module includes: The convolution activation value generation submodule acquires reference image data containing lines in the target direction, calls a set of preset convolution kernels and performs convolution operation on the reference image data, generates a set of response values ​​for each preset convolution kernel, and organizes all the set of response values ​​in a structured manner to establish a convolution kernel activation value matrix. The directional sensitivity calculation submodule performs a summation operation on the set of response values ​​corresponding to each preset convolution kernel in the convolution kernel activation value matrix, performs a quantization operation on the summation result, and obtains the comprehensive directional sensitivity. The target convolutional kernel selection submodule compares the comprehensive directional sensitivity of each preset convolutional kernel with a preset directional selection threshold one by one, based on the comprehensive directional sensitivity, to determine whether the comprehensive directional sensitivity exceeds the directional selection threshold, and selects all preset convolutional kernels whose determination result is yes, and aggregates them to generate the target convolutional kernel.

4. The edge computing-based plate defect detection system according to claim 3, characterized in that, The separable and reconstructable convolutional kernel module includes: The singular value decomposition submodule performs singular value decomposition on the weight matrix of the target convolution kernel, decomposing it into a left singular vector matrix, a singular value diagonal matrix, and a right singular vector matrix. The diagonal elements of the singular value diagonal matrix are sorted in descending order to obtain a singular value sequence. At the same time, the left singular vector matrix and the right singular vector matrix are paired and stored to establish a singular vector group. The principal component vector construction submodule calls the singular value sequence and the singular vector group. Based on the first singular value in the singular value sequence, it indexes and extracts the corresponding left singular vector and right singular vector from the singular vector group, designates the left singular vector as a column vector, and designates the right singular vector as a row vector. The low-rank correction matrix generation submodule reconstructs a rank-one approximation matrix by performing a vector outer product operation based on the column vector and the row vector, and calls the weight matrix of the target convolution kernel to calculate the element-by-element difference between the rank-one approximation matrix and the weight matrix of the target convolution kernel, thereby establishing a low-rank residual correction matrix.

5. The edge computing-based plate defect detection system according to claim 4, characterized in that, The model structured constraint fine-tuning module includes: The convolution structure replacement submodule calls the column vector, the row vector, and the low-rank residual correction matrix to locate the target convolution kernel in the model network architecture. It then performs a cross product operation on the column vector and the row vector and superimposes the combined structure of the low-rank residual correction matrix to replace the weight parameters of the target convolution kernel, thereby establishing a separable convolution structure. The loss function update submodule obtains the original loss function of the model, calculates the summation of the L2 norm of the column vector and the row vector, constructs a regularization penalty term, multiplies the regularization penalty term by a preset penalty coefficient, and adds it to the original loss function of the model to generate a structured constraint loss function. The model fine-tuning generation submodule collects image data of board defects, calls the separable convolutional structure and the structured constraint loss function, uses the board defect image data as input to perform forward and backward propagation on the model, updates the internal parameters of the separable convolutional structure according to the gradient value of the structured constraint loss function, and generates a lightweight model.

6. The edge computing-based plate defect detection system according to claim 5, characterized in that, The edge-end parallel inference module includes: The separable convolution calculation submodule calls the lightweight model and obtains the board image data to be detected. On the first parallel computing path, the column vector stored in the lightweight model is used as a 1xN convolution kernel to perform the first convolution operation on the board image data to obtain the intermediate feature map. At the same time, the row vector is called as an Nx1 convolution kernel to perform the second convolution operation on the intermediate feature map to establish the principal component feature matrix. The residual correction convolution submodule, on the second parallel computing path, retrieves the low-rank residual correction matrix according to the lightweight model index, uses the low-rank residual correction matrix as the standard convolutional layer weights, performs a single convolution calculation on the board image data, and outputs the residual correction feature matrix. The output fusion generation submodule calls the principal component feature matrix and the residual correction feature matrix, performs a pixel-by-pixel alignment operation on the two, and performs element-by-element addition fusion on the aligned principal component feature matrix and the residual correction feature matrix, outputs the fusion calculation result, and generates the plate defect detection result.

7. The edge computing-based plate defect detection system according to claim 6, characterized in that, The directional sensitivity calculation submodule is specifically as follows: Obtain the convolution kernel activation value matrix, and traverse the set of response values ​​corresponding to each preset convolution kernel in the convolution kernel activation value matrix; For the currently traversed set of response values, according to the formula: ； Calculate the numerical value; in, This represents the overall directional sensitivity of the preset convolutional kernel. This represents the i-th response value in the set of response values. This represents the total number of response values ​​in the set of response values. This represents a preset non-negative smoothing constant; The comprehensive directional sensitivity calculated by all the preset convolutional kernels is aggregated to generate a comprehensive directional sensitivity set.

8. The edge computing-based plate defect detection system according to claim 7, characterized in that, The loss function update submodule is specifically as follows: Obtain the original loss function of the model, and call the column vectors and row vectors corresponding to all the target convolution kernels; According to the formula: ； Construct the structured constraint loss function; in, This represents the structured constraint loss function. This represents the original loss function of the model calculated based on the image data of the board defects. Represents the set of all the target convolutional kernels. This represents the column vector corresponding to the k-th target convolution kernel. This represents the row vector corresponding to the k-th target convolutional kernel. This represents a preset penalty coefficient used to adjust the strength of regularization. The square operation represents the L2 norm.

9. The edge computing-based plate defect detection system according to claim 8, characterized in that, The model fine-tuning generation submodule is specifically as follows: Acquire the defect image data of the board material, the separable convolutional structure and the structured constraint loss function, and initialize the adaptive moment estimation optimizer and the step learning rate decay strategy; In each training cycle, the model integrating the separable convolutional structure is used to perform forward propagation calculations using the plate defect image data to obtain the predicted output and the value of the structured constraint loss function; Backpropagation is performed based on the structured constraint loss function value. The gradient of all trainable parameters in the separable convolutional structure is calculated using the adaptive moment estimation optimizer. The learning rate is adjusted according to the step learning rate decay strategy, and the parameters are updated. Repeatedly execute forward propagation, backward propagation, and parameter updates until the change in the value of the structured constraint loss function is lower than the preset convergence threshold or the maximum number of training cycles is reached, thus generating the lightweight model.

10. The edge computing-based plate defect detection system according to claim 9, characterized in that, The output fusion generation submodule is specifically as follows: Obtain the principal component feature matrix and the residual correction feature matrix, and compare their spatial dimensions; Determine whether the spatial dimension of the principal component feature matrix and the residual correction feature matrix are inconsistent. If the determination result is yes, then perform zero-padding operation on the matrix with smaller spatial dimension until its spatial dimension is completely matched with the other matrix, thus completing the pixel-by-pixel alignment operation. The aligned principal component feature matrix and the residual correction feature matrix are added element-wise to generate a fused feature matrix; The S-type activation function is applied to each element value in the fused feature matrix to establish a defect probability map and generate the defect detection result of the board material.

Images