Image recognition-based pcba defect positioning method and system

Abstract

The application relates to the technical field of industrial machine vision detection, and discloses a PCBA defect positioning method and system based on image recognition. The method obtains a circuit board image to be detected and carries out multi-level feature extraction to obtain a deep feature map; a response matrix is constructed according to pixel distribution values of each feature channel, and an abnormal response score is calculated to obtain a channel contribution score sequence; the channel contribution score sequence is compared with a preset score threshold value, and channels with insufficient contribution are removed to obtain a target feature channel set; feature reconstruction is carried out on the target feature channel set to obtain a reconstructed feature map, and a spatial difference matrix of the deep feature map and the reconstructed feature map is calculated; a continuous pixel connected region greater than a preset difference threshold value is extracted to obtain initial position information containing a micro defect, and reverse mapping is carried out in combination with a network step ratio and a receptive field size to obtain a spatial positioning coordinate. The application effectively suppresses irrelevant channel interference and improves the positioning precision of a PCBA micro defect.

Description

Technical Field

[0001] This invention relates to the field of industrial machine vision inspection technology, specifically to a PCBA defect localization method and system based on image recognition. Background Technology

[0002] Printed circuit board (PCB) assemblies are the core components of various electronic devices, densely covered with tiny solder joints, pins, and traces. Even tiny solder ball sputtering or trace cracks can cause short circuits or malfunctions in the entire device. In the mass production of consumer electronics and automotive electronics, using industrial machine vision inspection to perform image recognition-based defect detection on the PCB surface before shipment has become a crucial step in ensuring product reliability.

[0003] Currently, unsupervised identification methods are commonly used to detect defects on circuit boards. These methods extract deep features through convolutional neural networks to identify microscopic anomalies such as solder balls and cracks. However, as convolutional layers are stacked, the spatial size of the feature map is compressed step by step, and the location information is diluted, resulting in a blurred and coarse response area. At the same time, the response strength of each channel in the deep features to defects is not the same, making it difficult to determine which channels are truly related to the anomaly. The diluted spatial response is prone to deviating from the true defect in complex backgrounds, causing the positioning box to fall on a normal solder joint next to it.

[0004] Therefore, existing technologies cannot restore the spatial location of defects while preserving deep semantic information, nor can they measure the contribution of each channel to the abnormal response, resulting in insufficient accuracy in locating minor defects. Summary of the Invention

[0005] This invention provides a PCBA defect localization method and system based on image recognition, which solves the problem in the prior art that it is impossible to restore the spatial location of defects while retaining deep semantic information and to measure the contribution of each channel to the abnormal response, resulting in insufficient localization accuracy of small defects.

[0006] Firstly, to address the aforementioned problems, this invention provides a PCBA defect localization method based on image recognition, comprising: Obtain an image of the circuit board to be inspected, and perform multi-level feature extraction on the circuit board image to obtain a deep feature map; Extract the pixel distribution values ​​of the deep feature map in the spatial dimension, construct the response matrix corresponding to each feature channel based on the pixel distribution values, calculate the abnormal response score of each feature channel based on the response matrix, and obtain the channel contribution score sequence. The values ​​in the channel contribution score sequence are compared with a preset score threshold, and feature channels corresponding to values ​​not greater than the preset score threshold are removed to obtain the target feature channel set. The target feature channel set is reconstructed to obtain a reconstructed feature map with the same dimension as the deep feature map. The pixel difference between the deep feature map and the reconstructed feature map at corresponding spatial locations is calculated to obtain a spatial difference matrix. Extract the connected regions of consecutive pixels in the spatial difference matrix whose values ​​are greater than a preset difference threshold to obtain initial location information containing minute defects. Based on the initial location information, perform reverse mapping on the deep feature map to obtain the spatial positioning coordinates after location restoration.

[0007] Furthermore, the step of performing multi-level feature extraction on the circuit board image to obtain a deep feature map includes: The circuit board image is channel merged and convolutional to obtain a shallow edge feature map; Pooling is performed on the shallow edge feature map to obtain a dimension-reduced feature map, and nonlinear mapping is performed on the dimension-reduced feature map to obtain a high-dimensional abstract feature map; Semantic information fusion is performed on the high-dimensional abstract feature map to obtain a deep feature map containing semantic information.

[0008] Further, the step of extracting the pixel distribution values ​​of the deep feature map in the spatial dimension and constructing the response matrix corresponding to each feature channel based on the pixel distribution values ​​includes: The deep feature map is separated into multiple independent feature channels, and pixel distribution values ​​are extracted based on the pixel coordinates of the independent feature channels. The pixel distribution values ​​are weighted and suppressed, and the response intensity is calculated on the weighted and suppressed pixel distribution values ​​to obtain a grayscale mapping sequence. Matrix construction is performed based on the grayscale mapping sequence to obtain response matrices corresponding to multiple feature channels.

[0009] Further, the step of calculating the abnormal response score of each feature channel based on the response matrix to obtain the channel contribution score sequence includes: The discreteness of the matrix elements of the response matrix is ​​calculated to obtain the spatial distribution feature sequence; The absolute deviation is calculated based on the spatial distribution characteristic sequence to obtain the abnormal response score; The abnormal response scores are weighted and suppressed according to a preset abnormality judgment threshold, and the weighted and suppressed abnormal response scores are linearly normalized to obtain a score distribution sequence. The channel contribution score sequence is obtained by calculating the weight allocation based on the score distribution sequence.

[0010] Further, the step of comparing the values ​​in the channel contribution score sequence with a preset score threshold and removing feature channels corresponding to values ​​not greater than the preset score threshold to obtain a target feature channel set includes: When the value in the channel contribution score sequence is greater than the preset score threshold, a retention flag is added to the corresponding feature channel. Remove feature channels corresponding to values ​​not greater than the preset score threshold, and obtain the target feature channel set based on the feature channels with the retention identifier.

[0011] Further, the step of reconstructing the target feature channel set to obtain a reconstructed feature map with the same dimension as the deep feature map includes: The target feature channel set is dimensionality reduced and mapped to obtain the hidden layer feature vector; The target reconstruction dimension is determined based on the hidden layer feature vector; The hidden layer feature vectors are up-dimensioned and restored according to the target reconstruction dimension to obtain the initial pixel distribution matrix; When the dimension of the initial pixel distribution matrix does not match the dimension of the target reconstruction, the initial pixel distribution matrix is ​​interpolated and padded to obtain a reconstructed feature map with the same dimension as the deep feature map.

[0012] Further, the step of calculating the pixel difference between the deep feature map and the reconstructed feature map at corresponding spatial locations to obtain a spatial difference matrix includes: Obtain the alignment coordinates between the deep feature map and the reconstructed feature map, and calculate the absolute deviation based on the alignment coordinates; The absolute deviation is mapped to grayscale to obtain the pixel difference value at the corresponding spatial location; When the pixel difference is greater than a preset intensity threshold, the response intensity corresponding to the pixel difference is extracted to obtain a spatial difference matrix containing abnormal response intensity.

[0013] Further, the step of extracting continuous pixel connected regions with values ​​greater than a preset difference threshold from the spatial difference matrix to obtain initial location information containing minute defects, and then performing reverse mapping on the deep feature map based on the initial location information to obtain the spatial positioning coordinates after location restoration, includes: Based on the spatial difference matrix, target pixels with values ​​greater than a preset difference threshold are extracted, and pixel clustering is performed according to a preset neighborhood range to obtain continuous pixel connected regions. When the area of ​​the connected region is greater than a preset area threshold, the connected region is determined as a candidate defect region, and the geometric center coordinates of the candidate defect region are calculated. Global coordinate mapping is performed based on the area of ​​the connected region and the coordinates of the geometric center to obtain the initial location information containing the minor defects; Based on the initial position information, the pixel range corresponding to the deep feature map is extracted, and the scaling factor and boundary compensation value corresponding to the pixel range are calculated by combining the network stride ratio and the receptive field size. The pixel range is reverse-mapped using the scaling factor and the boundary compensation value, and the extreme coordinates after mapping are extracted to obtain the spatial positioning coordinates after position restoration.

[0014] Secondly, the present invention provides a PCBA defect localization system based on image recognition, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the PCBA defect localization method based on image recognition as described above.

[0015] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the image recognition-based PCBA defect localization method as described above.

[0016] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention extracts the initial position information containing minute defects by extracting the connected regions of continuous pixels, and then performs reverse mapping on the initial position information by combining the network stride ratio, receptive field size and boundary compensation value to obtain the spatial positioning coordinates in the original circuit board image coordinate system. This scheme accurately restores the defect position in the deep feature map coordinate system to the original image coordinate system, compensates for the dilution of position information caused by convolution downsampling, and improves the positioning accuracy of defects such as tiny solder balls, cracks and cold solder joints.

[0017] (2) This invention obtains a deep feature map by performing multi-level feature extraction on the image of the circuit board to be inspected, and then performs channel separation on the deep feature map, constructs the response matrix of each feature channel, calculates the abnormal response score, forms a channel contribution score sequence, and then removes feature channels with insufficient contribution according to a preset score threshold. This scheme quantifies and filters the response strength of each channel in the deep feature to defects, retains sensitive channels that have stable responses to anomalies such as solder balls and cracks, and removes irrelevant channels that only respond to normal background textures, thereby reducing the impact of background interference on defect localization.

[0018] (3) This invention obtains a reconstructed feature map with the same dimension as the deep feature map by performing dimensionality reduction mapping, dimensionality increase restoration, and interpolation filling on the filtered target feature channel set. Then, it calculates the pixel difference between the deep feature map and the reconstructed feature map at corresponding spatial locations to obtain a spatial difference matrix. Combined with preset difference thresholds, neighborhood clustering, and area thresholds, it filters continuous pixel connected regions. This scheme, based on target channel reconstruction and spatial difference detection, highlights the response deviation of abnormal locations under the same spatial coordinates, distinguishes the continuous response of real defects from the discrete response of isolated noise, reduces false alarms, and improves the accuracy of defect region extraction. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a PCBA defect localization method based on image recognition according to the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are all within the protection scope of the present invention.

[0021] like Figure 1 As shown, an embodiment of the present invention provides a PCBA defect localization method based on image recognition, including steps S11 to S15: S11, acquire the circuit board image to be detected, and perform multi-level feature extraction on the circuit board image to obtain a deep feature map; S12, extract the pixel distribution values ​​of the deep feature map in the spatial dimension, construct the response matrix corresponding to each feature channel based on the pixel distribution values, calculate the abnormal response score of each feature channel based on the response matrix, and obtain the channel contribution score sequence. S13, compare the values ​​in the channel contribution score sequence with a preset score threshold, and remove the feature channels corresponding to values ​​not greater than the preset score threshold to obtain the target feature channel set; S14, Perform feature reconstruction on the target feature channel set to obtain a reconstructed feature map with the same dimension as the deep feature map, calculate the pixel difference between the deep feature map and the reconstructed feature map at the corresponding spatial positions, and obtain the spatial difference matrix; S15, extract the continuous pixel connected regions in the spatial difference matrix whose values ​​are greater than a preset difference threshold to obtain the initial position information containing the minor defects, and perform reverse mapping on the deep feature map based on the initial position information to obtain the spatial positioning coordinates after position restoration.

[0022] In step S11, an image of the circuit board to be detected is acquired, and multi-level feature extraction is performed on the circuit board image to obtain a deep feature map, including: The circuit board image is channel merged and convolutional to obtain a shallow edge feature map; Pooling is performed on the shallow edge feature map to obtain a dimension-reduced feature map, and nonlinear mapping is performed on the dimension-reduced feature map to obtain a high-dimensional abstract feature map; Semantic information fusion is performed on the high-dimensional abstract feature map to obtain a deep feature map containing semantic information.

[0023] Specifically, an industrial camera is fixed above the inspection station, with its lens optical axis perpendicular to the circuit board surface. The circuit board is held in a preset posture by a positioning fixture. The industrial camera acquires an image of the circuit board surface under preset lighting conditions, obtaining an image of the circuit board to be inspected. The acquired color circuit board image contains three color channels: red, green, and blue. To reduce the dimensionality of subsequent data processing, a weighted average grayscale conversion method is used to merge the channels of the circuit board image. The weight ratios of the green, red, and blue channels are set to 59%, 30%, and 11%, respectively, converting the color pixel values ​​into single-channel brightness values ​​to obtain a single-channel image.

[0024] The aforementioned weights are based on a fixed ratio calculated using industrial image brightness. The green channel has the highest weight, used to preserve the brightness differences of traces, pads, and silkscreen on the green solder mask layer. The red and blue channels are used to supplement copper surface reflections and character edge information. In this embodiment, the weights remain unchanged at 59%, 30%, and 11% during the training, threshold calibration, and production inspection stages, ensuring that the input images at each stage use the same grayscale value.

[0025] Divide each brightness value of the single-channel image by 255 to obtain a normalized single-channel image with values ​​ranging from 0 to 1. The same normalization operation is performed during training, threshold calibration, and production detection. The auxiliary decoding path uses the normalized single-channel image as the reconstruction target, ensuring that the input target and the Sigmoid output share the same value range.

[0026] After the normalized single-channel image input convolutional feature extraction network, the initial convolutional layer uses a 3×3 kernel, stride 1, and padding 1 to map one input channel to 32 output channels. Following convolution, batch normalization and ReLU activation are performed sequentially to obtain a shallow edge feature map with dimensions of 32×512×512. This shallow edge feature map primarily reflects copper wire edges, via contours, pad boundaries, and silkscreen boundaries, providing geometric morphological basis for subsequent identification of micro-cracks and defects.

[0027] In this article, the outline refers to the closed geometric line formed by the via or the shape of the device, and the boundary refers to the gray-scale transition line between pads, silkscreens, or material areas; the two respectively characterize the overall shape of the object and the boundary between adjacent areas, and are not used interchangeably in the subsequent feature description.

[0028] It should be noted that a 3×3 max pooling process with a stride of 2 and padding of 1 is performed on the shallow edge feature map to obtain a dimensionality-reduced feature map of size 32×256×256. Subsequently, three downsampling convolutional layers are set sequentially, with kernel sizes of 5×5, 5×5 and 3×3, stride of 2, and padding of 2, 2 and 1 respectively, and output channels of 64, 128 and 256 respectively. Each convolutional layer is followed by batch normalization and ReLU activation, with output spatial dimensions of 128×128, 64×64 and 32×32 respectively.

[0029] Specifically, a 1×1 convolution is used to map the shallow edge feature map, the reduced feature map, and the output channels of the three downsampling convolutional layers to 256, which are then uniformly converted to 32×32 using bilinear interpolation. Finally, the corresponding coordinates are summed element-wise to obtain a 256×32×32 deep feature map. The network's four downsampling operations consist of one max-pooling layer and three downsampling convolutional layers, with a cumulative stride of 16.

[0030] The receptive field is recursively calculated as "the current receptive field plus the product of the convolution kernel size minus 1 and the previous cumulative stride". After the initial 3×3 convolution, the receptive field is 3 and the cumulative stride is 1; after 3×3 pooling, it is 5 and 2; after 5×5 convolution, it is 13 and 4, 29 and 8 respectively; after the final 3×3 convolution, it is 45 and 16. Therefore, in this embodiment, each spatial position of the deep feature map corresponds to a 16-pixel movement interval in the original image, and the effective receptive field size is 45.

[0031] The training and calibration samples were all from three consecutive production batches of the same production line and board type. The training set consisted of 60 normal images, covering 3 exposure levels and 2 camera gain levels, with 10 images for each combination; the calibration and validation sets each contained 20 images, with the three sets of images corresponding to different circuit boards and being mutually exclusive. The reference sample set referred to below is the 20 calibration images.

[0032] Ten repeating images were set for each exposure and gain combination to estimate the mean and standard deviation of the response under that acquisition condition; 20 calibration images were used to generate 20,480 threshold statistics at a spatial scale of 32×32, which were sufficient to determine the ranking positions of the 99th and 99.5th percentiles; 20 validation images were used only to check the false alarm boundary to avoid the same image being used for both calibration and validation at the same time.

[0033] The inclusion criteria for samples were: passing electrical tests, no solder balls, cracks, cold solder joints, or short circuits observed during manual inspection, an image Laplacian variance of not less than 120, a saturation pixel ratio of less than 1% for grayscale values ​​of 0 or 255, detection of all four fixture positioning points, and an effective circuit board area of ​​98% to 102% of the template area. Images with fixture obstruction exceeding 10% of the effective area, incorrect board type markings, or failure to meet any of the above criteria were excluded.

[0034] The convolutional feature extraction network in step S11 includes an encoding path and an auxiliary decoding path during the training phase. The auxiliary decoding path sequentially sets up four 4×4 transposed convolutional layers with a stride of 2 and padding of 1, resulting in output channels of 128, 64, 32, and 1 respectively. The first three layers use ReLU activation, and the last layer uses Sigmoid activation, restoring the 256×32×32 deep feature map to a 1×512×512 reconstructed image. The training loss is the pixel-wise mean square error between the reconstructed image and the normalized single-channel image.

[0035] Step S11 uses the Adam optimizer with an initial learning rate of 0.001 and a batch size of 6. Each batch contains one image from six different exposure and gain combinations; each epoch contains 10 batches. Training terminates when the training error decreases by less than 0.001 for 10 consecutive epochs or when training reaches 200 epochs. During the inference phase, auxiliary decoding paths are removed, and the convolutional kernels, batch normalization parameters, and semantic fusion parameters of the encoding paths are frozen.

[0036] Training and calibration are performed in a fixed order: first, the convolutional feature extraction network in step S11 is trained and the encoding path is frozen; then, steps S11 and S12 are performed on the benchmark sample set to calibrate the preset anomaly detection threshold and preset score threshold; subsequently, the target feature channel set is filtered according to the fixed threshold and the convolutional autoencoder in step S14 is trained; finally, the two trained networks are used to calibrate the preset intensity threshold, preset difference threshold and preset area threshold, and 20 independent verification images are used to verify the false alarm boundary.

[0037] In step S12, the pixel distribution values ​​of the deep feature map in the spatial dimension are extracted. A response matrix corresponding to each feature channel is constructed based on the pixel distribution values. The abnormal response score of each feature channel is calculated based on the response matrix to obtain a channel contribution score sequence, including: The deep feature map is separated into multiple independent feature channels, and pixel distribution values ​​are extracted based on the pixel coordinates of the independent feature channels. The pixel distribution values ​​are weighted and suppressed, and the response intensity is calculated on the weighted and suppressed pixel distribution values ​​to obtain a grayscale mapping sequence. Matrix construction is performed based on the grayscale mapping sequence to obtain response matrices corresponding to multiple feature channels.

[0038] Specifically, the deep feature map is decoupled along the channel dimension, resulting in 256 independent feature channels. For the c-th independent feature channel, its 32×32 spatial coordinates are traversed, and the activation value at each coordinate is obtained, forming a pixel distribution numerical sequence containing 1024 elements. This sequence preserves the response intensity of the c-th channel to various regions of the circuit board.

[0039] It should be noted that after freezing the benchmark sample set in step S11, the standard deviation of 20 activation values ​​is calculated for each channel and each spatial coordinate, resulting in 256×32×32 positional standard deviations. These 262,144 positional standard deviations are then sorted in ascending order, with the 5th percentile value being 13,108 and the 95th percentile value being 249,037; in this embodiment, these values ​​are 0.012 and 0.086, respectively.

[0040] For the positional standard deviation corresponding to the activation value to be processed, it is first truncated to 0.012 to 0.086, and then the normalized standard deviation is calculated. The background weight is obtained by subtracting the normalized standard deviation from 1, and the activation value is multiplied by 1 and then the background weight is subtracted to obtain the suppressed value. The smaller the standard deviation, the larger the background weight and the lower the retention ratio. When two quantile values ​​are equal, the background weight is set to 0 and the original activation value is retained to avoid division by zero.

[0041] For each channel, the 20×1024 suppressed values ​​of the benchmark sample set are arranged in ascending order. The value at the 5th percentile (index 1024) is taken as the lower bound, and the value at the 95th percentile (index 19456) is taken as the upper bound. In this embodiment, the lower bound is 0.03 and the upper bound is 0.81. When the value to be processed is not greater than the lower bound, it is mapped to 0; when it is not less than the upper bound, it is mapped to 255; when it is in between, it is linearly mapped to 0 to 255 according to its proportion, and then filled into a 32×32 response matrix according to the original coordinate order.

[0042] The extraction order of pixel distribution values ​​is consistent with the storage order of feature maps. When storing deep feature maps in a channel-first manner, the channel indices are fixed first, and then the row and column indices are traversed sequentially; when storing them in a space-first manner, the spatial coordinates are fixed first, and then the channel values ​​are read. When constructing the response matrix, the values ​​are uniformly rearranged to unique positions determined by the channel, row, and column to prevent coordinate misalignment caused by changes in the storage order.

[0043] In one implementation, the step of calculating the abnormal response score of each feature channel based on the response matrix to obtain the channel contribution score sequence includes: The discreteness of the matrix elements of the response matrix is ​​calculated to obtain the spatial distribution feature sequence; The absolute deviation is calculated based on the spatial distribution characteristic sequence to obtain the abnormal response score; The abnormal response scores are weighted and suppressed according to a preset abnormality judgment threshold, and the weighted and suppressed abnormal response scores are linearly normalized to obtain a score distribution sequence. The channel contribution score sequence is obtained by calculating the weight allocation based on the score distribution sequence.

[0044] Specifically, the arithmetic mean of the 1024 elements of each 32×32 response matrix is ​​calculated, and then the absolute difference between each element and the mean is taken as the average to obtain the dispersion value of that channel. The dispersion values ​​of the 256 channels are arranged by channel index to obtain a spatial distribution feature sequence; the median of this sequence is taken, and the absolute difference between each dispersion value and the median is calculated to obtain the abnormal response score of the corresponding channel.

[0045] The preset anomaly detection threshold is determined based on 20 × 256 anomaly response scores from the benchmark sample set. When the mean of the 5120 scores is 0.18 and the standard deviation is 0.07, the mean plus three times the standard deviation yields an initial threshold of 0.39. Verification is performed using 20 independent verification images: if more than 8 channels in a single normal image exceed the threshold, the threshold is increased by 0.01; if no channels in five consecutive images exceed the threshold, the threshold is decreased by 0.01. The adjustment range is limited to 0.36 to 0.42; in this embodiment, 0.40 is used.

[0046] The abnormal response score is compared with 0.40 for each channel. Scores below the threshold are multiplied by 0.1, while scores above or equal to the threshold remain unchanged, resulting in the suppressed score. The minimum and maximum values ​​of all suppressed scores for the current image are extracted. Each score is subtracted from the minimum value and then divided by the difference between the maximum and minimum values, linearly mapped to 0 to 1 to obtain the channel contribution score sequence. When the maximum value equals the minimum value, the contribution score of each channel is set to 0.

[0047] In one example, the channel contribution scores of channels 12 and 45 are 0.82 and 0.67 respectively, both greater than the preset score threshold of 0.60, and therefore a retention flag is added; the channel with a contribution score of 0.55 is not added a retention flag. Each channel contribution score represents an independent relative response intensity, and the sum is not required to equal 1. Different channels can exceed the preset score threshold simultaneously.

[0048] In step S13, the values ​​in the channel contribution score sequence are compared with a preset score threshold, and feature channels corresponding to values ​​not greater than the preset score threshold are removed to obtain a target feature channel set, including: When the value in the channel contribution score sequence is greater than the preset score threshold, a retention flag is added to the corresponding feature channel. Remove feature channels corresponding to values ​​not greater than the preset score threshold, and obtain the target feature channel set based on the feature channels with the retention identifier.

[0049] Specifically, the contribution scores of each of the 256 channels in the benchmark sample set are calculated for each image, resulting in 5120 values ​​which are then sorted in ascending order. The value at index 3584, corresponding to the 70th percentile, is taken as the initial score threshold, which is 0.60 in this embodiment. Twenty independent verification images are used for validation. If a single normal image retains more than 96 channels, the threshold is increased by 0.02; if five consecutive images retain fewer than 64 channels, the threshold is decreased by 0.02. The adjustment range is limited to 0.56 to 0.64, and the final value is 0.60.

[0050] A preset anomaly detection threshold is applied to the original anomaly response score to suppress low-response channels by 0.1 times; a preset score threshold is applied to the normalized channel contribution score to ultimately retain the target feature channels. The former uses the mean plus three times the standard deviation to avoid prematurely deleting potential anomaly channels, while the latter uses the 70th percentile to further compress the number of channels, achieving a balance between defect response retention rate and reconstruction computational cost.

[0051] It should be noted that the process iterates through each value in the channel contribution score sequence, comparing each value with a preset score threshold. When the channel contribution score of a feature channel is greater than the preset score threshold, a retention flag is added to that feature channel; when the channel contribution score of a feature channel is not greater than the preset score threshold, that feature channel is removed from subsequent processing. After completing the traversal and comparison of all feature channels, the feature channels with retention flags are aggregated and recombined according to their original index order, outputting the target feature channel set. The channels in this set are used for subsequent feature reconstruction and spatial difference detection processes for defects such as short circuits and open circuits on the circuit board surface.

[0052] In step S14, feature reconstruction is performed on the target feature channel set to obtain a reconstructed feature map with the same dimension as the deep feature map. The pixel difference between the deep feature map and the reconstructed feature map at corresponding spatial locations is calculated to obtain a spatial difference matrix, including: The target feature channel set is dimensionality reduced and mapped to obtain the hidden layer feature vector; The target reconstruction dimension is determined based on the hidden layer feature vector; The hidden layer feature vectors are up-dimensioned and restored according to the target reconstruction dimension to obtain the initial pixel distribution matrix; When the dimension of the initial pixel distribution matrix does not match the dimension of the target reconstruction, the initial pixel distribution matrix is ​​interpolated and padded to obtain a reconstructed feature map with the same dimension as the deep feature map.

[0053] Specifically, the number of target feature channels varies with the channel contribution score of the current image. Based on the original index of the target feature channels, the retained channels are written into a fixed input tensor of 256×32×32, and the corresponding positions of the unretained channels are set to 0. In the detection example, 128 channels are retained, but the number of input channels of the convolutional autoencoder is always 256, ensuring a consistent data structure across different images.

[0054] The encoding network consists of four 3×3 convolutional layers with a stride of 2 and padding of 1, producing output channels of 256, 192, 160, and 128 respectively. Each convolution is followed by batch normalization and ReLU activation, with spatial dimensions of 16×16, 8×8, 4×4, and 2×2 respectively. The 128×2×2 hidden feature tensor is flattened by channel, row, and column order to obtain a hidden feature vector of length 512.

[0055] The target reconstruction dimension is read from the metadata of the deep feature map and determined to be 256 channels, 32 rows, and 32 columns. During decoding, the hidden layer feature vector is first restored to a 128×2×2 tensor, and then four transposed convolutional layers with a stride of 2 and padding of 1 are set in sequence, with the number of output channels being 160, 192, 256, and 256 respectively, and the spatial dimensions being 4×4, 8×8, 16×16, and 32×32 respectively; the first three layers use ReLU activation, and the last layer uses linear output.

[0056] In this embodiment, using the above parameters, an initial pixel distribution matrix of 256×32×32 is obtained, consistent with the target reconstruction dimension. For implementations where input size changes or transposed convolution boundary rounding leads to inconsistent sizes, bilinear interpolation is performed in both row and column directions, and boundary copying is applied to the edges until the number of channels, rows, and columns are all consistent with the deep features. Figure 1 Thus, the reconstructed feature map is obtained.

[0057] Step S14 involves training after fixing the encoding path, preset anomaly detection threshold, and preset score threshold in step S11. Steps S11 to S13 are executed sequentially on 60 normal training images, using a fixed input tensor as network input and the corresponding 256×32×32 deep feature map as the reconstruction target. The Adam optimizer is used with an initial learning rate of 0.0005, a batch size of 6, and the training objective is to minimize the mean squared error between two feature maps.

[0058] After each training round, the average feature reconstruction error is calculated on the benchmark sample set. The training terminates and saves the parameters when the error decreases by less than 0.001 for 10 consecutive rounds or when 200 rounds of training are reached. After threshold calibration, the convolutional autoencoder parameters are no longer updated to keep the reconstruction accuracy fixed during the production detection stage.

[0059] In one implementation, calculating the pixel difference between the deep feature map and the reconstructed feature map at corresponding spatial locations to obtain a spatial difference matrix includes: Obtain the alignment coordinates between the deep feature map and the reconstructed feature map, and calculate the absolute deviation based on the alignment coordinates; The absolute deviation is mapped to grayscale to obtain the pixel difference value at the corresponding spatial location; When the pixel difference is greater than a preset intensity threshold, the response intensity corresponding to the pixel difference is extracted to obtain a spatial difference matrix containing abnormal response intensity.

[0060] Specifically, both the deep feature map and the reconstructed feature map use a channel, row, and column indexing order. The 256 channel values ​​of the two feature maps are read point-by-point along a 32×32 spatial coordinate system. The absolute deviation is calculated for each channel, and the arithmetic mean is taken to obtain the average absolute deviation for each spatial location. Same channel indices and same row and column indices form aligned coordinates, thus preventing channel or position mismatches during the comparison process.

[0061] The grayscale mapping uses a segmented mapping determined by the baseline sample quantiles and the maximum value. The 20480 mean absolute deviations of the baseline sample set are arranged in ascending order. The 95th percentile is taken from the value at index 19456, the 99.5th percentile from the value at index 20378, and the last maximum value is extracted; in this embodiment, these three values ​​are 0.24, 0.61, and 1.23, respectively. When the mean absolute deviation is not greater than 0.24, it is mapped to 0 to 100 according to its ratio to 0.24; when it is greater than 0.24 and less than 1.23, it is mapped to 100 to 255 according to its ratio between 0.24 and 1.23; when it is not less than 1.23, it is mapped to 255. When the mean absolute deviation is 0.61, the mapped value is rounded to 158.

[0062] When the 95th percentile equals the maximum value, the minimum and maximum values ​​of the average absolute deviation of this batch are linearly mapped to 0 to 255; when the maximum and minimum values ​​are still equal, all pixel differences are set to 0. This mapping only changes the range of response intensity representation, without changing the 32×32 spatial coordinates.

[0063] The preset intensity threshold is determined using the 99th percentile of the pixel difference in normal samples. 32×32 pixel differences are calculated for each reference sample set, resulting in 20480 values, which are then sorted in ascending order. The value with index 20276 is taken as the initial threshold; in this embodiment, the initial threshold is 148. Twenty independent verification images are then used to ensure that no more than three non-zero abnormal response points are found in each normal image. The adjustment range is limited to 145 to 155, and a final value of 150 is selected. The coordinates of the pixel difference value of 180 are retained.

[0064] During calibration, the camera position, image cropping range, and network parameters are kept constant to ensure that the 20480 pixel differences use the same units and spatial aperture. The 99th percentile value is used to cover most of the differences caused by normal texture, pad edges, and slight reflections, retaining the top 1% of the response in the spatial difference matrix. If there are more than 3 non-zero response points in an independent verification image, the threshold is increased by 1 gray level until the quantity boundary is met; if there are no non-zero response points in 5 consecutive verification images, the threshold is decreased by 1 gray level to preserve the response sensitivity to minor defects.

[0065] In step S15, continuous pixel connected regions with values ​​greater than a preset difference threshold are extracted from the spatial difference matrix to obtain initial location information containing minute defects. Based on this initial location information, the deep feature map is reverse-mapped to obtain the spatially reconstructed coordinates, including: Based on the spatial difference matrix, target pixels with values ​​greater than a preset difference threshold are extracted, and pixel clustering is performed according to a preset neighborhood range to obtain continuous pixel connected regions. When the area of ​​the connected region is greater than a preset area threshold, the connected region is determined as a candidate defect region, and the geometric center coordinates of the candidate defect region are calculated. Global coordinate mapping is performed based on the area of ​​the connected region and the coordinates of the geometric center to obtain the initial location information containing the minor defects; Based on the initial position information, the pixel range corresponding to the deep feature map is extracted, and the scaling factor and boundary compensation value corresponding to the pixel range are calculated by combining the network stride ratio and the receptive field size. The pixel range is reverse-mapped using the scaling factor and the boundary compensation value, and the extreme coordinates after mapping are extracted to obtain the spatial positioning coordinates after position restoration.

[0066] Specifically, the preset difference threshold is determined using the 99.5th percentile value of all response intensities in the difference matrix of the normal sample space. 32×32 response intensities were obtained from the benchmark sample set, merged to obtain 20480 values, and sorted in ascending order. The value with index 20378 was taken as the initial threshold. The three production batches contained 7, 7, and 6 calibration images, respectively, corresponding to 99.5 percentiles of 156, 158, and 157. The maximum value of 158 was taken, and one grayscale protection factor was added, resulting in 159.

[0067] Then, using 20 independent verification images, the number of connected regions is counted image by image. If a normal image produces more than two connected regions, the threshold is increased by one gray level. If no connected regions are formed in five consecutive images, the threshold is decreased by one gray level. The adjustment range is limited to 156 to 164. In this embodiment, 160 is ultimately chosen to ensure that the number of consecutive connected regions produced by normal images does not exceed one.

[0068] A preset intensity threshold controls whether a single spatial location enters the spatial difference matrix, while a preset difference threshold controls whether the response after entering the matrix participates in connected component construction. The former is set at the 99th percentile to reduce pixel-level false negatives, while the latter is set at the 99.5th percentile to compress weak connected components formed by normal textures. During verification, the number and area of ​​connected components are counted for each image. When a normal image shows more than two connected components, the preset difference threshold is increased; when five consecutive images show no connected components, the threshold is decreased, maintaining a stable relationship between sensitivity and false alarms between the two threshold levels.

[0069] Traverse the spatial difference matrix and extract the coordinates of pixels with a response intensity greater than 160 as target pixels to construct a discrete set of abnormal pixels. Perform pixel clustering based on eight neighborhoods: using any target pixel as a seed point, search for eight neighborhoods in the horizontal, vertical, and diagonal directions, grouping adjacent target pixels into the same connected region. Repeat the search until the region boundary no longer expands, obtaining a continuous pixel connected region.

[0070] It should be noted that the total number of pixels contained in each connected region is used as the region area. After filtering by the difference threshold, the benchmark sample set formed 96 normal connected regions. The areas of the 96 regions were sorted in ascending order, and the 95th percentile was taken as the value of the index 92, which is 5 pixels. The 95th percentiles of the three production batches were 4, 5, and 5 pixels respectively. The combined statistical value was consistent with the maximum value of the batch statistics, so the preset area threshold was set to 5.

[0071] On 20 independent verification images, no more than two candidate regions with an area greater than 5 were found per image, so this threshold was maintained. A connected region containing 8 pixels was identified as a candidate defect region. The arithmetic mean of the horizontal and vertical coordinates of the pixels within the region was calculated to obtain the geometric center coordinates; the region area and the geometric center coordinates together constitute the initial position information in the deep feature map coordinate system.

[0072] The initial position information is represented by 32×32 deep feature map coordinates, which are used to extract the horizontal and vertical extreme value indices of connected regions of consecutive pixels to form the pixel range. When the input circuit board image size is 512×512, the stride of the max pooling layer and the three downsampling convolutional layers is 2, and the network stride ratio is 16. The receptive fields are recursively calculated according to the structure disclosed in S11 as 3, 5, 13, 29, and 45. Half of 45 is taken and rounded down to obtain the boundary compensation value of 22.

[0073] A reverse mapping of the pixel index set is performed using coordinate scaling and boundary compensation. The horizontal index range of the connected regions is set to the minimum to the maximum horizontal index, and the vertical index range is set to the minimum to the maximum vertical index. The four extreme indices are multiplied by 16, and the minimum value is subtracted by 22, while the maximum value is added by 22, resulting in rectangular bounding boxes. The horizontal coordinates and vertical coordinates are restricted to 0 to 511, yielding the reconstructed spatial positioning coordinates. A bounding box is output for each connected region, and isolated response points with an area not exceeding a preset area threshold are removed from the positioning results as noise.

[0074] The inverse mapping uses the extremum indices of connected regions instead of a single center point as input, thus preserving the horizontal and vertical ranges of fine cracks and irregular pad defects. The network stride ratio determines that each index movement in the feature map corresponds to a 16-pixel movement in the original image, and the boundary compensation value covers the effective receptive field around that index. Coordinate cropping only applies to the localization boxes near the edges of the image, without changing the region center or connectivity. When the board size or camera resolution changes, the scaling factor is recalculated based on the input size, deep feature map size, and network layer stride, and the mapping deviation is verified using a calibration image.

[0075] For example, the input image size is 512×512. After channel merging, a single-channel image of 512×512 is formed. A max-pooling layer and three downsampling convolutional layers yield a deep feature map of 256×32×32. Channel contribution filtering retains 128 target feature channels. The encoding network outputs a hidden feature vector of length 512, and the decoding network outputs a reconstructed feature map of 256×32×32. In the spatial difference matrix, 12 pixels have a response intensity exceeding 160, of which 8 pixels are connected within their eight-neighborhoods.

[0076] The horizontal index range of this connected region is 18 to 19, and the vertical index range is 20 to 23. The minimum horizontal index 18 multiplied by 16 and subtracted by 22 gives 266, and the maximum horizontal index 19 multiplied by 16 and added by 22 gives 326; the minimum vertical index 20 multiplied by 16 and subtracted by 22 gives 298, and the maximum vertical index 23 multiplied by 16 and added by 22 gives 390. Thus, the bounding boxes at the top left (266, 298) and bottom right (326, 390) are obtained.

[0077] For example, the defect is a crack extending along the trace direction. Abnormal pixels in the spatial difference matrix form connected regions that are long and narrow, with the area exceeding a preset area threshold. After calculating the geometric center of the region to form initial position information, the horizontal and vertical extreme value indices of the connected regions are further extracted. Inverse mapping is then performed according to scaling factors in the horizontal and vertical directions, and receptive field boundary compensation is used to expand the localization range, ensuring the localization box covers the edge responses at both ends of the crack. Since the connected regions are composed of adjacent pixels, the localization result does not change significantly due to grayscale fluctuations of individual pixels, making it suitable for stable localization of slender cracks.

[0078] For example, the defect is a cold solder joint near the pad. The response in the cold solder joint area is mainly concentrated at the pad boundary. Feature channels with higher contribution scores retain the pad outline and local brightness variations. After removing low contribution channels, the normal pad structure in the reconstructed feature map remains intact. The deep feature map and the reconstructed feature map show a large pixel difference at the cold solder joint location. After filtering by intensity threshold and connected region area, the bounding box is output. This result demonstrates that this method does not rely on the fixed appearance of a particular defect, but rather utilizes the spatial difference between the target feature channel set and the reconstructed feature map to identify anomalies.

[0079] It should be noted that the channel contribution screening and spatial difference extraction follow the same process for different defect types. For solder balls, the abnormal response typically manifests as localized highlights and forms a small connected region; for trace cracks, the abnormal response forms a long, thin connected region along the trace direction; for cold solder joints, the abnormal response is concentrated at the edge of the pad and presents an irregular shape. Since steps S12 to S15 are all based on characteristic responses and spatial continuity, it is not necessary to establish different location rules for each defect; only the threshold needs to be calibrated based on the minimum defect size and the false alarm rate on site.

[0080] In this embodiment, the processing is executed on an edge computing device, industrial control computer, or server. An industrial camera acquires images of the circuit board, while the processor performs feature extraction, response matrix construction, channel contribution calculation, feature reconstruction, difference matrix calculation, and coordinate inverse mapping. The memory stores network parameters, threshold configurations, intermediate matrices, and localization results. For real-time detection scenarios, the image feature extraction process is performed in batches, and then channel filtering and localization output are completed image by image. For low-latency scenarios, a pipeline approach is used, so that while feature reconstruction is being performed on one frame, feature extraction is being performed on the next frame simultaneously.

[0081] It should be understood that each processing operation in steps S11 to S15 is implemented by software, hardware, or a combination of software and hardware. The program code used to perform these operations is stored in a computer-readable storage medium and is read and executed by a processor. The specific language, network framework, and operating platform of the program code do not constitute a limitation of the present invention, as long as the target feature channel set, spatial difference matrix, and spatial positioning coordinates are obtained according to the data flow sequence described in this embodiment, they fall within the scope of the present invention.

[0082] It should be noted that after step S15 is completed, the stability of the results of adjacent acquisition frames of the same circuit board can be verified. Ten circuit boards with known defect locations are selected, and four frames are continuously acquired from each board to form 30 pairs of adjacent frames. The center distances of the same defect in adjacent frames are arranged in ascending order. The distance corresponding to the 95th percentile and the serial number 29 is 8 pixels. An additional 2 pixels of protection are added to obtain a preset distance threshold of 10 pixels.

[0083] The intersection-union ratio (IUR) of the corresponding localization boxes for the 30 pairs of adjacent frames is calculated and sorted in ascending order. The value corresponding to the 5th percentile of the index 2 is 0.54, which is rounded down to obtain the preset overlap ratio of 0.5. A smaller distance threshold or a larger overlap ratio will split the slight localization jitter of the same defect into different results, while a more relaxed threshold may merge adjacent defects. Therefore, the above two boundary conditions are adopted. When the center distance is less than 10 pixels and the IUR is greater than 0.5, a stable localization result is output.

[0084] For industrial cameras with different resolutions, the network stride ratio and receptive field size are automatically calculated using network structure parameters. The network stride ratio is equal to the product of the strides of each downsampling layer, and the receptive field size is calculated recursively based on the convolutional kernel size, stride, and the receptive field size of the previous layer. Taking an input image size of 1024×768 and a deep feature map size of 64×48 as an example, the horizontal scaling factor is obtained by dividing 1024 by 64, resulting in 16, and the vertical scaling factor is obtained by dividing 768 by 48, also resulting in 16. If both are the same, a uniform scaling factor is used; if they are different, the horizontal and vertical scaling factors are saved separately and applied to the horizontal and vertical coordinates respectively during inverse mapping.

[0085] It should be noted that the baseline sample set contains 6 typical occlusion images, 2 from each of the 3 production batches. The occlusion comes from the shadows formed by the fixture pressure block at 3 fixed positions, with the occlusion area ranging from 1% to 5% of the effective area of ​​the circuit board. The occluded circuit boards have been confirmed to be defect-free through electrical testing and manual inspection. Six images are included to ensure duplicate samples for each batch and each fixture position, and to maintain consistency with the exclusion criteria of occlusion exceeding 10%.

[0086] The six images are used only to incorporate the stable response of the fixture shadow into the background weight. During detection, connected regions that do not overlap with the effective region mask of the circuit board are marked as invalid regions; for partially overlapping regions, only the geometric center and extreme coordinates of the overlapping pixels are retained. The effective region mask is jointly determined by the closed region formed by the outer contour of the circuit board and the four fixture positioning points, without changing the feature channel selection and spatial difference calculation process.

[0087] In terms of parameter version management, each time the preset score threshold, preset difference threshold, or preset area threshold is updated, a new version identifier is generated for the corresponding configuration file, and the number of calibration samples, acquisition time, camera parameters, and verification results are recorded. When the processor outputs spatial positioning coordinates, it also outputs the configuration version identifier, allowing maintenance personnel to trace the thresholds and network parameters used at the time of the update based on the positioning results. If the detection task switches to another board type, the corresponding configuration is loaded first according to the board type identifier, and then step S11 is executed; if a matching configuration cannot be found, the automatic positioning results are stopped, and a parameter calibration prompt is given to avoid using incompatible coordinate mapping parameters for the new board type.

[0088] It should be noted that during the result verification stage, the positioning bounding box is superimposed on the input circuit board image for visualization, and the spatial difference matrix is ​​saved as a grayscale image for traceability. If the verification result indicates that the positioning bounding box is too large, the preset area threshold is increased or the expansion range of the boundary compensation value is decreased; if the positioning bounding box is too small and does not completely cover the crack, the preset difference threshold is decreased or the boundary compensation value is increased. Parameter adjustments are completed through independent verification samples, and the version number is re-recorded after adjustment to avoid using untraceable parameter combinations in different batches.

[0089] In this embodiment, each parameter is stored as a configuration item corresponding to the board type, including channel merging weights, convolutional network parameters, pooling stride, preset anomaly detection threshold, preset score threshold, preset intensity threshold, preset difference threshold, preset area threshold, network stride ratio, receptive field size, and boundary compensation value. After reading the configuration items, the processor executes steps S11 to S15 sequentially and saves the intermediate results according to the image number. Even if only the final spatial positioning coordinates are output, the corresponding channel contribution score sequence and spatial difference matrix can be read back according to the image number to support the interpretation and auditing of defect positioning results.

[0090] It should be noted that, to verify the coordinate restoration process, a circuit board image with known pixel position markers was selected as the verification image. First, the coordinates of the top-left, bottom-right, and center corners of the marked area were recorded in the input image. Then, the corresponding spatial positioning coordinates were output according to steps S11 to S15, and the lateral and longitudinal deviations between the output coordinates and the recorded coordinates were calculated. If the deviation exceeds the allowable range corresponding to the receptive field boundary compensation value, the input image cropping position, the stride of each downsampling layer, the convolution padding method, and the deep feature map size were checked, and the scaling factor and boundary compensation value were recalculated.

[0091] After verification, the coordinate mapping parameters, along with the board type identifier, camera resolution, and network parameters, are saved together. During production testing, the same set of parameters is called according to the board type identifier to ensure consistency in the process of mapping connected regions from the deep feature map coordinate system to the original image coordinate system.

[0092] In the threshold verification process, normal samples, solder ball samples, crack samples, and cold solder joint samples are processed using the same chain. The number of pixels exceeding the preset difference threshold, the area of ​​connected regions, and the coordinate deviation after inverse mapping are recorded in the spatial difference matrix. Normal samples are used to observe false alarms caused by isolated responses and background textures, while defect samples are used to observe whether the target area forms a continuous pixel connected region. If there are many isolated responses in normal samples while the defect area remains continuous, the preset area threshold is adjusted first; if the grayscale ranges of normal textures and defect responses overlap, the preset intensity threshold and preset difference threshold are adjusted. The adjusted parameters are then run again on independent verification samples, and the statistical results are retained.

[0093] It should be noted that when the image acquired by the industrial camera has local overexposure, pixel values ​​exceeding 255 are truncated to 255 before channel merging in step S11. When the image has slight rotation, the coordinates of three non-collinear positioning reference points on the circuit board in the acquired image are extracted, and these coordinates are combined with the three corresponding coordinates in the template to form six linear equations, which are then used to solve for the six parameters of the two-dimensional affine transformation.

[0094] The affine transformation is applied to the pixel coordinates of the acquired image to obtain the pose-corrected image; if the determinant of the transformation matrix is ​​not zero, its inverse matrix is ​​calculated and saved. When outputting the spatial positioning coordinates, the mapping from the deep feature map coordinates to the pose-corrected image coordinates is first completed, then the four corner points of the positioning box are substituted into the inverse transformation, and the extreme values ​​of the transformed horizontal and vertical coordinates are taken to obtain the original acquired image coordinates. The pose correction does not change the processing order of steps S12 to S15.

[0095] On the production line, the positioning result output is a record containing the defect number, defect center coordinates, rectangular boundary coordinates, area, and abnormal response intensity. The defect number is generated according to the image acquisition order and the connected region sorting order. The rectangular boundary coordinates are represented by the coordinates of the upper left and lower right corners. The area is recorded using both the number of pixels in the deep feature map coordinate system and the converted area in the original image coordinate system. This record is passed to the rework station to guide operators in reviewing and repairing the circuit board.

[0096] In summary, this invention discloses a PCBA defect localization method based on image recognition. The method involves acquiring an image of the circuit board to be inspected, performing multi-level feature extraction on the image, extracting pixel distribution values ​​from the deep feature map and calculating the channel contribution score sequence, filtering the target feature channel set, reconstructing the target feature channel set, calculating the spatial difference between the deep feature map and the reconstructed feature map, extracting connected regions of continuous pixels, and obtaining the spatial localization coordinates through inverse mapping. This invention achieves accurate localization of minute defects in PCBAs through channel contribution filtering, feature reconstruction, spatial difference detection, and coordinate inverse mapping.

[0097] The second embodiment of the present invention provides a PCBA defect localization system based on image recognition, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the PCBA defect localization method based on image recognition as described above.

[0098] It should be noted that the PCBA defect localization system based on image recognition provided in this embodiment of the invention is used to execute all the process steps of the PCBA defect localization method based on image recognition in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0099] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, and variations made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A PCBA defect localization method based on image recognition, characterized in that, include: A circuit board image to be inspected is acquired, and multi-level feature extraction is performed on the circuit board image to obtain a deep feature map; Extract the pixel distribution values ​​of the deep feature map in the spatial dimension, construct the response matrix corresponding to each feature channel based on the pixel distribution values, calculate the abnormal response score of each feature channel based on the response matrix, and obtain the channel contribution score sequence. The values ​​in the channel contribution score sequence are compared with a preset score threshold, and feature channels corresponding to values ​​not greater than the preset score threshold are removed to obtain the target feature channel set. The target feature channel set is reconstructed to obtain a reconstructed feature map with the same dimension as the deep feature map. The pixel difference between the deep feature map and the reconstructed feature map at corresponding spatial locations is calculated to obtain a spatial difference matrix. Extract the connected regions of consecutive pixels in the spatial difference matrix whose values ​​are greater than a preset difference threshold to obtain initial location information containing minute defects. Based on the initial location information, perform reverse mapping on the deep feature map to obtain the spatial positioning coordinates after location restoration.

2. The PCBA defect localization method based on image recognition according to claim 1, characterized in that, The step of performing multi-level feature extraction on the circuit board image to obtain a deep feature map includes: The circuit board image is channel merged and convolutional to obtain a shallow edge feature map; Pooling is performed on the shallow edge feature map to obtain a dimension-reduced feature map, and nonlinear mapping is performed on the dimension-reduced feature map to obtain a high-dimensional abstract feature map; Semantic information fusion is performed on the high-dimensional abstract feature map to obtain a deep feature map containing semantic information.

3. The PCBA defect localization method based on image recognition according to claim 1, characterized in that, The step of extracting the pixel distribution values ​​of the deep feature map in the spatial dimension and constructing the response matrix corresponding to each feature channel based on the pixel distribution values ​​includes: The deep feature map is subjected to channel separation to obtain multiple independent feature channels, and Pixel distribution values ​​are extracted based on the pixel coordinates of the independent feature channels; The pixel distribution values ​​are weighted and suppressed, and the response intensity is calculated on the weighted and suppressed pixel distribution values ​​to obtain a grayscale mapping sequence. Matrix construction is performed based on the grayscale mapping sequence to obtain response matrices corresponding to multiple feature channels.

4. The PCBA defect localization method based on image recognition according to claim 1, characterized in that, The step of calculating the abnormal response score of each feature channel based on the response matrix to obtain the channel contribution score sequence includes: The discreteness of the matrix elements of the response matrix is ​​calculated to obtain the spatial distribution feature sequence; The absolute deviation is calculated based on the spatial distribution characteristic sequence to obtain the abnormal response score; The abnormal response scores are weighted and suppressed according to a preset abnormality judgment threshold, and the weighted and suppressed abnormal response scores are linearly normalized to obtain a score distribution sequence. The channel contribution score sequence is obtained by calculating the weight allocation based on the score distribution sequence.

5. The PCBA defect localization method based on image recognition according to claim 1, characterized in that, The step of comparing the values ​​in the channel contribution score sequence with a preset score threshold and removing feature channels corresponding to values ​​not greater than the preset score threshold yields a target feature channel set, including: When the value in the channel contribution score sequence is greater than the preset score threshold, a retention flag is added to the corresponding feature channel. Remove feature channels corresponding to values ​​not greater than the preset score threshold, and obtain the target feature channel set based on the feature channels with the retention identifier.

6. The PCBA defect localization method based on image recognition according to claim 1, characterized in that, The step of reconstructing the target feature channel set to obtain a reconstructed feature map with the same dimensions as the deep feature map includes: The target feature channel set is dimensionality reduced and mapped to obtain the hidden layer feature vector; The target reconstruction dimension is determined based on the hidden layer feature vector; The hidden layer feature vectors are up-dimensioned and restored according to the target reconstruction dimension to obtain the initial pixel distribution matrix; When the dimension of the initial pixel distribution matrix does not match the dimension of the target reconstruction, the initial pixel distribution matrix is ​​interpolated and padded to obtain a reconstructed feature map with the same dimension as the deep feature map.

7. The PCBA defect localization method based on image recognition according to claim 1, characterized in that, The step of calculating the pixel difference between the deep feature map and the reconstructed feature map at corresponding spatial locations to obtain a spatial difference matrix includes: Obtain the alignment coordinates between the deep feature map and the reconstructed feature map, and calculate the absolute deviation based on the alignment coordinates; The absolute deviation is mapped to grayscale to obtain the pixel difference value at the corresponding spatial location; When the pixel difference is greater than a preset intensity threshold, the response intensity corresponding to the pixel difference is extracted to obtain a spatial difference matrix containing abnormal response intensity.

8. The PCBA defect localization method based on image recognition according to claim 7, characterized in that, The step of extracting connected regions of consecutive pixels with values ​​greater than a preset difference threshold from the spatial difference matrix to obtain initial location information containing minute defects, and then performing reverse mapping on the deep feature map based on the initial location information to obtain the spatial positioning coordinates after location restoration, includes: Based on the spatial difference matrix, target pixels with values ​​greater than a preset difference threshold are extracted, and pixel clustering is performed according to a preset neighborhood range to obtain continuous pixel connected regions. When the area of ​​the connected region is greater than a preset area threshold, the connected region is determined as a candidate defect region, and the geometric center coordinates of the candidate defect region are calculated. Global coordinate mapping is performed based on the area of ​​the connected region and the coordinates of the geometric center to obtain the initial location information containing the minor defects; Based on the initial position information, the pixel range corresponding to the deep feature map is extracted, and the scaling factor and boundary compensation value corresponding to the pixel range are calculated by combining the network stride ratio and the receptive field size. The pixel range is reverse-mapped using the scaling factor and the boundary compensation value, and the extreme coordinates after mapping are extracted to obtain the spatial positioning coordinates after position restoration.

9. A PCBA defect localization system based on image recognition, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the PCBA defect localization method based on image recognition as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the PCBA defect location method based on image recognition as described in any one of claims 1 to 8.

Images