Surface defect detection method based on linear array camera

Abstract

The invention discloses a surface defect detection method based on a line-scan digital camera, and relates to the technical field of image data processing and image analysis. A space-time fusion module, a dynamic up-sampling module and a multi-scale feature pyramid network are introduced into an image sequence formed by continuous scanning of the line-scan digital camera; the characterization capability on the defects of the micro solder balls under the condition of high-speed acquisition is integrally enhanced; time attention and space attention are jointly modeled, multi-frame information is referred to at the same space position, weights are adaptively distributed according to inter-frame similarity and time intervals, transient defect signals such as cracks and bridging occurring in a short time are amplified, stable background constraints are provided by long-distance frames, and the accuracy and the reliability of the system are improved. Therefore, high sensitivity and discrimination capability to real defects are still kept when motion blur and noise exist.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing and image analysis, and particularly to a surface defect detection method based on a linear array camera. BACKGROUND

[0002] With the development of electronic devices towards miniaturization and high density, the size of micro components such as solder balls on printed circuit boards (PCB) is continuously reduced, and the manufacturing process has increasingly higher requirements for soldering quality. In order to realize online quality monitoring, a surface defect detection method based on a linear array camera is generally used in the production line to continuously scan the surface of the high-speed moving PCB to obtain a high-resolution image sequence to adapt to the beat requirements of the automatic production line.

[0003] In the prior art, for defects such as missing solder, bridging and false solder, the automatic detection of defects is usually realized by pre-processing, feature extraction and defect judgment on the images collected by the linear array camera. With the development of deep learning technology, some schemes introduce a small target detection model based on a convolutional neural network, such as a one-stage target detection network or a detection network combined with a Transformer structure, to realize end-to-end positioning and classification of solder defects, and combine multi-scale feature fusion, attention mechanism and data enhancement to improve the adaptability of the model to complex backgrounds.

[0004] However, under high-speed production rhythm, the relative motion speed between the PCB and the linear array camera is high, and limited by the exposure time and lighting conditions, motion blur and noise interference are prone to occur in the collected images, which weakens the fine texture and contrast of micron-level solder defect. The existing deep learning detection model has insufficient sensitivity to the microstructure details of the feature extraction layer when processing small and low-contrast defect targets, and it is difficult to reliably separate the fine defect region from the background texture. In addition, the linear array scanning produces a continuous image sequence over time, but many existing methods mainly detect based on a single frame of image and fail to fully utilize the spatio-temporal correlation information between images, which easily leads to missed detection when the defect is in the critical boundary or appears instantaneously.

[0005] In summary, the existing surface defect detection method based on a linear array camera still has problems such as motion blur, insufficient expression of small target features, and insufficient use of spatio-temporal information when detecting fine solder defects at high speed online, which makes it difficult to balance the detection speed and the detection rate of small defects. Therefore, how to improve the detection accuracy and stability of fine solder defects under high-speed linear array acquisition conditions has become a technical problem to be solved by those skilled in the art. SUMMARY

[0006] In view of the above existing problems, the present application is proposed.

[0007] The application provides a surface defect detection method based on a linear array camera to solve the problem that the solder ball detection of the existing linear array camera is prone to blurring under high-speed acquisition and is not sensitive to small low-contrast defects.

[0008] To solve the above technical problems, the application provides the following technical solutions. The embodiment of the application provides a surface defect detection method based on a linear array camera, which comprises the following steps: S1, acquiring a continuous image sequence of a surface to be detected by using a linear array camera; S2, extracting features, inputting the continuous image sequence into a neural network model, wherein the neural network model comprises a space-time fusion module and a dynamic upsampling module, performing multi-frame feature extraction on the continuous image sequence by using the space-time fusion module to obtain intermediate features representing space-time evolution characteristics of defects; S3, enhancing resolution, performing resolution enhancement processing on the intermediate features by using the dynamic upsampling module to obtain enhanced features; S4, outputting a defect detection result based on the enhanced features.

[0009] As a preferred scheme of the surface defect detection method based on the linear array camera, in the acquisition step, the linear array camera is scanned along the moving direction of the surface to be detected, image data is acquired row by row during relative motion of the measured object, and adjacent image rows are spliced into multiple images in the order of acquisition time to form the continuous image sequence.

[0010] As a preferred scheme of the surface defect detection method based on the linear array camera, the space-time fusion module is implemented based on a Transformer architecture, and the Transformer architecture comprises a time attention layer, which models the inter-frame relationship in the continuous image sequence. When the time attention layer models the inter-frame relationship in the continuous image sequence, the modeling is implemented in the space-time fusion module in the following manner: for the continuous image sequence formed by the linear array camera during the relative motion of the measured object, two-dimensional spatial feature maps are first extracted on each frame, then a time feature line is constructed along the time dimension at each spatial position, and the time feature line is weighted and aggregated by the time attention layer to obtain a time sequence feature representation representing the evolution of defects over time.

[0011] As a preferred scheme of the surface defect detection method based on the linear array camera, the space-time fusion module further comprises a spatial attention layer, which is used to give a higher weight to a suspected defect area in each frame of image.

[0012] As a preferred scheme of the surface defect detection method based on the linear array camera according to the application, wherein: the dynamic up-sampling module is integrated in the feature pyramid network, and the dynamic up-sampling module performs up-sampling processing on the low-resolution feature map through adaptive interpolation operation. When the dynamic up-sampling module performs adaptive interpolation on the low-resolution feature map, the process is refined into interpolation weight prediction and weighted up-sampling reconstruction steps in the feature pyramid network; the dynamic up-sampling module first predicts a set of interpolation weights adaptive to local structures for each spatial position on the low-resolution feature map, then combines the neighborhood features by using the set of interpolation weights to generate corresponding high-resolution output pixels, so as to enlarge the spatial resolution while retaining small target and weak texture details.

[0013] As a preferred scheme of the surface defect detection method based on the linear array camera according to the application, wherein: the feature pyramid network comprises a multi-scale fusion layer, and the multi-scale fusion layer performs multi-scale fusion of the feature maps from different levels through top-down and horizontal connection.

[0014] As a preferred scheme of the surface defect detection method based on the linear array camera according to the application, wherein: the neural network model is a pre-trained model, and when the neural network model is trained, a data enhancement strategy is used to generate synthetic defect samples, and the synthetic defect samples and real acquisition samples are used together for training.

[0015] As a preferred scheme of the surface defect detection method based on the linear array camera according to the application, wherein: the detection output step comprises: target detection decoding is performed on the enhanced features to obtain spatial position information and defect category information of the defect region, and structured detection information including defect position coordinates, defect category labels and corresponding confidence scores is generated.

[0016] As a preferred scheme of the surface defect detection method based on the linear array camera according to the application, wherein: the method is applied to the solder ball defect detection scene of a printed circuit board, the surface to be detected comprises fine solder ball structures arranged on pads, and the defect detection result comprises bridge, false soldering, missing or surface micro-crack defect types of the solder balls.

[0017] As a preferred scheme of the surface defect detection method based on the linear array camera according to the application, wherein: the detection output step further comprises: time sequence association and consistency constraint processing is performed on a plurality of detection results located in the same physical region in the continuous image sequence, and according to a pre-set confidence threshold and time sequence consistency rule, repeated detection or critical detection is merged and screened.

[0018] The application has the beneficial effects that: the application introduces a space-time fusion module, a dynamic up-sampling module and a multi-scale feature pyramid network on the image sequence formed by continuous scanning of a linear array camera, and the overall representation ability of micro solder ball defects under high-speed acquisition conditions is enhanced. On the one hand, the time attention and the space attention are jointly modeled, the same space position simultaneously refers to multiple frame information, the weight is adaptively allocated according to the frame similarity and the time interval, the transient defect signals such as cracks and bridging appearing in a short time are amplified, the stable background constraint is provided by the long-distance frame, and thus the high sensitivity and discrimination to the real defects are maintained even when there is motion blur and noise. On the other hand, the dynamic up-sampling module predicts the interpolation weight according to the local content in the feature pyramid structure, and the low-resolution features are adaptively up-sampled and reconstructed, so that the solder ball edge, small size defect and weak texture micro crack still have clear contours and gray details after amplification, and fine spatial clues are provided for small target detection.

[0019] In combination with the top-down and lateral fusion of multi-level feature maps, the application simultaneously utilizes large-scale context and small-scale details to improve the robustness in a complex background. Further, synthetic defect samples are constructed in the model training stage and are trained jointly with real samples, which alleviates the problem of few micro defect samples in actual production, so that the network can learn more rich and representative defect patterns. Finally, in cooperation with the time correlation and consistency constraint of the detection result, the application can suppress the fluctuations caused by single-frame accidental false detection and boundary conditions, and output structured information containing defect position, type and confidence, which is suitable for printed circuit board solder ball online detection and other high-speed continuous production scenes. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope of the present application.

[0021] Figure 1 The flowchart of the surface defect detection method based on the linear array camera in the embodiments. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the present application more clear, the following will further illustrate the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and should not be regarded as limiting the present application.

[0023] All the terms used in the present application (including technical and scientific terms) have the meanings generally understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted to have meanings consistent with the context of the present specification, and should not be interpreted in an idealized or overly formal manner.

[0024] The terms "first", "second", and the like as used herein are only intended to distinguish between similar objects, and are not used to describe a particular sequential or chronological order, nor are they used to indicate or imply relative importance.

[0025] The present application proposes a surface defect detection method based on a linear array camera, which combines Figure 1 As shown in the figure, the method comprises: S1, a collection step, using a linear array camera to collect a continuous image sequence of a surface to be detected; S2, a feature extraction step, inputting the continuous image sequence into a neural network model, the neural network model comprising a space-time fusion module and a dynamic upsampling module, performing multi-frame feature extraction on the continuous image sequence by the space-time fusion module to obtain intermediate features representing the space-time evolution characteristics of defects; S3, a resolution enhancement step, performing resolution enhancement processing on the intermediate features by the dynamic upsampling module to obtain enhanced features; S4, a detection output step, outputting a defect detection result based on the enhanced features; In this embodiment, the continuous image sequence can be understood as a group of image frames obtained in sequence at fixed time intervals during the passage of the measured object through the detection area, each frame being formed by the linear array camera accumulating a number of rows of data at a preset exposure time and line frequency, and the trigger period of the linear array camera can be synchronized with the encoder signal of the production line conveying mechanism to ensure that adjacent frames have a stable displacement relationship in physical space. In order to enable the space-time fusion module to cover the typical solder ball size and its surrounding background, the number of continuous frames used for single time series modeling in this embodiment is set to about 8 to 16 frames by default, which can be adjusted within the range of not less than 4 frames and not more than 32 frames according to the production line speed, camera line frequency, and length of the area to be detected, and the adjustment basis is to ensure that a single solder ball is covered by at least a number of frames while taking into account the requirement for calculation delay. In the detection output step, target detection decoding can be realized based on existing one-stage detection heads, and the internal threshold can be determined by an offline verification dataset, for example, the threshold for filtering low-confidence candidate boxes is initially set between 30% and 70%, and then fine-tuned according to the actual false negative and false positive situations. In engineering implementation, when a frame of image is severely underexposed or has obvious stripe noise, causing the basic quality detection to fail, the frame can be optionally removed from the continuous image sequence, or the features of the last normal image are used instead, to ensure that the subsequent space-time fusion and detection output steps still have stable input and usable detection results in extreme cases.

[0026] In one embodiment, during the acquisition step, the line scan camera scans along the moving direction of the surface to be detected, acquires image data line by line during the relative movement of the object being measured, and stitches adjacent image lines into multiple frames in the order of acquisition time to form a continuous image sequence. In one embodiment, the spatiotemporal fusion module is implemented based on the Transformer architecture, which includes a temporal attention layer. The temporal attention layer models the inter-frame relationships in a continuous image sequence to extract temporal features that reflect the changes of defects over time. When the temporal attention layer models the inter-frame relationships in a continuous image sequence, it is implemented in the spatiotemporal fusion module as follows: For a continuous image sequence formed by a line scan camera during the relative motion of the object being measured, a two-dimensional spatial feature map is first extracted in each frame, and then a temporal feature line is constructed along the time dimension at each spatial location. The temporal feature line is then weighted and aggregated by the temporal attention layer to obtain a temporal feature representation of the evolution of defects over time. In one implementation, for a continuous image sequence with a time index of... The temporal feature aggregation of the target frame at a given spatial location can be represented as: , in, This indicates that under the time attention branch, the time index is... The temporal feature vector of the target frame at its current spatial location. This represents the time index of the target frame in a continuous image sequence, and is no less than 1 and no greater than 1. integers, This indicates that the feature or weight belongs to the temporal attention branch, used to distinguish it from the spatial attention branch. This represents the number of consecutive frames involved in the modeling process during temporal attention computation. The frame index used as a reference is not less than 1 and not greater than 1. integers, This indicates that, under the temporal attention branch, the target frame index is... With reference frame index The temporal attention weights between reference frames are used to measure the proportion of contribution of the reference frames in the temporal feature aggregation. This indicates that in the time attention branch, the time index is... The value vector of the reference frame at the current spatial location is obtained by linear mapping from the local features of that frame; To obtain the above time attention weights Inter-frame relationships can be modeled using a scaled dot product attention approach that introduces a relative temporal bias, allowing adjacent frames and distant frames to have different weighting emphases. In one implementation, the weight calculation is expressed as: in, This indicates that, under the temporal attention branch, the target frame index is... With reference frame index Time attention weight, This represents the time index of the target frame in a continuous image sequence. This indicates the reference frame index that is of interest in the molecular part. This indicates that the corresponding vector, weights, and function belong to the temporal attention branch. This indicates that in the temporal attention branch, the target frame index is... The query vector is obtained by linearly mapping the features at the current spatial location of the frame. This indicates that in the time attention branch, the time index is... The reference frame key vector, The feature dimension of the temporal attention subspace is represented by a positive real number, used for scaling the dot product of the query vector and the key vector. This represents the relative time bias coefficient in the time attention branch. It is a learnable scalar used to control the influence of the time interval term in the total weight. The time interval mapping function in the temporal attention branch can, in one implementation, be a monotonically non-increasing function with respect to the input, used to decay larger time intervals. This indicates that in the temporal attention branch, the target frame index is... With reference frame index The normalized time interval between them can be obtained by linearly normalizing the difference between their indices according to a preset maximum time span. This represents the frame index that iterates through all reference frames in the summation term of the denominator, and is not less than 1 and not greater than... integers, This represents the number of consecutive frames that participate in normalization during the temporal attention computation. Through the aforementioned temporal attention weight calculation and temporal feature aggregation, at each spatial location, the temporal attention layer can adaptively adjust the contribution of each reference frame based on the similarity of inter-frame features and the time interval. This results in adjacent frames having a higher response to dynamic defect regions, while distant frames tend to provide more background stability information, thereby achieving [the desired effect]. Simultaneously incorporating short-term local changes and long-term contextual constraints; in engineering implementation, the above-mentioned temporal attention operation can be performed in parallel on all spatial locations on the entire frame feature map, and the resulting temporal enhanced feature map can be input into the spatial attention layer or dynamic upsampling module, so that the appearance, development and disappearance process of defects such as solder ball bridging, cold solder joints, missing parts and surface microcracks on the continuous scanning trajectory can be prominently expressed in the feature space; Specifically, the modeling of inter-frame relationships here involves constructing feature sequences along the temporal dimension at each spatial location and performing weighted aggregation through attention weights. By assigning different weights to each time frame, the model can simultaneously reference information from multiple frames at the same location, allowing defect signals that change abruptly in a short time and stable background patterns over a long time to participate in feature formation. After introducing a bias term related to the time interval, neighboring frames closer to the target frame are more likely to receive higher weights, more sensitively reflecting the rapid changes in solder ball surface defects during the scanning process, while farther frames are mainly used to provide global background and noise suppression effects. The temporal attention layer encodes temporal correlations into the feature representation in this way. In conjunction with the subsequent spatial attention and resolution enhancement modules, it can enhance the sensitivity to small, low-contrast, and intermittent defects while maintaining detection stability, providing more discriminative feature inputs for subsequent defect location and category determination. Specifically, in the implementation of the temporal attention layer, the consecutive frames involved in the calculation are typically selected as a fixed-length time window centered on the target frame or primarily along the forward temporal direction. The length of this time window can be set based on the average dwell time of the tested solder ball in the camera's field of view; in this embodiment, it is set to approximately 10 frames by default, and can be adjusted within the range of 8 to 16 frames according to the process cycle. The time interval mapping function can be implemented in engineering using a lookup table or a lightweight nonlinear transformation. Its output range can be limited to 0-1, maintaining the monotonicity that the larger the time interval, the smaller the function output. The relative time bias coefficient can be selected within a limited positive range during the offline parameter tuning stage through grid search or expert experience, so that adjacent frames receive significantly higher weights in attention allocation than distant frames. The linear mapping layer used to generate query vectors and key vectors can compress or expand the input features to a medium-sized feature dimension, which in this embodiment is typically selected to be between tens and hundreds, in order to control the computational load while ensuring expressive power. For frames located at the edge of the time window, due to insufficient available historical or future frames, the sequence length can be maintained by shortening the window or copying and padding missing frames. The normalization of the time interval can be achieved by mapping the frame index difference to a dimensionless value between 0 and 1 based on the actual inter-frame sampling period. When strong single-point noise occurs in several consecutive frames and spatial consistency is not met, a simple threshold-based anomaly masking logic can be optionally added before the temporal attention layer to mark the obviously abnormal feature locations as low-weight inputs, so as to avoid the anomalies being mistakenly amplified in multi-frame aggregation.

[0027] In one embodiment, the spatiotemporal fusion module further includes a spatial attention layer, which is used to assign higher weights to suspected defect regions within each frame of the image to enhance the feature representation ability of small defects and low-contrast regions. In one embodiment, the dynamic upsampling module is integrated into the feature pyramid network. The dynamic upsampling module performs upsampling processing on the low-resolution feature map through adaptive interpolation operation to improve the spatial resolution of the feature map while maintaining the feature information of small targets. When the dynamic upsampling module performs adaptive interpolation on the low-resolution feature map, the process is refined into two closely linked steps in the feature pyramid network: interpolation weight prediction and weighted upsampling reconstruction. The dynamic upsampling module first predicts a set of interpolation weights that are adaptive to the local structure for each spatial location on the low-resolution feature map, and then uses the set of interpolation weights to perform weighted combination of neighborhood features to generate the corresponding high-resolution output pixels, thereby preserving small targets and weak texture details while amplifying the spatial resolution. In the interpolation weight prediction stage, the position index in the low-resolution feature map can be... The local feature vector is input to the weight prediction branch to obtain the adaptive interpolation kernel weight vector corresponding to that position, which can be expressed as: , in, This indicates that in the adaptive upsampling branch, the spatial location index of the low-resolution feature map is... The interpolation weight vector has a length corresponding to the number of sampling points of the preset interpolation kernel. This represents the discrete spatial index of the low-resolution feature map in the vertical direction. This represents the discrete spatial index of the low-resolution feature map in the horizontal direction. This indicates that the corresponding value belongs to the adaptive upsampling branch, used to distinguish it from other network branches. This represents an adaptive interpolation weight prediction function, which in one implementation can consist of several convolutional layers and nonlinear activation units, used to map local features to normalized interpolation weights. This indicates that in the low-resolution feature map branch, the spatial location index is... The local feature vector can be obtained by concatenating the multi-channel feature values ​​at that location with its neighborhood information. This indicates that the corresponding quantity belongs to the low-resolution feature map branch; During the upsampling reconstruction stage, the position of a certain output pixel in the high-resolution feature map can be indexed as... The points correspond to the anchor points in the low-resolution feature map. And select within the neighborhood of that anchor point For each sampling point, the features of these sampling points are weighted and summed using the aforementioned interpolation weights to obtain the value of the high-resolution output pixel in the feature dimension. In one implementation, this can be represented as: in, This indicates that in the adaptive upsampling branch, the spatial location index of the high-resolution feature map is... Channel index is The output feature value, This represents the discrete spatial index of the high-resolution feature map in the vertical direction. This represents the discrete spatial index of the high-resolution feature map in the horizontal direction. This indicates that the corresponding quantity belongs to the high-resolution feature map branch. An index representing a feature channel, used to distinguish different semantic channels. This indicates the dimension of the feature channel corresponding to the subscript. The value represents the size of the adaptive interpolation kernel in a single spatial dimension, and is a positive integer. The total number of sampling points is... , This represents the discrete index in the direction perpendicular to the interpolation kernel, and is no less than 1 and no greater than 1. integers, This represents the discrete index along the horizontal direction of the interpolation kernel, and is no less than 1 and no greater than 1. integers, This indicates the low-resolution anchor point position in the adaptive upsampling branch. and interpolation kernel index The corresponding interpolation weight scalar, the value of which comes from the vector. The unfolding and rearrangement, This indicates that in the low-resolution feature map branch, the spatial location index is... Channel index is The input feature values, Indicates the vertical index of the interpolation kernel The corresponding vertical offset is a preset integer used to offset the data from the anchor point. Selecting neighboring sampling points upwards or downwards, Indicates the horizontal index of the interpolation kernel. The corresponding horizontal offset is a preset integer used to offset the anchor point. Select neighboring sampling points to the left or right. This indicates that the corresponding quantity belongs to the low-resolution feature map branch. This indicates that the corresponding quantity belongs to the adaptive upsampling branch; high-resolution index. With low-resolution anchor index A correspondence is established based on a fixed magnification factor. In implementation, the source anchor point can be determined by dividing the high-resolution index into integers. By combining the above-mentioned adaptive interpolation weight prediction and weighted upsampling reconstruction, the dynamic upsampling module generates content-related interpolation kernels for different spatial locations in the feature pyramid network, so that complex texture areas, small-sized solder balls and low-contrast defect areas still maintain clear structural expression in the upsampled feature map, which is beneficial for the subsequent defect detection head to distinguish defect types such as bridging, cold solder joints, missing parts and surface microcracks. Specifically, this section refines the adaptive interpolation operation, breaking down the internal mechanism of the dynamic upsampling module into two closely related processes: weight prediction and weighted reconstruction. Each location in the low-resolution feature map first extracts a local description through a dedicated branch, then maps it to a set of interpolation weights. These weights are closely related to the structure and texture of the surrounding neighborhood, thus automatically obtaining differentiated interpolation kernels for different regions. Based on this, each output pixel in the high-resolution feature map no longer relies on a fixed bilinear or deconvolution template, but instead returns to its corresponding low-resolution anchor point. A set of sampling points is selected from the anchor point's neighborhood, and linearly combined using the previously predicted interpolation weights to complete the upsampling reconstruction. In this way, the network can generate interpolation methods that better match the local morphology for sensitive areas such as solder ball edges, small cracks, and slight bridging, reducing the blurring and detail loss caused by traditional fixed-kernel interpolation. When this upsampling result is transferred and fused across different scales of the feature pyramid, it can provide richer and clearer spatial cues for small target defects, thereby improving the overall defect detection performance in conjunction with the spatiotemporal fusion module and the detection head. Furthermore, in the dynamic upsampling module, the spatial sampling interval of the low-resolution feature map can be determined based on the level of the feature pyramid and the initial resolution of the input image. In this embodiment, the magnification factor corresponding to the key level at the solder ball scale is usually set to two or four times. The side length of the interpolation kernel can be configured as an odd number between 3 and 5 to cover sufficient local neighborhood while avoiding excessive smoothing introduced by an overly large receptive field. The interpolation weight prediction function can be in the form of several layers of convolution and nonlinear activation units stacked sequentially. Its receptive field coverage is preferably slightly larger than the neighborhood of the interpolation kernel so as to take into account local texture trends and edge directions when predicting weights. The output interpolation weights can be normalized to ensure that the sum of all weights at each high-resolution location is one. The neighborhood sampling points used for upsampling reconstruction can be obtained by indexing on the low-resolution feature map with integer offsets during actual calculation. When the target sampling position is close to the edge of the feature map, the missing neighborhood points can be handled by boundary mirroring, boundary copying, or cropping to avoid out-of-bounds access and ensure that each output position can be obtained by combining complete weights and input features. If real-time performance is more critical in certain production line configurations, while the ability to distinguish extremely small defects is relatively less critical, the interpolation kernel size of the dynamic upsampling module can be configured to a smaller value, or in scenarios with extremely limited resources, it can be degraded to a fixed-weight interpolation method, while keeping the amplification factor and the number of output feature channels unchanged. This reduces the computational load and maintains the existing interface constraints without changing the overall network structure.

[0028] In one embodiment, the feature pyramid network includes a multi-scale fusion layer that fuses feature maps from different levels in a top-down and lateral manner to generate an enhanced feature representation for defect detection. In one embodiment, the neural network model is a pre-trained model. When training the neural network model, a data augmentation strategy is used to generate synthetic defect samples. The synthetic defect samples and real collected samples are used together for training to expand the distribution of training data and improve the model's ability to detect minute defects. In one embodiment, the output detection step includes: The enhanced features are decoded for target detection to obtain the spatial location information and defect category information of the defect region, and to generate structured detection information including defect location coordinates, defect category labels and corresponding confidence scores. In one embodiment, the method is applied to the solder ball defect detection scenario of printed circuit boards. The surface to be detected includes a micro solder ball structure arranged on the pads. The defect detection results include defect types such as solder ball bridging, cold solder joints, missing solder balls, or surface microcracks. In one embodiment, the output detection step further includes: Multiple detection results located in the same physical region in a continuous image sequence are subjected to temporal correlation and consistency constraint processing. Duplicate or critical detections are merged and screened according to a preset confidence threshold and temporal consistency rules to improve the stability of defect detection results. For example, in temporal correlation and consistency constraint processing, the defect candidate regions detected in each frame can be regarded as target instances with location information, category information, and confidence level. By comparing the spatial proximity and overlap of candidate regions between adjacent frames or multiple adjacent frames, candidate regions whose center positions are within a preset pixel range and whose overlap area ratio exceeds a preset lower limit are grouped into the same temporal tracking chain. The confidence levels of each frame in the same tracking chain can be weighted averaged or the maximum value is taken as the final result. In this embodiment, the threshold for filtering low-confidence candidates can be initially set in the range of 30%-70%, and fine-tuned based on the actual production line missed detection and false detection statistics; the lower limit for judging the spatial overlap ratio of the same physical region can be selected according to the projected area of ​​the solder ball in the image, generally between 30%-60%, so as to suppress the erroneous association of irrelevant candidates without missing slight positional drift. When a candidate region appears only in a single frame or a few consecutive frames and its confidence level is consistently lower than the typical level of a stable defect, it can be optionally identified as a critical detection and marked as a low-priority alarm, or suppressed in the final output. When no candidate is detected in the intermediate frame but a stable detection chain exists in the preceding and following frames, the continuity of the tracking chain can be maintained by allowing a small number of consecutive missing frames, thereby avoiding the mistaken identification of a temporary stop caused by accidental occlusion or local noise as the disappearance of the defect.

[0029] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0030] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of this application and form different embodiments. For example, all the embodiments above can be used in any combination. The information disclosed in this background section is intended only to enhance the understanding of the general background of this application and should not be construed as an admission or in any way implying that such information constitutes prior art known to those skilled in the art.

Claims

1. A surface defect detection method based on a linear array camera, characterized by, The method comprises: S1, a collection step, using a linear array camera to collect a continuous image sequence of a surface to be detected; S2, a feature extraction step, inputting the continuous image sequence into a neural network model, the neural network model comprising a space-time fusion module and a dynamic upsampling module, performing multi-frame feature extraction on the continuous image sequence by the space-time fusion module to obtain intermediate features representing the space-time evolution characteristics of defects; S3, a resolution enhancement step, performing resolution enhancement processing on the intermediate features by the dynamic upsampling module to obtain enhanced features; S4, a detection output step, outputting a defect detection result based on the enhanced features.

2. The line array camera-based surface defect detection method of claim 1, wherein, In the collection step, the linear array camera scans along the movement direction of the surface to be detected, and image data is collected row by row during the relative movement of the measured object, and adjacent image rows are spliced in time sequence to form a plurality of images to form the continuous image sequence.

3. The line array camera-based surface defect detection method of claim 2, wherein, The space-time fusion module is implemented based on a Transformer architecture, and the Transformer architecture comprises a time attention layer, which models the inter-frame relationship in the continuous image sequence; When the time attention layer models the inter-frame relationship in the continuous image sequence, the modeling is implemented in the space-time fusion module in the following manner: for the continuous image sequence formed by the linear array camera during the relative movement of the measured object, two-dimensional spatial feature maps are first extracted on each frame, then a time feature line is constructed along the time dimension at each spatial position, and the time feature line is weighted and aggregated by the time attention layer to obtain a time sequence feature representation representing the evolution of defects over time.

4. The line array camera-based surface defect detection method of claim 3, wherein, The space-time fusion module further comprises a spatial attention layer, which is used to give a higher weight to the suspected defect area in each frame of image.

5. The line camera based surface defect detection method of claim 1, wherein, The dynamic upsampling module is integrated in a feature pyramid network, and the dynamic upsampling module performs upsampling processing on the low-resolution feature map through adaptive interpolation operation; When the dynamic upsampling module performs adaptive interpolation on the low-resolution feature map, the process is refined into interpolation weight prediction and weighted upsampling reconstruction steps in the feature pyramid network; the dynamic upsampling module first predicts a set of interpolation weights adaptive to the local structure for each spatial position on the low-resolution feature map, then uses the set of interpolation weights to weight and combine the neighborhood features to generate corresponding high-resolution output pixels, thereby preserving small targets and weak texture details while enlarging the spatial resolution.

6. The line array camera-based surface defect detection method of claim 5, wherein, The feature pyramid network comprises a multi-scale fusion layer, which performs multi-scale fusion of feature maps from different levels in a top-down and horizontal connection manner.

7. The line camera based surface defect detection method according to any one of claims 1 to 6, wherein The neural network model is a pre-trained model, and during training of the neural network model, a data augmentation strategy is used to generate synthetic defect samples, and the synthetic defect samples and real acquisition samples are used together for training.

8. The line camera based surface defect detection method of claim 1, wherein, The detection output step comprises: performing target detection decoding on the enhanced features to obtain spatial position information and defect category information of the defect area, and generating structured detection information comprising defect position coordinates, defect category labels and corresponding confidence scores.

9. The line camera based surface defect detection method according to any one of claims 1 to 8, wherein, The method is applied to a solder ball defect detection scene of a printed circuit board, the surface to be detected includes a micro solder ball structure arranged on a pad, and the defect detection result includes a bridge, a virtual solder, a missing, or a surface micro crack defect type of the solder ball.

10. The line camera based surface defect detection method according to any one of claims 1 to 9, wherein, The detection output step further includes: The multiple detection results located in the same physical region in the continuous image sequence are subjected to time sequence association and consistency constraint processing, and repeated detection or critical detection is combined and screened according to a preset confidence threshold and a time sequence consistency rule.

Images