Large-format object surface defect detection method, device and system based on ZYNQ

Abstract

This invention discloses a method, apparatus, and system for detecting surface defects on large-format objects based on ZYNQ, comprising: a ZYNQ processing module receiving real-time surface images of the object under test sent by a linear scan camera; parsing the real-time surface images to obtain real-time image data; extracting the real-time image data and comparing it with preset reference data to obtain difference data; identifying defect image data based on the difference data and multiple preset thresholds; fusing the multi-field defect image data to obtain a preprocessed image; and sending the preprocessed image to a host computer. This invention preprocesses the surface image using the ZYNQ processing module to obtain a preprocessed image containing defects; this eliminates the need for the host computer to process the original surface image, allowing it to directly process the preprocessed image, reducing the workload of the host computer, lowering the performance requirements of the host computer, and thus reducing costs.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and in particular to a method, apparatus and system for detecting surface defects of large-format objects based on ZYNQ. Background Technology

[0002] Machine vision uses machines to replace human eyes for measurement and judgment, and it is widely used in defect detection of large-format materials. A machine vision system uses machine vision products to capture images of the surface of large-format materials, converting them into image signals and transmitting them to a dedicated image processing system. The image processing system performs correlation operations on the image data based on pixel grayscale values ​​and their distribution, extracts target features, and then obtains the corresponding defect detection results to monitor the surface condition of large-format materials in real time.

[0003] However, current defect detection equipment for inspecting and identifying defects on the surface of large-format materials typically employs an "industrial camera + image acquisition card + host computer" processing model. After the industrial camera acquires an image, the image acquisition card directly transmits it to the host computer for processing. This means all image processing is done on the host computer, placing high demands on the CPU (Central Processing Unit) and GPU (Graphics Processing Unit) performance, resulting in high costs. If preliminary image processing is required, a high-performance acquisition card is needed, which is also costly; furthermore, the acquisition card requires plug-and-play installation, leading to poor reliability. Large-format materials generally require 4-8 cameras to cover the inspection area while maintaining sufficient accuracy. The amount of raw data is enormous, placing an excessive load on the PC's CPU and GPU, making it difficult to meet real-time requirements. Multiple PCs with CPUs / GPUs are needed to handle the computational demands of such large amounts of data, further increasing costs.

[0004] Therefore, there is still an urgent need for a method for detecting surface defects of large-format objects that can reduce costs, improve real-time performance, and enhance reliability. Summary of the Invention

[0005] The main objective of this invention is to propose a method, apparatus, and system for detecting surface defects of large-format objects based on ZYNQ, thereby solving the problems of high cost, poor real-time performance, and poor reliability of existing defect detection equipment when processing large-format objects.

[0006] To achieve the above objectives, this invention proposes a method for detecting surface defects of large-format objects based on ZYNQ, the method comprising:

[0007] The ZYNQ processing module receives real-time surface images of the object under test sent by the line scan camera, and analyzes the real-time surface images to obtain real-time image data. The object under test is a large-format object. Each ZYNQ processing module is connected to one 8K line scan camera or two 4K line scan cameras. Multiple sets of line scan cameras and ZYNQ processing modules work in parallel to cover the detection area of ​​the large-format object.

[0008] Extract the real-time image data and compare it with preset reference data to obtain the difference data;

[0009] Defective image data is identified based on the difference data and multiple preset thresholds;

[0010] The defective image data from multiple fields are fused to obtain a preprocessed image, which is then sent to the host computer.

[0011] In some embodiments, extracting the real-time image data and comparing it with preset reference data to obtain difference data includes:

[0012] Several rows of image data are extracted, and the pixel gray values ​​are averaged using a filtering algorithm to obtain reference row data that reflects the background information of the image. The filtering algorithm includes statistical averaging and moving average.

[0013] The grayscale value of the real-time image data is compared with the grayscale value of the reference row data to obtain the difference data.

[0014] In some embodiments, identifying defective image data based on the difference data and multiple preset thresholds includes:

[0015] The absolute value of the difference data is used to obtain the calculation result;

[0016] The calculation result is compared with multiple preset thresholds to obtain a comparison result, and the defective image data is identified based on the comparison result.

[0017] In some embodiments, the step of combining and comparing the calculation result with multiple preset thresholds to obtain a comparison result, and identifying the defective image data based on the comparison result, includes:

[0018] If the calculation result is greater than T1, the surface image corresponding to the difference data is identified as an overexposed image;

[0019] If the calculation result is less than T0, the surface image corresponding to the difference data is identified as an interfered image;

[0020] If the calculation result is greater than T0 and less than T1, then the surface image corresponding to the difference data is identified as a valid defect image, and the defect image data is obtained.

[0021] Wherein, T1 is the grayscale reference value of the real-time image data, and T0 is less than T1.

[0022] In some embodiments, fusing the defect image data from multiple fields to obtain a preprocessed image includes:

[0023] The surface of the object under test is illuminated from multiple angles by a striped light source to obtain surface images from each angle, thereby obtaining multiple defect image data.

[0024] The preprocessed image is generated by fusing multiple defect image data.

[0025] In some embodiments, the multi-field defect image data is fused using one or more of the following methods to generate a preprocessed image:

[0026] Maximum value fusion method: For each pixel position, take the maximum real-time row gray value in all scenes as the fusion result;

[0027] Minimum value fusion method: For scenes where the stripes do not overlap, compare their reference row gray values ​​and select the real-time row gray value of the scene with the smaller value as the fusion result;

[0028] Additive synthesis method: For each pixel position, calculate the sum of gray values ​​of two adjacent fields, and take the maximum or minimum value from all sums as the fusion result;

[0029] Average synthesis method: For each pixel location, after removing the maximum and minimum values ​​in all fields, the average value of the remaining grayscale values ​​is taken as the fusion result;

[0030] Weighted fusion method: Calculate the weights based on the sharpness and contrast of the images in each session, and then perform a weighted average of all sessions to obtain the fusion result.

[0031] In some embodiments, after generating the preprocessed image, the method further includes:

[0032] Flat field correction: Using the gray value curve of the reference row image, the correction coefficient is calculated, and flat field correction is performed on the preprocessed image to eliminate illumination non-uniformity and obtain corrected image data;

[0033] Filtering and edge detection: Mean filtering is applied to the corrected image data to suppress noise, and the Sobel operator is used for edge detection. An edge detection threshold is set, significant edge features are extracted, and the edge detection results are obtained.

[0034] Image segmentation: The edge detection results are subjected to threshold segmentation and binarization to obtain a black and white binary image;

[0035] Defect analysis: Connected component labeling is performed on the black and white binary image, Blob feature extraction is performed, and the defect location coordinates are recorded to obtain defect feature data. The defect feature data includes the location coordinates and circumscribed rectangle of the defect region, the shape feature parameters of the defect region, and the corresponding original grayscale image data.

[0036] This invention also proposes a large-format object surface defect detection device based on ZYNQ, comprising:

[0037] At least one ZYNQ processing module, the ZYNQ processing module comprising a core board and a carrier board;

[0038] The core board contains a ZYNQ chip, as well as multiple DDR memory, EMMC memory and QSPI Nor Flash memory connected to the chip;

[0039] The carrier board includes a Camera Link interface for line scan camera input, a gigabit network interface, I / O input / output interfaces, a clock circuit, a power interface, and a power management circuit.

[0040] The memory includes DDR memory, EMMC memory, and QSPI Nor Flash memory. The memory stores instructions that are executed by the at least one ZYNQ processing module. The instructions are executed by the at least one ZYNQ processing module to enable the at least one ZYNQ processing module to perform the ZYNQ-based large-format object surface defect detection method according to any one of claims 1 to 7.

[0041] The present invention also proposes a large-format object surface defect detection system based on ZYNQ, the large-format object surface defect detection system based on ZYNQ includes a line scan camera, a host computer and a ZYNQ processing module; multiple line scan cameras and multiple ZYNQ processing modules are configured, and the ZYNQ processing module is the ZYNQ processing module described above.

[0042] In some embodiments, the ZYNQ module and the host computer use dual-port gigabit network communication. One port is used to transmit various commands and configuration parameters between the ZYNQ processing module and the host computer, and the other port is used to transmit image data to the host computer.

[0043] This invention uses a ZYNQ processing module to preprocess the surface image of the object under test, obtaining a preprocessed image containing defect data. This eliminates the need for the host computer to process the original surface image, allowing it to directly process the preprocessed image, reducing the workload and performance requirements of the host computer, thereby lowering costs. Preprocessing the surface image of the object under test accelerates the next step of the host computer's inspection process and improves the system's real-time performance. Furthermore, the ZYNQ processing module performs preliminary processing of the surface image, eliminating the need to plug and unplug high-performance acquisition cards, thus improving system reliability. Attached Figure Description

[0044] Figure 1 This is a schematic flowchart of the large-format object surface defect detection method based on ZYNQ according to the present invention;

[0045] Figure 2 This is another flowchart illustrating the ZYNQ-based method for detecting surface defects in large-format objects according to the present invention.

[0046] Figure 3 This is another flowchart illustrating the ZYNQ-based method for detecting surface defects in large-format objects according to the present invention.

[0047] Figure 4 This is another flowchart illustrating the ZYNQ-based method for detecting surface defects in large-format objects according to the present invention.

[0048] Figure 5 This is another flowchart illustrating the ZYNQ-based method for detecting surface defects in large-format objects according to the present invention.

[0049] Figure 6 This is a schematic diagram illustrating the application scenario of the large-format object surface defect detection method based on ZYNQ according to the present invention.

[0050] Figure 7 This is a schematic diagram of the structure of the large-format object surface defect detection device based on ZYNQ according to an embodiment of the present invention.

[0051] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0052] The solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0054] It should also be noted that when a component is described as "fixed to" or "set on" another component, it can be directly on the other component or there may be an intervening component present. When a component is described as "connected to" another component, it can be directly connected to the other component or there may be an intervening component present.

[0055] Furthermore, the use of terms such as "first" and "second" in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed by this invention.

[0056] To achieve the above objectives, this invention proposes a method for detecting surface defects in large-format objects. The method includes:

[0057] Step S110: The ZYNQ processing module receives the real-time surface image of the object under test sent by the line scan camera, analyzes the real-time surface image, and obtains real-time image data.

[0058] Among them, the object to be tested is a large-format object. Each ZYNQ processing module is connected to one 8K line scan camera or two 4K line scan cameras. Multiple sets of line scan cameras and ZYNQ processing modules work in parallel to cover the detection area of ​​the large-format object.

[0059] Step S120: Extract real-time image data and compare the real-time image data with preset reference data to obtain difference data;

[0060] Step S130: Identify defective image data based on the difference data and multiple preset thresholds;

[0061] Step S140: Fuse the multi-field defect image data to obtain a preprocessed image, and send the preprocessed image to the host computer.

[0062] In some embodiments, refer to Figure 1 and Figure 6The method for detecting surface defects of large-format objects is applied to a large-format object surface defect detection system. The large-format object surface defect detection system may include a line scan camera, a host computer, and a ZYNQ processing module. Each ZYNQ processing module is connected to one 8K line scan camera or two 4K line scan cameras. Multiple sets of line scan cameras and ZYNQ processing modules work in parallel to cover the detection area of ​​the large-format object.

[0063] It should be noted that the ZYNQ processing module of the present invention has the following characteristics:

[0064] Processing capacity: 8K mode, one CameraLink Full interface connects to one 8K line scan camera; Dual 4K mode, two CameraLink Base interfaces connect to two 4K line scan cameras.

[0065] Interface configuration: CameraLink interface (Base / Medium / Full); Dual Gigabit Ethernet ports;

[0066] Storage support: 1GB DDR3L on PS side + 1GB DDR3L on PL side + 1GB EMMC + QSPI Nor Flash.

[0067] The object under test is a large-format object, which can be a photovoltaic glass, lithium battery separator, fabric, roll material, or other objects with a large surface area. Because the surface area of ​​the object under test is relatively large, it is difficult for a single line scan camera to fully cover the width of the object's surface when taking pictures. In this case, multiple line scan cameras are needed to take pictures simultaneously to fully cover the width of the object's surface.

[0068] The ZYNQ processing module can perform preliminary processing on surface images captured by a line scan camera to obtain defect image data and send it to the host computer. This eliminates the need for the host computer to process the raw surface images; it can directly process the defect image data, reducing the workload and performance requirements of the host computer, thereby lowering costs. Furthermore, multiple ZYNQ processing modules can be configured; for example, one ZYNQ processing module can process surface images captured by one line scan camera. Multiple ZYNQ processing modules can improve the efficiency of surface image processing.

[0069] It is understood that, in this embodiment, the ZYNQ processing module is a scalable processing module that tightly integrates the software programmability of the processor with the hardware programmability of the field-programmable gate array.

[0070] First, the line scan camera needs to capture real-time images of the object under test to obtain real-time surface images. Then, the line scan camera sends the real-time surface images to the ZYNQ processing module. After receiving the real-time surface images, the ZYNQ processing module performs serial parsing and serial-to-parallel conversion on the real-time surface images in real time to reconstruct the data structure of the real-time surface images and obtain the real-time image data.

[0071] In a preferred embodiment, the line scan camera is connected to the ZYNQ processing module via a Camera Link interface.

[0072] Understandably, in this embodiment, the line scan camera can send real-time surface images to the ZYNQ processing module via the Camera Link Base / Medium / Full transmission protocol. The ZYNQ processing module serially parses the real-time surface images according to the Camera Link Base / Medium / Full transmission protocol; then, it performs serial-to-parallel data conversion to reassemble the data structure of the real-time surface images to obtain real-time image data.

[0073] The ZYNQ processing module incorporates DDR (Double Data Rate) memory. After receiving real-time image data, the ZYNQ processing module first stores the real-time image data in the DDR memory. The DDR memory also stores preset reference data. The ZYNQ processing module then retrieves the real-time image data from the DDR memory and compares it with the preset reference data to obtain the difference data.

[0074] The ZYNQ processing module identifies and judges the difference data and multiple preset thresholds to obtain defect image data. It also fuses multi-field defect image data to obtain a pre-processed image. Multi-field defect image data refers to surface images acquired from various angles of a striped light source illuminating the surface of the object under test, which are then processed to obtain the defect image data. Finally, the pre-processed image is sent to the host computer.

[0075] The host computer processes the defect image data to obtain defect information such as defect type, defect location, and defect size. The host computer also displays this defect information so that the user can process the object under test based on the defect information.

[0076] This embodiment uses the ZYNQ processing module to preprocess the surface image of the object under test, obtaining a preprocessed image containing defect data. This eliminates the need for the host computer to process the original surface image, allowing it to directly process the preprocessed image, reducing the workload and performance requirements of the host computer, thereby lowering costs. Preprocessing the surface image of the object under test accelerates the next step of the host computer's inspection work and improves the system's real-time performance. Furthermore, the ZYNQ processing module can perform preliminary processing of the surface image, eliminating the need to plug and unplug high-performance acquisition cards, thus improving the system's reliability.

[0077] In some embodiments, the extraction of real-time image data and the comparison of the real-time image data with preset reference data to obtain difference data includes:

[0078] Step S150: Extract several rows of image data, and use a filtering algorithm to average the pixel gray values ​​to obtain reference row data that reflects the background information of the image. The filtering algorithm includes statistical averaging and moving average.

[0079] Step S151: Compare the grayscale values ​​of the real-time image data with the grayscale values ​​of the reference row data to obtain the difference data.

[0080] In this embodiment, refer to Figure 2 After obtaining real-time image data, the ZYNQ processing module first stores the real-time image data in DDR memory. When real-time image data from multiple time periods are all stored in DDR memory, historical image data is formed. The ZYNQ processing module can then extract several rows of image data from the historical image data.

[0081] For example, the ZYNQ processing module extracts 1000 rows of image data from historical image data; it uses a filtering algorithm (measurement noise covariance R = 0.1, process noise covariance Q = 0.01) to average the gray values, obtaining reference row data reflecting image background information such as light source, lens, and environment; the reference row data includes standard gray values ​​of intact images under different image background conditions, and the filtering algorithm is not limited to statistical averaging, moving average, etc. When there are defects on the surface of the object under test, the gray values ​​of the image data corresponding to the defects will change. The gray values ​​of the real-time image data are compared with the gray values ​​of the reference row data to obtain the difference data.

[0082] In some embodiments, the identification of defective image data based on difference data and multiple preset thresholds includes:

[0083] Step S160: Perform an absolute value operation on the difference data to obtain the calculation result;

[0084] Step S161: Combine and compare multiple preset thresholds of the calculation results to obtain a comparison result, and identify defective image data based on the comparison result.

[0085] In this embodiment, refer to Figure 3 After obtaining the difference data, the ZYNQ processing module performs absolute value operations on the difference data to convert negative difference data into positive difference data, thus obtaining the calculation result. This result is then compared with multiple preset thresholds. These preset thresholds can be data obtained by the host computer through image processing. The host computer can also send multiple preset thresholds to the ZYNQ processing module for storage in DDR memory; this yields the comparison result. The comparison result is then used to identify defect image data.

[0086] In some embodiments, the foregoing comparison of the calculation result with multiple preset thresholds to obtain a comparison result, and the identification of defective image data based on the comparison result includes:

[0087] If the calculation result is greater than T1, the surface image corresponding to the difference data will be identified as an overexposed image.

[0088] If the calculation result is less than T0, the surface image corresponding to the difference data will be identified as the disturbed image.

[0089] If the calculation result is greater than T0 and less than T1, the surface image corresponding to the difference data is identified as a valid defect image, and defect image data is obtained.

[0090] Where T1 is the grayscale reference value of the real-time image data, and T0 is less than T1.

[0091] In this embodiment, T0 and T1 are obtained by the host computer through processing the standard image. For example, the host computer acquires image data, performs statistical analysis on the grayscale values ​​of the image data, and takes the average of the grayscale values ​​corresponding to the top 5% of grayscale values ​​as the threshold T1. The host computer collects 100 images as samples to obtain the corresponding image data; for each image, the pixel values ​​of the image data are subtracted from the pixel values ​​of the reference row data, adaptively calculated using the OTSU algorithm, and the average is used as the threshold T2. If the calculation result is greater than T1, the surface image corresponding to the difference data is identified as an overexposed image, and defect information cannot be extracted from overexposed images. If the calculation result is less than T0, the surface image corresponding to the difference data is identified as an interfered image, and defect information cannot be accurately extracted from interfered images. If the calculation result is greater than T0 and less than T1, the surface image corresponding to the difference data is identified as a valid defect image, and defect image data is obtained. By comparing the calculation result with the preset threshold, meaningful defect images can be effectively extracted, while irrelevant interference is filtered out, greatly improving the accuracy and efficiency of detection.

[0092] In some embodiments, the foregoing fusion of multi-field defect image data to obtain a preprocessed image includes:

[0093] Step S170: The striped light source illuminates the surface of the object under test from multiple angles to obtain surface images of each angle, so as to obtain multi-field defect image data.

[0094] Step S171: Fuse the multi-field defect image data to generate a preprocessed image.

[0095] In this embodiment, refer to Figure 4 Step S170 specifically involves the striped light source illuminating the surface of the object under test from multiple angles using a time-division strobe illumination sequence. A line scan camera acquires surface images under different illumination conditions. The ZYNQ processing module receives and processes these multiple images to obtain multi-field defect image data. When the light source module emits striped light to illuminate the surface of the object under test from multiple angles, it can more completely reveal the defect information of the object's surface. The striped light illuminates the surface of the object under test from multiple angles, acquiring a surface image at each angle to capture image defects, resulting in surface images from each angle. These surface images are then processed to obtain multi-field defect image data. The ZYNQ processing module fuses the defect image data to obtain a pre-processed image.

[0096] In some other embodiments, step S170 includes:

[0097] The object under test is illuminated by time-division stroboscopic light at various angles. The ZYNQ processing module receives surface images of the object under test from various angles of illumination sent by the linear scan camera. The surface images of each angle are analyzed to obtain image data. Each image data is extracted and compared with preset reference data to obtain difference data. Based on the difference data, multi-field defect image data is obtained.

[0098] In some embodiments, one or more of the following methods are used to fuse multi-field defect image data to generate a preprocessed image:

[0099] Maximum value fusion method: For each pixel position, take the maximum real-time row gray value in all scenes as the fusion result;

[0100] Minimum value fusion method: For scenes where the stripes do not overlap, compare their reference row gray values ​​and select the real-time row gray value of the scene with the smaller value as the fusion result;

[0101] Additive synthesis method: For each pixel position, calculate the sum of gray values ​​of two adjacent fields, and take the maximum or minimum value from all sums as the fusion result;

[0102] Average synthesis method: For each pixel position, after removing the maximum and minimum values in all sessions, the average value of the remaining gray values is taken as the fusion result;

[0103] Weighted fusion method: Weights are calculated based on the clarity and contrast of the images in each session, and the weighted average of all sessions is used as the fusion result.

[0104] In this embodiment, there are various fusion methods for fusing multi-session defect image data. According to the actual detection requirements and material characteristics, a single fusion method can be selected for fusion or multiple fusion methods can be combined for fusion. The fusion methods include the maximum value fusion method, the minimum value fusion method, the addition synthesis method, the average synthesis method, and the weighted fusion method.

[0105] Maximum value fusion method: For each pixel position, the maximum gray value in all sessions is taken as the fusion result. The mathematical expression of this method is F(x) = max{I1(x), I2(x),..., In(x)}, where: F(x) is the fusion result, Ii(x) is the gray value of the i-th session, and n is the total number of sessions. The advantage is that it can retain defect information to the greatest extent and is suitable for detecting微小 defects. The disadvantage is that it is easy to introduce environmental noise and is sensitive to light fluctuations.

[0106] Minimum value fusion method: For sessions with non-overlapping stripes, compare the gray values of their reference rows, and select the gray value of the real-time row of the session corresponding to the smaller value as the fusion result. The mathematical expression of this method is F(x) = {I1(x), when R1(x) < R2(x); I2(x), when R1(x) ≥ R2(x)}. Where: Ri(x) is the gray value of the reference row of the i-th session, and Ii(x) is the gray value of the real-time row of the i-th session. The advantage is that it can effectively suppress environmental interference and the detection result is stable. The disadvantage is that it may lose some defect information and is not suitable for detecting low-contrast defects.

[0107] Addition synthesis method: For each pixel position, calculate the sum of the gray values of adjacent sessions, and take the maximum or minimum value from all the sum values as the fusion result. The mathematical expression of this method is F(x) = max / min{S1(x), S2(x),..., Sn-1(x)} where: Si(x) = Ii(x) + Ii+1(x) is the sum of the gray values of adjacent sessions. The advantage is that it superimposes the defect gray information and enhances the defect response, and is suitable for detecting continuous defects. The disadvantage is that it may amplify local interference.

[0108] Average synthesis method: For each pixel location, after removing the maximum and minimum values ​​from all scenes, the average of the remaining grayscale values ​​is taken as the fusion result. The mathematical expression of this method is F(x)=mean{sort{I1(x),...,In(x)}[2:n-1]}, where: sort is the grayscale value sorting, [2:n-1] represents the sequence after removing the maximum and minimum values, and mean is the average value calculation. The advantage is that it balances the defect detection rate and anti-interference ability, and the result is the most stable. The disadvantage is that the defect response is not so obvious.

[0109] Weighted fusion method: This method calculates weights based on the sharpness and contrast of images from each scene, and then performs a weighted average across all scenes to obtain the fusion result. The mathematical expression for this method is F(x) = Σ(wi × Ii(x)), i = 1 to n; where wi is the weight of the i-th scene, calculated from sharpness Si and contrast Ci, and the sum of the weight coefficients is 1 (Σwi = 1). The advantage is that the fusion strategy can be adjusted according to image quality. The disadvantage is high computational complexity and the need for additional image quality assessment.

[0110] In some embodiments, after generating the preprocessed image, the method further includes:

[0111] Step S180, Flat field correction: Using the gray value curve of the reference row image, calculate the correction coefficient, perform flat field correction on the preprocessed image, eliminate illumination non-uniformity, and obtain corrected image data;

[0112] Step S181, Filtering and Edge Detection: The corrected image data is subjected to mean filtering to suppress noise, the Sobel operator is used for edge detection, an edge detection threshold is set, significant edge features are extracted, and the edge detection result is obtained.

[0113] Step S182, Image segmentation: Threshold segmentation and binarization are performed on the edge detection results to obtain a black and white binary image;

[0114] Step S183, Defect Analysis: Connected component labeling is performed on the black and white binary image, Blob feature extraction is performed, defect location coordinates are recorded, and defect feature data is obtained;

[0115] The defect feature data includes the location coordinates and circumscribed rectangle of the defect area, the shape feature parameters of the defect area, and the corresponding original grayscale image data.

[0116] In this embodiment, refer to Figure 5 After fusing multiple defect image data to obtain a preprocessed image, the ZYNQ processing module will also analyze the preprocessed image to extract defect information.

[0117] Using a reference row image of a defect-free sample as a flat-field image, the preprocessed image is corrected to eliminate the influence of factors such as uneven illumination and nonlinear camera response, thereby improving image quality and obtaining corrected image data. Specifically, F(x) = I(x) × M / R(x); where I(x) is the image to be corrected, R(x) is the gray value of the flat-field image, and M is the average gray value of the flat-field image.

[0118] The ZYNQ processing module can perform mean / median filtering and edge detection on the corrected image data using preset template sizes; for example, using template window sizes of 3×3, 5×5, and 7×7, mean / median filtering can be performed on the corrected image data. Sobel edge detection: A 3×3 operator is used to calculate the horizontal gradient Gx and vertical gradient Gy on the filtered image. The gradient magnitudes are calculated as the sum of their absolute values, G = |Gx| + |Gy|. A fixed threshold is set based on the image mean, and the Sobel calculation results are thresholded to extract defect edge information, thus obtaining the edge detection result.

[0119] The ZYNQ processing module performs adaptive thresholding and binarization on the edge detection results. It uses the OTSU algorithm to adaptively calculate the optimal threshold, or sets a fixed threshold based on experience. The edge detection results are then subjected to thresholding and binarization to obtain a black-and-white binary image. Specifically, the binarization process outputs 255 when the image grayscale value G(x) > Threshold, and 0 otherwise.

[0120] The ZYNQ processing module performs connected component identification and blob analysis on the black and white binary image to obtain defect feature data for fusion analysis. Specifically: Connected component labeling: 4-neighborhood connectivity analysis is used to label independent defect regions. Feature extraction: Location information, including starting point coordinates (X1, Y1) and ending point coordinates (X2, Y2); basic features, including area (total number of pixels), length (X2-X1), and width (Y2-Y1).

[0121] In a preferred embodiment, after performing step S173, the method further includes:

[0122] Determine whether the number and features of defects within the defect feature data are within the threshold range;

[0123] If the number and features of defects are less than the threshold, the image region corresponding to the defect feature data is determined to be a non-defect region, and the defect feature data is not sent to the host computer.

[0124] If the number and features of defects exceed the threshold, the image blob region corresponding to this defect feature data is defined as the defect region, and this defect feature data is sent to the host computer.

[0125] In this embodiment, after obtaining the defect feature data, the ZYNQ processing module can further determine the number and characteristics of defects in the defect feature data. When the number and characteristics of defects are less than a threshold, the image region corresponding to this defect feature data is determined to be a non-defect region, and the defect feature data is not sent to the host computer. When the number and characteristics of defects are greater than the threshold, the image blob region corresponding to this defect feature data is defined as a defect region, and the defect feature data is sent to the host computer.

[0126] In a preferred embodiment, the ZYNQ processing module includes a dual-port gigabit network communication unit. One port is dedicated to sending and receiving various commands and configuration parameters between the ZYNQ module system and the host computer, and the other port is dedicated to transmitting image data to the host computer. The image data may include pre-processed images.

[0127] The ZYNQ processing module sends image data to the host computer based on a dual-network port communication unit.

[0128] In this embodiment, after generating the preprocessed image, if the number of defects is not within the processable range, the ZYNQ processing module will not process the preprocessed image further and will directly send it to the host computer. At this time, the data volume of the preprocessed image is relatively large, and to quickly transmit the preprocessed image to the host computer, a dual-port gigabit network communication unit is required to send the data.

[0129] If the number of defects is within a manageable range, the ZYNQ processing module will further analyze and process the preprocessed image to obtain defect feature data. At this point, the defect feature data is further processed, which greatly reduces the amount of defect feature data.

[0130] The dual-port gigabit network communication unit of this invention is implemented using a ZYNQ chip, and specifically includes the following technical solutions:

[0131] The ZYNQ chip's processing system (PS) integrates two Gigabit Ethernet media access controllers (GMACs), which are connected to different physical layer (PHY) chips, forming a dual-port architecture; among which:

[0132] The first GMAC interface is connected to PHY chip 1 and establishes a reliable communication link with the host computer through gigabit network port 1. It is mainly used to receive control commands and configuration parameters issued by the host computer and to feed back execution status information to the host computer.

[0133] The second GMAC interface is connected to PHY chip 2 and is dedicated to high-capacity network data transmission via Gigabit Ethernet port 2. Specifically, after the ZYNQ chip's programmable logic (PL) section completes data acquisition and preprocessing, the processed data is transmitted to the PS section via the on-chip AXI high-speed bus, and then the preprocessed image data is transmitted to the host computer via Gigabit Ethernet port 2.

[0134] Alternatively, Gigabit Ethernet Port 2 can also directly implement the User Datagram Protocol (UDP) communication stack in the PL section to achieve direct data transmission with the host computer, further improving data transmission efficiency.

[0135] This dual-port separation design can effectively avoid mutual interference between control commands and large-capacity data transmission, improving the overall communication reliability and real-time performance of the system.

[0136] This invention uses a ZYNQ processing module to preprocess the surface image of the object under test, obtaining a preprocessed image containing defect data. This eliminates the need for the host computer to process the original surface image, allowing it to directly process the preprocessed image, reducing the workload and performance requirements of the host computer, thereby lowering costs. Preprocessing the surface image of the object under test accelerates the next step of the host computer's inspection process and improves the system's real-time performance. Furthermore, the ZYNQ processing module performs preliminary processing of the surface image, eliminating the need to plug and unplug high-performance acquisition cards, thus improving system reliability.

[0137] This invention also proposes a large-format object surface defect detection device based on ZYNQ, see reference. Figure 7 , Figure 7 This is a schematic diagram of the structure of a large-format object surface defect detection device based on ZYNQ in the hardware operating environment involved in the embodiments of the present invention.

[0138] The ZYNQ-based surface defect detection device for large-format objects in this embodiment of the invention can be a ZYNQ processing module capable of running a ZYNQ-based surface defect detection method for large-format objects. For example... Figure 7 As shown, the ZYNQ-based large-format object surface defect detection device may include:

[0139] At least one ZYNQ processing module, the ZYNQ processing module comprising a core board and a carrier board;

[0140] The core board contains a ZYNQ chip, as well as multiple DDR memory, EMMC memory and QSPI Nor Flash memory connected to the chip;

[0141] The carrier board includes a Camera Link interface for line scan camera input, a gigabit network interface, I / O input / output interfaces, a clock circuit, a power interface, and a power management circuit.

[0142] The memory includes DDR memory, EMMC memory, and QSPI Nor Flash memory. The memory stores instructions that are executed by the at least one ZYNQ processing module. The instructions are executed by the at least one ZYNQ processing module to enable the at least one ZYNQ processing module to execute any of the above-described ZYNQ-based methods for detecting surface defects of large-format objects.

[0143] In this embodiment, the core board consists of a ZYNQ chip (XC7Z035 / XC7Z045 / XC7Z100), which integrates the ARM processor PS section and the programmable logic PL section; DDR memory stores Linux system data and image data; EMMC memory stores the operating system and applications; and QSPI Nor Flash stores FPGA configuration files.

[0144] Carrier board: Camera Link interface and related circuitry, supporting Base / Medium / Full configurations for receiving image data from line scan cameras; dual Gigabit network interfaces for network communication with a host computer; power management circuitry to provide power to each module; clock circuitry to provide the clock for the system and individual chips; I / O input / output interfaces to receive I / O trigger signals from signal sources and to output I / O signals to control other devices.

[0145] The ZYNQ chip executes the program in its memory to implement the aforementioned method for detecting surface defects on large-format objects. The programmable logic (PL) section is mainly used for image data acquisition and preprocessing, while the ARM processor is responsible for software flow control, receiving parameter configurations from the host computer, and transmitting commands and data with the host computer over the network.

[0146] Those skilled in the art will understand that Figure 7 The structure of the ZYNQ-based large-format object surface defect detection device shown in the figure does not constitute a limitation on the ZYNQ-based large-format object surface defect detection device. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0147] The present invention also proposes a large-format object surface defect detection system based on ZYNQ. The large-format object surface defect detection system based on ZYNQ includes a line scan camera, a host computer, and a ZYNQ processing module. Multiple line scan cameras and multiple ZYNQ processing modules are configured. The ZYNQ processing module is the aforementioned ZYNQ processing module.

[0148] In this embodiment, refer to Figure 6The line scan camera is connected to the ZYNQ processing module, which in turn is connected to the host computer; the line scan camera can be configured as one or two. Figure 6 (Only one is shown in the image).

[0149] In some embodiments, the ZYNQ module and the host computer use dual-port gigabit network communication. One port is used to transmit various commands and configuration parameters between the ZYNQ processing module and the host computer, and the other port is used to transmit image data to the host computer.

[0150] In this embodiment, the ZYNQ module and the host computer use dual-port gigabit network communication. Specifically, the ZYNQ module communicates with the host computer through two gigabit network ports: one port is used to transmit various commands and configuration parameters between the ZYNQ processing module and the host computer, and the other port is used to transmit image data to the host computer. Multiple ZYNQ modules are connected to one host computer via a gigabit network switch.

[0151] Based on the computer program proposed in the foregoing embodiments, the present invention also proposes a storage medium storing the computer program, which, when executed by a controller, implements the ZYNQ-based method for detecting surface defects of large-format objects as described in the foregoing embodiments.

[0152] The present invention also proposes a storage medium storing a computer program, which, when executed by a processor, implements the steps of the ZYNQ-based method for detecting surface defects of large-format objects as described in any of the above technical solutions.

[0153] The above description is only a part or preferred embodiment of the present invention. Neither the text nor the drawings should limit the scope of protection of the present invention. All equivalent structural transformations made using the content of the present invention specification and drawings under the overall concept of the present invention, or direct / indirect applications in other related technical fields, are included within the scope of protection of the present invention.

Claims

1. A method for detecting surface defects of large-format objects based on ZYNQ, characterized in that, The ZYNQ-based method for detecting surface defects in large-format objects includes: The ZYNQ processing module receives real-time surface images of the object under test sent by the line scan camera, and analyzes the real-time surface images to obtain real-time image data. The object under test is a large-format object. Each ZYNQ processing module is connected to one 8K line scan camera or two 4K line scan cameras. Multiple sets of line scan cameras and ZYNQ processing modules work in parallel to cover the detection area of ​​the large-format object. Extracting the real-time image data and comparing it with preset reference data to obtain difference data specifically includes: extracting several rows of image data, averaging the pixel gray values ​​using a filtering algorithm to obtain reference row data reflecting image background information, the filtering algorithm including statistical averaging and moving average; comparing the gray values ​​of the real-time image data with the gray values ​​of the reference row data to obtain the difference data; The method for identifying defective image data based on the difference data and multiple preset thresholds specifically includes: performing an absolute value operation on the difference data to obtain a calculation result; comparing the calculation result with multiple preset thresholds to obtain a comparison result; and identifying the defective image data based on the comparison result, specifically including: if the calculation result is greater than T1, then the surface image corresponding to the difference data is identified as an overexposed image; if the calculation result is less than T0, then the surface image corresponding to the difference data is identified as an interfered image; if the calculation result is greater than T0 and less than T1, then the surface image corresponding to the difference data is identified as a valid defective image, thereby obtaining the defective image data; wherein, T0 and T1 are grayscale reference values ​​of real-time image data, and T0 is less than T1; A preprocessed image is obtained by fusing the defect image data from multiple fields. The step of fusing the defect image data from multiple fields to obtain the preprocessed image includes: The surface of the object under test is illuminated from multiple angles by a striped light source to obtain surface images from each angle, thereby obtaining multiple defect image data. The preprocessed image is generated by fusing the defect image data from multiple fields. After generating the preprocessed image, the process further includes: Flat field correction: Using the gray value curve of the reference row image, the correction coefficient is calculated, and flat field correction is performed on the preprocessed image to eliminate illumination non-uniformity and obtain corrected image data; Filtering and edge detection: The corrected image data is subjected to mean filtering to suppress noise, and the Sobel operator is used for edge detection. An edge detection threshold is set, significant edge features are extracted, and the edge detection results are obtained. Image segmentation: The edge detection results are subjected to adaptive threshold segmentation and binarization to obtain a black and white binary image; Defect analysis: Connected component labeling is performed on the black and white binary image, Blob feature extraction is performed, and the defect location coordinates are recorded to obtain defect feature data. The defect feature data includes the location coordinates and bounding rectangle of the defect region, the shape feature parameters of the defect region, and the corresponding original grayscale image data. The ZYNQ processing module, after obtaining the defect feature data, further includes: judging the number and features of defects in the defect feature data; when the number and features of defects are less than a threshold, the image region corresponding to the defect feature data is determined as a non-defect region and the defect feature data is not sent to the host computer; when the number and features of defects are greater than the threshold, the image Blob region corresponding to the defect feature data is defined as a defect region and the defect feature data is sent to the host computer. The ZYNQ processing module and the host computer use dual-port gigabit network communication. One port is used to transmit various commands and configuration parameters between the ZYNQ processing module and the host computer, and the other port is used to transmit defect feature data to the host computer.

2. The method for detecting surface defects of large-format objects based on ZYNQ according to claim 1, characterized in that, The fusion of multiple defect image data to generate a preprocessed image can be achieved using one or more of the following methods: Maximum value fusion method: For each pixel position, take the maximum real-time row gray value in all scenes as the fusion result; Minimum value fusion method: For scenes where the stripes do not overlap, compare their reference row gray values ​​and select the real-time row gray value of the scene with the smaller value as the fusion result; Additive synthesis method: For each pixel position, calculate the sum of gray values ​​of two adjacent fields, and take the maximum or minimum value from all sums as the fusion result; Average synthesis method: For each pixel location, after removing the maximum and minimum values ​​in all fields, the average value of the remaining grayscale values ​​is taken as the fusion result; Weighted fusion method: Calculate the weights based on the sharpness and contrast of the images in each session, and then perform a weighted average of all sessions to obtain the fusion result.

3. A large-format object surface defect detection device based on ZYNQ, characterized in that, include: ZYNQ processing module, the ZYNQ processing module includes a core board and a carrier board; The core board contains a ZYNQ chip, as well as multiple DDR memory, EMMC memory and QSPINor Flash memory connected to the chip; The carrier board includes a Camera Link interface for line scan camera input, a gigabit network interface, I / O input / output interfaces, a clock circuit, a power interface, and a power management circuit. The memory includes DDR memory, EMMC memory, and QSPI Nor Flash memory. The memory stores instructions that are executed by the ZYNQ processing module. The instructions are executed by the ZYNQ processing module to enable the ZYNQ processing module to perform the large-format object surface defect detection method based on ZYNQ as described in any one of claims 1 to 2.

4. The ZYNQ-based large-format object surface defect detection device according to claim 3, characterized in that, It also includes a line scan camera and a host computer, and both the line scan camera and the ZYNQ processing module are configured with multiple sets.

Images