Lithium electrode plate burr high-speed detection method based on DPU heterogeneous platform

Abstract

The invention discloses a lithium electrode plate burr high-speed detection method based on a DPU heterogeneous platform, and relates to the technical field of automatic detection, the method is used in the process that an industrial camera transmits an image to a host through a network card, the industrial camera reads out a single image line by line after image exposure and transmits the image to the network card, and the image is detected through the network card. The network card receives the image data through an internal FPGA (Field Programmable Gate Array), synchronously performs filtering preprocessing and defect detection on the image data in the receiving process, distinguishes a defect image, establishes direct communication from the network card to a host GPU (Graphics Processing Unit), directly transmits the defect image to the host GPU by utilizing the FPGA, and performs subsequent image reinspection operation; according to the technical scheme provided by the invention, filtering and defect detection operations originally required to be completed in the CPU of the host are preposed to the data transmission channel between the camera and the host, so that the advantage of high-speed transmission of the image in the data transmission channel is fully utilized, the image processing time is greatly shortened, and the problem of high load occupation of the CPU is solved.

Description

Technical Field

[0001] The present application relates to the technical field of automated detection, and in particular to a high-speed detection method for burrs on lithium electrode sheets based on a DPU heterogeneous platform. Background Art

[0002] One of the main culprits causing lithium battery explosions and fires is the micron-level burrs between the positive and negative electrodes and the diaphragm of the lithium battery, which can only be seen clearly under a microscope. Whether it is hardware slitting or laser die-cutting, it is difficult to avoid the generation of burrs.

[0003] The strong demand and pursuit of inspection quality have also made inspection speed and inspection accuracy the core key indicators that are constantly pursued in lithium battery burr industrial visual inspection equipment. This also means that the inspection visual equipment must have higher image resolution, higher acquisition speed, and higher processing speed. Such demands have also brought the following difficulties to the processing performance of the PC host:

[0004] The speed of lithium battery burr detection is generally required to be at least 100 meters / minute. Taking 120 meters / minute real-time full inspection as an example, the corresponding 200W camera (resolution of 1936*1000) frame rate of about 510FPS (10% overlap of field of view) can meet the detection requirements; one inspection station requires two cameras, one end-shot camera is used to detect the slitting burrs on the side, and the other overhead camera is used to detect coating defects. In this way, the transmission bandwidth of a single camera is required to be 7.35G, and the total transmission bandwidth of the two cameras reaches 14.7G. At present, only the CXP interface and the 10 Gigabit network interface in industrial vision can meet such transmission speeds. At the same time, the PC needs to process images at 510 frames per second, and it takes 1.9ms to complete the detection of a 1936*1000 resolution image. At present, neither traditional algorithms nor deep learning algorithms can achieve full inspection, and can only skip frames for sampling inspection. Such acquisition and transmission speeds place very high demands on the transmission bandwidth and processing performance of the PC host. Either the speed must be reduced, or 100% full detection cannot be achieved, which also greatly increases the cost of the visual system.

[0005] In the traditional lithium electrode slitting burr detection process, the solution is to manually conduct secondary inspections of the electrode burrs on a regular basis. Due to technical limitations, it is difficult to achieve full inspection, and the consequence is that burrs will inevitably slip through the net, burying a great hidden danger to lithium battery safety.

[0006] After machine vision technology solved this problem that has plagued the lithium battery industry for many years, the existing inspection system has achieved online real-time full inspection through a solution that combines traditional algorithms with deep learning technology. Traditional algorithms are used for detection and measurement, supplemented by AI algorithms to complete defect classification and defect review; however, the inspection speed can often only reach a full inspection speed of about 50 meters / minute, and the host's CPU is always under high load.

[0007] The main problems of the above detection system include: 1) After the camera collects images, they are all copied to the filter driver of the network camera through the network card and the network card driver. Among them, the images with burrs only account for less than 1% of the total images. This process will cause high CPU load; in addition, in this process, the filter driver also needs to bear the parsing function of the network protocol header and the visual transmission protocol header, which will reduce the transmission performance and further increase the CPU usage. 2) Since all images are directly given to users, the speed of the user's detection algorithm must also match the acquisition and transmission speed of the front-end image. Let's take a 200W camera (resolution 1936*1000) with a frame rate of 510FPS as an example. The user needs 1.9ms to complete the detection of a 1936*1000 resolution image, which obviously cannot meet the requirements, because a basic full-image filtering of a 200M image on a PC takes at least 4ms after instruction set acceleration. In addition, the user's traditional detection algorithm is also running on the PC's CPU, which will also cause high CPU load occupancy; in addition, there are heterogeneous copy images between the CPU and the GPU, and the time consumed by heterogeneous copy images also increases the total time consumed by the image processing algorithm. Summary of the invention

[0008] The purpose of this application is to provide a high-speed detection method and system for lithium electrode sheet burrs based on a DPU heterogeneous platform, adopting the solution of sinking computing power, first performing the analysis and unloading of the network protocol and the visual transmission protocol through the DPU processor of the network card FPGA, and completing the burr detection in the DPU, and only transmitting the pictures with burrs (usually less than 1%) to the PC for subsequent defect classification and defect review, thereby unloading the data processing pressure on the PC, and effectively solving the problem that the detection speed of the PC in the current high-speed detection system for lithium electrode sheet cutting burrs cannot meet the purpose of high-speed real-time detection speed.

[0009] The technical solution of the present application is to provide a high-speed detection method for lithium electrode sheet burrs based on a DPU heterogeneous platform. The method is used in the process of an industrial camera transmitting an image to a host through a network card. The network card includes an FPGA. The network card performs burr detection on the transmitted image through the FPGA and transmits the defective image to the host. The method includes:

[0010] Step 1: register the shared buffer of the host GPU to the driver of the network card, establish direct communication from the network card to the host GPU, so that the network card can directly transfer the image data to the shared buffer of the GPU; Step 2: after the image is exposed, the industrial camera reads out the image data corresponding to each pixel in a single image line by line, and synchronously transmits the image data to the network card in the form of a data stream during the readout process, wherein a single image includes several data packets, and the data packet is the basic unit in the data stream; Step 3: the network card receives the image data through the internal FPGA, and synchronously pre-processes and detects defects on the image data during the receiving process. Distinguishing defective images specifically includes: step 31, the network card receives the image data corresponding to each pixel row by row through the FPGA, and when the number of received row-direction pixels reaches a predetermined row filter size, starts row filtering, and when the number of received column-direction pixels reaches a predetermined column filter size, starts column filtering; step 32, reads the grayscale value of the pixel after column filtering, searches for the jump position of the pixel grayscale value with the direction of the pixel column as the scanning direction, extracts the target boundary point, and distinguishes the defective image based on the coordinate value of the target boundary point; step 4, uses the FPGA to directly transmit the defective image to the GPU of the host for subsequent image re-inspection operations.

[0011] Furthermore, in the pixel coordinate system where the image data is located, the pixel point in the upper left corner is taken as the origin, the direction to the right of the origin is taken as the positive direction of the row coordinate, and the direction downward of the origin is taken as the positive direction of the column coordinate. The step 32 of extracting the target boundary point specifically includes: taking the direction of the pixel column as the scanning direction, taking the first pixel coordinate point detected that satisfies the grayscale value greater than the average grayscale threshold of the coating and less than the average grayscale threshold of the foil as the coating starting point, taking the last pixel coordinate point detected that satisfies the grayscale value greater than the average grayscale threshold of the coating and less than the average grayscale threshold of the foil as the coating ending point, taking the first pixel coordinate point detected that satisfies the grayscale value greater than the average grayscale threshold of the foil as the foil starting point, and taking the last pixel coordinate point detected that satisfies the grayscale value greater than the average grayscale threshold of the foil as the foil ending point.

[0012] Further, the defect type is determined based on the coordinate value of the target boundary point in step 32, specifically including: setting a foil break length threshold, when a first type of pixel column is detected and the number of first type of pixel columns arranged continuously reaches the foil break length threshold, determining that a foil break defect exists at the position where the first type of pixel columns are arranged continuously, wherein the first type of pixel column is a pixel column where the foil starting point and the foil ending point are not detected;

[0013] A class A burr length threshold is set, and when a second class pixel column is detected and the number of the second class pixel columns arranged continuously reaches the class A burr length threshold, it is determined that a class A burr defect exists at the position where the second class pixel columns arranged continuously are located, wherein the second class pixel columns are pixel columns whose starting column coordinates of the foil are less than the starting column coordinates of the coating or whose ending column coordinates of the foil are greater than the ending column coordinates of the coating;

[0014] A Class B burr length threshold is set. When a third type of pixel column is detected and the number of consecutively arranged third type pixel columns reaches the Class B burr length threshold, it is determined that a Class B burr defect exists at the location of these consecutively arranged third type pixel columns, wherein the third type of pixel column is a pixel column that satisfies the condition that the difference between the foil starting column coordinate and the foil starting column coordinate mean is greater than the Class B burr distance threshold, or the difference between the foil ending column coordinate and the foil ending column coordinate mean is greater than the Class B burr distance threshold.

[0015] Furthermore, the foil starting column coordinates are averaged from m-NeighborLen / 2 columns to m+

[0016] The average value of the starting column coordinates of all foils in the NeighborLen / 2 column, and the average value of the ending column coordinates of the foil is the average value of the ending column coordinates of all foils in the m-NeighborLen / 2 column to the m+NeighborLen / 2 column, where m is the current column and NeighborLen is the neighborhood length, expressed as:

[0017] NeighborLen=Width*view overlap ratio

[0018] Where Width is the width of the entire image, and the field of view overlap rate is the ratio of the overlapping parts of two adjacent images in the acquisition process to a single image.

[0019] Furthermore, step 1 specifically includes: setting a shared buffer in the GPU of the host, using the shared buffer as a memory area for receiving the defective image transmitted by the network card, adding a dedicated interface in the user interface of the network card, providing the address data of the shared buffer in the GPU to the driver of the network card through the dedicated interface, so that the network card can access the shared buffer, and after receiving the image data, can directly transfer the image data to the shared buffer of the GPU.

[0020] Furthermore, in step 3, the network card receives the image data through the internal FPGA, which specifically includes: the network card receives the corresponding data packets in sequence through the FPGA in the order in which the industrial camera reads the image data, parses and unloads the network protocol and the visual transmission protocol for a single data packet, extracts the image data from the data packet, and unloads the protocol header information in the data packet, including the network protocol information and the visual transmission protocol information, and passes the image data to the DPU processor of the FPGA.

[0021] The beneficial effects of this application are:

[0022] First, the technical solution in the present application adopts the idea of ​​sinking computing power, and adopts a network card with FPGA to transmit images between the industrial camera and the host. During the image transmission process, the FPGA is used to synchronously perform image filtering and defect detection, screen out images with defects, and finally transmit the defective images, which account for a relatively small proportion of the collected images, to the GPU of the host for re-inspection. Compared with the method in the prior art of directly transmitting all images to the host for filtering, defect detection and re-inspection, in the technical solution in the present application, the host only needs to complete the re-inspection of the defective images, which reduces the number of images entering the host and avoids high CPU load. At the same time, the parsing of the network protocol header and the visual transmission protocol header is completed by the network card, which further reduces the CPU occupancy, thereby unloading the data processing pressure on the host side.

[0023] Second, the technical solution in the present application pre-places the filtering and defect detection operations that originally need to be completed in the host CPU to the data transmission channel between the camera and the host, making full use of the advantage of high-speed transmission of images in the data transmission channel. For a single image, the main body of FPGA processing is the rows of image data corresponding to the pixels. When the number of pixels received by the FPGA meets the filtering requirements, the image processing can begin. Compared with the method in the prior art where the processing main body is the entire image and the CPU needs to receive the entire image before starting processing, the technical solution in the present application does not need to wait for the entire image to be transmitted. After the image is transmitted from the camera to the FPGA, it only needs to delay a few line cycles to complete filtering and defect detection, which greatly shortens the image processing time, improves the efficiency of the entire process from image acquisition to defect detection, and reduces the running time of the entire user program; the technical solution in the present application reduces the bandwidth of subsequent processing through FPGA pre-screening, thereby alleviating the processing pressure on the back end and achieving the purpose of high-speed real-time detection.

[0024] Third, in the preferred implementation of the present application, the data driven by the network card kernel is directly transmitted to the GPU, without the need for heterogeneous copying from the CPU to the GPU at the user level, further reducing the running time of the user program. Compared with the method in the prior art of transferring the image to the GPU for subsequent processing through heterogeneous copying from the CPU to the GPU, the network card in the technical solution of the present application will directly transmit the defective image to the GPU of the host. This process does not require heterogeneous copying from the CPU to the GPU, reducing the total time of the image processing algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The advantages of the above and / or additional aspects of the present application will become apparent and easily understood in the description of the embodiments in conjunction with the following drawings, in which:

[0026] Figure 1 It is a schematic diagram of the system structure for defect detection using a high-speed detection method for lithium electrode sheet burrs based on a DPU heterogeneous platform according to an embodiment of the present application;

[0027] Figure 2 is a schematic flow chart of a high-speed detection method for lithium electrode sheet burrs based on a DPU heterogeneous platform according to an embodiment of the present application;

[0028] Figure 3 is a schematic diagram of row filtering and column filtering according to an embodiment of the present application;

[0029] Figure 4 is a schematic diagram of various defect types according to an embodiment of the present application;

[0030] Figure 5 is a schematic diagram of a target point detection process based on a real-shot image according to an embodiment of the present application;

[0031] Figure 6 is a schematic diagram of detection logic in defect detection according to an embodiment of the present application;

[0032] Figure 7 is a schematic diagram of a foil fracture defect determination process based on a real shot according to an embodiment of the present application;

[0033] Figure 8 is a schematic diagram of a Class A burr defect determination process based on a real shot according to an embodiment of the present application;

[0034] Fig. 9 is a schematic diagram of a Class B burr defect determination process based on a real-shot image according to an embodiment of the present application;

[0035] Fig.10is a comparison diagram of various time points involved in processing an image in an existing method according to an embodiment of the present application and in the method of the present invention;

[0036] Fig.11 is a schematic diagram of an image data transmission path after establishing direct communication from a network card to a host GPU according to an embodiment of the present application;

[0037] Fig.12 is a schematic diagram of a system structure for defect detection using an existing method according to an embodiment of the present application;

[0038] Fig.13 The original defect image when defect detection is performed using an existing method according to an embodiment of the present application and the defect image after straight line measurement are performed;

[0039] Fig.14 It is a schematic diagram of determining a Class A burr defect after determining the upper and lower boundary points when performing defect detection using an existing method according to an embodiment of the present application;

[0040] Fig.15 It is a schematic diagram of the system structure for setting the burr detection solution in the smart network card in an industrial camera to complete defect detection according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0042] In the following description, many specific details are elaborated to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present application is not limited to the specific embodiments disclosed below.

[0043] like Figure 1As shown, from the process of industrial camera collecting images to the completion of burr defect detection, the entire system used includes three parts: industrial camera, smart network card and host. The industrial camera communicates with the host through the network card. The industrial camera is used to collect images after die-cutting or slitting of lithium electrode sheets (die-cutting and slitting are two cutting methods after coating of battery electrodes, slitting is cutting by a hardware slitting machine, and die-cutting is cutting by laser). The processing unit in the host includes a CPU and a GPU. The GPU processing unit is used to re-inspect the image for defects. The smart network card is an intelligent network acquisition card with FPGA, which can filter and perform burr detection on the image during image transmission through the DPU processor of FPGA, and finally only transmits the image with burr defects to the GPU of the host for subsequent defect classification and defect review, thereby unloading the data processing pressure of the host. In this embodiment, the network card with embedded FPGA and capable of completing the image pre-processing function through FPGA is called a smart network card.

[0044] like Figures 2 to 9 As shown, this embodiment provides a high-speed detection method for lithium electrode sheet burrs based on a DPU heterogeneous platform. The method is used in the process of an industrial camera transmitting an image to a host through a network card. An FPGA is set in the network card. The network card performs burr detection on the transmitted image through the FPGA and transmits the defective image to the host. The high-speed detection method for lithium electrode sheet burrs based on a DPU heterogeneous platform includes:

[0045] Step 1: register the shared buffer of the host GPU to the driver of the smart network card, establish direct communication from the smart network card to the host GPU, so that the smart network card can directly transmit image data to the shared buffer of the GPU.

[0046] A shared buffer is set in the GPU of the host, and the shared buffer is used as a memory area for receiving the defective image transmitted by the network card. A dedicated interface is added to the user interface of the smart network card (i.e., a programming interface provided to the application for interacting with the smart network card hardware) (a dedicated interface program is defined in the smart network card application by programming). The address data of the shared buffer in the GPU is provided to the driver of the network card through the dedicated interface, so that the smart network card can access the shared buffer, and after receiving the image data, the image data can be directly transferred to the shared buffer of the GPU without passing through the host CPU or host memory and then entering the GPU.

[0047] like Fig.11As shown, this embodiment registers the GPU's shared buffer to the driver of the smart network card by adding a shared buffer interface in the user interface of the smart network card, so that after receiving the image data, the smart network card can directly access the shared buffer and write the image data into the shared buffer, realizing zero-copy data transmission between the GPU and the network card, avoiding the involvement of the CPU and host memory, and once the data in the GPU memory is written, the GPU can directly process it without waiting for the CPU to copy or synchronize the data, reducing the delay of data copying and optimizing the performance of data transmission. In this embodiment, the GPU's shared buffer can be registered to the driver of the smart network card with the help of NVIDIA GPUDirect for Video technology.

[0048] Step 2: After the image is exposed, the industrial camera reads out the image data corresponding to each pixel in a single image line by line, and synchronously transmits the image data to the network card in the form of a data stream during the reading process, wherein a single image includes several data packets, and a data packet is the basic unit in the data stream;

[0049] In this embodiment, the industrial camera collects images including the process of image exposure and the process of image reading (i.e., the process of generating image data). Specifically, for a single image, the industrial camera will first receive light through its sensor pixel array within a predetermined exposure time, and convert the light signal into an electrical signal. After the image exposure is completed, the electrical signal corresponding to each row of pixels is read row by row in order (from the first row to the last row), and the electrical signal data is converted into a digital signal to obtain the grayscale value corresponding to each pixel, wherein the grayscale values ​​corresponding to all pixels belong to the exposure results at the same time point.

[0050] In this embodiment, in the process of reading a single image, the industrial camera packages the image data read line by line with the protocol header information to form a number of data packets, and synchronously transmits these data packets to the network card in the form of a data stream (i.e., transmitting a continuous data sequence in time order); wherein the protocol header information includes network protocol information and visual transmission protocol information.

[0051] Step 3: The network card receives the image data through the internal FPGA, and during the receiving process, the image data is preprocessed and defect detected to distinguish images with defects.

[0052] The network card receives the corresponding data packets in sequence through the FPGA in the order in which the industrial camera reads the image data, parses and unloads the network protocol and visual transmission protocol for a single data packet, extracts the image data from the data packet, and unloads the protocol header information in the data packet, including the network protocol information and the visual transmission protocol information, and passes the image data to the DPU processor of the FPGA.

[0053] In this embodiment, the process of parsing and unloading the network protocol and the visual transmission protocol of the data packet is that after the FPGA inside the network card obtains the data packet of the image data, the data packet is parsed to extract the protocol header information and the image data, and after the parsing is completed, the protocol header information is removed and the image data is retained to complete the unloading, and then the DPU inside the FPGA gradually reads out the remaining image data.

[0054] During the receiving process, the image data is preprocessed and defect detected synchronously, which specifically includes the following steps:

[0055] Step 31, the network card receives the image data corresponding to each pixel row by row through the FPGA, and starts row filtering when the number of received row-direction pixels reaches a predetermined row filter size, and starts column filtering when the number of received column-direction pixels reaches a predetermined column filter size;

[0056] The row filter size and column filter size are set. The network card receives the image data corresponding to the pixels row by row through the DPU processor of the FPGA. In the direction of the pixel row, when the number of pixels reaches the preset row filter size, the image data starts to be filtered by row to remove the noise interference in the row direction; in the direction of the pixel column, when the number of pixels reaches the preset column filter size, the image data starts to be filtered by column to remove the noise interference in the column direction.

[0057] In this embodiment, the image data is filtered and preprocessed by using a spatial neighborhood filtering method, wherein the row filtering of the image data may be performed by a mean filtering method, and the column filtering of the image data may be performed by a Gaussian smoothing filtering method.

[0058] It should be noted that the row filter size is set to N, and the column filter size is set to M, where N and M are positive integers. Preferably, N is 4, and M is 5, 7, or 9. The coefficients of different column filter sizes are shown in the filter template of Table 1:

[0059] Table 1 Filter template

[0060] SG5 0 0 -351 1404 1989 1404 -351 0 0 SG7 0 -390 585 1170 1365 1170 585 -390 0 SG9 -372 248 692 958 1046 958 692 248 -372

[0061] In the table, SG5, SG7, and SG8 in the first column represent column filter sizes of 5, 7, and 9, and the value of each row in the table represents the weight matrix corresponding to the column filter size.

[0062] In this embodiment, since the image data corresponding to the pixels are transmitted row by row, after the DPU processor of the FPGA receives the image data, it can delay directional filtering by a maximum of N-1 pixels. Similarly, it can delay column filtering by a maximum of M-1 pixel rows. This filtering method realizes the simultaneous reception of image data and filtering. Compared with the existing method of filtering after the image transmission is completed, it saves image processing time and improves the efficiency of the entire detection process.

[0063] Step 32, read the gray value of the pixel after column filtering, find the jump position of the pixel gray value with the direction of the pixel column as the scanning direction, extract the target boundary point, and determine the defect type based on the coordinate value of the target boundary point, wherein the target boundary point includes the coating starting point, the coating ending point, the foil starting point and the foil ending point, and the pixel coordinate system where the image data is located takes the pixel point in the upper left corner as the origin (0, 0), the direction to the right of the origin as the positive direction of the row coordinate, and the direction downward from the origin as the positive direction of the column coordinate.

[0064] Set the average grayscale threshold of the coating and the average grayscale threshold of the foil. Read the grayscale value of the pixel after each column filtering is completed. Take the direction of the pixel column as the scanning direction. The first pixel coordinate point detected in the scanning direction that satisfies the grayscale value greater than the average grayscale threshold of the coating and less than the average grayscale threshold of the foil is used as the starting point of the coating. The last pixel coordinate point detected in the scanning direction that satisfies the grayscale value greater than the average grayscale threshold of the coating and less than the average grayscale threshold of the foil is used as the ending point of the coating. The first pixel coordinate point detected in the scanning direction that satisfies the grayscale value greater than the average grayscale threshold of the foil is used as the starting point of the foil. The last pixel coordinate point detected in the scanning direction that satisfies the grayscale value greater than the average grayscale threshold of the foil is used as the ending point of the foil.

[0065] In this embodiment, the average grayscale threshold of the coating and the average grayscale threshold of the foil can be set based on existing experience, or a predetermined number of images can be taken before image detection, and the average grayscale threshold of the coating and the average grayscale threshold of the foil can be set according to the average grayscale values ​​of the coating and the average grayscale values ​​of the foil read from these images. For each pixel column, the initial default column coordinate value of the coating starting point, coating ending point, foil starting point, and foil ending point defined in the system is -1. After the FPGA performs pixel grayscale detection in the scanning direction, each point is assigned a value according to the detection result, and the column coordinate values ​​of the coating starting point, coating ending point, foil starting point, and foil ending point are updated to the detection value. If no point that meets the conditions is detected, the column coordinate value will not be updated. Coating starting point, coating ending point, foil starting point, foil ending point

[0066] A foil break length threshold is set, and whether there is a foil break defect is determined based on the column coordinate detection results of the foil starting point and the foil ending point. When the first-category pixel column is detected and the number of consecutive arrangements of the first-category pixel columns reaches the foil break length threshold, it is determined that there is a foil break defect at the location of these consecutively arranged first-category pixel columns, and the current image has a foil break defect, wherein the first-category pixel column is: a pixel column where the foil starting point and the foil ending point are not detected, and the starting row coordinates of the foil break defect are recorded (that is, the row coordinates of the first first-category pixel column at the foil break defect are taken as the starting row coordinates of the foil break defect and recorded); when the first-category pixel column is not detected or the number of consecutive arrangements of the first-category pixel column is less than the foil break length threshold, it is determined that there is no foil break defect in the current image.

[0067] A Class A burr length threshold is set, and whether there is a Class A burr defect is determined according to the column coordinate detection results of the coating starting point, the coating ending point, the foil starting point, and the foil ending point. When a second-class pixel column is detected and the number of consecutively arranged second-class pixel columns reaches the Class A burr length threshold, it is determined that there is a Class A burr defect at the location of these consecutively arranged second-class pixel columns, and a Class A burr defect exists in the current image, wherein the second-class pixel column is: a pixel column whose starting column coordinate of the foil is less than the starting column coordinate of the coating or whose ending column coordinate of the foil is greater than the ending column coordinate of the coating, and the starting row coordinate of the Class A burr defect is recorded (that is, the row coordinate of the first second-class pixel column at the Class A burr defect is taken as the starting row coordinate of the Class A burr defect and recorded); when no second-class pixel column is detected or the number of consecutively arranged second-class pixel columns is less than the Class A burr length threshold, it is determined that the current image does not have the Class A burr length threshold.

[0068] A Class B burr length threshold is set, and whether there is a Class B burr defect is determined according to the column coordinate detection results of the coating starting point, the coating ending point, the foil starting point, and the foil ending point. When the third-type pixel column is detected and the number of consecutively arranged third-type pixel columns reaches the Class B burr length threshold, it is determined that there is a Class B burr defect at the location of these consecutively arranged third-type pixel columns, and the current image has a Class B burr defect, wherein the third-type pixel column is: a pixel column that satisfies the difference between the foil starting column coordinate and the foil starting column coordinate mean value greater than the Class B burr distance threshold, or a pixel column that satisfies the difference between the foil ending column coordinate and the foil ending column coordinate mean value greater than the Class B burr distance threshold, and the starting row coordinate of the Class B burr defect is recorded (that is, the row coordinate of the first third-type pixel column at the Class B burr defect is taken as the starting row coordinate of the Class B burr defect and recorded); when the third-type pixel column is not detected or the number of consecutively arranged third-type pixel columns is less than the Class B burr length threshold, it is determined that the current image does not have the Class B burr length threshold.

[0069] The mean value of the starting column coordinate of the foil is the mean value of the starting column coordinates of all foils in the columns (m-NeighborLen / 2) to (m+NeighborLen / 2), and the mean value of the ending column coordinates of the foil is the mean value of the ending column coordinates of all foils in the columns (m-NeighborLen / 2) to (m+NeighborLen / 2), where m is the current column, NeighborLen is the neighborhood length, and NeighborLen is:

[0070] NeighborLen=Width*view overlap ratio

[0071] Where Width is the width of the entire image, and the field of view overlap rate is the ratio of the overlapping parts of two adjacent images to a single image during the acquisition process. The calculation process of the foil ending column coordinate mean is the same as the calculation process of the foil starting column coordinate mean.

[0072] It should be noted that in the image processing process of burr detection, in order to avoid the burrs on the edge of the image from being missed, in practice the user usually sets a certain field of view overlap between the previous image and the next image to avoid the burrs on the edge of the image from being missed. Therefore, the actual overlapping width between the previous image and the next image can be expressed as Width*field of view overlap rate, that is, the actual overlapping width is used as the neighborhood length.

[0073] In this embodiment, Class A burrs are burrs that protrude from the coating, and Class B burrs are burrs that do not protrude from the coating but can increase the risk of lithium battery explosion and fire; the foil break length threshold is the minimum foil break length set by the user, the Class A burr length threshold is the minimum length of Class A burrs set by the user, and the Class B burr length threshold is the minimum length of Class B burrs set by the user. The Class B burr distance threshold is used to determine whether the pixel column belongs to the third type of pixel column (some burrs in the lithium electrode sheet do not protrude from the coating. Although these burrs do not protrude from the coating, if the length in the column direction is greater than the predetermined Class B burr distance threshold, such as 7 to 15 pixels, it may also increase the risk of lithium battery explosion and fire. Therefore, the burrs that do not protrude from the coating and whose length in the column direction is greater than the predetermined Class B burr distance threshold are defined as Class B burrs). In practice, these thresholds need to be set according to specific quality inspection requirements.

[0074] In the present embodiment, for pixel columns, the grayscale value of the pixel point can be read after each column filtering is completed, and then the grayscale value of the current row of the column is compared with the grayscale value of the previous row of the column to find the jump position of the pixel grayscale value and perform defect detection. It can be seen that for a complete image, defect detection can be started after the pixel grayscale value is read, and there is no need to wait for the entire image to be transmitted. The process of FPGA receiving image data and the process of defect detection are carried out synchronously (that is, detection can be carried out synchronously during the process of image data transmission). Compared with the existing method of performing image detection after the entire image is transmitted, the detection time is shortened and the efficiency of defect detection is improved.

[0075] Step 4: Use the FPGA of the network card to directly transmit the defect image to the GPU of the host for subsequent image re-inspection operations.

[0076] Each time a single image is inspected for defects, the FPGA of the network card determines whether the current image is a defective image. If it is a defective image, the current image is directly transmitted to the GPU of the host for subsequent image re-inspection operations. Otherwise, it is not transmitted to the host.

[0077] In this embodiment, Fig.15 As shown in the figure, by adopting the solution of sinking computing power, the FPGA can also be set up in the camera for burr detection. The network card directly uses an ordinary network card and is only used to transmit image data. As shown in the figure, after the industrial camera is exposed, filtering and defect detection can be performed in the process of reading out the image data line by line, and finally the screened defective image is transmitted to the host GPU through the network card for re-inspection.

[0078] The FPGA in the camera is used to detect burrs, and the defective image is transmitted to the host through the network card. The specific process is as follows:

[0079] S101, registering the shared buffer of the host GPU to the driver of the camera, establishing direct communication from the camera to the host GPU, so that the camera can directly transmit image data to the shared buffer of the GPU;

[0080] S102, after the image is exposed, the industrial camera reads out the image data corresponding to each pixel in the single image line by line, and synchronously transmits the image data to the FPGA of the camera in the form of a data stream during the reading process, wherein a single image includes a plurality of data packets, and a data packet is a basic unit in the data stream;

[0081] S103, the FPGA of the camera receives the image data, and during the receiving process, pre-processes and detects defects on the image data to distinguish defective images, specifically including:

[0082] S1031, the camera receives image data corresponding to each pixel row by row through the FPGA, and starts row filtering when the number of received row-direction pixels reaches a predetermined row filter size, and starts column filtering when the number of received column-direction pixels reaches a predetermined column filter size;

[0083] S1032, reading the grayscale value of the pixel after column filtering, finding the jump position of the pixel grayscale value with the direction of the pixel column as the scanning direction, extracting the target boundary point, and distinguishing the defective image based on the coordinate value of the target boundary point;

[0084] S104, using FPGA to directly transmit the defect image to the GPU of the host for subsequent image re-inspection operations.

[0085] The process of using the FPGA in the camera for glitch detection is the same as that of using the FPGA in the network card for glitch detection, which will not be described here.

[0086] The method of the present invention is compared with the prior art method. The prior art method directly transmits all images to the host for filtering, defect detection and re-inspection. The whole process has low processing efficiency and high load occupancy of the host CPU. The method of the present invention adopts a computing power sinking method, uses the FPGA in the network card to filter and detect defects on the image, and then transmits the pre-screened defective image to the host for re-inspection, thereby improving processing efficiency and reducing the high load occupancy of the host CPU.

[0087] like Fig.12 As shown, in the prior art, all images captured by the camera will be copied to the filter driver of the network camera in the host through the network card and the network card driver, and the proportion of defective images is less than 1% of all images. In the entire process from image acquisition to defect detection, the user detection program of the host needs to complete the burr detection of all images and the re-inspection of the defective images. Among them, the amount of image data is large, and the burr detection of all images will cause a high load on the CPU. The filter driver also needs to bear the parsing function of the network protocol header and the visual transmission protocol header, which will reduce the transmission performance and further increase the CPU occupancy. In addition, when re-inspecting the defective image, a heterogeneous copy of the defective image from the CPU to the GPU is required, and the heterogeneous copy process will also increase the total time consumption of the image processing algorithm.

[0088] Among them, the host uses the existing algorithm to detect burrs on the image through the principle of two-dimensional measurement. Taking the burr detection in the following figure as an example, the defect detection process is as follows:

[0089] S201: First, find the upper and lower boundaries of the coating through two straight line measurements. The specific measurement principle is: for the upper straight line, use countless small calipers perpendicular to the straight line to find the change point from dark to bright in the image. In order to avoid noise interference, smooth the image in the small caliper. For the lower straight line, find the change point from dark to bright in the image from the opposite direction of the image. Fig.13 As shown, Fig.13 The left picture in the figure is the original defect picture, and the right picture is the defect picture after straight line measurement;

[0090] S202: Perform straight line fitting on the upper and lower boundary points respectively, and output two straight lines;

[0091] S203: Traverse the upper and lower boundary points respectively, calculate the deviation distance from the boundary point to the straight line, and if it exceeds the set threshold, it is a Class A burr, such as Fig.14 As shown;

[0092] S204: Cut out the image between the two straight lines, and then perform two straight line measurements to measure the upper and lower boundaries of the foil respectively, so as to calculate whether the foil is broken or has Class B burrs. The principle of this part is the same as that of detecting Class A burrs.

[0093] The whole process above includes: two full-image traversals for straight-line measurement (including image filtering), two traversals of cropped small images (within the upper and lower boundaries of the coating) for straight-line measurement (including image filtering), four straight-line fittings, four boundary point traversals, and a series of threshold judgments. Taking a 200W camera (resolution of 1936*1000) with a frame rate of 510FPS as an example, in order to match the acquisition and transmission speed of the front-end image, the host needs to complete the detection of a 1936*1000 resolution image within 1.9ms. In the whole process, for a 1936*1000 resolution image, the time for a full-image traversal for straight-line measurement (including image filtering) is more than 20ms on a PC with a typical configuration (including a 64-bit operating system and 16GB of RAM). Such a detection speed cannot match the acquisition and transmission speed of the front-end image.

[0094] like Figure 1As shown, in the method of the present invention, an intelligent network card with FPGA is first used to filter and detect defects in the image data during transmission, and defective images are pre-screened, which is equivalent to stringing the FPGA in the acquisition chain as a key processing node in the entire image data acquisition chain, so that the image processing process (including filtering and defect detection) that originally needs to be performed after the image acquisition is completed and transmitted to the host CPU is directly completed in the process of transmitting the image from the camera to the network card, wherein the proportion of defective images is small. In the entire process from image acquisition to defect detection, the user detection program of the host only needs to complete the re-inspection of a small number of defects, thereby avoiding high CPU load occupancy. At the same time, the parsing of the network protocol header and the visual transmission protocol header is completed by the network card, further reducing the CPU occupancy. In addition, when the defective image is re-inspected, the network card will directly transmit the defective image to the GPU of the host. This process does not require heterogeneous copying from the CPU to the GPU, thereby reducing the total time consumption of the image processing algorithm.

[0095] In the FPGA of the network card, the image processing method based on spatial neighborhood often only needs a few line cycles. The line cycle here refers to the time it takes to read out a line of the image sensor in the camera. The time range is from a few microseconds to tens of microseconds. For example, for a camera with a resolution of 1280*1024 and a frame rate of 100FPS, a line cycle is about 10us. Therefore, it only takes a few microns to perform image processing using FPGA. Compared with image detection on the PC side, using FPGA greatly reduces the running time of the user program.

[0096] When the FPGA is used to process images in the present invention, the main body processed is the image data rows corresponding to the pixel points. For a single image, the image processing can be started when the number of transmitted pixels meets the filtering requirements, and there is no need to wait for the entire image to be transmitted. This processing method fully utilizes the advantage of high-speed transmission of images in the data transmission channel. Filtering and defect detection can be performed synchronously during the image transmission process, which greatly improves the efficiency of the entire process from image acquisition to defect detection. When the existing method processes images, the main body processed is the entire image, and filtering and defect detection can only be started after the entire image is transmitted to the host CPU. During the whole process, all images collected by the camera must be transmitted to the CPU for processing, which not only takes a long time to run, but also increases the pressure on the CPU, resulting in high CPU load occupancy.

[0097] like Fig.10As shown in the figure, image processing in industry emphasizes real-time performance. The processing speed usually refers to the speed from the start of image reading or exposure to the completion of processing (i.e., T2-T5 or T2-T5 in the figure), rather than the processing speed of a processor. Image processing in industry focuses on the overall efficiency from image acquisition to processing completion. Since FPGA is connected in the acquisition chain, the starting point of FPGA processing time is almost the starting point of image generation (only a few lines from T2 to T3), and with the completion of image transmission, image processing is also completed (similarly, only a few lines from T4 to T5). For existing methods, the image may have just entered the host memory or video memory at this time. The image processing on the host starts with T4 as the starting point for image processing, and its speed is much slower than the speed of image processing using FPGA, i.e., (T4-T5)<<(T4-T6). It can be seen that FPGA is connected in the acquisition link, and the processing based on spatial neighborhood greatly shortens the image processing time, making it have outstanding advantages in spatial domain processing tasks with relatively simple calculations.

[0098] The steps in this application can be adjusted in order, combined, and deleted according to actual needs.

[0099] The units in the device of the present application can be combined, divided and deleted according to actual needs.

[0100] Although the present application is disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely exemplary and are not intended to limit the application of the present application. The scope of protection of the present application is defined by the appended claims and may include various modifications, alterations and equivalents made to the invention without departing from the scope and spirit of the present application.

Claims

1. A high-speed detection method for lithium electrode burrs based on a DPU heterogeneous platform, which is used in the process of industrial cameras transmitting images to a host through a network card, characterized in that: The network card includes an FPGA, and the network card performs burr detection on the transmitted image through the FPGA, and transmits the defective image to the host. The method includes: Step 1: register the shared buffer of the host GPU to the driver of the network card, establish direct communication from the network card to the host GPU, so that the network card can directly transfer image data to the shared buffer of the GPU; Step 2: After the image is exposed, the industrial camera reads out the image data corresponding to each pixel in a single image line by line, and synchronously transmits the image data to the network card in the form of a data stream during the reading process, wherein a single image includes several data packets, and a data packet is the basic unit in the data stream; Step 3: The network card receives the image data through the internal FPGA, and performs preprocessing and defect detection on the image data during the receiving process to distinguish defective images, including: Step 31, the network card receives the image data corresponding to each pixel row by row through the FPGA, and starts row filtering when the number of received row-direction pixels reaches a predetermined row filter size, and starts column filtering when the number of received column-direction pixels reaches a predetermined column filter size; Step 32, read the grayscale value of the pixel after column filtering, find the jump position of the pixel grayscale value with the direction of the pixel column as the scanning direction, extract the target boundary point, and distinguish the defective image based on the coordinate value of the target boundary point; Step 4: Use FPGA to directly transmit the defect image to the GPU of the host for subsequent image re-inspection operations.

2. The high-speed detection method for lithium electrode sheet burrs based on DPU heterogeneous platform according to claim 1 is characterized in that: In the pixel coordinate system where the image data is located, the pixel point in the upper left corner is taken as the origin, the direction to the right of the origin is the positive direction of the row coordinate, and the direction downward from the origin is the positive direction of the column coordinate. The step 32 of extracting the target boundary point specifically includes: Taking the direction of pixel column as the scanning direction, the first pixel coordinate point detected that satisfies the grayscale value greater than the average grayscale threshold of the coating and less than the average grayscale threshold of the foil is taken as the starting point of the coating, the last pixel coordinate point detected that satisfies the grayscale value greater than the average grayscale threshold of the coating and less than the average grayscale threshold of the foil is taken as the ending point of the coating, the first pixel coordinate point detected that satisfies the grayscale value greater than the average grayscale threshold of the foil is taken as the starting point of the foil, and the last pixel coordinate point detected that satisfies the grayscale value greater than the average grayscale threshold of the foil is taken as the ending point of the foil.

3. The high-speed detection method for lithium electrode sheet burrs based on DPU heterogeneous platform as claimed in claim 2 is characterized in that: Determining the defect type based on the coordinate value of the target boundary point in step 32 specifically includes: A foil breakage length threshold is set. When a first type of pixel column is detected and the number of consecutively arranged first type of pixel columns reaches the foil breakage length threshold, it is determined that a foil breakage defect exists at the position where the consecutively arranged first type of pixel columns are located. The first type of pixel column is a pixel column where the foil starting point and the foil ending point are not detected. A class A burr length threshold is set, and when a second class pixel column is detected and the number of the second class pixel columns arranged continuously reaches the class A burr length threshold, it is determined that a class A burr defect exists at the position where the second class pixel columns arranged continuously are located, wherein the second class pixel columns are pixel columns whose starting column coordinates of the foil are less than the starting column coordinates of the coating or whose ending column coordinates of the foil are greater than the ending column coordinates of the coating; A Class B burr length threshold is set. When a third type of pixel column is detected and the number of consecutively arranged third type pixel columns reaches the Class B burr length threshold, it is determined that a Class B burr defect exists at the location of these consecutively arranged third type pixel columns, wherein the third type of pixel column is a pixel column that satisfies the condition that the difference between the foil starting column coordinate and the foil starting column coordinate mean is greater than the Class B burr distance threshold, or the difference between the foil ending column coordinate and the foil ending column coordinate mean is greater than the Class B burr distance threshold.

4. The high-speed detection method for lithium electrode sheet burrs based on DPU heterogeneous platform as claimed in claim 3 is characterized in that: The average value of the starting column coordinates of the foil is the average value of the starting column coordinates of all foils from m-NeighborLen / 2 columns to m+NeighborLen / 2 columns, and the average value of the ending column coordinates of the foil is the average value of the ending column coordinates of all foils from m-NeighborLen / 2 columns to m+NeighborLen / 2 columns, where m is the current column and NeighborLen is the neighborhood length, which is expressed as: NeighborLen=Width*view overlap ratio Where Width is the width of the entire image, and the field of view overlap rate is the ratio of the overlapping parts of two adjacent images in the acquisition process to a single image.

5. The high-speed detection method for lithium electrode sheet burrs based on DPU heterogeneous platform as claimed in claim 1, characterized in that: The step 1 specifically includes: A shared buffer is set in the GPU of the host, and the shared buffer is used as a memory area for receiving the defective image transmitted by the network card. A dedicated interface is added to the user interface of the network card. The address data of the shared buffer in the GPU is provided to the driver of the network card through the dedicated interface, so that the network card can access the shared buffer and, after receiving the image data, can directly transmit the image data to the shared buffer of the GPU.

6. The high-speed detection method for lithium electrode sheet burrs based on DPU heterogeneous platform as claimed in claim 1, characterized in that: In step 3, the network card receives the image data through the internal FPGA, which specifically includes: The network card receives the corresponding data packets in sequence through the FPGA in the order in which the industrial camera reads the image data, parses and unloads the network protocol and visual transmission protocol for a single data packet, extracts the image data from the data packet, and unloads the protocol header information in the data packet, including the network protocol information and the visual transmission protocol information, and passes the image data to the DPU processor of the FPGA.

Images