A method and system for single board deep defect recognition based on line laser image
Through the depth defect recognition method based on line laser images, the random forest algorithm and edge extraction technology are used to solve the problem of low detection accuracy of depth defect detection in wood veneer, and automatic depth defect recognition and efficient recognition effects are achieved.
Patent Information
- Application Number
- CN202311714377.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-12
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2043-12-12
AI Technical Summary
Existing machine vision detection technology is difficult to effectively identify depth defects in wood veneer, such as wrinkles, cuts, holes and cracks, because the color characteristics of these defects are not obvious and are easily confused with wood texture, resulting in low detection accuracy.
The depth defect recognition method based on line laser images is adopted, and the line laser image is acquired, edge extraction and depth value calculation are performed. The depth defect recognition model trained by the random forest algorithm is used to automatically identify the depth defects on the surface of the veneer.
Automatic identification of depth defects of veneer board is realized, detection accuracy and speed are improved, and the type and position of depth defects on the surface of veneer board can be accurately identified, and the recognition speed and accuracy are higher than traditional methods.
Smart Images

Figure CN117953268B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of machine vision, and in particular relates to a single board depth defect recognition method and system based on line laser images. Background Art
[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.
[0003] During the production and processing of veneers, it is necessary to identify and classify the surface depth information to ensure that the veneer products have a smooth processing surface and good mechanical properties. In addition to defects such as scars and pollution with obvious color characteristics, the surface of wood veneers also has depth defects with less obvious color characteristics. Usually, depth defects refer to defects in the uneven surface of the veneer, that is, defects along the thickness direction of the veneer, such as shrinkage, knife marks, holes, cracks, etc. Figure 1(a)-Figure 1(d) As shown, these deep defects are more serious defects in certain wood materials.
[0004] Nowadays, the current machine vision detection technologies for veneer defects are all based on color and shape features, and have good detection accuracy for defects such as knots and pollution. However, for deep defects such as wrinkles and knife marks, the color features are not obvious, and the color and shape are easily confused with the inherent texture of the wood veneer. Therefore, the existing detection technologies based on color or shape features are not suitable for the detection and identification of deep defects. Summary of the invention
[0005] In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a method and system for identifying veneer depth defects based on line laser images, thereby realizing automatic detection of wood veneer depth defects.
[0006] To achieve the above object, a first aspect of the present invention provides a single board depth defect recognition method based on line laser image, comprising:
[0007] Acquire a line laser image of the board to be inspected;
[0008] Extracting edges of the line laser image and refining the image to obtain single-pixel edge lines;
[0009] Calculate the depth value of each pixel in the single-pixel edge line;
[0010] Divide the depth value of the single-pixel edge line into sub-regions, each sub-region contains N continuous depth values, and perform depth defect recognition on each sub-region based on the single-board depth defect recognition model;
[0011] Among them, the single board deep defect recognition model is based on the single board defect feature data set and is trained using a random forest algorithm; the single board defect feature data set includes multiple defect feature data, and each defect feature data includes N continuous depth value data and a defect type.
[0012] In some embodiments, after the line laser image of the board to be inspected is acquired, Gaussian filtering is performed on the line laser image.
[0013] In some embodiments, edge extraction of the line laser image specifically includes:
[0014] Based on the Canny edge detection algorithm, the line laser in the line laser image is roughly positioned;
[0015] Obtaining the optimal step intensity threshold of the line laser image by using the OTSU algorithm;
[0016] Based on the optimal step intensity threshold, edge detection is performed on the roughly positioned line laser image based on a Zernike moment edge detection algorithm.
[0017] In some embodiments, calculating the depth value of each pixel in the single-pixel edge line specifically includes:
[0018] Calculate the pixel equivalent of the image based on the number of image pixels, image resolution, and the actual length of the laser line on the image;
[0019] According to the horizontal inclination angle of the line laser projection, the number of pixels offset by each pixel point in the single-pixel edge line relative to the reference line laser image is calculated;
[0020] According to the number of offset pixels and pixel equivalent, the actual offset of each pixel point in the single-pixel edge line is calculated;
[0021] According to the line laser projection angle and the actual offset of each pixel point, the depth value of each pixel point in the single-pixel edge line is calculated.
[0022] In some embodiments, the number of sub-decision trees of the random forest algorithm is N.
[0023] In some embodiments, when the depth value of the single-pixel edge line is divided into sub-regions, each sub-region is also numbered, and for a sub-region where a depth defect is identified, the defect type and number of the sub-region are obtained.
[0024] A second aspect of the present invention provides a single board depth defect recognition system based on line laser images, comprising:
[0025] An image acquisition module is configured to acquire a line laser image of the single board to be inspected;
[0026] An edge extraction module is configured to extract edges from the line laser image and refine it to obtain single-pixel edge lines;
[0027] A depth conversion module configured to calculate a depth value of each pixel in a single-pixel edge line;
[0028] A defect recognition module is configured to divide the depth value of the single-pixel edge line into sub-regions, each sub-region includes N continuous depth values, and perform depth defect recognition on each sub-region based on the single-board depth defect recognition model;
[0029] Among them, the single board deep defect recognition model is based on the single board defect feature data set and is trained using a random forest algorithm; the single board defect feature data set includes multiple defect feature data, and each defect feature data includes N continuous depth value data and a defect type.
[0030] The third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the single board depth defect recognition method based on line laser images is implemented.
[0031] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the single board depth defect recognition method based on line laser images.
[0032] A fifth aspect of the present invention provides a single board depth defect recognition system, comprising: a line laser module, an image acquisition module, a host computer and a signal control module;
[0033] The line laser module is arranged on one side of the image shooting area, and is used to project the line laser on the surface of the single board; the image acquisition module is used to acquire the single board surface image containing the line laser, and send it to the host computer;
[0034] The host computer is used to execute the single board depth defect recognition method based on line laser image;
[0035] The signal control module is used to control the speed of the transmission mechanism and / or the shooting frequency of the image acquisition module.
[0036] One or more of the above technical solutions have the following beneficial effects:
[0037] By introducing line laser scanning technology, the depth defects on the surface of the single board are expressed in the form of bending changes of the laser line. The edge information is extracted through the bent laser line, and the type of depth defects is judged based on the edge information, thereby realizing automatic identification of single board depth defects.
[0038] Compared with the traditional direct-incidence triangulation method, the position of the line laser is swapped with that of the industrial camera. The line laser projects the line laser obliquely onto the surface of the single board, and the camera is installed above the measured board to collect images. By adjusting the position and angle of the line laser, line lasers of different lengths and incident angles can be obtained, making the entire system more flexible. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0040] Figure 1(a) shows an example of shrinkage defects on a single board;
[0041] Figure 1(b) shows an example of a knife mark defect on a single board;
[0042] Figure 1(c) shows an example of a hole defect on a single board;
[0043] Figure 1(d) shows an example of a crack defect on a single plate;
[0044] Figure 2 A single board depth defect system in one or more embodiments of the present invention;
[0045] Figure 3 A schematic diagram of an image acquisition platform in one or more embodiments of the present invention;
[0046] Figure 4 It is an overall framework diagram of a single board deep defect detection method in one or more embodiments of the present invention;
[0047] Figure 5 A flow chart of a single board deep defect detection method in one or more embodiments of the present invention;
[0048] Figure 6 A diagram showing the positional relationship of a linear laser, an image acquisition module, and a single board in one or more embodiments of the present invention;
[0049] Figure 7(a) is the original line laser image;
[0050] FIG7( b ) is a diagram showing the effect of the line laser image after mean filtering;
[0051] FIG7( c ) is a diagram showing the effect of the line laser image after median filtering;
[0052] Figure 7(d) is the effect of the line laser image after bilateral filtering;
[0053] FIG7( e ) is a diagram showing the effect of the line laser image after Gaussian filtering;
[0054] Figure 8 A flow chart of an edge detection method according to one or more embodiments of the present invention;
[0055] Figure 9(a) is a diagram of the classification results based on the random forest algorithm;
[0056] Figure 9(b) is a diagram showing the classification results based on the BP neural network algorithm;
[0057] Figure 9(c) is a diagram of the classification results based on the convolutional neural network algorithm;
[0058] FIG9( d ) is a diagram showing the classification results of a single board deep defect recognition method provided according to one or more embodiments of the present invention. DETAILED DESCRIPTION
[0059] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used in one or more embodiments of the present invention have the same meanings as those generally understood by those skilled in the art to which the present invention belongs.
[0060] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.
[0061] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.
[0062] According to the reference veneer national standard (GB / T13010-2020), actual production needs and production process requirements, the deep defects on the veneer are divided into four basic defect types: wrinkles, knife marks, holes and cracks.
[0063] From the above analysis, it can be seen that traditional image processing methods are difficult to apply to the detection and identification of deep defects on the wood surface. One or more embodiments of the present invention introduce line laser scanning technology, and provide a single board depth defect identification method and system based on line laser images. The depth defects on the single board surface are expressed in the form of bending changes of laser lines, and edge information is extracted by bending laser lines. Then, the edge information of the laser lines is identified by machine learning technology to determine the type of depth defects, thereby realizing the detection and identification of single board depth defects.
[0064] Specifically, the single board deep defect recognition system cooperates with the image acquisition platform, including a line laser module, an image acquisition module, a host computer and a signal control module. The line laser module is used to project a line laser on the surface of the single board; the image acquisition module is used to acquire the single board surface image containing the line laser and send it to the host computer; the host computer is used to receive the control instructions of the transmission mechanism and the image acquisition module and send them to the signal control module, and perform deep defect recognition based on the line laser on the single board surface image; the signal control module is used to control the transmission mechanism and / or the image acquisition module in response to the control instructions of the host computer. The shooting frequency of the image acquisition module is adapted to the speed of the transmission mechanism, so that the line laser image acquired for the same single board can cover the entire single board surface.
[0065] The overall architecture of the deep defect recognition system is as follows: Figure 2 As shown, the image acquisition platform includes a conveying mechanism, the conveying mechanism includes two conveyor belts, a reference plane is provided between the two conveyor belts, the reference plane is made of an aluminum plate, a line laser module is provided on one side of the reference plane, and an image acquisition module is provided above the reference plane. As a specific implementation method, Figure 3 As shown, brackets are provided on both sides of the reference plane, and the two brackets are connected by a crossbeam. The line laser module is installed on the bracket on one side, at a fixed angle with the aluminum plate, and projects a line laser onto the aluminum plate. The length of the line laser is equal to the length of the reference plane; the image acquisition module is installed directly above the aluminum plate. The image acquisition module, the signal control module and the host computer establish a local area network through a switch. The speed of the conveyor belt can be set and sent to the signal control module via the host computer, and it can also be used as an image processing module for deep defect recognition of the single-board line laser image.
[0066] The signal control module controls the speed of the single board conveyor belt to reach the production setting speed. The single board follows the conveyor belt to the base surface position. The line laser module projects the line laser on the surface of the single board. The image acquisition module then obtains the line laser image and transmits it to the host computer through the switch. The host computer performs machine vision processing on the line laser image to obtain the category and location of the deep defect, and displays the processed data information through a graphical interface. Finally, it is fed back to the staff to complete the single board surface defect identification work. Compared with traditional manual identification, the single board surface defect identification system has the advantages of fast identification speed and high identification accuracy.
[0067] In addition to identifying the image acquisition area, the reference plane is also used to determine the line laser reference line. In this system, determining the line laser reference line is a key technology. As a ruler for subsequent image processing, it has high requirements for the flatness of the reference line laser line plane, that is, the line laser can form a straight, continuous and uniform laser line on the plane that supports the wooden board to ensure the accuracy and reliability of subsequent image processing and detection results. Therefore, a reference plane is required in the image acquisition module. In order to achieve a high-quality line laser reference line, an aluminum plate with a flatness of 0.005mm was selected as the reference plane for the wooden board, and the line laser reference line image was collected on this plane. In this way, we can ensure that the light beam emitted by the line laser can be imaged on a flat and stable surface, thereby obtaining a high-quality line laser reference line image. This reference line can not only provide an accurate measurement reference, but also provide a high-quality ruler for subsequent image processing, ensuring the accuracy and reliability of the entire system.
[0068] In order to ensure that the collected line laser image more accurately reflects the depth defect of the single board, the imaging of the laser beam on the single board should be uniform, stable and straight. Based on this, the optical prism inside the line laser is selected. In some embodiments, a line laser transmitter with a Powell prism can be selected. The image acquisition module uses an industrial camera for image acquisition. The two important bases for the selection of industrial cameras are resolution and acquisition frame rate: at a higher resolution, the captured image is clearer and the single board image has more details; and the acquisition frame rate directly affects the acquisition speed. The higher the acquisition frame rate, the faster the image acquisition speed, which affects the detection speed of the system to a certain extent. The signal control module mainly includes a photoelectric encoder, a photoelectric switch and a corresponding drive circuit. The control command is sent to the signal control module through the host computer or PLC to control the transmission mechanism of the single board defect recognition system to run at a set speed, and to control the image acquisition module to take pictures at a set frequency. In the process of image acquisition, the signal control module is required to control the camera shooting speed to adapt to the moving speed of the single board to ensure that the image acquisition work is completed when the single board is acquired at the specified position.
[0069] Those skilled in the art will appreciate that the options of the line laser module, image acquisition module and signal control module can be selected according to requirements, as long as the above functions can be achieved, and no specific limitation is made here.
[0070] The working process of the single board deep defect recognition system is as follows:
[0071] (1) The signal control module controls the conveying device to transfer the single board to the shooting area at a uniform speed;
[0072] (2) The line laser module projects a uniform straight line laser onto the surface of the board. For the surface of the board with deep defects, the line laser will be deflected. The image acquisition module, under the control of the signal control module, captures the line laser image in coordination with the speed of the conveyor.
[0073] (3) The image acquisition module transmits the collected line laser image to the image processing module through the local area network, and the image processing module performs image processing and detection, identifies and determines the type and location of the depth defect on the image and records it, providing data support for subsequent production and processing.
[0074] One or more embodiments of the present invention provide a single board depth defect recognition method based on line laser image, such as Figure 4 As shown, deep defect recognition is performed based on a single board deep defect recognition model.
[0075] The overall idea of constructing a single-board deep defect recognition model is as follows: first, adjust the line laser to a suitable angle to facilitate the acquisition of a line laser image containing deep defect details; then use an industrial camera to capture the line laser image, preprocess the line laser image, and obtain sub-pixel coordinates from the line laser image. Compare the obtained sub-pixel coordinates with the pre-acquired line laser baseline to obtain the offset of each pixel, and then calculate the depth value of each pixel, using the pixel depth value as sample data to build and optimize the data set; finally, train and optimize the data classification machine learning algorithm on the data set to obtain the optimal classification network model. The method for constructing a single-board deep defect recognition model specifically includes:
[0076] Step 1: Construct a single board deep defect dataset. Step 1 specifically includes:
[0077] Step 101: Correct the angle of the laser emitted by the laser module to obtain a line laser sample image, where the line laser sample image covers all defect types to be identified.
[0078] Compared with the traditional direct-incidence triangulation method, this system swaps the position of the line laser with the position of the industrial camera. The line laser projects the line laser obliquely to the surface of the single board, and the camera is installed above the measured board to collect images. By adjusting the position and angle of the line laser, line lasers of different lengths and incident angles can be obtained, making the entire system more flexible; at the same time, the fixed position of the camera can determine an accurate image accuracy, making the calculation of the acquired image size more accurate. Through the improved design of the line laser structure, it is more suitable for the needs of the single board production process.
[0079] like Figure 6As shown, when determining the position of the line laser, the distance from the line laser to the camera and the inclination angle of the line laser must be considered; secondly, the line laser cannot miss the surface to be measured, so the initial position of the line laser projection must be within the camera's field of view, and at the same time, the length of the line laser must be sufficient to cover the entire width of the surface to be measured.
[0080] Through experiments, it is found that at a larger tilt angle, the change of the line laser is easier to detect for more obvious deep defects such as holes. However, due to the presence of relatively small deep defects on the surface of the single board, such as knife marks and small cracks, the change of the line laser at these deep defect positions is not obvious. According to formula (1), the horizontal distance L of the line laser to the camera at different heights is equal to the ratio of the height H of the surface of the object being measured to the tangent value of the tilt angle θ between the line laser and the horizontal plane. When the distance between the line laser and the industrial camera is determined, a smaller tilt angle will have a larger tangent value, thereby amplifying the bending degree of the emitted line laser on the uneven single board surface in the image taken by the industrial camera, and obtaining a line laser image with more obvious bending features.
[0081]
[0082] According to formula (1), the surface height of the object under test can also be calculated based on the number of pixels on the image. In the experiment, it is first necessary to determine the reference plane for shooting the line laser and obtain the pixel coordinates of the line laser image on the reference plane. The pixel coordinates of the line laser on the reference plane are used as the standard to determine the degree of bending of the line laser projected on the surface of the single board. By adjusting the projection angle and obtaining line laser images with different inclination angles, it can be determined that when the inclination angle of the line laser is 15°, the bending degree of the line laser is obvious enough to reflect the relatively small crack defects on the surface of the single board, and the width of the line laser does not exceed the width requirement.
[0083] When the shooting height of the industrial camera is determined, the coordinates of the laser points on the image can be converted into depth data, including the following steps:
[0084] (1) The relationship between the image resolution and the actual length is calculated by shooting the standard length. The pixel equivalent of the image is calculated based on the number of image pixels, image resolution, and the actual length of the laser line on the image.
[0085] (2) When the horizontal inclination angle θ is determined, the number of pixels offset between each point on the line laser and the reference line laser on the image is calculated according to the horizontal inclination angle of the line laser projection;
[0086] (3) According to the number of offset pixels and pixel equivalent, the actual offset l of each point on the line laser is calculated, as shown in formula (2).
[0087] (4) According to the trigonometric function relationship, the height difference h between each point on the line laser and the reference surface, that is, the depth value, is calculated, as shown in formula (3). From this, the depth information of the single board surface reflected by the line laser image can be calculated.
[0088] l=Δp×a (2)
[0089] h=l×tanθ (3)
[0090] In formula (2), l is the actual distance line laser offset, Δp is the pixel difference between the baseline and the corresponding point, and a is the pixel equivalent of the image; in formula (3), h is the height difference between this position and the reference surface.
[0091] Step 102: pre-processing the line laser sample image.
[0092] The depth information of the single board surface is obtained from the edge information of the line laser. In the image acquisition system, due to the optical properties of the line laser itself and the influence of environmental factors, the obtained line laser often has a large noise, especially the large number of small jagged edges at the edge position, which has a significant impact on the extraction of the line laser edge; secondly, when acquiring images in a large field of view, the image accuracy corresponding to the camera resolution will be seriously reduced, and the relatively subtle depth defects on the surface of the single board put forward higher requirements for the accuracy of the image. Therefore, when acquiring the edge, first of all, it is necessary to filter these noise information to complete the preprocessing of the image.
[0093] According to the pixel distribution of the single-board online laser image and the feature that red is the main color, the R channel of the image is separated and several commonly used filtering methods are used for experiments. The filter size is uniformly set to 7×7. The comparison results are shown in the figure below. Figure 7(a)-Figure 7(e) While the mean filter and the median filter smooth the edges, a lot of detail information is lost. Compared with the original image, the edges are not well smoothed after the bilateral filter. Compared with the mean filter, the median filter and the bilateral filter, the Gaussian filter achieves a smoothing effect on the edges without changing the edge direction of the original image.
[0094] Step 103: extracting edges from the preprocessed line laser sample image.
[0095] In one or more embodiments of the present invention, the line laser projects a complete line on the flat surface of the single board, which should correspond to a continuous function. Therefore, the line laser image provided by one or more embodiments of the present invention meets the requirements for applying sub-pixel algorithms, and sub-pixel algorithms can be applied to perform high-precision processing on the line laser image.
[0096] The Zernike orthogonal moment edge detection algorithm has good noise resistance because it is not affected by image rotation changes. It has good adaptability for detecting fine edges of line lasers. According to the detection accuracy requirements, the Zernike moment edge detection algorithm takes two thresholds: distance threshold δ and step intensity threshold k. Threshold operations are performed on parameters d and k. When |Z11|≥k, d≤δ, the detected pixel point is considered to be an edge point, and the output (xs, ys) calculated at this time has sub-pixel accuracy. One or more embodiments of the present invention provide a 5×5 Zernike moment as a calculation template.
[0097] In order to reduce the amount of computation of the sub-pixel edge detection algorithm and improve the accuracy of the judgment conditions of edge points, a fast algorithm based on the rough positioning of the Canny operator and an optimization of the grayscale threshold based on the OTSU algorithm are used. Specifically, the Canny edge detection algorithm after Gaussian operator filtering is used as a rough positioning algorithm to obtain the pixel-level image edge coordinates. The optimal k threshold of each line laser image is automatically obtained through the OTSU algorithm, and the Zernike moment sub-pixel edge detection algorithm is improved. The improved sub-pixel edge detection process is as follows Figure 8 shown.
[0098] In view of the characteristic that there is a bright line on the black background of the single-board line laser image, one or more embodiments of the present invention first use the Canny operator to perform pixel-level edge extraction on the line laser image after Gaussian filtering. When the Canny operator extracts edge coordinates for the line laser image after Gaussian filtering, there will be no more than three edge points at the same horizontal coordinate position after the Canny operator edge extraction. Therefore, the number of pixels that the Zernike moment edge detection algorithm needs to calculate is reduced from 4092×252 pixels of the original image to no more than 4092×3 pixels, which greatly reduces the calculation amount of line laser edge detection. At the same time, the Canny operator has good pixel-level edge extraction accuracy, and is suitable as a coarse positioning algorithm for the line laser Zernike moment sub-pixel edge extraction algorithm provided by one or more embodiments of the present invention, thereby improving the detection speed and detection accuracy of the sub-pixel edge algorithm.
[0099] According to the calculation formulas (2) and (3) proposed by one or more embodiments of the present invention for calculating the depth from the pixel to the actual veneer surface, the sub-pixel edge extraction algorithm and Canny edge detection operator provided by one or more embodiments of the present invention are compared with the actual veneer surface depth, where the measured value is the height from the wooden board surface to the reference plane. The results are shown in Table 1 in millimeters. It can be seen that the sub-pixel edge detection algorithm provided by one or more embodiments of the present invention is the most consistent with the actual height of the veneer surface.
[0100] Table 1 Comparison of actual detection results
[0101]
[0102] Step 104: Convert the image coordinates of the edge points on the edge data into depth data to obtain a one-dimensional data feature of a line laser sample image.
[0103] The edge data obtained by the sub-pixel edge extraction algorithm is refined to obtain a single-pixel edge line, and the depth value corresponding to each pixel is calculated. The obtained depth value is used as a one-dimensional data feature and written into a data table to obtain a line laser sample depth data set.
[0104] Step 105: Construct a single board deep defect dataset.
[0105] The depth information extracted by edge detection is discrete single point information. The defects identified by line laser cannot be judged by independent point data, but need to reflect the defect type on the single board through a continuous data as input signal in a certain area. For a complete line laser edge coordinate height difference data, the data volume is large, resulting in a large number of one-dimensional features, making it difficult for the classification algorithm to fit and converge. At the same time, the time consumption of machine learning will also be longer, and the hardware requirements will also become higher, which does not meet the actual production requirements. In addition, there may be multiple depth defect information in a complete line laser, which makes it difficult to mark the defects and it is difficult to determine the specific location of the defect on the line laser.
[0106] In response to the above problems, in some embodiments, the data of a line of laser is divided into multiple sub-areas according to the idea of sliding window algorithm, that is, sliding windows of equal length are moved on the edge data of the line laser, and the number of data in each unit frame is calculated by non-overlapping sliding windows for equal distance division. Each sub-area is numbered and defect-labeled, and it is calculated that when 93 consecutive pixels are taken as the length of a sub-area, the actual length corresponding to the single board is 8 mm, which meets the length requirement of defect detection. Therefore, it is planned to take 93 consecutive data points as input features, build a classification algorithm according to the construction requirements of different algorithms, and set 5 output layers to obtain classification results, which are knife marks, wrinkles, holes, cracks and normal single boards.
[0107] According to the requirements of the one-dimensional data classification algorithm for data, the height difference between the acquired line laser and the baseline is used as a feature, and numbers are used to represent the corresponding classifications to annotate the data set, where 1 is a knife mark defect, 2 is a shrinkage defect, 3 is a hole defect, 4 is a crack defect, and 5 is a normal single board. The defect types corresponding to different height differences are as follows:
[0108] (1) Knife mark defect: According to the formula in Chapter 3, the number of pixels corresponding to 0.05 mm is 0.592. Considering the thickness of the single board is 2 mm, the row data of the height difference pixel data in the interval of (0, 23.083] and (24.267, 35.513] are marked as 1.
[0109] (2) Shrinkage defect: The row data with height difference pixel data greater than 35.513 is marked as 2.
[0110] (3) Hole defect: There is 0 data in the middle of a row of data, and the number of consecutive 0 data is greater than 18, which is marked as 3. (4) Crack defect: There is 0 data in the middle of a row of data, and the number of consecutive 0 data is less than 18, which is marked as 4.
[0111] (5) Normal single board: The row data of the height difference pixel data in the interval [23.083, 24.267] is marked as 5.
[0112] Finally, 1500 pieces of deep defect feature information are obtained, and each piece of defect feature data includes 93 pixel points corresponding to the depth data and a category identifier. As an example, table storage can be used, where each row of the table corresponds to a piece of defect feature data, with a total of 94 columns. The first 93 columns are the depth data corresponding to the 93 pixel points, and the last column identifies the category of this piece of data.
[0113] Subsequently, the line laser image to be inspected is divided into multiple sub-areas based on the same division rule, and defect identification and classification are performed on each sub-area. By obtaining the number of each sub-area, the category information and number information of the sub-area, the location of the defect can be determined.
[0114] Step 2: Based on the single board deep defect dataset, the random forest algorithm is used to train a single board deep defect recognition model.
[0115] The machine learning classification algorithm has good performance in solving data classification problems. According to the characteristics of line laser edge information data reflecting single board deep defects, one or more embodiments of the present invention provide a machine learning classification algorithm as a key technology for completing the single board deep defect classification work.
[0116] For the random forest algorithm, the main factors affecting the accuracy of the random forest algorithm are the number and depth of trees, the randomness of obtaining data from the data set, and the number of leaf nodes. The number and depth of sub-decision trees contained in the random forest algorithm have a great impact on the accuracy of the model. Generally, the more sub-decision trees there are, the higher the accuracy of the model, but it will also increase the computational complexity. The depth of the random forest is too large or too small, which will affect the accuracy of the model. The optimal number of trees and forest depth need to be determined based on the specific problem. The randomness of the random forest algorithm also affects the accuracy of the model. Its randomness mainly includes the randomness of data sampling and feature selection. The introduction of randomness can improve the generalization ability and stability of the model. The number of leaf nodes, as an important algorithm parameter of the random forest algorithm, also has an important impact on the accuracy of the algorithm.
[0117] According to the characteristics of the single-board dataset, the algorithm is set to contain 93 sub-decision trees, and each decision tree contains a minimum number of leaves of 5. By randomly assigning the dataset to training sets and test sets, the randomness of data sampling and feature selection is improved, and the importance of the evaluation features is recalculated each time training.
[0118] On the basis of enhancing the randomness of the algorithm, it is proposed to use genetic algorithm to optimize the parameters of random forest algorithm. Genetic algorithm is an optimization algorithm based on the principle of biological evolution. It simulates the process of biological evolution, selects the optimal solution from the initial population, and generates new individuals through genetic operations (such as crossover, mutation, etc.), and finally achieves the optimization goal. Genetic algorithm has good performance in solving complex nonlinear optimization problems. Specifically, the number of decision trees and the number of forest layers of random forest are used as optimized hyperparameters for genetic algorithm optimization, and the optimization design of random forest algorithm is completed.
[0119] In order to verify the advantages of the proposed single board deep defect classification algorithm, one or more embodiments of the present invention provide a random forest algorithm, a BP neural network algorithm, a convolutional neural network algorithm and a single board deep defect recognition algorithm provided by one or more embodiments of the present invention through MATLAB software, and a classification comparison experiment of single board surface deep defect data is performed. In one or more embodiments of the present invention, according to the requirements of defect detection, a single depth information point cannot indicate the defect information in the area. By calculation, when 93 continuous pixel point coordinates are taken, the actual length corresponding to the single board is 8mm, which meets the length requirements of defect detection. Therefore, in one or more embodiments of the present invention, 93 continuous points are taken as input features, and a classification algorithm is built according to the requirements of different algorithms. Single board deep defect recognition is a five-classification problem, and numbers are used to represent the corresponding classifications, where 1 is a knife mark defect, 2 is a shrinkage defect, 3 is a hole defect, 4 is a crack defect, and 5 is a normal single board without depth defects. Five output layers are set to obtain the classification results, and the neural network model in the experiment is saved.
[0120] The experiment divides the acquired data set into a training set and a test set in the ratio of 8:2. The accuracy of the four classification algorithms on the test set is as follows: Figure 9(a)-Figure 9(d) As shown, the asterisks in the figure are the identification and classification results of the training set, and the circles are the identification and classification results of the test set. It can be seen from the figure that the single board deep defect recognition algorithm provided by one or more embodiments of the present invention has the lowest discreteness and the best recognition effect.
[0121] Table 2 shows the comparison of the classification results of the four classification algorithms, and the experimental results of the four classification algorithms are analyzed. Compared with the random forest algorithm, the proposed single-board deep defect classification algorithm provided by one or more embodiments of the present invention has an accuracy rate of 0.91% higher than the random forest algorithm on the training set, 5.5% higher than the BP neural network algorithm, and 1% higher than the convolutional neural network algorithm; compared with the random forest algorithm on the test set, the accuracy rate is 2.67% higher, the precision rate is 1.83% higher, the recall rate is 2.84% higher, and the F1 score is 2.33% higher; compared with the BP neural network algorithm, the accuracy rate is 14.67% higher, the precision rate is 13.26% higher, the recall rate is 14.39% higher, and the F1 score is 13.82% higher; compared with the convolutional neural network algorithm, the accuracy rate is 5.67% higher, the precision rate is 5.43% higher, the recall rate is 5.59% higher, and the F1 score is 5.5% higher.
[0122] Table 2 Comparison of experimental results
[0123]
[0124] According to the mixed matrix accuracy comparison table of experimental results, in the single board surface depth defect data classification problem, the single board depth defect classification algorithm based on improved random forest provided by one or more embodiments of the present invention performs best in terms of precision, recall, F1 score, training set accuracy and training set accuracy.
[0125] Based on the above single board deep defect recognition model, such as Figure 5 As shown, the single board depth defect recognition method specifically includes the following steps:
[0126] Step 1: Obtain a line laser image of the board to be inspected;
[0127] Step 2: extracting edges of the line laser image and refining them to obtain single-pixel edge lines;
[0128] Step 3: Calculate the depth value of each pixel in the single-pixel edge line;
[0129] Step 4: Divide the depth value of the single-pixel edge line into sub-regions, each sub-region contains N continuous depth values, and perform depth defect recognition on each sub-region based on the single-board depth defect recognition model.
[0130] After the line laser image of the board to be inspected is acquired in step 1, Gaussian filtering is performed on the line laser image.
[0131] The edge extraction of the line laser image in step 2 specifically includes: roughly locating the line laser in the line laser image based on the Canny edge detection algorithm; obtaining the optimal step intensity threshold of the line laser image through the OTSU algorithm; and based on the optimal step intensity threshold, performing edge detection on the roughly located line laser image based on the Zernike moment edge detection algorithm.
[0132] The step 3 specifically includes:
[0133] Step 301: Calculate the pixel equivalent of the image according to the number of image pixels, image resolution and the actual length of the laser line on the image;
[0134] Step 302: Calculate the number of pixels that each pixel point in the single-pixel edge line is offset from the reference line laser image according to the horizontal inclination angle of the line laser projection;
[0135] Step 303: Calculate the actual offset of each pixel point in the single-pixel edge line according to the number of offset pixels and the pixel equivalent;
[0136] Step 304: Calculate the depth value of each pixel point in the single-pixel edge line according to the line laser projection angle and the actual offset of each pixel point.
[0137] Step 305: Identify the category on the image through the classification network model, and output the area number of the defect position on the image to complete the single board surface deep defect recognition task.
[0138] The specific methods of the above filtering, edge detection and depth value calculation are the same as the processing methods for the line laser sample image mentioned above, and please refer to the above description for details.
[0139] When the depth value of the single pixel edge line is divided into sub-regions in step 4, each sub-region is also numbered, and for a sub-region where a depth defect is identified, the defect type and number of the sub-region are obtained.
[0140] Those skilled in the art can understand that the surface of the same single board is reflected by a plurality of continuous line laser images, and by performing depth defect recognition on the plurality of line laser images, all the depth defects on the single board can be accurately recognized.
[0141] Based on the above method, one or more embodiments of the present invention further provide a single board depth defect recognition system based on line laser images, characterized in that it includes:
[0142] An image acquisition module is configured to acquire a line laser image of the single board to be inspected;
[0143] An edge extraction module is configured to extract edges from the line laser image and refine it to obtain single-pixel edge lines;
[0144] A depth conversion module configured to calculate a depth value of each pixel in a single-pixel edge line;
[0145] A defect recognition module is configured to divide the depth value of the single-pixel edge line into sub-regions, each sub-region includes N continuous depth values, and perform depth defect recognition on each sub-region based on the single-board depth defect recognition model;
[0146] Among them, the single board deep defect recognition model is based on the single board defect feature data set and is trained using a random forest algorithm; the single board defect feature data set includes multiple defect feature data, and each defect feature data includes N continuous depth value data and a defect type.
[0147] The system further comprises a preprocessing module configured to perform Gaussian filtering on the line laser image after acquiring the line laser image of the single board to be inspected.
[0148] One or more embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a single board depth defect recognition method based on line laser images is implemented.
[0149] In some embodiments, the single board depth defect recognition method based on line laser image can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program can be loaded and / or installed into an electronic device via RAM and / or ROM and / or a communication unit. When the computer program is loaded into RAM and / or ROM and executed by the electronic device, one or more steps of the method described above can be performed.
[0150] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.
Claims
1. A single board depth defect recognition method based on line laser image, characterized in that: include: Acquire a line laser image of the board to be inspected; Extracting edges of the line laser image and refining the image to obtain single-pixel edge lines; Calculate the depth value of each pixel in the single-pixel edge line; The step of calculating the depth value of each pixel in the single-pixel edge line specifically includes: Calculate the pixel equivalent of the image based on the number of image pixels, image resolution, and the actual length of the laser line on the image; According to the horizontal inclination angle of the line laser projection, the number of pixels offset by each pixel point in the single-pixel edge line relative to the reference line laser image is calculated; According to the number of offset pixels and pixel equivalent, the actual offset of each pixel point in the single-pixel edge line is calculated; Calculate the depth value of each pixel in the single-pixel edge line according to the line laser projection angle and the actual offset of each pixel; Divide the depth value of the single-pixel edge line into sub-regions, each sub-region contains N continuous depth values, and perform depth defect recognition on each sub-region based on a single-board depth defect recognition model; Among them, the single board deep defect recognition model is based on the single board defect feature data set and is trained using a random forest algorithm; the single board defect feature data set includes multiple defect feature data, and each defect feature data includes N continuous depth value data and a defect type.
2. The single board deep defect recognition method based on line laser image according to claim 1, characterized in that: After the line laser image of the single board to be inspected is acquired, Gaussian filtering is performed on the line laser image.
3. The single board deep defect recognition method based on line laser image according to claim 1 or 2, characterized in that: The edge extraction of the line laser image specifically includes: Based on the Canny edge detection algorithm, the line laser in the line laser image is roughly positioned; Obtaining the optimal step intensity threshold of the line laser image by using the OTSU algorithm; Based on the optimal step intensity threshold, edge detection is performed on the roughly positioned line laser image based on a Zernike moment edge detection algorithm.
4. The single board deep defect recognition method based on line laser image according to claim 1, characterized in that: The number of child decision trees of the random forest algorithm is N.
5. The single board deep defect recognition method based on line laser image according to claim 1, characterized in that: When the depth value of the single-pixel edge line is divided into sub-regions, each sub-region is also numbered, and for a sub-region where a depth defect is identified, the defect type and number of the sub-region are obtained.
6. A single board deep defect recognition system based on line laser image, characterized in that: include: An image acquisition module is configured to acquire a line laser image of the single board to be inspected; An edge extraction module is configured to extract edges from the line laser image and refine it to obtain single-pixel edge lines; A depth conversion module configured to calculate a depth value of each pixel in a single-pixel edge line; The step of calculating the depth value of each pixel in the single-pixel edge line specifically includes: Calculate the pixel equivalent of the image based on the number of image pixels, image resolution, and the actual length of the laser line on the image; According to the horizontal inclination angle of the line laser projection, the number of pixels offset by each pixel point in the single-pixel edge line relative to the reference line laser image is calculated; According to the number of offset pixels and pixel equivalent, the actual offset of each pixel point in the single-pixel edge line is calculated; Calculate the depth value of each pixel in the single-pixel edge line according to the line laser projection angle and the actual offset of each pixel; A defect recognition module is configured to divide the depth value of the single-pixel edge line into sub-regions, each sub-region contains N continuous depth values, and perform depth defect recognition on each sub-region based on a single-board depth defect recognition model; Among them, the single board deep defect recognition model is based on the single board defect feature data set and is trained using a random forest algorithm; the single board defect feature data set includes multiple defect feature data, and each defect feature data includes N continuous depth value data and a defect type.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the single board depth defect recognition method based on line laser image is implemented as described in any one of claims 1-5.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the single board depth defect recognition method based on line laser image as described in any one of claims 1 to 5 is implemented.
9. A single board deep defect recognition system, characterized in that: include: Line laser module, image acquisition module, host computer and signal control module; The line laser module is arranged on one side of the image shooting area, and is used to project the line laser on the surface of the single board; the image acquisition module is used to acquire the single board surface image containing the line laser, and send it to the host computer; The host computer is used to execute the single board depth defect recognition method based on line laser image according to any one of claims 1 to 5; The signal control module is used to control the speed of the transmission mechanism and / or the shooting frequency of the image acquisition module.
