Plate edge sealing defect online detection method and device based on machine vision

Abstract

The invention discloses a machine vision-based plate edge sealing defect online detection method and device, and the method comprises the steps: carrying out the image collection operation of a target plate, and obtaining a collected image; performing block processing on the acquired image to obtain a plurality of block images; constructing an instance segmentation model according to a plate area and a defect area in the block image; performing online prediction on the plate area and the defect area according to the instance segmentation model to obtain a prediction result set; and performing post-processing operation on the prediction result set to obtain a target detection result. The method can improve the precision and speed of plate edge sealing defect detection, and can be widely applied to the technical field of machine vision detection.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision detection, and in particular to an online detection method and device for edge banding defects of plates based on machine vision. Background Art

[0002] Edge banding of plates is a crucial process in furniture manufacturing, and its quality directly affects the appearance and service life of furniture. Traditional detection of edge banding defects mainly relies on manual visual inspection, which has problems such as low efficiency, high omission rate, and being greatly affected by subjective factors. With the development of machine vision technology, automatic detection methods based on machine vision have gradually become a research hotspot.

[0003] In recent years, machine vision technology has developed rapidly, providing a new solution for the automatic detection of edge banding defects of plates.

[0004] However, the existing methods for detecting edge banding defects of plates based on machine vision still have the following deficiencies:

[0005] 1. The detection accuracy is not high: Affected by factors such as illumination and background, it is difficult to accurately identify subtle defects.

[0006] 2. The real-time performance is insufficient: The detection speed is slow, and it is difficult to meet the requirements of high-speed operation of the production line. Summary of the Invention

[0007] In view of this, the main purpose of the embodiments of the present invention is to provide an online detection method and device for edge banding defects of plates based on machine vision, in order to solve at least one of the problems in the prior art. The present invention can improve the detection accuracy and detection speed of edge banding defects of plates.

[0008] To achieve the above object, on the one hand, an embodiment of the present invention provides an online detection method for edge banding defects of plates based on machine vision, and the method includes:

[0009] Performing an image acquisition operation on a target plate to obtain an acquired image;

[0010] Performing a block processing on the acquired image to obtain a plurality of block images;

[0011] Constructing an instance segmentation model according to the plate region and defect region in the block images;

[0012] Performing an online prediction on the plate region and the defect region according to the instance segmentation model to obtain a set of prediction results;

[0013] Performing a post-processing operation on the set of prediction results to obtain a target detection result.

[0014] In some embodiments, the operation of collecting an image of the target board to obtain a collected image includes the following steps:

[0015] Obtain the incoming and outgoing signals of the target board through the IO module and the travel switch;

[0016] Trigger a plurality of line array cameras according to the incoming and outgoing signals;

[0017] Synchronously collect an image of the target board through the line array cameras to obtain the collected image.

[0018] In some embodiments, the operation of performing block processing on the collected image to obtain a plurality of block images includes the following steps:

[0019] Obtain the image width and the first image height of the collected image;

[0020] Preset the block height of a single block image;

[0021] Set the overlapping area height according to the block height;

[0022] Perform block processing on the collected image according to the first image height, the block height, and the overlapping area height to obtain a plurality of the block images.

[0023] In some embodiments, the operation of performing block processing on the collected image to obtain a plurality of block images further includes the following steps:

[0024] Perform a splicing operation on the block images to obtain a spliced image;

[0025] Obtain the second image height of the spliced image;

[0026] Determine whether the second image height is equal to the first image height. If the second image height is not equal to the first image height, perform a filling operation on the last block image so that the third image height of the last block image is equal to the block height.

[0027] In some embodiments, constructing an instance segmentation model according to the board area and the defect area in the block image includes the following steps:

[0028] Perform board category annotation on the board area of the block image and perform defect category annotation on the defect area of the block image to obtain an image data set;

[0029] Train the YOLO model according to the image data set and the training parameters to obtain the instance segmentation model.

[0030] In some embodiments, the post - processing operation on the set of prediction results to obtain the target detection results includes the following steps:

[0031] Perform a significance analysis on the defect features in the set of prediction results to obtain an analysis result;

[0032] Filter the set of prediction results according to the analysis result;

[0033] Preset a defect size threshold condition, and filter the set of prediction results according to the defect size threshold condition;

[0034] Optimize the set of prediction results according to the positional relationship of the defect regions on the target plate;

[0035] Filter the set of prediction results through the non - maximum suppression algorithm.

[0036] In some embodiments, the filtering of the set of prediction results through the non - maximum suppression algorithm includes the following steps:

[0037] Obtain the confidence scores of several defect regions in the set of prediction results;

[0038] Arrange several defect regions according to the confidence scores, and select the defect region with the highest confidence score as the first target region;

[0039] Obtain several second target regions according to the remaining several defect regions;

[0040] Obtain the area of the intersection and the area of the union of the first target region and each second target region;

[0041] Obtain each overlap degree according to the area of the intersection and the area of the union;

[0042] Take the second target regions with the overlap degree less than the system - preset threshold as the third target regions;

[0043] Cover the defect regions according to the third target regions.

[0044] In some embodiments, the formulas used for obtaining the area of the intersection and the area of the union of the first target region and each second target region include:

[0045] A int =max(0,x rt -x lb )×max(0,y rt -y ln )

[0046] A uni = w1×h1 + w i ×h i - A int

[0047] In the formula,

[0048]

[0049] wherein, A int represents the area of the regional intersection; A uni represents the area of the regional union; (x lb , y lb ) represents the lower left coordinate of the intersection area formed by the first target area and the second target area; (x rt , y rt ) represents the upper right coordinate of the intersection area formed by the first target area and the second target area; w1 represents the width of the first target area; h1 represents the height of the first target area; (x1, y1) represents the center point coordinate of the first target area; w i represents the width of the second target area; h i represents the height of the second target area; (x i , y i ) represents the center point coordinate of the second target area; i = 2,..., n; max(.,.) represents the maximum value between two numerical values; min(.,.) represents the minimum value between two numerical values.

[0050] In some embodiments, the formula used to obtain each overlap degree according to the area of the regional intersection and the area of the regional union includes:

[0051]

[0052] wherein, IoU represents the overlap degree; A int represents the area of the regional intersection; A uni represents the area of the regional union.

[0053] To achieve the above object, on the other hand, an embodiment of the present invention proposes an on-line detection device for sheet edge sealing defects based on machine vision, and the device includes:

[0054] A first module, configured to perform an image acquisition operation on a target sheet to obtain an acquired image;

[0055] A second module, configured to perform a block processing on the acquired image to obtain a plurality of block images;

[0056] A third module, configured to construct an instance segmentation model according to the sheet area and the defect area in the block image;

[0057] A fourth module, configured to perform online prediction on the sheet area and the defect area according to the instance segmentation model to obtain a set of prediction results;

[0058] A fifth module, configured to perform post-processing operations on the set of prediction results to obtain a target detection result.

[0059] To achieve the above object, another aspect of the embodiments of the present invention provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the above-mentioned online detection method for sheet edge sealing defects based on machine vision.

[0060] To achieve the above object, another aspect of the embodiments of the present invention provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the above-mentioned online detection method for sheet edge sealing defects based on machine vision.

[0061] To achieve the above object, another aspect of the embodiments of the present invention provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device can read the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, the computer device implements the above-mentioned online detection method for sheet edge sealing defects based on machine vision.

[0062] The embodiments of the present invention at least include the following beneficial effects: The present invention provides an online detection method and device for sheet edge sealing defects based on machine vision. This solution performs image acquisition operations on the target sheet to obtain a captured image; performs block processing on the captured image to obtain several block images, which can significantly improve the detection speed and meet the requirements of high-speed online detection on the production line; constructs an instance segmentation model according to the sheet area and the defect area in the block images; performs online prediction on the sheet area and the defect area according to the instance segmentation model to obtain a set of prediction results; performs post-processing operations on the set of prediction results to obtain a target detection result, which can improve the accuracy and detection speed of sheet edge sealing defect detection. Description of the Drawings

[0063] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0064] Figure 1 It is a flowchart of an online detection method for edge banding defects of sheets based on machine vision provided by an embodiment of the present invention;

[0065] Figure 2 It is a schematic diagram of the working process of an online detection system for edge banding defects provided by an embodiment of the present invention;

[0066] Figure 3 It is a schematic diagram of the specific visual detection process for edge banding defects provided by an embodiment of the present invention;

[0067] Figure 4 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0068] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the embodiments of the present invention. They are only examples of devices and methods consistent with some aspects of the embodiments of the present invention detailed in the appended claims.

[0069] It should be noted that although functional module division is performed in the system schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the system or the order in the flowchart. The terms "first / S100" and "second / S200" in the description, claims and the above-mentioned drawings can be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present invention, the first information can also be called the second information, and similarly, the second information can also be called the first information. Depending on the context, the words "if" and "when" as used herein can be interpreted as "when" or "while" or "in response to determining".

[0070] The terms "at least one", "multiple", "each", "any one", etc. used in the present invention, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any one refers to any one of the multiple.

[0071] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this invention belongs. The terms used herein are for the purpose of describing embodiments of the invention only and are not intended to limit the invention.

[0072] In recent years, machine vision technology has developed rapidly, providing a new solution for the automated detection of edge banding defects of sheets.

[0073] However, the existing methods for detecting edge banding defects of sheets based on machine vision still have the following deficiencies:

[0074] 1. Low detection accuracy: Affected by factors such as illumination and background, it is difficult to accurately identify subtle defects.

[0075] 2. Insufficient real-time performance: The detection speed is slow, making it difficult to meet the requirements of high-speed operation of the production line.

[0076] In view of this, as Figure 1 shown, embodiments of the present invention provide an online detection method for edge banding defects of sheets based on machine vision, which may include but is not limited to steps S100 to S500:

[0077] Step S100, perform an image acquisition operation on the target sheet to obtain an acquired image;

[0078] Step S200, perform a block processing on the acquired image to obtain a number of block images;

[0079] Step S300, construct an instance segmentation model according to the sheet area and defect area in the block images;

[0080] Step S400, perform an online prediction on the sheet area and the defect area according to the instance segmentation model to obtain a set of prediction results;

[0081] Step S500, perform a post-processing operation on the set of prediction results to obtain a target detection result.

[0082] In steps S100 to S500 of some embodiments, by performing image acquisition on the target sheet and analyzing the acquired image using a deep learning algorithm, it is predicted and identified whether there are defects on the sheet. At the same time, through a post-processing algorithm, the false detections that may be generated by deep learning are effectively suppressed, the detection accuracy is significantly improved, and the reliability and accuracy of the sheet quality detection are ensured.

[0083] In some embodiments, step S100 may include but is not limited to steps S110 to S130:

[0084] Step S110, obtain the in-out signals of the target sheet through the IO module and the travel switch;

[0085] Step S120: Trigger a plurality of line array cameras according to the incoming and outgoing signals.

[0086] Step S130: Synchronously collect images of the target board through the line array cameras to obtain the collected images.

[0087] In steps S110 to S130 of some embodiments, the system captures the incoming and outgoing signals of the target board in real time through the IO module and the travel switch to ensure that image collection is triggered when the target board enters the detection area. Three line array cameras, namely the upper, lower, and middle cameras, are used to synchronously collect images of the upper surface edge, lower surface edge, and side edge banding area of the target board respectively to obtain the collected images. Among them, the line array cameras have the characteristics of high resolution and high frame rate, and can stably capture clear and detailed images under the condition that the board moves at high speed in the edge banding machine. Optionally, the upper line array camera and the lower line array camera respectively cover an area of about 30 mm near the edge positions of the upper and lower surfaces of the target board, and the middle line array camera covers the field of view range of boards with a thickness of 18 mm to 60 mm, so as to ensure that the details of the edge banding area can be captured.

[0088] In some embodiments, due to the installation distance between the vision device (such as a line array camera) and the travel switch, there will be a time delay in signal transmission and response. If the trigger signals of the light source and the camera are not precisely delay-controlled, it may occur that the camera starts to expose when the light source has not been fully lit or has already gone out. The system controls the turning on and off of the light source by setting the parameters of delaying the turning on and off of the light source, effectively solving the imaging problem caused by the installation distance between the vision device and the travel switch, and can ensure that the light source reaches the ideal brightness at the appropriate moment of camera exposure, thereby avoiding imaging blurring or insufficient contrast caused by time delay and meeting the requirements of high-precision recognition.

[0089] The size range of the boards produced in customized home furnishing varies greatly, usually between 40 mm and 2800 mm. For the width of the image, the upper and lower cameras need to cover an area of about 30 mm near the edge positions of the upper and lower surfaces of the board, and the middle camera needs to cover the area of boards with a thickness of 18 mm to 60 mm. For the height of the image, due to the uncertainty of the board length, the imaging height is dynamically adjusted according to the actual length of the board.

[0090] In order to meet the requirements of deep learning for high-resolution images, the pixel values of the images collected by the system usually reach the level of hundreds of millions. However, directly using such high-resolution images for training and prediction will significantly increase the computational time consumption. For this reason, in step S200 of some embodiments, the embodiment of the present invention introduces an image segmentation technology to segment the high-resolution image into multiple sub-images with fixed specifications and combines multi-threading technology for parallel processing, thereby significantly reducing the time overhead of image prediction.

[0091] In some embodiments, step S200 may include, but is not limited to, steps S210 to S240:

[0092] Step S210, obtaining the image width and the first image height of the acquired image;

[0093] Step S220, presetting the block height of a single block image;

[0094] Step S230, setting the overlapping region height according to the block height;

[0095] Step S240, performing block processing on the acquired image according to the first image height, the block height, and the overlapping region height, to obtain a plurality of block images.

[0096] Exemplarily, assume that the size of the complete image obtained for the sheet material is W*H, where W represents the image width and H represents the first image height. The first image height H will vary dynamically with the length of the sheet. The system sets the block height value of a single block image to H img , thereby restricting the image height for a single deep learning defect prediction. However, using traditional blocking will cause some defect targets to be incomplete. Therefore, in the system of the embodiments of the present invention, by setting the overlapping region height H dup , there is a certain overlapping region between adjacent image blocks, reducing the false detection and missed detection caused by defect targets being segmented into different images. Optionally, the overlapping region height H dup is generally set to 5% of the block height H img .

[0097] Therefore, the complete acquired image of a target sheet material will be blocked into N block images of W*H img , and there is the following expression:

[0098]

[0099] where N represents the number of block images; H represents the first image height of the acquired image; H img represents the block height of the block image; H dup represents the overlapping region height; represents rounding up.

[0100] In some embodiments, step S200 may further include, but is not limited to, steps S250 to S270:

[0101] Step S250, performing a splicing operation on the block images to obtain a spliced image;

[0102] Step S260: Obtain the second image height of the spliced image;

[0103] Step S270: Determine whether the second image height is equal to the first image height. If the second image height is not equal to the first image height, perform a filling operation on the last piece of the segmented image to make the third image height of the last piece of the segmented image equal to the segmentation height.

[0104] In some embodiments, the expression for the second image height of the spliced image after splicing the segmented images is:

[0105] H res = H img × N - H dup × (N - 1)

[0106] where H res represents the second image height of the spliced image.

[0107] Next, determine whether the second image height of the spliced image is equal to the first image height of the captured image. When i.e., when the second image height is inconsistent with the first image height, the last piece of the segmented image needs to be filled with black pixels RGB(0, 0, 0) to make the height of the last piece of the segmented image satisfy H img , and the filling height can be expressed as:

[0108] H fill = H res - H

[0109] where H fill represents the filling height.

[0110] In some embodiments, step S300 may include but is not limited to steps S310 to S320:

[0111] Step S310: Label the sheet material category of the sheet material area of the segmented image and label the defect category of the defect area of the segmented image to obtain an image data set;

[0112] Step S320: Train the YOLO model according to the image data set and training parameters to obtain the instance segmentation model.

[0113] In step S310 of some embodiments, an image dataset can be obtained by performing board category annotation on the board area of the segmented image and defect category annotation on the described defect area of the segmented image. Exemplarily, the system uses LabelImg as an annotation tool to annotate the images. Only the board category is annotated for the board area, while for the defect area, defect categories such as long tape, short tape, delamination, and dirt are annotated according to customer requirements. Optionally, the annotation information is saved as an annotation file in YOLO format, which contains bounding box coordinates and class labels for subsequent model training. Among them, long tape means that the edge band fed into the edge banding machine is too long, resulting in the length of the edge band being greater than the side length of the board after edge banding; short tape means that the edge band fed into the edge banding machine is too short, resulting in the length of the edge band being less than the side length of the board after edge banding.

[0114] In step S320 of some embodiments, based on the obtained image dataset and training parameters such as batch size and number of iterations provided, the YOLO model is trained to obtain an instance segmentation model. Exemplarily, the YOLO model is trained with the annotated image dataset so that it can accurately identify the board area and the defect area. The YOLO training model provides parameters such as batch size and number of iterations. According to the scenario requirements, the parameter values and the training model are adjusted, and a validation dataset is used to verify the image results, so that the trained model has good generalization ability.

[0115] In step S400 of some embodiments, based on the instance segmentation model, online prediction is performed on the board area and the defect area in the image to be detected, and a prediction result set can be obtained. Exemplarily, during the operation of the production line, the system collects images in real time and uses the instance segmentation model to perform online prediction on the board area and the defect area. The YOLO prediction results will output information such as the class of the target, bounding box coordinates, bounding box size, and confidence. After the system extracts this information, a result set of the board area and the defect area is generated, which is convenient for subsequent defect classification, positioning, and processing.

[0116] In some embodiments, in the case where the edge features of the target board area are obvious, the embodiments of the present invention also perform downsampling processing on the images of the board area for annotation and the predicted images. By reducing the pixel values of the training set images, the efficiency of model training and prediction is improved:

[0117] 1) Downsampling processing: The system performs downsampling on the images of the board area, converting high-resolution images into low-resolution images. This process reduces the amount of data processed by the model by reducing the pixel values of the images, while retaining the key features of the board area.

[0118] 2) Improvement in model training efficiency: Since the amount of image data after downsampling is reduced, the computational resources required by the model during training are significantly reduced, the training speed is significantly accelerated, and the recognition accuracy of the model is not significantly affected.

[0119] 3) Improvement in prediction efficiency: In the online prediction stage, the downsampling process also reduces the amount of computation required for a single prediction, enabling the system to complete the prediction of the sheet area in a shorter time and meet the requirements of high-speed operation of the production line.

[0120] In step S500 of some embodiments, based on the YOLO algorithm, a variety of post-processing algorithms are integrated to suppress false detections caused by deep learning algorithms, further optimize the defect detection results, and reduce false detections.

[0121] In some embodiments, step S500 may include but is not limited to steps S510 to S550:

[0122] Step S510, perform a saliency analysis on the defect features in the prediction result set to obtain an analysis result;

[0123] Step S520, filter the prediction result set according to the analysis result;

[0124] Step S530, preset a defect size threshold condition, and filter the prediction result set according to the defect size threshold condition;

[0125] Step S540, optimize the prediction result set according to the positional relationship of the defect area on the target sheet;

[0126] Step S550, filter the prediction result set through the non-maximum suppression algorithm.

[0127] In steps S510 to S520 of some embodiments, a saliency analysis is performed on the defect features in the prediction result set, and the prediction result set is filtered according to the result of the saliency analysis. Exemplarily, filtering is performed through the saliency of the defect features. For example, if a short band appears on the side of the plate and the imaging feature of the middle camera is more obvious than that of the upper and lower cameras, then the short band is only detected by the middle camera. Although the long band appears on the side of the plate, the imaging features of the upper and lower cameras are more obvious than those of the middle camera, so the long band is only detected by the upper and lower cameras. Optionally, the following are some characteristics of defect imaging:

[0128] 1. For short tapes: The length of the edge band is less than the length of the edge-bonded board. The thickness of the edge band is generally 0.5 mm - 1 mm, and short tapes are generally defined as those with a length of more than 1 mm. After imaging by the upper and lower cameras, a tiny notch will appear at the edge position of the board (usually rectangular), which is likely to misdetect or miss defects. For the middle camera, the thickness of the board is generally 18 mm, and the detection size is at least 18 mm * 1 mm. Moreover, when short tapes appear, the middle camera can show the internal structure of the wood.

[0129] 2. For long tapes: The length of the edge band is greater than the length of the edge-bonded board. The imaging on the middle camera and normal edge bonding both appear as rectangles, making it difficult to judge the occurrence of long tape defects. However, after imaging by the upper and lower cameras, a line similar to an extension line will appear on the side length of the rectangle, protruding from the rectangular area. Therefore, the upper and lower cameras have a higher judgment accuracy.

[0130] In step S530 of some embodiments, by setting defect size threshold conditions, too small or too large misdetection areas are filtered according to these threshold conditions. Exemplarily, the system provides minimum height, maximum height, minimum width, maximum width, minimum area, and maximum area for users to maintain the filtering conditions for defects. When the height of the detected defect is less than the set minimum height, or the height is greater than the set maximum height, or the width of the defect is less than the set minimum width, or the width is greater than the set maximum width, or the area of the defect is less than the set minimum area, or the area is greater than the set maximum area, the defect area will be filtered out.

[0131] In step S540 of some embodiments, since there is a certain probability of misdetection in the YOLO algorithm, some misdetected content can be filtered according to the positional relationship of the defects. Therefore, according to the positional relationship of the defects on the board, the detection results can be further optimized. Exemplarily, for long tapes, the system checks whether the position where the defect is detected is in the height direction of the board area. If it is detected that the defect position is not in the height direction of the board area, then the defect area will be filtered out; for short tapes, it checks whether the position where the defect is detected is inside the board area. If it is detected that the defect position is not inside the board area, then the defect area will be filtered out.

[0132] In step S550 of some embodiments, since the YOLO algorithm may generate multiple bounding boxes for the same defect, the system uses the non-maximum suppression algorithm to retain the optimal result. Among them, the defect detection result set contains n defect areas, where each defect includes the center point x, y coordinates, width w, height h, and confidence score s. This set U can be expressed as:

[0133] {(x1,y1,w1,h1,s1),(x2,y2,w2,h2,s2),…,(x n ,y n, w n , h n , s n )}

[0134] Among them, w1 represents the width of the first target area; h1 represents the height of the first target area; s1 represents the confidence score of the first target area; (x1, y1) represents the center point coordinates of the first target area; w i represents the width of the second target area; h i represents the height of the second target area; s i represents the confidence score of the second target area; (x i , y i ) represents the center point coordinates of the second target area; i = 2, …, n.

[0135] In some embodiments, step S550 may include but is not limited to steps S551 to S557:

[0136] Step S551, obtain the confidence scores of several of the defect areas in the prediction result set;

[0137] Step S552, arrange several of the defect areas according to the confidence scores, and select the defect area with the highest confidence score as the first target area;

[0138] Step S553, obtain several second target areas based on the remaining several defect areas;

[0139] Step S554, obtain the intersection area and union area of the first target area and each of the second target areas;

[0140] Step S555, obtain each overlap degree according to the intersection area and the union area;

[0141] Step S556, use the second target areas with overlap degrees less than the system preset threshold as the third target areas;

[0142] Step S557, cover the defect areas according to the third target areas.

[0143] In steps S551 to S553 of some embodiments, obtain the confidence scores of several defect areas in the prediction result set, sort them from high to low according to the confidence, and select the defect area with the highest confidence as the first target area. Then, use the defect areas remaining in the prediction result set except the first target area as each second target area.

[0144] In steps S554 to S557 of some embodiments, the intersection over union (IoU) of the first target region and each second target region is calculated, and the defective regions with an IoU less than the system preset threshold are retained as the third target regions. Then, the retained third target regions are used to cover the set U, and the steps of obtaining the area of the intersection and the area of the union of the first target region and each of the second target regions are returned until all defective regions are processed.

[0145] Exemplarily, the IoU of the region with the highest confidence and other defective regions is calculated, and the defective regions with an IoU less than the system preset threshold are retained. The IoU can be given by the following formula:

[0146]

[0147] where IoU represents the intersection over union; A int represents the area of the intersection of regions; A uni represents the area of the union of regions.

[0148] Assume that the defective region with the highest confidence is (x1, y1, w g , h1, s1), and one of the other defective regions is (x i , y i , w i , h i , s i ), where i represents any defective region from 2 to n. Then, the lower left coordinates of the intersection region formed by these two defects are:

[0149]

[0150] The upper right coordinates are:

[0151]

[0152] Then, the formula for the area of the intersection of regions A int is as follows:

[0153] A int = max(0, x rt - x lb ) × max(0, y rt - y lb )

[0154] And the formula for the area of the union of regions A uni is as follows:

[0155] A uni = w1 × h1 + w i × h i - A int

[0156] Among them, (x lb , y lb ) represents the lower left corner coordinates of the intersection area formed by the first target area and the second target area; (x rt , y rt ) represents the upper right corner coordinates of the intersection area formed by the first target area and the second target area; w1 represents the width of the first target area; h1 represents the height of the first target area; (x1, y1) represents the center point coordinates of the first target area; w i represents the width of the second target area; h i represents the height of the second target area; (x i , y i ) represents the center point coordinates of the second target area; i = 2, …, n; max(.,.) represents the maximum value between two numerical values; min(.,.) represents the minimum value between two numerical values.

[0157] Next, cover the set U with the remaining defective area, and return the step of calculating the overlap degree between the area with the highest confidence and other defective areas until all defective areas are processed.

[0158] In some embodiments, the object detection results of the system can generate the following data:

[0159] 1) The original image file collected by the system;

[0160] 2) The image file after the system divides the blocks;

[0161] 3) The image file in which the system detects defects, with defect information marked in text in the image and defective areas marked with rectangles.

[0162] Among them, the object detection results of the system will be saved into the local database. For the defective plates, the system can also give alarm prompts through three-color lights and a display screen, and can also be docked with a labeling device or a coding device to mark the defective plates for subsequent operators to process.

[0163] Such as Figure 2As shown in the figure, the online edge - banding defect monitoring system of the embodiment of the present invention captures the incoming and outgoing signals of the board in real - time through the IO module and the travel switch, and during the effective period of the signal, triggers the upper, lower, and middle linear array cameras to collect images of about 30mm regions on the upper and lower surfaces of the board and the edge - banding region on the side surface respectively. Subsequently, the system uses deep - learning algorithms to analyze the collected images, predict and identify whether there are defects such as long strips, short strips, glue opening, and dirt on the board. At the same time, by integrating a variety of post - processing algorithms, it effectively suppresses the false detections that may be generated by deep learning, significantly improves the detection accuracy, and ensures the reliability and accuracy of the board quality inspection. After the detection process is completed, the system light source is turned off and the detection results are recorded and output. According to the detection results, for the boards with defects, the system can give alarm prompts through the three - color lamp and the display screen.

[0164] In some embodiments, the embodiment of the present invention provides an online detection method for board edge - banding defects based on machine vision to improve the detection accuracy, adaptability, and real - time performance. The core modules of the present invention include five modules: an image acquisition module, an image segmentation module, a board and defect prediction module, a post - processing algorithm module, and a result output module. As Figure 3 shown, combining the five modules, the specific edge - banding defect vision detection process of the embodiment of the present invention is as follows:

[0165] 1. Image acquisition module

[0166] The system captures the incoming and outgoing signals of the board in real - time through the IO module and the travel switch to ensure that image acquisition is triggered when the board enters the detection area. Three linear array cameras, namely the upper, lower, and middle ones, are used to synchronously collect images of the upper - surface edge, lower - surface edge, and side - edge - banding region of the board respectively.

[0167] 2. Image segmentation module

[0168] The system introduces image segmentation technology to divide the high - resolution image into multiple sub - images with fixed specifications and combines multi - thread technology for parallel processing, thus significantly reducing the time overhead of image prediction.

[0169] 3. Board and defect prediction module

[0170] The system uses the YOLO algorithm to train the model and make online predictions for the board area of the image and the defect area of the segmented image. Among them, the core steps of the prediction module are divided into image annotation, model training, and online prediction:

[0171] 1) Image annotation: The system uses LabelImg as the annotation tool to annotate the images collected in the early stage;

[0172] 2) Model training: Through the annotated image data set, train the YOLO model so that it can accurately identify the board area and the defect area;

[0173] 3) Online prediction: During the operation of the production line, the system collects images in real time and uses the trained YOLO model to perform online prediction on the sheet area and the defect area, generating a result set of the sheet area and the defect area for subsequent defect classification, positioning, and processing.

[0174] 4. Post-processing algorithm module

[0175] Based on the YOLO algorithm, the system integrates multiple post-processing algorithms to further optimize the defect detection results and reduce false detections. Among them, the system integrating multiple post-processing algorithms includes defect name conditions, size threshold conditions, defect position relationships, and defect suppression algorithms:

[0176] 1) Defect name condition: Filter by the significance of defect features;

[0177] 2) Size threshold condition: Set a threshold condition according to the size of the defect to filter out misdetected areas that are too small or too large;

[0178] 3) Defect position relationship: Further optimize the detection results according to the position relationship of the defects on the sheet;

[0179] 4) Defect suppression algorithm: Adopt the non-maximum suppression algorithm to retain the optimal results.

[0180] 5. Result output module

[0181] The system detection results will generate the following data:

[0182] 1) The original image file collected by the system;

[0183] 2) The image file after the system divides the blocks;

[0184] 3) The image file in which the system detects defects. The defect information is marked with text in the image, and the defect area is marked with a rectangle;

[0185] Among them, the system detection results will be saved into the local database.

[0186] The embodiment of the present invention also provides an online detection device for the edge sealing defects of sheets based on machine vision, which can implement the above-mentioned online detection method for the edge sealing defects of sheets based on machine vision. The device includes:

[0187] The first module is used to perform image acquisition operations on the target sheet to obtain the acquired image;

[0188] The second module is used to perform block processing on the acquired image to obtain several block images;

[0189] A third module, configured to construct an instance segmentation model according to the sheet area and the defect area in the segmented image;

[0190] A fourth module, configured to perform online prediction on the sheet area and the defect area according to the instance segmentation model to obtain a set of prediction results;

[0191] A fifth module, configured to perform post-processing operations on the set of prediction results to obtain a target detection result.

[0192] It can be understood that the content in the above method embodiments is applicable to the device embodiments. The functions specifically implemented in the device embodiments are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those in the above method embodiments.

[0193] An embodiment of the present invention further provides an electronic device, which includes a processor and a memory. The memory stores a computer program, and when the processor executes the computer program, the above-mentioned online detection method for sheet edge sealing defects based on machine vision is implemented. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0194] It can be understood that the content in the above method embodiments is applicable to the device embodiments. The functions specifically implemented in the device embodiments are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those in the above method embodiments.

[0195] Refer to Figure 4 , Figure 4 which schematically shows the hardware structure of an electronic device in another embodiment. The electronic device includes:

[0196] A processor 601, which can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is configured to execute relevant programs to implement the technical solutions provided by the embodiments of the present invention;

[0197] The memory 602 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 602 can store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 602 and are called by the processor 601 to execute an online detection method for edge-sealing defects of plates based on machine vision according to an embodiment of the present invention;

[0198] The input / output interface 603 is used to implement information input and output;

[0199] The communication interface 604 is used to implement communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.);

[0200] The bus 605 transmits information between various components of the device (such as the processor 601, the memory 602, the input / output interface 603, and the communication interface 604);

[0201] Among them, the processor 601, the memory 602, the input / output interface 603, and the communication interface 604 are communicatively connected to each other inside the device through the bus 605.

[0202] An embodiment of the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the above-mentioned online detection method for edge-sealing defects of plates based on machine vision is implemented.

[0203] It can be understood that the content in the above method embodiments is applicable to the embodiments of this storage medium. The functions specifically implemented by the embodiments of this storage medium are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0204] An embodiment of the present invention also provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions. The computer instructions are stored in a computer-readable storage medium. The processor of the computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned online detection method for edge-sealing defects of plates based on machine vision.

[0205] In summary, an online detection method and device for edge-sealing defects of plates based on machine vision according to an embodiment of the present invention have the following advantages:

[0206] 1. The embodiments of the present invention adopt deep learning algorithms, which can accurately identify various types of edge sealing defects, including tiny defects and defects under complex backgrounds, and have the advantage of high-precision detection.

[0207] 2. The embodiments of the present invention adopt segmented sampling of images, reducing the pixel amount of a single prediction, significantly improving the detection speed, meeting the requirements of high-speed online detection on the production line, and having the characteristics of high efficiency and real-time performance.

[0208] 3. The embodiments of the present invention establish multiple post-processing algorithms according to the defect characteristics to suppress the false detections caused by deep learning algorithms, and have strong adaptability.

[0209] 4. The embodiments of the present invention can realize the full-process automation from image acquisition to defect recognition, positioning, and result output, reduce manual intervention, and improve production efficiency and product quality.

[0210] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order mentioned in the operating diagrams. For example, depending on the functions / operations involved, two consecutive blocks shown may actually be executed substantially simultaneously or the blocks can sometimes be executed in the reverse order. In addition, the embodiments presented and described in the flowcharts of the present invention are provided by way of example for the purpose of providing a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated, where the order of various operations is changed and where sub-operations described as part of a larger operation are executed independently.

[0211] In addition, although the present invention has been described in the context of functional modules, it should be understood that one or more of the functions and / or features described, unless otherwise stated to the contrary, may be integrated in a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It can also be understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More precisely, considering the attributes, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the modules will be understood within the ordinary skills of an engineer. Therefore, those skilled in the art can implement the present invention as set forth in the claims without undue experimentation. It can also be understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0212] If the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0213] The logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a predefined sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in combination with an instruction execution system, apparatus, or device.

[0214] More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection part with one or more wirings (electronic device), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber device, and portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, then editing, interpreting, or processing it in other suitable ways as necessary, and then storing it in a computer memory.

[0215] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), and the like.

[0216] In the description of this specification, the descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0217] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the claims and their equivalents.

[0218] The above has specifically described the preferred embodiments of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art can also make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included within the scope defined by the claims of the present invention.

Claims

1. A method for online detection of plate edge defects based on machine vision, characterized in that: The following steps are involved: Performing image acquisition operation on the target plate to obtain an acquired image; Performing block processing on the collected image to obtain a plurality of block images; constructing an instance segmentation model according to the plate area and the defect area in the block image; According to the instance segmentation model, online prediction is performed on the plate area and the defect area to obtain a set of prediction results; Post-processing is performed on the prediction result set to obtain a target detection result.

2. The method for online detection of sheet edge defects based on machine vision according to claim 1, characterized in that: The step of performing an image acquisition operation on the target plate to obtain an acquired image comprises the following steps: Obtain the in and out signals of the target plate through the IO module and the travel switch; triggering a plurality of linear array cameras according to the in / out signals; The target plate is synchronously imaged by the linear array camera to obtain the image.

3. The method for online detection of sheet edge defects based on machine vision according to claim 1, characterized in that: The step of performing block processing on the acquired image to obtain a plurality of block images comprises the following steps: Obtaining an image width and a first image height of the acquired image; Presetting a block height of a single block image; According to the block height, setting the overlapping area height; The acquired image is processed into blocks according to the first image height, the block height and the overlapping area height to obtain a plurality of the block images.

4. The method for online detection of sheet edge defects based on machine vision according to claim 3, characterized in that: The block processing of the acquired image to obtain a plurality of block images also includes the following steps: Performing a stitching operation on the block images to obtain a stitched image; Acquire a second image height of the stitched image; Determine whether the second image height is equal to the first image height. If the second image height is not equal to the first image height, perform a padding operation on the last block image so that the third image height of the last block image is equal to the block height.

5. The method for online detection of sheet edge defects based on machine vision according to claim 1, characterized in that: The example segmentation model is constructed according to the plate area and defect area in the block image, comprising the following steps: Performing plate type labeling on the plate area of ​​the segmented image, and performing defect type labeling on the defect area of ​​the segmented image, to obtain an image data set; The YOLO model is trained according to the image data set and the training parameters to obtain the instance segmentation model.

6. The method for online detection of plate edge defects based on machine vision according to claim 1, characterized in that: The post-processing operation is performed on the prediction result set to obtain the target detection result, including the following steps: Performing a significance analysis on the defect features in the prediction result set to obtain an analysis result; According to the analysis result, filtering the prediction result set; Presetting a defect size threshold condition, and filtering the prediction result set according to the defect size threshold condition; Optimizing the set of prediction results according to the positional relationship of the defective area on the target plate; The prediction result set is filtered by a non-maximum suppression algorithm.

7. The method for online detection of plate edge defects based on machine vision according to claim 6, characterized in that: The filtering of the prediction result set by the non-maximum suppression algorithm comprises the following steps: Obtaining confidence scores of a plurality of the defect areas in the prediction result set; Arranging the plurality of defect regions according to the confidence scores, and selecting the defect region with the highest confidence score as the first target region; According to the remaining defective regions, a plurality of second target regions are obtained; Obtaining an area of ​​intersection and an area of ​​union of the first target area and each of the second target areas; Obtaining respective overlapping degrees according to the intersection area of ​​the regions and the union area of ​​the regions; The second target area whose overlap degree is less than a preset threshold of the system is used as the third target area; The defective area is covered according to the third target area.

8. The method for online detection of plate edge defects based on machine vision according to claim 7, characterized in that: The formula used to obtain the intersection area and union area of ​​the first target area and each of the second target areas includes: TO int =max(0,x rt -x lb )×max(0,y rt -and lb ) A uni =w1×h1+w i ×h i -A int In the formula, Among them, A int Represents the area of ​​regional intersection; A uni represents the area of ​​the union of regions; (x lb ,y lb ) represents the lower left corner coordinate of the intersection area formed by the first target area and the second target area; (x rt ,y rt ) represents the coordinates of the upper right corner of the intersection area formed by the first target area and the second target area; w1 represents the width of the first target area; h1 represents the height of the first target area; (x1, y1) represents the coordinates of the center point of the first target area; w i Indicates the width of the second target area; h i Indicates the height of the second target area; (x i ,y i ) represents the center point coordinates of the second target area; i=2,…,n; max(.,.) represents the maximum value between two values; min(.,.) represents the minimum value between two values.

9. The method for online detection of plate edge defects based on machine vision according to claim 7, characterized in that: The formulas used to obtain the overlapping degrees according to the intersection area of ​​the regions and the union area of ​​the regions include: Among them, IoU represents the overlap; A int Represents the area of ​​regional intersection; A uni Represents the area of ​​the union of regions.

10. An online detection device for plate edge defects based on machine vision, characterized in that: include: The first module is used to perform image acquisition operation on the target plate to obtain the acquired image; The second module is used to perform block processing on the collected image to obtain a plurality of block images; The third module is used to construct an instance segmentation model according to the plate area and the defect area in the block image; A fourth module is used to perform online prediction on the plate area and the defect area according to the instance segmentation model to obtain a set of prediction results; The fifth module is used to perform post-processing operations on the prediction result set to obtain the target detection result.

Images