Glue path quality detection method and glue path quality detection device

Abstract

The invention discloses a glue path quality detection method and a glue path quality detection device. The glue path quality detection method comprises the following steps: acquiring a to-be-detected image; detecting a first type of defects of the to-be-detected image based on the instance segmentation model; performing glue path extraction on the to-be-detected image based on the semantic segmentation model, and outputting a glue path binary image; setting a glue detection area in the glue path binary image, wherein the glue detection area comprises a first center track; in the glue detection area, glue path edge detection is conducted in the direction perpendicular to the first center track, and a plurality of edge point pairs distributed on the two sides of the first center track are obtained; detecting and measuring a second type of defects in the glue detection area according to the plurality of edge points; and outputting the detection result of the first type of defects and the detection and measurement results of the second type of defects. Therefore, all defects related to the glue path can be detected, the detection precision can reach the limit of human eyes, and the glue path quality detection in complex scenes such as glue path reflection, color unevenness, chromatic aberration and background interference is compatible.

Description

Technical Field

[0001] The present application relates to the technical field of defect detection, and in particular to a glue road quality detection method, a glue road quality detection device, a glue road quality detection system and a computer-readable storage medium. Background Art

[0002] Glue path quality inspection is an important inspection task in the industrial manufacturing of smart phones, smart watches, speakers, motors, lenses, etc. Its purpose is to accurately and timely inspect the quality of the glue path of the product after the glue is dispensed, and detect defective glue paths to ensure the smooth progress of the subsequent product assembly process. At present, due to factors such as reflection in the product glue path imaging (caused by high reflection of the product material or glue), uneven color (caused by inconsistent thickness of glue dispensed), color difference (the glue composition of the factory cannot be completely consistent), and background interference (the background texture or color is similar to the glue), it is impossible to accurately inspect the glue path quality. Summary of the invention

[0003] The embodiments of the present application provide a glue road quality detection method, a glue road quality detection device, a glue road quality detection system and a computer-readable storage medium to solve at least one of the above-mentioned technical problems.

[0004] The method for detecting the quality of a rubber road according to the embodiment of the present application comprises:

[0005] Acquire an image to be detected, wherein the image to be detected includes a glue path area;

[0006] Performing detection of first-category defects on the image to be detected based on an instance segmentation model;

[0007] Extracting glue paths from the image to be detected based on a semantic segmentation model, and outputting a binary image of the glue paths;

[0008] Setting a glue inspection area in the glue path binary image, wherein the glue inspection area includes a first center track;

[0009] In the glue inspection area, glue path edge detection is performed along a direction perpendicular to the first center track to obtain a plurality of edge point pairs distributed along both sides of the first center track;

[0010] Detecting and measuring the second type of defects in the glue inspection area according to the plurality of edge points;

[0011] Outputting the detection result of the first type of defects of the image to be detected, and the detection and measurement results of the second type of defects.

[0012] In some embodiments, the first type of defects includes any one or more of wire drawing defects, foreign body defects, and bubble defects, and the detecting of the first type of defects on the image to be detected based on the instance segmentation model includes:

[0013] Obtaining the instance segmentation model;

[0014] Splitting the image to be detected into a plurality of regions to be detected, each of the regions to be detected having an overlapping area with an adjacent region to be detected in a first preset ratio;

[0015] Inputting a plurality of the regions to be inspected into the instance segmentation model to determine the first type of defects in each of the regions to be inspected;

[0016] Restoring the plurality of the to-be-detected areas to the to-be-detected image, and merging the first-type defects that overlap and have the same defect type in adjacent to the to-be-detected areas;

[0017] Output the first type of defects in the image to be detected, and the minimum circumscribed rectangular frame and defect type corresponding to each of the first type of defects.

[0018] In some embodiments, the step of extracting glue paths from the image to be detected based on a semantic segmentation model and outputting a binary image of the glue paths includes:

[0019] Acquire the semantic segmentation model;

[0020] Determine a plurality of regions of interest in the image to be detected, each of the regions of interest having an overlapping area with an adjacent region of interest of a second preset ratio;

[0021] Inputting the plurality of regions of interest into the semantic segmentation model to perform glue path extraction on the plurality of regions of interest;

[0022] The plurality of regions of interest are restored to the image to be detected, and an OR operation is performed on the overlapping glue paths in adjacent regions of interest to output a binary image of the glue paths.

[0023] In some embodiments, the second type of defects include excess glue defects, insufficient glue defects, glue break defects, glue length defects, glue quantity defects, and glue deviation defects, and the detection and measurement of the second type of defects in the glue inspection area according to the plurality of edge points includes:

[0024] Determine a corresponding glue path center point according to each edge point pair;

[0025] Determine a first width corresponding to each edge point according to the plurality of edge point pairs and the corresponding plurality of glue path center points, so as to detect and measure the excess glue defect and the insufficient glue defect;

[0026] Correcting the first center trajectory according to the plurality of center points of the glue path to determine a second center trajectory;

[0027] Determine whether the two nearest pixel points on both sides of the second center track belong to glue path pixel points, so as to detect and measure the glue breaking defect;

[0028] Detecting and measuring the glue length defects according to the multiple glue path center points;

[0029] Analyzing the glue path pixels according to a connected domain analysis algorithm to detect and measure the glue quantity defects;

[0030] The centroids of the plurality of glue path pixel points are calculated to detect and measure the glue deviation defects.

[0031] In some embodiments, the determining of the first width corresponding to each edge point according to the plurality of edge point pairs and the corresponding plurality of glue path center points to detect and measure the excess glue defect and the insufficient glue defect includes:

[0032] Calculate the first width corresponding to each edge point according to the plurality of edge point pairs and the corresponding plurality of glue path center points;

[0033] When the first width of a plurality of edge points exceeding a first predetermined number in succession is greater than half of a first preset glue width, it is determined that the positions corresponding to the plurality of edge points have the excess glue defect;

[0034] When the first width of a plurality of edge points exceeding a second predetermined number in succession is less than half of a second preset glue width, it is determined that the glue deficiency defect exists at positions corresponding to the plurality of edge points;

[0035] Determine a multi-glue defect point set according to a plurality of edge points located at and adjacent to the multi-glue defect positions, and output a starting point, an end point, a highest point, a length, the multi-glue defect point set, a defect region, and a defect area of ​​each multi-glue defect;

[0036] A glue shortage defect point set is determined based on a plurality of edge points located at and adjacent to the glue shortage defect position, and the starting point, end point, lowest point, length, glue shortage defect point set, defect region and defect area of ​​each glue shortage defect are output.

[0037] In some embodiments, the glue break defect includes a straight glue break defect and an oblique glue break defect, the second center track includes a plurality of center track points, and the determining whether the two nearest pixel points on both sides of the second center track belong to glue path pixel points to detect and measure the glue break defect includes:

[0038] Determine whether the two nearest pixel points on both sides of each of the central track points along the second central track belong to the glue path pixel points;

[0039] Determine whether each of the central trajectory points has a corresponding edge point pair;

[0040] When the two nearest pixel points on both sides of the second central track of a plurality of the central track points exceeding a third predetermined number continuously do not belong to the glue path pixel points, and the plurality of the central track points do not have corresponding edge point pairs, it is determined that the positions corresponding to the plurality of the central track points have the straight-mouth glue break defect;

[0041] When the two nearest pixel points on both sides of the second central trajectory of a plurality of the central trajectory points exceeding the third predetermined number continuously do not belong to the glue path pixel points, and the plurality of the central trajectory points have corresponding edge point pairs, the plurality of the central trajectory points are added to the set of points to be detected, so as to detect and measure the oblique glue breaking defect;

[0042] A straight-mouth glue break defect point set is determined according to the glue path pixel points located on both sides of the straight-mouth glue break defect position, and the starting point, end point, length, straight-mouth glue break defect point set and defect area of ​​each straight-mouth glue break defect are output.

[0043] In some embodiments, adding the plurality of center track points to a set of points to be detected to detect and measure the oblique glue break defect includes:

[0044] Determine a central detection point according to the set of points to be detected, and perform edge point search along a first direction according to the central detection point to determine a first detection point pair and a first detection line corresponding to the first detection point pair;

[0045] Move the first detection line along a second direction with a predetermined step length, and perform edge point search again along the first direction to determine a plurality of second detection point pairs and second detection lines corresponding to the second detection point pairs, wherein the second direction is perpendicular to the first direction;

[0046] For each of the second detection lines, the number of glue path pixels on the second detection line is calculated;

[0047] When the number of pixels of the glue path is equal to the preset number of pixels, it is determined that the position corresponding to the second detection line does not have the oblique glue break defect, and the detection and measurement of the oblique glue break defect are exited, wherein the preset number of pixels is the sum of the number of pixels of the second detection line and twice the preset tolerance number of pixels;

[0048] When the number of pixels of the glue path is less than the preset number of pixels, it is determined that the oblique glue break defect exists at the position corresponding to the second detection line;

[0049] The oblique glue break defect point set is determined according to the first detection point pair and the second detection point pair, and the starting point, end point, length, the oblique glue break defect point set and the defect area of ​​each oblique glue break defect are output.

[0050] In some embodiments, after determining whether the central track point has a corresponding edge point pair, detecting and measuring the second type of defects in the glue inspection area according to the plurality of edge point pairs further includes:

[0051] When the two nearest pixel points on both sides of the second central track of a plurality of the central track points exceeding a fourth predetermined number continuously do not belong to the glue path pixel points, and a corresponding edge point exists at the plurality of the central track points, it is determined that there is a glue shortage defect at the position corresponding to the central track point;

[0052] A glue deficiency defect point set is determined based on a plurality of edge points located at and adjacent to the glue deficiency defect position, so as to output the starting point, end point, lowest point, length, glue deficiency defect point set, defect region and defect area of ​​each glue deficiency defect.

[0053] In certain embodiments, the calculating of the centroids of the plurality of glue path pixel points to detect and measure glue deviation defects includes:

[0054] Calculating the centroid of a plurality of the glue path pixel points;

[0055] Determine a detection reference for the adhesive deviation defect, wherein the detection reference includes any one or more of a reference point, a reference line, and a reference profile;

[0056] Calculating a first minimum distance from the centroid to the detection reference;

[0057] When the first minimum distance is greater than a preset distance, it is determined that the glue deviation defect exists in the glue inspection area, and the first minimum distance is output as the glue deviation amount;

[0058] When the detection benchmark includes the benchmark contour, the second minimum distance and the maximum distance between multiple edge points and the benchmark contour, and the third minimum distance between the center point of the minimum circumscribed rectangular frame of multiple glue path pixel points and the benchmark contour are calculated and output as the glue deviation.

[0059] The glue road quality detection device of the embodiment of the present application includes:

[0060] An acquisition module, used for acquiring an image to be detected, wherein the image to be detected includes a glue path area;

[0061] A first defect detection module, used for detecting a first type of defect on the image to be detected based on an instance segmentation model;

[0062] An extraction module, used for extracting the glue path from the image to be detected based on a semantic segmentation model, and outputting a binary image of the glue path;

[0063] A determination module, configured to set a glue detection area in the glue path binary image, wherein the glue detection area includes a first center track;

[0064] An edge detection module is used to perform glue path edge detection in the glue detection area along a direction perpendicular to the first central track to obtain a plurality of edge point pairs distributed along both sides of the first central track;

[0065] A second defect detection module, used for detecting and measuring a second type of defect in the glue inspection area according to the plurality of edge points;

[0066] An output module is used to output the detection result of the first type of defects in the image to be detected, and the detection and measurement results of the second type of defects.

[0067] The glue road quality detection system of the implementation mode of the present application includes one or more processors and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the glue road quality detection method of any of the above implementation modes is implemented.

[0068] The computer-readable storage medium of the embodiment of the present application stores a computer program thereon, and when the program is executed by a processor, the glue path quality detection method of any of the above-mentioned embodiments is implemented.

[0069] The glue road quality detection method, glue road quality detection device, glue road quality detection system and computer-readable storage medium of the embodiment of the present application use a deep learning model to detect the quality of the glue road, detect the first type of defects in the glue road area based on the instance segmentation model, and detect and measure the second type of defects in the glue road area based on the semantic segmentation model. In this way, all defects related to the glue road can be detected, and the detection accuracy can reach the limit of the human eye, and it is compatible with the glue road quality detection in complex scenes such as glue road reflection, color unevenness, color difference and background interference.

[0070] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0072] Figure 1 It is a schematic flow chart of a method for detecting the quality of a rubber road according to certain embodiments of the present application;

[0073] Figure 2 It is a schematic flow chart of a method for detecting the quality of a rubber road according to certain embodiments of the present application;

[0074] Figure 3 is a schematic diagram of determining multiple regions of interest in an image to be detected according to certain embodiments of the present application;

[0075] Figure 4 is a schematic diagram of setting a glue inspection area in a glue path binary image according to certain embodiments of the present application;

[0076] Figure 5 is a schematic diagram of setting a glue inspection area in a glue path binary image according to certain embodiments of the present application;

[0077] Figure 6 is a schematic diagram of performing glue path edge detection along a direction perpendicular to the first center track according to certain embodiments of the present application;

[0078] Figure 7 It is a schematic flow chart of a method for detecting the quality of a rubber road according to certain embodiments of the present application;

[0079] Figure 8 It is a schematic flow chart of a method for detecting the quality of a rubber road according to certain embodiments of the present application;

[0080] Fig. 9 It is a schematic flow chart of a method for detecting the quality of a rubber road according to certain embodiments of the present application;

[0081] Fig.10 is a schematic diagram of a glue path defect in certain embodiments of the present application;

[0082] Fig.11 It is a schematic flow chart of a method for detecting the quality of a rubber road according to certain embodiments of the present application;

[0083] Fig.12 It is a schematic diagram of detection and measurement of multiple glue defects in certain embodiments of the present application;

[0084] Fig.13 It is a schematic diagram of detecting and measuring glue deficiency defects in certain embodiments of the present application;

[0085] Fig.14 It is a schematic flow chart of a method for detecting the quality of a rubber road according to certain embodiments of the present application;

[0086] Fig.15 It is a schematic diagram of a straight-mouth glue break defect and an oblique-mouth glue break defect in certain embodiments of the present application;

[0087] Fig.16 It is a schematic diagram of detecting straight-mouth glue break defects in certain embodiments of the present application;

[0088] Fig.17 It is a schematic diagram of detecting the oblique glue break defect in certain embodiments of the present application;

[0089] Fig.18 It is a schematic flow chart of a method for detecting the quality of a rubber road according to certain embodiments of the present application;

[0090] Fig.19 It is a schematic diagram of detecting straight-mouth glue break defect and glue deficiency defect in certain embodiments of the present application;

[0091] Fig. 20 It is a schematic flow chart of a method for detecting the quality of a rubber road according to certain embodiments of the present application;

[0092] Fig.21 It is a schematic diagram of detecting the oblique glue break defect in certain embodiments of the present application;

[0093] Fig. 22 It is a schematic diagram of a connected glue path existing at a position corresponding to a detection line in certain embodiments of the present application;

[0094] Fig.23 It is a schematic diagram of determining a defect area of ​​an oblique glue break defect in certain embodiments of the present application;

[0095] Fig.24 It is a schematic flow chart of a method for detecting the quality of a rubber road according to certain embodiments of the present application;

[0096] Fig.25 It is a schematic diagram of detection and measurement of adhesive deviation defects when the detection benchmark includes a benchmark point in certain embodiments of the present application;

[0097] Fig.26 It is a schematic diagram of detection and measurement of adhesive deviation defects when the detection reference includes a reference line in certain embodiments of the present application;

[0098] Fig. 27 It is a schematic diagram of detecting and measuring adhesive deviation defects when the detection benchmark includes a benchmark profile in certain embodiments of the present application;

[0099] Fig.28 It is a schematic diagram of a module of a glue road quality detection device according to certain embodiments of the present application;

[0100] Fig.29 It is a schematic diagram of a module of a glue road quality detection device according to certain embodiments of the present application;

[0101] Fig.30 It is a schematic diagram of the connection status between a computer-readable storage medium and a processor in certain embodiments of the present application. DETAILED DESCRIPTION

[0102] The following is a further description of the embodiments of the present application in conjunction with the accompanying drawings. The same or similar reference numerals in the accompanying drawings represent the same or similar elements or elements with the same or similar functions from beginning to end. In addition, the embodiments of the present application described below in conjunction with the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the present application.

[0103] See also Figure 1 and Figure 2 The embodiment of the present application provides a method for detecting the quality of a glue road. The method for detecting the quality of a glue road includes:

[0104] 010: Acquire the image to be detected, which includes the glue path area;

[0105] 020: Detect the first type of defects in the image to be inspected based on the instance segmentation model;

[0106] 030: Extract glue paths from the image to be detected based on the semantic segmentation model and output a binary image of the glue paths;

[0107] 040: setting a glue inspection area in the glue path binary image, wherein the glue inspection area includes a first center track;

[0108] 050: In the glue inspection area, glue path edge detection is performed along a direction perpendicular to the first center track to obtain a plurality of edge point pairs distributed along both sides of the first center track;

[0109] 060: Detect and measure the second type of defects in the glue inspection area based on multiple edge points;

[0110] 070: Output the detection result of the first type of defects of the image to be detected, and the detection and measurement results of the second type of defects.

[0111] The glue road quality detection method of the embodiment of the present application adopts a deep learning model to detect the quality of the glue road, detects the first type of defects in the glue road area based on an instance segmentation model, and detects and measures the second type of defects in the glue road area based on a semantic segmentation model. In this way, all defects related to the glue road can be detected, and the detection accuracy can reach the limit of the human eye, and it is compatible with the glue road quality detection in complex scenes such as glue road reflection, color unevenness, color difference and background interference.

[0112] Specifically, after the glue is applied to products such as smart phones, smart watches, speakers, motors, lenses, etc., they are photographed to obtain the image to be detected. Figure 3The image to be detected includes a glue path area, which is the area corresponding to the glue path formed by glue dispensing in the image to be detected. Figure 3 The annular area in the middle is also the glue path area.

[0113] The defects of the rubber road quality inspection can be divided into the first type of defects and the second type of defects. For the first type of defects, it is only necessary to detect whether the defects exist. For the second type of defects, in addition to detecting whether the defects exist, it is also necessary to accurately measure the starting point, end point, length, area, etc. of the defects.

[0114] After acquiring the image to be inspected, the first type of defects can be detected on the image to be inspected based on the instance segmentation model. The instance segmentation model can adopt any deep learning instance segmentation model, without limitation here. For example, the YOLOv8-seg model, RF-Next model or Mask R-CNN model can be adopted.

[0115] Based on the semantic segmentation model, the second type of defects can be detected and measured for the image to be inspected. The semantic segmentation model can adopt any deep learning semantic segmentation model without limitation. For example, the Segformer model, BiSeNetV2 model or DNLNet model can be adopted.

[0116] In the implementation mode of the present application, a deep learning model is used to detect the quality of the rubber road, the first type of defects in the rubber road area are detected based on the instance segmentation model, and the second type of defects in the rubber road area are detected and measured based on the semantic segmentation model. The deep learning model has a strong feature extraction and generalization capability. In the rubber road quality detection, the deep learning model can learn the shape, texture, color and other features of the rubber road, as well as the change patterns of these features under different lighting, color unevenness, color difference and background interference. In this way, all types of defects related to the rubber road can be detected, and the detection accuracy can reach the limit of the human eye, and it is compatible with the rubber road quality detection in complex scenes such as rubber road reflection, color unevenness, color difference and background interference.

[0117] The specific process of detecting and measuring the second type of defects in the image to be inspected based on the semantic segmentation model is as follows: First, the glue path is extracted from the image to be inspected based on the semantic segmentation model, the glue path area in the image to be inspected is extracted, and a binary image of the glue path is output. In the binary image of the glue path, a glue inspection area can be set according to the glue path area to detect and measure the second type of defects in the glue path area. The center line of the glue inspection area is the first center trajectory.

[0118] It should be noted that since the product glue path can be in various shapes such as annular, linear, S-shaped, etc., the glue path area can also be in various shapes such as annular, linear, S-shaped, etc. The glue inspection area can be set in different shapes according to different settings of the glue path area, and the glue inspection area can also be set in sections. For example, when the glue path area is linear, the glue inspection area can be set as a rectangular area, and when the glue path area is annular, the glue inspection area can be set as a ring area, or it can be set in sections as multiple ring areas (such as Figure 4 In addition, the length, width and other dimensions of the segmented glue inspection area may be the same or different, such as Figure 5 As shown, the width of the glue inspection area 1 is smaller than the width of the inspection area 2. In this way, the quality of different special-shaped glue paths can be inspected.

[0119] After setting the glue inspection area, in the glue inspection area, the glue path edge detection is performed on both sides of the first center track along the direction perpendicular to the first center track. It should be noted that the first center track can be a straight line or a curve. When the first center track is a curve, the direction perpendicular to the first center track is also the normal direction of the first center track. For annular or S-shaped special-shaped glue path areas, edge detection is performed along the direction perpendicular to the first center track, which can more accurately capture the edge of the special-shaped glue path, thereby improving the accuracy of subsequent defect detection.

[0120] like Figure 6 As shown, the two sides of the first center track are divided into the left side and the right side. Starting from any point on the first center track, the edge detection of the glue path is performed to the left along the direction perpendicular to the first center track. The point that transitions from light (grayscale value is 255) to dark (grayscale value is 0) is detected as the left edge point; the edge detection of the glue path is performed to the right along the direction perpendicular to the first center track. The point that transitions from light to dark is detected as the right edge point. A left edge point and a corresponding right edge point constitute an edge point pair distributed along the two sides of the first center track.

[0121] If the grayscale values ​​of the pixels on the left and / or right of the first center track in the glue inspection area are both 255, the leftmost and / or rightmost pixels are used as left edge points and / or right edge points.

[0122] Afterwards, the second type of defects can be detected and measured in the glue inspection area according to the plurality of edge point pairs. Finally, the detection results of the first type of defects of the image to be inspected and the detection and measurement results of the second type of defects are output.

[0123] See also Figure 7 In some embodiments, the first type of defects include any one or more of wire drawing defects, foreign body defects, and bubble defects. Detecting the first type of defects on the image to be detected based on the instance segmentation model (i.e., 020) includes:

[0124] 021: Get instance segmentation model;

[0125] 022: Split the image to be detected into a plurality of regions to be detected, each region to be detected having an overlapping area with an adjacent region to be detected in a first preset ratio;

[0126] 023: Input multiple regions to be inspected into the instance segmentation model to determine the first type of defects in each region to be inspected;

[0127] 024: restore multiple to-be-detected areas to the to-be-detected image, and merge the first-category defects of the same defect type that overlap in adjacent to-be-detected areas;

[0128] 025: Output the first-class defects in the image to be detected, as well as the minimum bounding rectangle and defect type corresponding to each first-class defect.

[0129] Specifically, the first type of defects includes any one or more of wire drawing defects, foreign body defects, and bubble defects. For example, the first type of defects may include only wire drawing defects, or only foreign body defects, or only bubble defects; the first type of defects may also include any two of wire drawing defects, foreign body defects, and bubble defects, or the first type of defects may include all three defects of wire drawing defects, foreign body defects, and bubble defects.

[0130] First, a model training is performed based on the first sample image to obtain an instance segmentation model suitable for glue road quality inspection. The model training process is as follows: a plurality of first sample images including the glue road area are obtained as the first training set, and the glue road area in the first sample image has a first type of defect. In the first sample image, the first type of defects are annotated to obtain a first annotated image. For the three defects included in the above-mentioned first type of defects, the first training set may include a first sub-training set, a second sub-training set, and a third sub-training set. The first sub-training set includes a plurality of first sample images with wiredrawing defects, the second sub-training set includes a plurality of first sample images with foreign matter defects, and the third sub-training set includes a first sample image with bubble defects. There is no limit on the number of first sample images. In an example, each sub-training set may include 10-50 first sample images.

[0131] It can be understood that the glue paths are mostly annular, the glue dispensing positions are relatively fixed, and the defect positions are random. If the complete first sample image is used for full image training, the model training accuracy is low; therefore, the first sample image is split, and each first sample image is split into N×N first sample areas, N≥4. When splitting, each first sample area has a first predetermined ratio of overlapping areas with the first sample areas adjacent to the left and right, and the top and bottom. There is no limit on the size of the first predetermined ratio. In one example, the first predetermined ratio is 25%.

[0132] Select any deep learning instance segmentation model as the first initial model, input each first sample area into the first initial model for model training, the first initial model detects the first sample area, annotates the detected first type of defects with a minimum bounding rectangle and outputs the first sample image. Restore the output first sample area to the first sample image, merge the defects in the overlapping area, and recalculate the minimum bounding rectangle. Adjust the model parameters of the first initial model according to the error between the minimum bounding rectangle obtained by detection and the corresponding annotation box of the first annotated image to minimize the error. After training, an instance segmentation model suitable for rubber road quality inspection is obtained.

[0133] Afterwards, according to the splitting rules during model training, the image to be detected is split into multiple areas to be detected, and each area to be detected has an overlapping area with an adjacent area to be detected of a first preset ratio. The multiple areas to be detected are input into the instance segmentation model, and the instance segmentation model infers the areas to be detected to determine the first type of defects and the specific defect type in each area to be detected, and annotate them with a minimum bounding rectangle. The multiple areas to be detected are restored to the image to be detected. For the second type of defects in the overlapping areas, if the defect types are the same, they are merged and the minimum bounding rectangle is recalculated. Finally, the first type of defects in the image to be detected, as well as the minimum bounding rectangle and defect type corresponding to each first type defect are output. In this way, the detection of the first type of defects is completed.

[0134] See also Figure 8 In some embodiments, the glue path is extracted from the image to be detected based on the semantic segmentation model, and a glue path binary image (i.e., 030) is output, including:

[0135] 031: Get the semantic segmentation model;

[0136] 032: Determine a plurality of regions of interest in the image to be detected, each region of interest having an overlapping area with an adjacent region of interest of a second preset ratio;

[0137] 033: Input multiple regions of interest into the semantic segmentation model to extract glue paths for multiple regions of interest;

[0138] 034: Restore multiple regions of interest to the image to be detected, and perform an OR operation on the overlapping glue paths in adjacent regions of interest to output a binary image of the glue paths.

[0139] Specifically, model training is performed based on the second sample image to obtain a semantic segmentation model suitable for glue road quality inspection. The model training process is as follows: a plurality of second sample images including the glue road area are obtained as the second training set. There is no limit on the number of second sample images. In one example, the number of second sample images is 100-200. The second sample images include normal sample images and abnormal sample images, and the ratio of normal sample images to abnormal sample images is 9:1. The glue road area in the normal sample image is a normal glue road area, and the glue road area in the abnormal sample image is an abnormal glue road area including the second type of defects. In the second sample image, the edges of the glue road area are annotated to obtain a second annotated image.

[0140] It can be understood that the glue path is mostly annular, the glue dispensing position is relatively fixed, and the defect position is random. If the complete second sample image is used for full-image training, the model training accuracy is low; therefore, the second sample image is split. A sample region of interest (ROI) is set along the glue path area. The sample ROI can be a rectangular area. The size of the sample ROI can be set according to the size of the second sample image. For example, the size of the sample ROI can be set to 512×512, 256×256, 128×128 or 64×64. In each second sample image, the number of sample ROIs can be 20-30. When setting the sample ROI, each second sample area has an overlapping area with an adjacent second sample area of ​​a second predetermined ratio. There is no limit on the size of the second predetermined ratio. In one example, the second predetermined ratio is 10%.

[0141] Select any deep learning semantic segmentation model as the second initial model, input each sample ROI into the second initial model for model training, and the second initial model extracts and outputs the glue path of the sample ROI. Restore the output sample ROI to the second sample image, and perform an OR operation on the edges of the overlapping area. Adjust the model parameters of the second initial model according to the error between the detected glue path edge and the corresponding annotated edge of the second annotated image to minimize the error. After training, a semantic segmentation model suitable for glue path quality detection is obtained.

[0142] Afterwards, in the image to be detected, multiple regions of interest are determined according to the glue path area. Similar to the sample ROI, the size of the region of interest can be set to 512×512, 256×256, 128×128 or 64×64, and the number of regions of interest can be 20-30. Each region of interest has an overlapping area with an adjacent region of interest at a second preset ratio. The multiple regions of interest are input into the semantic segmentation model, and the semantic segmentation model performs glue path extraction on the regions of interest to determine the glue path edge in each region of interest. The multiple regions of interest are restored to the image to be detected, and an OR operation is performed on the glue path edge in the overlapping area to extract the glue path in the image to be detected. The image to be detected is then binarized, the grayscale value of the glue path part is set to 255, and the grayscale value of the background is set to 0, and the glue path binary image (such as Figure 4 shown).

[0143] See also Fig. 9 In some embodiments, the second type of defects include excess glue defect, insufficient glue defect, glue break defect, glue length defect, glue quantity defect and glue deviation defect. The second type of defects are detected and measured in the glue inspection area according to multiple edge points (i.e. 060), including:

[0144] 061: Determine a corresponding glue path center point according to each edge point pair;

[0145] 062: Determine a first width corresponding to each edge point according to a plurality of edge point pairs and a plurality of corresponding glue path center points, so as to detect and measure excess glue defects and insufficient glue defects;

[0146] 063: Correct the first center trajectory according to multiple glue path center points to determine the second center trajectory;

[0147] 064: Determine whether the two nearest pixel points on both sides of the second center track belong to the glue path pixel points, so as to detect and measure the glue break defect;

[0148] 065: Detect and measure glue length defects based on multiple glue path center points;

[0149] 066: Analyze the glue path pixels according to the connected domain analysis algorithm to detect and measure glue quantity defects;

[0150] 067: Calculate the centroid of multiple glue path pixels to detect and measure glue deviation defects.

[0151] Specifically, Fig.10The figure shows the second type of common defects in the glue path, which includes any one or more of the defects of excess glue, insufficient glue, glue break, glue length, glue quantity and glue deviation. Excess glue defect means that the actual glue path partially overflows the edge of the template glue path. Insufficient glue defect means that the actual glue path is less than the template glue path. Glue break defect means that the template glue path is connected, but the actual glue path is disconnected. Glue length defect means that the length of the actual glue path is less than or greater than the length of the template glue path. Glue quantity defect means that the area of ​​the actual glue path is greater than or less than the area of ​​the template glue path. Glue deviation defect means that the actual glue path deviates from the template glue path in whole or in part.

[0152] Based on the edge point pairs, the above-mentioned second-type defects can be detected and measured separately in the glue inspection area. In the glue inspection area, the glue path edge detection is performed along the direction perpendicular to the first center track, and after obtaining multiple edge point pairs distributed along both sides of the first center track, for each edge point pair, a corresponding glue path center point is calculated.

[0153] The first width corresponding to each edge point is determined based on multiple edge point pairs and corresponding multiple glue path center points, and the detection and measurement of excess glue defects and insufficient glue defects can be performed based on the first width. Figure 6 As shown, the first width corresponding to the left edge point is widthL, and the first width corresponding to the right edge point is widthR. In addition, it can be understood that the calculated glue path center point is closer to the center point of the actual glue path, and the above-mentioned first center trajectory is determined according to the glue inspection area, and there may be a large error with the center trajectory of the actual glue path. Therefore, the first center trajectory can be corrected according to multiple glue path center points to determine the second center trajectory.

[0154] The correction process is as follows: the continuous point set composed of multiple glue path center points is the second center track of the corresponding glue inspection area. There may be glue break defect areas without edge point pairs in the glue inspection area, and without edge point pairs, the glue path center point cannot be calculated; at this time, the average deviation coordinates of all glue path center points and the first center track are calculated. For the glue break defect areas without edge point pairs, the first center track is translated according to the average deviation coordinates to obtain the corresponding second center track.

[0155] After determining the second center track, it is determined whether the two nearest pixels on both sides of the second center track belong to the glue path pixels. It can be understood that in the glue path binary image, the gray value of the glue path pixel is 255, that is, it is determined whether the gray value of the two nearest pixels on both sides of the second center track is 255, and the glue break defect can be detected and measured. The two sides of the second center track refer to the two sides in the direction perpendicular to the second center track.

[0156] Based on multiple glue path center points, glue length defects can also be detected and measured. The specific process is as follows: calculate the distance between every two consecutive and adjacent glue path center points. The sum of the distances between multiple consecutive glue path center points is the glue length. It can be understood that the two glue path center points on both sides of the glue break defect area are not continuous glue path center points and do not participate in the glue length calculation.

[0157] After the glue length is calculated, it is determined whether the glue length is greater than or equal to the preset minimum glue length, and less than or equal to the preset maximum glue length. When the glue length is less than the preset minimum glue length, or the glue length is greater than the preset maximum glue length, it is determined that there is a glue length defect in the glue inspection area, and the defective area and glue length of the glue length defect are output. In this way, secondary glue filling or correction can be performed according to the glue length. When the glue length is greater than or equal to the preset minimum glue length, and less than or equal to the preset maximum glue length, it is determined that there is no glue length defect in the glue inspection area.

[0158] For glue quantity defects, detection and measurement can be performed through a connected domain analysis algorithm, which can use a Blob analysis algorithm. The connected domain analysis algorithm can analyze the area composed of adjacent pixels with the same grayscale value in the glue inspection area, where the area composed of pixels with a grayscale value of 255 is the glue path area, and the area of ​​the glue path area is calculated as the glue quantity.

[0159] After the glue amount is calculated, it is determined whether the glue amount is greater than or equal to the preset minimum glue amount and less than or equal to the preset maximum glue amount. When the glue amount is less than the preset minimum glue amount, or the glue amount is greater than the preset maximum glue amount, it is determined that there is a glue amount defect in the glue inspection area, and the defective area and glue amount of the glue amount defect are output. In this way, secondary glue filling or correction can be performed according to the glue amount. When the glue amount is greater than or equal to the preset minimum glue amount, and less than or equal to the preset maximum glue amount, it is determined that there is no glue amount defect in the glue inspection area.

[0160] It should be noted that in different glue inspection areas, different settings can be made for the preset minimum glue length, the preset maximum glue length, the preset minimum glue quantity and the preset maximum glue quantity, and there is no restriction on the specific values.

[0161] For adhesive deviation defects, the centroid of multiple adhesive path pixel points can be calculated for detection and measurement. The following describes in detail the specific detection and measurement process of excess adhesive defects, insufficient adhesive defects, adhesive breakage defects and adhesive deviation defects.

[0162] See also Fig.11 In some embodiments, a first width corresponding to each edge point is determined according to a plurality of edge point pairs and a plurality of corresponding glue path center points to detect and measure excess glue defects and insufficient glue defects (i.e., 062), including:

[0163] 0621: Calculate a first width corresponding to each edge point according to a plurality of edge point pairs and a plurality of corresponding glue path center points;

[0164] 0622: When the first widths of a plurality of edge points exceeding a first predetermined number in succession are all greater than half of the first preset glue width, it is determined that there are multiple glue defects at positions corresponding to the plurality of edge points;

[0165] 0623: When the first widths of a plurality of edge points exceeding a second predetermined number in succession are all less than half of the second preset glue width, it is determined that there is a glue shortage defect at positions corresponding to the plurality of edge points;

[0166] 0624: Determine the multi-glue defect point set according to multiple edge points located at the multi-glue defect position and adjacent to each other, and output the starting point, end point, highest point, length, multi-glue defect point set, defect region and defect area of ​​each multi-glue defect;

[0167] 0625: Determine the glue deficiency defect point set based on multiple edge points located at the glue deficiency defect position and adjacent to each other, and output the starting point, end point, lowest point, length, glue deficiency defect point set, defect region and defect area of ​​each glue deficiency defect.

[0168] Specifically, the first width corresponding to each edge point can be calculated based on multiple edge point pairs and corresponding multiple glue path center points. For example, in an edge point pair, the distance between the left edge point and the glue path center point is calculated to obtain the first width corresponding to the left edge point; the distance between the right edge point and the glue path center point is calculated to obtain the first width corresponding to the right edge point.

[0169] After calculating the first width corresponding to each edge point, for the detection and measurement of multiple glue defects, first determine whether the first width corresponding to each edge point is greater than half of the first preset glue width, and the first preset glue width is the maximum allowable glue width. When the first width corresponding to the edge point is less than or equal to half of the first preset glue width, it means that there is no multiple glue defect at the position corresponding to the edge point. When the first width corresponding to the edge point is greater than half of the first preset glue width, it means that there may be multiple glue defects at the position corresponding to the edge point, and it is necessary to combine other adjacent edge points for judgment.

[0170] When the first width corresponding to multiple edge points that exceed the first predetermined number in succession is greater than half of the first preset glue width, it can be determined that there are multiple glue defects at the positions corresponding to the multiple continuous edge points. The first predetermined number is also the multiple glue length threshold. A multiple glue defect point set is determined based on multiple edge points that are located at the multiple glue defect position and are adjacent, that is, multiple edge points located at the same multiple glue defect position are regarded as a multiple glue defect point set. After determining the multiple glue defect point set, the starting point, end point, highest point, length, multiple glue defect point set, defect region and defect area of ​​each multiple glue defect can be output. In this way, the size information such as the starting point, end point, highest point, length, defect region and defect area of ​​the multiple glue defect is accurately measured, which is convenient for secondary glue filling or correction based on the size information.

[0171] Among them, Fig.12 As shown, the starting point is the first edge point in the multi-glue defect point set; the end point is the last edge point in the multi-glue defect point set; the highest point is the edge point with the largest first width in the multi-glue defect point set; the length is the number of edge points in the multi-glue defect point set; the defect area is the area where the minimum circumscribed rectangle of multiple edge points is located; the defect area is the sum of the first widths of all edge points in the multi-glue defect point set. In addition, the height of the multi-glue area can also be determined based on the defect area.

[0172] For the detection and measurement of insufficient glue defects, first determine whether the first width corresponding to each edge point is less than half of the second preset glue width, and the second preset glue width is the minimum allowable glue width. When the first width corresponding to the edge point is greater than or equal to half of the second preset glue width, it means that there is no insufficient glue defect at the position corresponding to the edge point. When the first width corresponding to the edge point is less than half of the second preset glue width, it means that there may be insufficient glue defect at the position corresponding to the edge point, and it is necessary to combine other adjacent edge points for judgment.

[0173] When the first width corresponding to multiple edge points that exceed the second predetermined number in succession is less than half of the second preset glue width, it can be determined that there is a lack of glue defect at the position corresponding to the multiple continuous edge points. The second predetermined number is also the threshold value of the length of the lack of glue. A set of lack of glue defect points is determined based on multiple edge points that are located at the position of the lack of glue defect and are adjacent, that is, multiple edge points located at the same position of the lack of glue defect are regarded as a set of lack of glue defect points. After determining the set of lack of glue defect points, the starting point, end point, lowest point, length, set of lack of glue defect points, defect region and defect area of ​​each lack of glue defect can be output. In this way, the size information such as the starting point, end point, highest point, length, defect region and defect area of ​​the lack of glue defect is accurately measured, so as to facilitate secondary glue filling or correction according to the size information.

[0174] Among them, Fig.13As shown in the figure, the starting point is the first edge point in the glue defect point set; the end point is the last edge point in the glue defect point set; the lowest point is the edge point with the smallest first width in the glue defect point set; the length is the number of edge points in the glue defect point set; the defect area is the area where the minimum circumscribed rectangle of multiple edge points is located; the defect area is the sum of the first widths of all edge points in the glue defect point set. In addition, the height of the glue defect area can also be determined based on the defect area.

[0175] It should be noted that, in different glue inspection areas, the first preset glue width, the second preset glue width, the first preset number, and the second preset number can be set differently, and there is no restriction on the specific values.

[0176] See also Fig.14 In some embodiments, the glue breaking defect includes a straight glue breaking defect and an oblique glue breaking defect. The second center track includes a plurality of center track points. Determining whether the two nearest pixel points on both sides of the second center track belong to glue path pixel points to detect and measure the glue breaking defect (ie, 064) includes:

[0177] 0641: Determine whether the two nearest pixel points on both sides of each center track point along the second center track belong to the glue path pixel points;

[0178] 0642: Determine whether there is a corresponding edge point pair for each center trajectory point;

[0179] 0643: When the two nearest pixel points on both sides of the second central track of a plurality of central track points exceeding a third predetermined number continuously do not belong to the glue path pixel points, and the plurality of central track points do not have corresponding edge point pairs, it is determined that there is a straight glue break defect at the positions corresponding to the plurality of central track points;

[0180] 0644: when the two nearest pixel points on both sides of the second central track of a plurality of central track points exceeding a third predetermined number continuously do not belong to the glue path pixel points, and the plurality of central track points have corresponding edge point pairs, the plurality of central track points are added to the set of points to be detected, so as to detect and measure the oblique glue break defect;

[0181] 0645: Determine the straight-mouth glue break defect point set based on the glue path pixel points on both sides of the straight-mouth glue break defect position, and output the starting point, end point, length, straight-mouth glue break defect point set and defect area of ​​each straight-mouth glue break defect.

[0182] Specifically, the glue breaking defect includes straight glue breaking defect and oblique glue breaking defect. Fig.15 As shown in (a), the oblique glue break defect is Fig.15 As shown in (b), therefore, the detection and measurement of glue break defects include the detection and measurement of straight glue break defects and the detection and measurement of oblique glue break defects.

[0183] The detection and measurement of glue breaking defects are performed based on the second center track. The second center track includes multiple center track points. It is judged whether the two nearest pixel points on both sides of each center track point along the second center track belong to the glue path pixel points. When the two nearest pixel points on both sides of the center track point along the second center track belong to the glue path pixel points, it means that there is no glue breaking defect at the position corresponding to the center track point; when the two nearest pixel points on both sides of the center track point along the second center track do not belong to the glue path pixel points, it means that there may be glue breaking defects at the position corresponding to the center track point, and it is necessary to combine other adjacent center track points for judgment. When the two nearest pixel points on both sides of the center track point along the second center track do not belong to the glue path pixel points, the center track point and the two pixels on both sides constitute a glue-free area.

[0184] When the two nearest pixel points along the two sides of the second center trajectory for a plurality of center trajectory points that continuously exceed the third predetermined number do not belong to the glue path pixel points, it is determined that there may be glue break defects at the positions corresponding to the plurality of center trajectory points. Furthermore, the straight glue break defects and the oblique glue break defects are judged to determine whether there are corresponding edge point pairs at the plurality of center trajectory points at the glue break defect positions. When at least some of the plurality of center trajectory points located at the glue break defect positions do not have corresponding edge point pairs, it is determined that there is a straight glue break defect at the positions corresponding to at least some of the center trajectory points. Then the judgment condition corresponding to the straight glue break defect is: the two nearest pixel points along the two sides of the second center trajectory for a plurality of center trajectory points that continuously exceed the third predetermined number do not belong to the glue path pixel points, and the plurality of center trajectory points do not have corresponding edge point pairs. Fig.16 As shown, the dotted boxes A1, A2 and A3 are all locations where straight-mouth glue break defects exist.

[0185] After that, edge detection is performed on the glue paths on both sides of the straight glue break defect position, and two columns of glue path pixel points on both sides of the straight glue break defect position are obtained as the straight glue break defect point set. After the straight glue break defect point set is determined, the starting point, end point, length, straight glue break defect point set and defect area of ​​each straight glue break defect are output. In this way, the size information such as the starting point, end point, length and defect area of ​​the straight glue break defect is accurately measured, which is convenient for secondary glue filling or correction based on the size information.

[0186] Among them, the starting point is the first central trajectory point of the straight-mouth glue break defect; the end point is the last central trajectory point of the straight-mouth glue break defect; the length is the number of pixels between the starting point and the end point (including the starting point and the end point); the defect area is the area where the minimum circumscribed rectangle of the straight-mouth glue break defect point set is located.

[0187] like Fig.17As shown, when there are corresponding edge point pairs for at least some of the multiple center trajectory points located at the glue breaking defect position, it means that the positions corresponding to at least some of the center trajectory points may have oblique glue breaking defects, and these multiple center trajectory points are added to the set of points to be detected to detect and measure the oblique glue breaking defects.

[0188] It should be noted that the third predetermined number can be set differently in different glue inspection areas, and there is no limitation on the specific value.

[0189] See also Fig.18 In some embodiments, after determining whether there is a corresponding edge point pair at the center track point (i.e., 0642), detecting and measuring the second type of defects in the glue inspection area according to the plurality of edge point pairs (i.e., 060) further includes:

[0190] 068: when the two nearest pixel points on both sides of the second central track of the plurality of central track points exceeding a fourth predetermined number continuously do not belong to the glue path pixel points, and the plurality of central track points have a corresponding edge point, it is determined that there is a glue shortage defect at the positions corresponding to the plurality of central track points;

[0191] 069: Determine a glue deficiency defect point set based on multiple edge points located at and adjacent to the glue deficiency defect position to output the starting point, end point, lowest point, length, glue deficiency defect point set, defect region and defect area of ​​each glue deficiency defect.

[0192] Specifically, in the process of detecting and measuring the glue breaking defect, when the two nearest pixel points on both sides of the second center track of the multiple center track points exceeding the fourth predetermined number continuously do not belong to the glue path pixel points, and there is only one corresponding edge point for the multiple center track points, it means that there is no glue breaking defect at the positions corresponding to the multiple center track points, such as Fig.19 As shown, it is determined that there is a glue shortage defect at the positions corresponding to the multiple center track points. The fourth predetermined number is a glue shortage length threshold, and the fourth predetermined number can be the same as or different from the second predetermined number. In different glue inspection areas, the fourth predetermined number can be set differently.

[0193] The glue deficiency defect point set is determined based on multiple edge points located at the glue deficiency defect position and adjacent to each other, that is, multiple edge points located at the same glue deficiency defect position are regarded as a glue deficiency defect point set. After the glue deficiency defect point set is determined, the starting point, end point, lowest point, length, glue deficiency defect point set, defect region and defect area of ​​each glue deficiency defect can be output.

[0194] Among them, the starting point is the first edge point in the low glue defect point set; the end point is the last edge point in the low glue defect point set; the lowest point is the edge point closest to the center track point in the low glue defect point set; the length is the number of edge points in the low glue defect point set; the defect area is the area where the minimum circumscribed rectangle of multiple edge points is located; in addition, the height of the low glue defect area can also be determined based on the defect area. The defect area can be calculated based on the number of all edge points in the low glue defect point set and the height of the low glue defect area.

[0195] See also Fig. 20 In some embodiments, multiple center trajectory points are added to the set of points to be detected to detect and measure the bevel glue break defect (i.e., 0644), including:

[0196] 06441: determining a central detection point according to the set of points to be detected, and searching for edge points along a first direction according to the central detection point to determine a first detection point pair;

[0197] 06442: Move the first detection point pair along the second direction with a predetermined step length, and perform edge point search again along the first direction to determine a plurality of second detection point pairs and detection lines corresponding to the second detection point pairs, wherein the second direction is perpendicular to the first direction;

[0198] 06443: For each detection line, calculate the number of glue path pixels on the detection line;

[0199] 06444: When the number of pixels of the glue path is equal to the preset number of pixels, it is determined that there is no oblique glue break defect at the position corresponding to the detection line, and the detection and measurement of the oblique glue break defect are exited, wherein the preset number of pixels is the sum of the number of pixels between the first detection point pair and twice the preset tolerance number of pixels;

[0200] 06445: When the number of pixels in the glue path is less than the preset number of pixels, it is determined that there is an oblique glue break defect at the position corresponding to the detection line;

[0201] 06446: Determine the oblique glue break defect point set according to the first detection point pair and the second detection point pair, and output the starting point, end point, length, oblique glue break defect point set and defect area of ​​each oblique glue break defect.

[0202] Specifically, Fig.21As shown, a central detection point O is determined according to multiple central track points in the set of points to be detected, and the central point of the glue-free area corresponding to the multiple central track points can be determined as the central detection point O. According to the central detection point O, an edge point search is performed along a first direction, and the first direction can be a direction parallel to the extension direction of the second central track. An edge search is performed along the first direction toward the glue path on both sides of the glue break defect, and the first glue path pixel point searched is determined as the first detection point. The two first detection points detected on both sides form a first detection point pair.

[0203] Along the second direction, the first detection point pair is moved to both sides of the line segment where the first detection point pair is located with a predetermined step length. The second direction is perpendicular to the first direction, and the second direction can be a direction perpendicular to the second center track. It can be understood that the second center track can be a straight line or a curve. When the second center track is a curve, the direction perpendicular to the second center track is also the normal direction of the second center track. Therefore, the second direction can be the normal direction of the second center track. The predetermined step length is in pixels. In an example, the predetermined step length can be one pixel.

[0204] Each time the first detection point pair moves, the first detection point pair is connected to obtain a connection line L1, and an edge search is performed again along the first direction on the connection line L1, toward the glue path on both sides of the glue break defect, and the two second detection points searched form a second detection point pair. A detection line L2 can be determined based on the second detection point pair. Then, each time the first detection point pair moves, a detection line L2 can be determined. The connection line L1 and the detection line L2 can be straight lines or curves.

[0205] It should be noted that the length of the detection line L2 is set to a fixed L1+2×T, where T is the number of preset tolerance pixels. Setting the preset tolerance pixels can improve the accuracy of the searched second detection point pair. In an example, the number of preset tolerance pixels is 2. When two second detection points are searched, the length of the determined detection line L2 is less than L1+2×T, and the edge search is continued along the first direction until the length of the determined detection line L2 is equal to L1+2×T. When only one second detection point is searched, the second detection point is used as the starting point, and L1+2×T pixels are determined along the connecting line L1 as the detection line L2. If the second detection point cannot be found, the search is stopped and the movement of the first detection point pair is stopped.

[0206] For each detection line L2, calculate the number of glue path pixels on the detection line L2 to determine whether there is a bevel glue break defect at the position corresponding to the detection line L2. Determine a preset number of pixels, which is the sum of the number of pixels between the first detection point pair (including the first detection point pair) and twice the preset tolerance pixel number. When the number of glue path pixels on the detection line L2 is equal to the preset number of pixels, Fig. 22 As shown, it means that each point on the detection line L2 is a glue path pixel point. If there is a connected glue path at the position corresponding to the detection line L2, it is determined that there is no oblique glue break defect at the position corresponding to the detection line, and the detection and measurement of the oblique glue break defect is exited.

[0207] When the number of glue path pixels on the detection line L2 is less than the preset number of pixels, it means that there is no connected glue path at the position corresponding to the detection line L2, and it is determined that there is an oblique glue break defect at the position corresponding to the detection line L2. According to the first detection point pair and the second detection point pair located at the same oblique glue break defect, an oblique glue break defect point set can be determined, and the starting point, end point, length, oblique glue break defect point set and defect area of ​​each oblique glue break defect are output. In this way, the size information such as the starting point, end point, length and defect area of ​​the oblique glue break defect is accurately measured, which is convenient for secondary glue filling or correction based on the size information.

[0208] Among them, Fig.23 As shown in the figure, the starting point is the first center track point of the oblique glue break defect; the end point is the last center track point of the oblique glue break defect; the length is the number of pixels between the starting point and the end point (including the starting point and the end point); the defect area can be the area where the minimum circumscribed rectangle of the oblique glue break defect point set is located, or it can be the area formed by connecting each detection point pair.

[0209] It should be noted that, in different glue inspection areas, the predetermined step length, the preset number of pixels, and the preset number of tolerance pixels can be set differently, and there is no restriction on the specific values.

[0210] See also Fig.24 In some embodiments, the centroid of a plurality of glue path pixels is calculated to detect and measure glue deviation defects (i.e., 067), including:

[0211] 0671: Calculate the centroid of multiple glue path pixels;

[0212] 0672: Determine the detection reference of the adhesive deviation defect, wherein the detection reference includes any one or more of a reference point, a reference line, and a reference profile;

[0213] 0673: Calculate the first minimum distance from the centroid to the detection benchmark;

[0214] 0674: When the first minimum distance is greater than the preset distance, it is determined that there is a glue deviation defect in the inspection area, and the first minimum distance is output as the glue deviation amount;

[0215] 0675: When the detection benchmark includes a benchmark contour, the second minimum distance and the maximum distance between multiple edge points and the benchmark contour, and the third minimum distance between the center point of the minimum circumscribed rectangular box of multiple glue path pixel points and the benchmark contour are calculated and output as glue deviation.

[0216] Specifically, in each inspection area, when there is no glue break defect in the glue inspection area, the glue inspection area only includes one glue path area, and the centroid of the connected domain of the glue path area, that is, the average value of the coordinates of multiple glue path pixel points, is calculated to obtain the centroids of multiple glue path pixel points; when there is a glue break defect in the glue inspection area, the glue inspection area includes multiple glue path areas, and the centroid of the connected domain of each glue path area is calculated separately, and then the average value of the centroid coordinates of the multiple glue path areas is calculated to obtain the centroids of multiple glue path pixel points.

[0217] Determine the detection benchmark of the glue deviation defect. The detection benchmark can be any one or more of the benchmark points, benchmark lines, and benchmark profiles. For example, only the benchmark points can be used as the detection benchmark, or only the benchmark lines can be used as the detection benchmark, or only the benchmark profile can be used as the detection benchmark. Any two of the benchmark points, benchmark lines, and benchmark profiles can also be used as the detection benchmarks. The benchmark points and benchmark lines are suitable for measuring the overall glue deviation of the glue path, and the benchmark profile is suitable for measuring the different glue deviations in each segmented glue inspection area.

[0218] Calculate the first minimum distance between the centroid and the detection reference, for example, when the detection reference includes reference points, such as Fig.25 As shown in , the distance between the centroid and the reference point is calculated as the first minimum distance. When the detection reference includes the reference line, such as Fig.26 As shown, the distance between the perpendicular point and the center of mass is the first minimum distance. The relationship between the first minimum distance and the preset distance is determined, and the preset distance is the maximum glue deviation threshold. When the first minimum distance is less than or equal to the preset distance, it is determined that there is no glue deviation defect in the inspection area. When the first minimum distance is greater than the preset distance, it is determined that there is a glue deviation defect in the glue path of the inspection area, and the first minimum distance is output as the glue deviation amount.

[0219] When the detection benchmark includes the benchmark profile, such as Fig. 27 As shown in the figure, in addition to calculating the first minimum distance, it is also necessary to calculate the second minimum distance between multiple edge points and the reference contour, the maximum distance between multiple edge points and the reference contour, and the third minimum distance between the center point of the minimum circumscribed rectangular frame of multiple glue path pixel points and the reference contour, and output them together with the first minimum distance as the glue deviation amount. In this way, the glue deviation amount and other dimensional information of the glue deviation defect can be accurately measured, which is convenient for secondary glue filling or correction based on the dimensional information.

[0220] In some embodiments, model reasoning deployment based on the central processing unit (CPU) can be achieved by converting the instance segmentation model and semantic segmentation model running on the graphics processing unit (GPU) into models that can be run on the CPU through Open ViNO. In this way, the hardware cost of reasoning can be reduced (from GPU to CPU) while the model accuracy is acceptable. In addition, the model can be pruned according to the project life cycle time (CycleTime, CT) requirements, thereby further reducing the model size and improving the reasoning efficiency of the model on the CPU.

[0221] See also Fig.28 , the embodiment of the present application also provides a glue road quality detection device 100. The glue road quality detection device 100 includes an acquisition module 10, a first defect detection module 20, an extraction module 30, a determination module 40, an edge detection module 50, a second defect detection module 60 and an output module 70. The acquisition module 10 is used to acquire an image to be detected, and the image to be detected includes a glue road area; the first defect detection module 20 is used to detect the first type of defects in the image to be detected based on an instance segmentation model. The extraction module 30 is used to extract the glue road in the image to be detected based on a semantic segmentation model and output a binary image of the glue road. The determination module 40 is used to set a glue inspection area in the binary image of the glue road, wherein the glue inspection area includes a first center track. The edge detection module 50 is used to perform glue road edge detection in the glue inspection area along a direction perpendicular to the first center track to obtain a plurality of edge point pairs distributed along both sides of the first center track. The second defect detection module 60 is used to detect and measure the second type of defects in the glue inspection area according to the plurality of edge point pairs. The output module 70 is used to output the detection results of the first type of defects of the image to be detected, and the detection and measurement results of the second type of defects.

[0222] In some embodiments, the first type of defects include any one or more of wire drawing defects, foreign body defects, and bubble defects. The first defect detection module 20 is specifically used to obtain an instance segmentation model; split the image to be detected into multiple areas to be detected, each area to be detected has an overlapping area with an adjacent area to be detected with a first preset ratio; input multiple areas to be detected into the instance segmentation model to determine the first type of defects in each area to be detected; restore the multiple areas to be detected to the image to be detected, and merge the first type of defects that overlap and have the same defect type in adjacent areas to be detected; output the first type of defects in the image to be detected, as well as the minimum circumscribed rectangular frame and defect type corresponding to each first type of defect.

[0223] In certain embodiments, the extraction module 30 is specifically used to obtain a semantic segmentation model; determine multiple regions of interest in the image to be detected, each region of interest having an overlapping area with an adjacent region of interest of a second preset ratio; input the multiple regions of interest into the semantic segmentation model to perform glue path extraction on the multiple regions of interest; restore the multiple regions of interest to the image to be detected, and perform an OR operation on the overlapping glue paths in adjacent regions of interest to output a binary image of the glue paths.

[0224] In some embodiments, the second type of defects include excess glue defects, insufficient glue defects, glue break defects, glue length defects, glue quantity defects, and glue deviation defects. The second defect detection module 60 is specifically used to determine a corresponding glue path center point according to each edge point pair; determine the first width corresponding to each edge point according to multiple edge point pairs and corresponding multiple glue path center points, so as to detect and measure excess glue defects and insufficient glue defects; correct the first center trajectory according to multiple glue path center points to determine the second center trajectory; determine whether the two nearest pixels on both sides of the second center trajectory belong to glue path pixels, so as to detect and measure glue break defects; detect and measure glue length defects according to multiple glue path center points; analyze glue path pixels according to the connected domain analysis algorithm to detect and measure glue quantity defects; calculate the centroid of multiple glue path pixels to detect and measure glue deviation defects.

[0225] In certain embodiments, the second defect detection module 60 is specifically used to calculate the first width corresponding to each edge point based on multiple edge point pairs and corresponding multiple glue path center points; when the first widths of multiple edge points that continuously exceed a first predetermined number are all greater than half of the first preset glue width, it is determined that there are multiple glue defects at the positions corresponding to the multiple edge points; when the first widths of multiple edge points that continuously exceed a second predetermined number are all less than half of the second preset glue width, it is determined that there are few glue defects at the positions corresponding to the multiple edge points; determine a multiple glue defect point set based on multiple edge points that are located at the multiple glue defect position and are adjacent to each other, and output the starting point, end point, highest point, length, multiple glue defect point set, defect region and defect area of ​​each multiple glue defect; determine a few glue defect point set based on multiple edge points that are located at the few glue defect position and are adjacent to each other, and output the starting point, end point, lowest point, length, few glue defect point set, defect region and defect area of ​​each few glue defect.

[0226] In some embodiments, the glue breaking defect includes a straight glue breaking defect and an oblique glue breaking defect. The second center track includes a plurality of center track points. The second defect detection module 60 is specifically used to determine whether the two nearest pixel points on both sides of each center track point along the second center track belong to the glue path pixel points; determine whether there is a corresponding edge point pair for each center track point; when the two nearest pixel points on both sides of the second center track of a plurality of center track points exceeding a third predetermined number in succession do not belong to the glue path pixel points, and there are no corresponding edge point pairs for the plurality of center track points, it is determined that there is a straight glue breaking defect at the position corresponding to the plurality of center track points; when the two nearest pixel points on both sides of the second center track of a plurality of center track points exceeding a third predetermined number in succession do not belong to the glue path pixel points, and there are corresponding edge point pairs for the plurality of center track points, the plurality of center track points are added to the point set to be detected to detect and measure the oblique glue breaking defect; determine the straight glue breaking defect point set according to the glue path pixel points located on both sides of the straight glue breaking defect position, and output the starting point, end point, length, straight glue breaking defect point set and defect area of ​​each straight glue breaking defect.

[0227] In some embodiments, the second defect detection module 60 is specifically used to determine a central detection point based on the set of points to be detected, and based on the central detection point, perform an edge point search along a first direction to determine a first detection point pair; move the first detection point pair along a second direction with a predetermined step length, and perform an edge point search again along the first direction to determine a plurality of second detection point pairs and detection lines corresponding to the second detection point pairs, wherein the second direction is perpendicular to the first direction; for each detection line, calculate the number of glue path pixel points on the detection line; when the number of glue path pixel points is equal to a preset image When the number of pixel points is less than the preset number of pixel points, it is determined that there is a bevel glue break defect at the position corresponding to the detection line, and the detection and measurement of the bevel glue break defect are exited, wherein the preset number of pixel points is the sum of the number of pixel points between the first detection point pair and twice the preset tolerance pixel point number; when the number of glue path pixel points is less than the preset number of pixel points, it is determined that there is a bevel glue break defect at the position corresponding to the detection line; the bevel glue break defect point set is determined according to the first detection point pair and the second detection point pair, and the starting point, end point, length, bevel glue break defect point set and defect area of ​​each bevel glue break defect are output.

[0228] In certain embodiments, after determining whether there is a corresponding edge point pair for a central trajectory point, the second defect detection module 60 is specifically used to determine that there is a low glue defect at the positions corresponding to the multiple central trajectory points when the two nearest pixel points on both sides of the second central trajectory for a plurality of central trajectory points exceeding a fourth predetermined number in succession do not belong to the glue path pixel points, and there is a corresponding edge point for the multiple central trajectory points; determine a low glue defect point set based on a plurality of edge points that are located at the position of the low glue defect and are adjacent to each other, and output the starting point, end point, lowest point, length, low glue defect point set, defect region and defect area of ​​each low glue defect.

[0229] In certain embodiments, the second defect detection module 60 is specifically used to calculate the centroid of multiple glue path pixel points; determine the detection benchmark for glue deviation defects, wherein the detection benchmark includes any one or more of a benchmark point, a benchmark line, and a benchmark contour; calculate the first minimum distance from the centroid to the detection benchmark; when the first minimum distance is greater than a preset distance, determine that there is a glue deviation defect in the inspection area, and output the first minimum distance as the glue deviation amount; when the detection benchmark includes a benchmark contour, calculate the second minimum distance, the maximum distance between multiple edge points and the benchmark contour, and the third minimum distance between the center point of the minimum circumscribed rectangle of multiple glue path pixel points and the benchmark contour, and output them as the glue deviation amount.

[0230] It should be pointed out that the explanation of the glue path quality detection method in the aforementioned embodiment is also applicable to the glue path quality detection device 100 in the embodiment of the present application, and will not be elaborated here.

[0231] See also Fig.29 The embodiment of the present application also provides a glue road quality detection system 200, which includes one or more processors 210 and a memory 220. The memory 220 stores a computer program. When the computer program is executed by the processor 210, the glue road quality detection method of any of the above embodiments is implemented.

[0232] For example, when the computer program is executed by the processor 210, the following glue path quality detection method is implemented:

[0233] 010: Acquire the image to be detected, which includes the glue path area;

[0234] 020: Detect the first type of defects in the image to be inspected based on the instance segmentation model;

[0235] 030: Extract glue paths from the image to be detected based on the semantic segmentation model and output a binary image of the glue paths;

[0236] 040: setting a glue inspection area in the glue path binary image, wherein the glue inspection area includes a first center track;

[0237] 050: In the glue inspection area, glue path edge detection is performed along a direction perpendicular to the first center track to obtain a plurality of edge point pairs distributed along both sides of the first center track;

[0238] 060: Detect and measure the second type of defects in the glue inspection area based on multiple edge points;

[0239] 070: Output the detection result of the first type of defects of the image to be detected, and the detection and measurement results of the second type of defects.

[0240] For another example, when the computer program is executed by the processor 210, the following glue path quality detection method is implemented:

[0241] 021: Get instance segmentation model;

[0242] 022: Split the image to be detected into a plurality of regions to be detected, each region to be detected having an overlapping area with an adjacent region to be detected in a first preset ratio;

[0243] 023: Input multiple regions to be inspected into the instance segmentation model to determine the first type of defects in each region to be inspected;

[0244] 024: restore multiple to-be-detected areas to the to-be-detected image, and merge the first-category defects of the same defect type that overlap in adjacent to-be-detected areas;

[0245] 025: Output the first-class defects in the image to be detected, as well as the minimum bounding rectangle and defect type corresponding to each first-class defect.

[0246] It should be pointed out that the explanations and descriptions of the glue road quality detection method and the glue road quality detection device 100 in the aforementioned embodiments are also applicable to the glue road quality detection system 200 of the embodiment of the present application, and will not be elaborated here.

[0247] See also Figures 1 to 3 The computer-readable storage medium 300 of the embodiment of the present application stores a computer program 310 thereon, and when the program is executed by the processor 320, the glue path quality detection method of any of the above-mentioned embodiments is implemented.

[0248] For example, when the computer program is executed by the processor 320, the following glue path quality detection method is implemented:

[0249] 010: Acquire the image to be detected, which includes the glue path area;

[0250] 020: Detect the first type of defects in the image to be inspected based on the instance segmentation model;

[0251] 030: Extract glue paths from the image to be detected based on the semantic segmentation model and output a binary image of the glue paths;

[0252] 040: setting a glue inspection area in the glue path binary image, wherein the glue inspection area includes a first center track;

[0253] 050: In the glue inspection area, glue path edge detection is performed along a direction perpendicular to the first center track to obtain a plurality of edge point pairs distributed along both sides of the first center track;

[0254] 060: Detect and measure the second type of defects in the glue inspection area based on multiple edge points;

[0255] 070: Output the detection result of the first type of defects of the image to be detected, and the detection and measurement results of the second type of defects.

[0256] For another example, when the computer program is executed by the processor 320, the following glue path quality detection method is implemented:

[0257] 021: Get instance segmentation model;

[0258] 022: Split the image to be detected into a plurality of regions to be detected, each region to be detected having an overlapping area with an adjacent region to be detected in a first preset ratio;

[0259] 023: Input multiple regions to be inspected into the instance segmentation model to determine the first type of defects in each region to be inspected;

[0260] 024: restore multiple to-be-detected areas to the to-be-detected image, and merge the first-category defects of the same defect type that overlap in adjacent to-be-detected areas;

[0261] 025: Output the first-class defects in the image to be detected, as well as the minimum bounding rectangle and defect type corresponding to each first-class defect.

[0262] It should be pointed out that the explanations and descriptions of the glue path quality detection method and the glue path quality detection device 100 in the aforementioned embodiments are also applicable to the computer-readable storage medium 300 in the embodiments of the present application, and will not be elaborated here.

[0263] In summary, the glue road quality detection method, glue road quality detection device 100, glue road quality detection system 200 and computer-readable storage medium 300 of the embodiments of the present application use a deep learning model to detect the quality of the glue road, detect the first type of defects in the glue road area based on an instance segmentation model, and detect and measure the second type of defects in the glue road area based on a semantic segmentation model. In this way, all defects related to the glue road can be detected, and the detection accuracy can reach the limit of the human eye, and it is compatible with the glue road quality detection in complex scenes such as glue road reflection, color unevenness, color difference and background interference.

[0264] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0265] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0266] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, a computer-readable storage medium can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable storage media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable storage medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0267] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0268] A person of ordinary skill in the art will appreciate that all or part of the steps carried by the method for implementing the above-mentioned embodiment can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment. In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a disk or an optical disk, etc.

[0269] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application. The scope of the present application is defined by the claims and their equivalents.

Claims

1. A method for detecting the quality of a rubber road, characterized in that: include: Acquire an image to be detected, wherein the image to be detected includes a glue path area; Performing detection of first-category defects on the image to be detected based on an instance segmentation model; Extracting glue paths from the image to be detected based on a semantic segmentation model, and outputting a binary image of the glue paths; Setting a glue inspection area in the glue path binary image, wherein the glue inspection area includes a first center track; In the glue inspection area, glue path edge detection is performed along a direction perpendicular to the first center track to obtain a plurality of edge point pairs distributed along both sides of the first center track; Detecting and measuring the second type of defects in the glue inspection area according to the plurality of edge points; Outputting the detection result of the first type of defects of the image to be detected, and the detection and measurement results of the second type of defects.

2. The method for detecting the quality of the rubber road according to claim 1, characterized in that: The first type of defects includes any one or more of wire drawing defects, foreign body defects, and bubble defects. The detecting of the first type of defects on the image to be detected based on the instance segmentation model includes: Obtaining the instance segmentation model; Splitting the image to be detected into a plurality of regions to be detected, each of the regions to be detected having an overlapping area with an adjacent region to be detected in a first preset ratio; Inputting a plurality of the regions to be inspected into the instance segmentation model to determine the first type of defects in each of the regions to be inspected; Restoring the plurality of the to-be-detected areas to the to-be-detected image, and merging the first-type defects that overlap and have the same defect type in adjacent to the to-be-detected areas; Output the first type of defects in the image to be detected, and the minimum circumscribed rectangular frame and defect type corresponding to each of the first type of defects.

3. The method for detecting the quality of the rubber road according to claim 1, characterized in that: The step of extracting glue paths from the image to be detected based on the semantic segmentation model and outputting a binary image of the glue paths includes: Acquire the semantic segmentation model; Determine a plurality of regions of interest in the image to be detected, each of the regions of interest having an overlapping area with an adjacent region of interest of a second preset ratio; Inputting the plurality of regions of interest into the semantic segmentation model to perform glue path extraction on the plurality of regions of interest; The plurality of regions of interest are restored to the image to be detected, and an OR operation is performed on the overlapping glue paths in adjacent regions of interest to output a binary image of the glue paths.

4. The method for detecting the quality of the rubber road according to claim 1, characterized in that: The second type of defects include excess glue defect, insufficient glue defect, glue break defect, glue length defect, glue quantity defect and glue deviation defect. The detection and measurement of the second type of defects in the glue inspection area according to the plurality of edge points includes: Determine a corresponding glue path center point according to each edge point pair; Determine a first width corresponding to each edge point according to the plurality of edge point pairs and the corresponding plurality of glue path center points, so as to detect and measure the excess glue defect and the insufficient glue defect; Correcting the first center trajectory according to the plurality of center points of the glue path to determine a second center trajectory; Determine whether the two nearest pixel points on both sides of the second center track belong to glue path pixel points, so as to detect and measure the glue breaking defect; Detecting and measuring the glue length defects according to the multiple glue path center points; Analyzing the glue path pixels according to a connected domain analysis algorithm to detect and measure the glue quantity defects; The centroids of the plurality of glue path pixel points are calculated to detect and measure the glue deviation defects.

5. The method for detecting the quality of the rubber road according to claim 4, characterized in that: The step of determining the first width corresponding to each edge point according to the plurality of edge point pairs and the corresponding plurality of glue path center points, so as to detect and measure the excess glue defect and the insufficient glue defect, comprises: Calculate the first width corresponding to each edge point according to the plurality of edge point pairs and the corresponding plurality of glue path center points; When the first width of a plurality of edge points exceeding a first predetermined number in succession is greater than half of a first preset glue width, it is determined that the positions corresponding to the plurality of edge points have the excess glue defect; When the first width of a plurality of edge points exceeding a second predetermined number in succession is less than half of a second preset glue width, it is determined that the glue deficiency defect exists at positions corresponding to the plurality of edge points; Determine a multi-glue defect point set according to a plurality of edge points located at and adjacent to the multi-glue defect positions, and output a starting point, an end point, a highest point, a length, the multi-glue defect point set, a defect region, and a defect area of ​​each multi-glue defect; A glue shortage defect point set is determined based on a plurality of edge points located at and adjacent to the glue shortage defect position, and the starting point, end point, lowest point, length, glue shortage defect point set, defect region and defect area of ​​each glue shortage defect are output.

6. The method for detecting the quality of the rubber road according to claim 4, characterized in that: The glue break defect includes a straight glue break defect and an oblique glue break defect, the second center track includes a plurality of center track points, and the determining whether the two nearest pixel points on both sides of the second center track belong to glue path pixel points to detect and measure the glue break defect includes: Determine whether the two nearest pixel points on both sides of each of the central track points along the second central track belong to the glue path pixel points; Determine whether each of the central trajectory points has a corresponding edge point pair; When the two nearest pixel points on both sides of the second central track of a plurality of the central track points exceeding a third predetermined number continuously do not belong to the glue path pixel points, and the plurality of the central track points do not have corresponding edge point pairs, it is determined that the positions corresponding to the plurality of the central track points have the straight-mouth glue break defect; When the two nearest pixel points on both sides of the second central trajectory of a plurality of the central trajectory points exceeding the third predetermined number continuously do not belong to the glue path pixel points, and the plurality of the central trajectory points have corresponding edge point pairs, the plurality of the central trajectory points are added to the set of points to be detected, so as to detect and measure the oblique glue breaking defect; A straight-mouth glue break defect point set is determined according to the glue path pixel points located on both sides of the straight-mouth glue break defect position, and the starting point, end point, length, straight-mouth glue break defect point set and defect area of ​​each straight-mouth glue break defect are output.

7. The method for detecting the quality of the rubber road according to claim 6, characterized in that: The step of adding the plurality of center track points to a set of points to be detected to detect and measure the oblique glue break defect includes: Determine a central detection point according to the set of points to be detected, and perform edge point search along a first direction according to the central detection point to determine a first detection point pair and a first detection line corresponding to the first detection point pair; Move the first detection line along a second direction with a predetermined step length, and perform edge point search again along the first direction to determine a plurality of second detection point pairs and second detection lines corresponding to the second detection point pairs, wherein the second direction is perpendicular to the first direction; For each of the second detection lines, the number of glue path pixels on the second detection line is calculated; When the number of pixels of the glue path is equal to the preset number of pixels, it is determined that the position corresponding to the second detection line does not have the oblique glue break defect, and the detection and measurement of the oblique glue break defect are exited, wherein the preset number of pixels is the sum of the number of pixels of the second detection line and twice the preset tolerance number of pixels; When the number of pixels of the glue path is less than the preset number of pixels, it is determined that the oblique glue break defect exists at the position corresponding to the second detection line; The oblique glue break defect point set is determined according to the first detection point pair and the second detection point pair, and the starting point, end point, length, the oblique glue break defect point set and the defect area of ​​each oblique glue break defect are output.

8. The method for detecting the quality of the rubber road according to claim 6, characterized in that: After determining whether the central track point has a corresponding edge point pair, detecting and measuring the second type of defects in the glue inspection area according to the plurality of edge point pairs further includes: When the two nearest pixel points on both sides of the second central track of a plurality of the central track points exceeding a fourth predetermined number continuously do not belong to the glue path pixel points, and a corresponding edge point exists at the plurality of the central track points, it is determined that there is a glue shortage defect at the position corresponding to the central track point; A glue deficiency defect point set is determined based on a plurality of edge points located at and adjacent to the glue deficiency defect position, so as to output the starting point, end point, lowest point, length, glue deficiency defect point set, defect region and defect area of ​​each glue deficiency defect.

9. The method for detecting the quality of the rubber road according to claim 6, characterized in that: The calculating of the centroids of the plurality of glue path pixel points to detect and measure glue deviation defects includes: Calculating the centroid of a plurality of the glue path pixel points; Determine a detection reference for the adhesive deviation defect, wherein the detection reference includes any one or more of a reference point, a reference line, and a reference profile; Calculating a first minimum distance from the centroid to the detection reference; When the first minimum distance is greater than a preset distance, it is determined that the glue deviation defect exists in the glue inspection area, and the first minimum distance is output as the glue deviation amount; When the detection benchmark includes the benchmark contour, the second minimum distance and the maximum distance between multiple edge points and the benchmark contour, and the third minimum distance between the center point of the minimum circumscribed rectangular frame of multiple glue path pixel points and the benchmark contour are calculated and output as the glue deviation.

10. A rubber road quality detection device, characterized in that: include: An acquisition module, used for acquiring an image to be detected, wherein the image to be detected includes a glue path area; A first defect detection module, used for detecting a first type of defect on the image to be detected based on an instance segmentation model; An extraction module, used for extracting glue paths from the image to be detected based on a semantic segmentation model, and outputting a binary image of the glue paths; A determination module, configured to set a glue detection area in the glue path binary image, wherein the glue detection area includes a first center track; An edge detection module is used to perform glue path edge detection in the glue detection area along a direction perpendicular to the first central track to obtain a plurality of edge point pairs distributed along both sides of the first central track; A second defect detection module, used for detecting and measuring a second type of defect in the glue inspection area according to the plurality of edge points; An output module is used to output the detection result of the first type of defects in the image to be detected, and the detection and measurement results of the second type of defects.

Images