Conveyor belt surface defect tracking method based on image mask rapid matching

Abstract

The invention discloses a conveyor belt surface defect tracking method based on image mask rapid matching, and relates to the field of automatic detection, which comprises the following steps: acquiring and processing to obtain a standard image; performing instance segmentation on the standard image, identifying a defect area and extracting a defect mask; establishing a defect database, distributing an identification ID for the defect area, and storing the identification ID and the defect mask in the defect database; in the next period, the collected real-time image is corrected and synchronously matched, a first defect mask is obtained, the first defect mask and the stored defect masks are matched in sequence, and interaction ratio data and structural similarity indexes between the defect masks are obtained through calculation; and when the interaction ratio data and the structural similarity index are both greater than a first threshold value, judging that the defect region belongs to the same defect, otherwise, judging that the defect region is a new defect region, and updating the defect database. According to the invention, the defect detection speed and accuracy of the surface area of the conveyor belt are improved, and repeated recording or missing detection is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of automated detection, and specifically, to a conveyor belt surface defect tracking method based on fast image mask matching. Background Art

[0002] With the continuous development of industrial automation, conveyor belts are widely used in various production lines. Especially in high-speed production environments, defects on the conveyor belt surface will directly affect production efficiency and product quality. Therefore, real-time monitoring and tracking of conveyor belt surface defects have become an important task in industrial automated detection. Most current defect detection methods rely on traditional image processing techniques and machine vision techniques to complete defect detection. A typical detection process includes steps such as image acquisition, image preprocessing, feature extraction, and defect recognition. However, when dealing with the real-time detection requirements of high-speed conveyor belts, these methods exhibit the following problems: 1) slow detection speed, the existing technology processes images at a relatively slow speed and cannot meet the real-time monitoring requirements of high-speed production lines; 2) insufficient accuracy, under complex lighting, dynamic background, or occlusion conditions, the existing technology is difficult to maintain high accuracy; 3) lack of an effective defect tracking mechanism, the existing technology usually cannot achieve continuous tracking and recording of the same defect, resulting in repeated detection or missed detection of defects.

[0003] In summary, there is an urgent need for a conveyor belt surface defect tracking method based on fast image mask matching to solve the problems of slow detection speed, insufficient accuracy, and lack of an effective defect tracking mechanism in the existing detection technology. Summary of the Invention

[0004] To solve the deficiencies of the above-mentioned existing technology, the present invention provides a conveyor belt surface defect tracking method based on fast image mask matching, including:

[0005] Collecting a conveyor belt surface image using a high-speed camera, processing the surface image to obtain a standard image;

[0006] Performing instance segmentation on the standard image using a first preset algorithm, identifying the defect areas on the conveyor belt surface, and generating defect masks corresponding to the defect areas;

[0007] Assigning a unique identification ID to all the already identified defect areas, establishing a defect database, and storing the identification ID and defect mask corresponding to the defect areas in the defect database;

[0008] In the next cycle, performing calibration and synchronous matching processing on the surface image collected by the high-speed camera to obtain a processing result, and based on the processing result, identifying and obtaining the defect mask corresponding to any defect area, denoted as the first defect mask;

[0009] The first defect mask is sequentially matched with the defect masks stored in the defect database by using a second preset algorithm, and the interaction ratio data and the structural similarity index between the first defect mask and the stored defect masks are respectively calculated;

[0010] A first threshold is set, and it is determined whether any set of interaction ratio data and structural similarity index are both greater than the first threshold. If so, it is determined that the first defect mask matches the corresponding defect mask in the defect database; if not, it is determined that the defect area corresponding to the first defect mask is a new defect area, denoted as the first defect area;

[0011] A unique identification ID is assigned to the first defect area, and the corresponding identification ID and the first defect mask are stored in the defect database to update the defect database.

[0012] The present invention is implemented by the following technical solutions: A high-speed camera is used to collect the surface image of the conveyor belt. Since the images of the conveyor belt are different under different camera positions, especially the geometric shape and position of the defect area, in the image acquisition stage, it is necessary to calibrate the collected surface image to obtain a standard image to ensure the unity of image acquisition. The standard image is subjected to instance segmentation by a first preset algorithm to identify the defect area on the conveyor belt and extract the defect mask corresponding to the defect area; A defect database is established, and a unique identification ID is assigned to each defect area, and then the identification ID corresponding to the defect area and the defect mask are stored in the defect database together.

[0013] In the next cycle, when performing defect mask matching, it is necessary to ensure that the processing of the collected image is carried out under the same conditions. Even if the angle of the camera or the motion state of the conveyor belt is different, the similarity of the defect masks can be effectively compared. Therefore, in the next cycle, the surface image collected by the high-speed camera is corrected and synchronously matched, and then the defect mask of the defect area is extracted, denoted as the first mask; By using a second preset algorithm, the first defect mask is sequentially matched with the masks stored in the defect database, specifically by calculating the interaction ratio data and the structural similarity index between the first defect mask and the stored defect masks. When any set of interaction ratio data and structural similarity index both satisfy being greater than the first threshold, it is determined that the first defect mask matches the corresponding defect mask in the defect database, that is, they belong to the same defect; when any set of interaction ratio data and structural similarity index do not satisfy being greater than the first threshold, it is determined that the defect area corresponding to the first defect mask is a new defect area, denoted as the first defect area; A unique identification ID is assigned to the first defect area, and at the same time, the identification ID and the first defect mask are stored in the defect database to update the defect database.

[0014] As an alternative technical solution, processing the surface image to obtain a standard image includes:

[0015] Processing the surface image using a perspective transformation formula to transform the surface image into a unified reference coordinate system to obtain the standard image. The perspective transformation formula is specifically:

[0016]

[0017] is the perspective transformation matrix, obtained by calibrating the high-speed camera; the coordinates of the surface image are (x, y); the coordinates of the standard image are (x ′ , y ′ ).

[0018] As an alternative technical solution, correcting the surface image collected by the high-speed camera includes:

[0019] Calibrating the high-speed camera to obtain corresponding internal and external parameters, and based on the internal and external parameters, obtaining the internal parameter matrix and external parameter matrix of the high-speed camera;

[0020] The internal parameter matrix includes:

[0021]

[0022] f x represents the horizontal focal length of the image, and f y represents the vertical focal length of the image; c x and c y are the principal point coordinates, representing the position of the center point of the image;

[0023] The external parameter matrix includes [R|t], where R is the rotation matrix and t is the translation vector;

[0024] When the angle of the high-speed camera changes, based on the internal parameter matrix and the external parameter matrix, transforming the surface image captured by the high-speed camera into a unified world coordinate system to complete the correction processing of the surface image.

[0025] As an alternative technical solution, synchronously matching the surface image collected by the high-speed camera includes:

[0026] Synchronously matching the surface image using a first formula, specifically:

[0027] p(t) = P(t) + Δp(t)

[0028] p(t) is the position of the defect area in the surface image, and P(t) is the position of the defect area on the conveyor belt; Δp(t) is the offset of any defect area during the movement of the conveyor belt, which is calculated based on the running speed of the conveyor belt and the initial position of the defect area;

[0029] Based on the first formula, the position of the defect area collected each time in the surface image is corrected, so that the position of the defect area in the surface image can be synchronously matched with the conveyor belt.

[0030] As an alternative technical solution, calculating the interaction ratio data using the second formula includes:

[0031]

[0032] IoU is the interaction ratio data between two defect masks, Area of Overlap is the intersection area between two defect masks, and Area of Union is the union area between two defect masks.

[0033] As an alternative technical solution, calculating the structural similarity index using the third formula includes:

[0034]

[0035] SSIM(i,j) is the structural similarity index between two defect masks corresponding to two images, μ i is the mean of image i, μ j is the mean of image j, σ ij is the covariance between image i and image j, is the variance of image i, is the variance of image j; C1 and C2 are both constants to avoid the denominator being zero.

[0036] As an alternative technical solution, the first preset algorithm is an image instance segmentation algorithm.

[0037] As an alternative technical solution, the second preset algorithm is a mask fast matching algorithm.

[0038] As an alternative technical solution, the first threshold is 0.95.

[0039] One or more technical solutions provided by the present invention have at least the following technical effects or advantages:

[0040] The present invention discloses a method for tracking surface defects of a conveyor belt based on fast image mask matching. By optimizing the mask matching algorithm, it realizes the rapid monitoring of the defect areas on the conveyor belt surface, meeting the real-time monitoring requirements of high-speed conveyor belts. By combining instance segmentation with similarity calculation (including the intersection ratio data and structural similarity index between defect masks), it can maintain high defect detection accuracy under complex lighting and background conditions. By assigning unique IDs to the defect areas and updating and maintaining the defect database, it can achieve continuous tracking of the same defect area, avoiding duplicate records or missed detections. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, and constitute a part of the present invention, but do not limit the embodiments of the present invention.

[0042] Figure 1 It is a schematic flowchart of a method for tracking surface defects of a conveyor belt based on fast image mask matching in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] In order to more clearly understand the above objects, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

[0044] In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0045] Embodiment

[0046] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a method for tracking surface defects of a conveyor belt based on fast image mask matching in the present invention, including:

[0047] Collect the surface image of the conveyor belt using a high-speed camera, process the surface image to obtain a standard image;

[0048] Perform instance segmentation on the standard image using a first preset algorithm to identify the defect areas on the conveyor belt surface and generate defect masks corresponding to the defect areas;

[0049] Assign a unique identification ID to all the identified defect areas, establish a defect database, and store the identification ID and defect mask corresponding to the defect areas in the defect database;

[0050] In the next cycle, the surface images collected by the high-speed camera are corrected and synchronously matched to obtain a processing result. Based on the processing result, a defect mask corresponding to any defect area is identified and obtained, denoted as the first defect mask;

[0051] The first defect mask and the defect masks stored in the defect database are sequentially matched using a second preset algorithm, and the interaction ratio data and the structural similarity index between the first defect mask and the stored defect masks are respectively calculated;

[0052] A first threshold is set to determine whether any set of interaction ratio data and structural similarity index are both greater than the first threshold. If so, it is determined that the first defect mask matches the corresponding defect mask in the defect database. If not, it is determined that the defect area corresponding to the first defect mask is a new defect area, denoted as the first defect area;

[0053] A unique identification ID is assigned to the first defect area, and the corresponding identification ID and the first defect mask are stored in the defect database to update the defect database.

[0054] Specific embodiments of the present invention are as follows:

[0055] Image acquisition and calibration processing:

[0056] The surface images of the conveyor belt are collected using a high-speed camera. To ensure the quality of the collected images, the camera position can be adjusted according to the specific application scenario, such as directly above or obliquely above the conveyor belt. Under different camera positions, the surface images of the conveyor belt collected will be different, especially the geometric shape and position of the defect area. Therefore, in the image acquisition stage, the collected surface images need to be calibrated to eliminate the influence of the camera installation angle on image acquisition. Processing the surface images to obtain standard images includes:

[0057] The surface images are processed using a perspective transformation formula to transform the surface images into a unified reference coordinate system to obtain the standard images. The perspective transformation formula is specifically:

[0058]

[0059] is the perspective transformation matrix, obtained by calibrating the high-speed camera; the coordinates of the surface image are (x, y); the coordinates of the standard image are (x ′ , y ′ ).

[0060] Among them, through the perspective transformation matrix, the image under the view of the high-speed camera is converted into the image in the standard coordinate system, so as to eliminate the influence brought by the view difference and ensure the consistency of subsequent image analysis.

[0061] Defect instance segmentation:

[0062] Use the image instance segmentation algorithm (different instance segmentation models can be selected according to different application scenarios, such as MaskR-CNN, MaskFormer, DeepLab, etc.) to perform instance segmentation processing on the standard image, identify the defect areas on the conveyor belt surface and generate corresponding defect masks; through precise pixel-level classification, the defect areas in the image can be accurately extracted and marked as defect masks, which serve as the basic data for subsequent defect matching and tracking.

[0063] Defect matching and tracking mechanism:

[0064] Establish a defect database, and assign a unique identification ID to each identified defect area. Store the identification ID and the corresponding defect mask in the defect database. Obtain the defect mask corresponding to the defect area in the next cycle (denoted as the first defect mask). When performing mask matching, it is necessary to ensure that each image is processed under the same conditions. Even if the angle of the camera or the movement state of the conveyor belt is different, the similarity of the defect masks can be effectively compared. For this purpose, the following measures are taken:

[0065] The correction process for the surface image collected by the high-speed camera includes:

[0066] Calibrate the high-speed camera to obtain the corresponding internal and external parameters. Based on the internal and external parameters, obtain the internal parameter matrix and the external parameter matrix of the high-speed camera;

[0067] The internal parameter matrix includes:

[0068]

[0069] f x represents the horizontal focal length of the image, and f y represents the vertical focal length of the image; c x and c y are the principal point coordinates, representing the position of the center point of the image;

[0070] The external parameter matrix includes [R|t], where R is the rotation matrix and t is the translation vector;

[0071] When the angle of the high-speed camera changes, based on the internal parameter matrix and the external parameter matrix, convert the surface image captured by the high-speed camera into a unified world coordinate system to complete the correction process of the surface image.

[0072] Among them, through the calibrated internal and external camera parameter matrices, the surface images captured by the camera are converted into a unified world coordinate system, thereby eliminating the influence of the camera installation angle difference on the matching result.

[0073] Performing synchronous matching processing on the surface images collected by the high-speed camera includes:

[0074] Performing synchronous matching processing on the surface images using the first formula, specifically:

[0075] p(t) = P(t) + Δp(t)

[0076] p(t) is the position of the defect area in the surface image, P(t) is the position of the defect area on the conveyor belt; Δp(t) is the offset of any defect area during the movement of the conveyor belt, which is calculated based on the running speed of the conveyor belt and the initial position of the defect area;

[0077] Based on the first formula, the position of the defect area collected in each surface image is corrected so that the position of the defect area in the surface image can be synchronously matched with the conveyor belt.

[0078] Among them, through the above method, the defect position collected in each surface image can be corrected according to the movement trajectory of the conveyor belt, so that the position of the defect area in the surface image can be matched with the position on the conveyor belt.

[0079] Through the mask fast matching algorithm, the first defect mask is sequentially matched with all the defect masks stored in the defect database. During the defect mask matching process, the intersection ratio data and the structural similarity index are used to calculate the similarity between each pair of defect masks;

[0080] Calculating the intersection ratio data using the second formula includes:

[0081]

[0082] IoU is the intersection ratio data between two defect masks, Area of Overlap is the intersection area between two defect masks, and Area of Union is the union area between two defect masks.

[0083] Calculating the structural similarity index using the third formula includes:

[0084]

[0085] SSIM(i,j) is the structural similarity index between two defect masks corresponding to two images, μ i is the mean value of image i, μ j is the mean value of image j, σ ij is the covariance between image i and image j, is the variance of image i, is the variance of image j; C1 and C2 are both constants to avoid a zero denominator.

[0086] Defect database management:

[0087] Determine whether any set of interaction ratio data and structural similarity index are both greater than 0.95. If so, it is regarded as the same defect; if not, it is considered that a new defect is detected, and a new identification ID is assigned to it, and the identification ID corresponding to the new defect and the defect mask are stored in the defect database, and the defect database is updated.

[0088] In this embodiment, by collecting the surface image of the conveyor belt and performing calibration processing, the unity of the collected images is ensured; then instance segmentation is performed on the processed standard image to identify the defect areas on the surface of the conveyor belt, and the corresponding defect masks are extracted; a defect database is established, and a unique identification ID is assigned to each defect area. The identification ID and defect mask corresponding to any defect area are stored in the defect database as a set of data; in the next cycle, in order to ensure the accuracy of defect mask matching, the collected real-time images are corrected and synchronously matched, and then the defect areas of the conveyor belt and the corresponding first defect masks are obtained. The first defect mask is sequentially matched with the defect masks in the defect database. Specifically, by calculating the interaction ratio data and structural similarity index between the two defect masks, only when a set of interaction ratio data and structural similarity index are both greater than the first threshold, it is determined to belong to the same defect; otherwise, it is determined to be a new defect area, and the defect database is updated.

[0089] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they know the basic creative concept. Therefore, the appended claims are intended to be interpreted to include the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0090] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. A conveyor belt surface defect tracking method based on fast matching of image masks, characterized in that include: A high-speed camera is used to collect a surface image of the conveyor belt, and the surface image is processed to obtain a standard image; Using a first preset algorithm to perform instance segmentation on the standard image, identify defect areas on the surface of the conveyor belt, and generate defect masks corresponding to the defect areas; Allocate unique identification IDs to all identified defect areas, establish a defect database, and store the identification IDs and defect masks corresponding to the defect areas in the defect database; In the next cycle, the surface image collected by the high-speed camera is corrected and synchronously matched to obtain a processing result, and based on the processing result, a defect mask corresponding to any defect area is identified and obtained, which is recorded as a first defect mask; Using a second preset algorithm to sequentially match the first defect mask with the defect masks stored in the defect database, and respectively calculate interaction ratio data and a structural similarity index between the first defect mask and the stored defect masks; Setting a first threshold, determining whether any set of interaction ratio data and structural similarity index are both greater than the first threshold, if so, determining that the first defect mask matches the corresponding defect mask in the defect database, if not, determining that the defect region corresponding to the first defect mask is a new defect region, recorded as the first defect region; A unique identification ID is allocated to the first defect area, and the corresponding identification ID and the first defect mask are stored in the defect database, and the defect database is updated.

2. The conveyor belt surface defect tracking method based on fast image mask matching according to claim 1, characterized in that, Processing the surface image to obtain a standard image includes: The surface image is processed by using a perspective transformation formula, and the surface image is converted into a unified reference coordinate system to obtain the standard image. The perspective transformation formula is specifically: is a perspective transformation matrix obtained by calibrating the high-speed camera; the coordinates of the surface image are (x, y); the coordinates of the standard image are (x ′ , y ′ ).

3. A conveyor belt surface defect tracking method based on fast image mask matching according to claim 1, characterized in that, Correcting the surface image collected by the high-speed camera includes: Calibrate the high-speed camera to obtain corresponding internal and external parameters, and obtain an internal parameter matrix and an external parameter matrix of the high-speed camera based on the internal and external parameters; The internal parameter matrix includes: f x represents the horizontal focal length of the image, f y represents the vertical focal length of the image; c x and c y are the principal point coordinates, representing the position of the center point of the image; The external parameter matrix includes [R|t], where R is a rotation matrix and t is a translation vector; When the angle of the high-speed camera changes, the surface image taken by the high-speed camera is converted into a unified world coordinate system based on the intrinsic parameter matrix and the extrinsic parameter matrix to complete the correction processing of the surface image.

4. A conveyor belt surface defect tracking method based on fast matching of image masks according to claim 1, characterized in that, The synchronous matching process of the surface image collected by the high-speed camera includes: The first formula is used to perform synchronous matching processing on the surface image, specifically: p(t)=P(t)+Δp(t) p(t) is the position of the defective area in the surface image, P(t) is the position of the defective area on the conveyor belt; Δp(t) is the offset of any defective area during the movement of the conveyor belt, which is calculated based on the running speed of the conveyor belt and the initial position of the defective area; Based on the first formula, the position of the defective area in each surface image acquisition is corrected so that the position of the defective area in the surface image can be synchronously matched with the conveyor belt.

5. A conveyor belt surface defect tracking method based on fast matching of image masks according to claim 1, characterized in that Calculating the interaction ratio data using the second formula includes: The IoU is the interaction ratio data between two defect masks, the Area of Overlap is the intersection area between two defect masks, and the Area of Union is the union area of two defect masks.

6. A conveyor belt surface defect tracking method based on fast matching of image masks according to claim 1, characterized in that Calculating the structural similarity index using the third formula includes: SSIM(i,j) is the structural similarity index between two defect masks corresponding to two images, μ i is the mean of image i, μ j is the mean of image j, σ ij is the covariance of image i and image j, is the variance of image i, is the variance of image j; C1 and C2 are both constants to avoid a zero denominator.

7. A conveyor belt surface defect tracking method based on fast matching of image masks according to claim 1, characterized in that, The first preset algorithm is an image instance segmentation algorithm.

8. A conveyor belt surface defect tracking method based on fast matching of image masks according to claim 1, characterized in that The second preset algorithm is a mask fast matching algorithm.

9. A conveyor belt surface defect tracking method based on fast matching of image masks according to claim 1, characterized in that, The first threshold is 0.95.

Images