A tape quality monitoring method and system based on image processing

Abstract

The present invention relates to the field of image processing technology, and more specifically, to a method and system for monitoring the quality of adhesive tape based on image processing, comprising: obtaining a target image of adhesive tape, graying the target image, and obtaining a first gray image; inputting the first gray image into an autoencoder network after optimizing the loss function, reconstructing a second gray image with adhesive tape characteristics, and calculating the offset angle of the adhesive tape, thereby determining whether the adhesive tape is offset, wherein the important factor is inversely correlated with the difference between the hue value of the pixel point in the HSV space of the target image and the reference hue value, positively correlated with the saturation of the pixel point, and inversely correlated with the brightness of the pixel point. The present invention grays and enhances the image through an autoencoder network, reduces the influence of non-tape factors in the image on the detection effect of the edge of the adhesive tape, improves the accuracy of the detection of the edge of the adhesive tape through image enhancement, and reduces the possibility of misjudgment during offset judgment.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically, to a method and system for monitoring the quality of adhesive tape based on image processing. Background Art

[0002] In the tape production scenario, it is necessary to detect whether the tape is offset during its travel to ensure the quality of tape production. However, in the early stage of tape offset, it is difficult for people to observe the change in angle with the naked eye, so we need to use algorithms to monitor the production process in real time to achieve the effect of early detection of problems.

[0003] Currently, the straight line detection of the tape edge is performed by detecting the tape edge in the tape production scenario. However, when the existing edge detection algorithm is performing tape edge detection, the collected images often contain more non-tape information, which causes interference. In addition, due to the reflective nature of the tape, highlights will appear on the tape surface, thereby affecting the tape edge detection result and causing misjudgment when judging the deviation of the tape during its movement. Summary of the invention

[0004] The present invention provides a tape quality monitoring method and system based on image processing, aiming to solve the problem in the related art that the collected images often contain more non-tape information, which causes interference, and the reflective nature of the tape will cause highlights on the tape surface, thereby affecting the tape edge detection results and causing misjudgment when judging the deviation of the tape during travel.

[0005] In a first aspect, the present invention provides a method for monitoring the quality of an adhesive tape based on image processing, comprising: obtaining a target image of an adhesive tape, graying the target image, and obtaining a first gray image; inputting the first gray image into an autoencoder network after optimizing a loss function, reconstructing a second gray image having features of the adhesive tape, and calculating the offset angle of the adhesive tape, thereby determining whether the adhesive tape is offset; wherein the expression of the optimized loss function is: ; In the formula, represents the optimized loss function, The first The gray value corresponding to the largest important factor in the neighborhood of each pixel. is the number of autoencoder networks in the current training process. The weight of the pixel gray value during reconstruction, For the The gray value of the pixel after reconstruction, is the number of all pixels; the important factor is inversely correlated with the difference between the hue value of the pixel and the reference hue value in the HSV space of the target image, is positively correlated with the saturation of the pixel, and is inversely correlated with the brightness of the pixel; the weight of the pixel gray value during reconstruction is positively correlated with the curvature of the binary classification boundary of the pixel, and is inversely correlated with the difference between the important factor of the pixel and the maximum important factor in the neighborhood.

[0006] In one embodiment, calculating the offset angle of the tape and then determining whether the tape is offset includes: performing straight line detection on the second grayscale image to obtain a first straight line parameter in the second grayscale image. ,in, is the perpendicular distance from the origin to the line, is the angle between the vertical line and the X-axis; whether the tape is offset is determined based on the degree of offset of the straight line in the second grayscale image at adjacent moments.

[0007] In one embodiment, judging whether the tape is offset according to the offset degree of the straight line in the second grayscale image at the adjacent moment includes: obtaining the next moment The second straight line parameters in the second grayscale image ; Calculate the degree of deviation of the tape according to the distance between the first straight line parameter and the second straight line parameter; In response to the degree of deviation being greater than a threshold value, determine that the tape is deviated.

[0008] In one embodiment, the degree of deviation of the tape is calculated according to the distance between the first straight line parameter and the second straight line parameter. , the calculation formula is: ; Among them, the second straight line parameter is , the first straight line parameter is .

[0009] In one embodiment, the calculation formula of the importance factor of a pixel is: ; In the formula, Indicates The important factor of pixels is Indicates The hue value of a pixel, Expressed as the base hue value, Expressed as The saturation value of each pixel, For the The brightness value of a pixel.

[0010] In one embodiment, a method for obtaining a reference hue value includes: converting the target image into an HSV space, constructing a frequency histogram of the hue with the hue as the horizontal coordinate and the frequency of occurrence of the hue as the vertical coordinate, and selecting the hue corresponding to the maximum frequency in the frequency histogram as the reference hue value.

[0011] In one embodiment, the weight of the grayscale value of the i-th pixel during reconstruction is calculated using the following formula: ; In the formula, Indicates the curvature of the binary classification boundary of the pixel point. Indicates The important factor of pixels is Represents the largest important factor in the neighborhood of the i-th pixel.

[0012] In one embodiment, a method for calculating the curvature of a binary classification boundary of a pixel point includes: establishing a window of a preset size with any pixel point as the center; performing binary classification on important factors corresponding to the pixel points in the window to obtain a classification result; performing edge detection on the classification result to obtain a classification boundary, and fitting the coordinate data of the pixel points on the classification boundary to obtain a fitting function; calculating the mean of the absolute values ​​of the second-order derivatives of all coordinate data, and using the mean as the curvature of the binary classification boundary of the pixel point.

[0013] In a second aspect of the present invention, there is also provided a tape quality monitoring system based on image processing, comprising a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the above-mentioned tape quality monitoring methods based on image processing.

[0014] Beneficial Effects

[0015] (1) The image is grayed out and enhanced through the autoencoder network to reduce the impact of non-tape factors in the image on the tape edge detection effect. The accuracy of tape edge detection is improved through image enhancement, and the possibility of misjudgment during offset judgment is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0017] Figure 1 is a flow chart schematically illustrating calculation of tape offset according to an embodiment of the present invention;

[0018] Figure 2 FIG. 1 is a diagram schematically showing a system structure according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0020] like Figure 1 As shown, the specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings.

[0021] In one embodiment, during the production of the adhesive tape, a conveyor belt is required to transport the adhesive tape and ensure that the adhesive tape moves in a straight line on the conveyor belt to avoid deviation during the process of the adhesive tape, thereby affecting the production quality of the adhesive tape. Therefore, the present invention monitors the process of the adhesive tape through an image processing method, finds problems in time, and reduces the occurrence of serious consequences. For the image processing method to monitor the process of the adhesive tape, a shooting device is required to monitor it in real time through the shooting device.

[0022] S101: Acquire a target image of the tape.

[0023] In one embodiment, a device for capturing images is disposed on a conveyor belt, and the capturing device is installed directly above the adhesive tape. Specifically, the camera device is installed directly above the central axis of the adhesive tape, and the adhesive tape accounts for the largest proportion in the captured image of the camera device. The camera device may be a camera or a mobile phone, etc. Then, the adhesive tape on the conveyor belt is captured by the camera device to obtain a target image of the adhesive tape, and the adhesive tape is grayed to obtain a first grayscale image.

[0024] S102: Calculate the importance factor of the pixel.

[0025] In one embodiment, in the collected target image, it is observed that the tape has color features that are obviously different from the background, and there are reflections on the surface of the tape and there are large interference factors, which will affect the subsequent edge detection and cause the tape offset to be calculated inaccurately. In order to better use the color features to distinguish the tape from the background and reduce the influence of the reflection factors, this embodiment converts the target image to the HSV space so as to quantitatively analyze the hue, saturation and brightness of the target image. According to the image features obtained by the quantitative analysis, the probability that each pixel belongs to the tape part is calculated, and this probability is called an important factor. The important factor is used to reflect the importance of the pixel in the image, which is helpful for the next step of segmentation judgment in the autoencoder.

[0026] In one embodiment, the importance factor of a pixel is calculated using the following formula: In the formula, Indicates The important factor of pixels is Indicates The hue value of a pixel, Expressed as the base hue value, Expressed as The saturation value of each pixel, For the The brightness value of a pixel.

[0027] In the formula, Indicates The color difference between the hue value of a pixel and the reference hue value of the tape. The larger the color difference, the smaller the probability that the pixel belongs to the tape. Therefore, it is inversely proportional to the importance factor. is a hyperparameter. To avoid exponential operations that make the value too small, The importance assessment effect of each pixel is not good; For the The saturation value of each pixel. Since the saturation indicates the purity of the tape color, the larger the value, the closer it is to the tape color. The value should be proportional to the important factor; For the The brightness of each pixel. When the image brightness is too high, the reflective features on the tape surface are enhanced, and the image is overexposed as a whole, which affects the accurate judgment of the tape. To avoid this situation, the important factor should be Negative correlation.

[0028] In one embodiment, a method for obtaining a reference hue value includes: converting the target image into an HSV space, and constructing a frequency histogram of the hue with the hue as the horizontal coordinate and the frequency of occurrence of the hue as the vertical coordinate. Since the color of the tape in the camera shot image occupies the largest proportion in the image, the hue corresponding to the maximum frequency in the frequency histogram is selected as the reference hue value.

[0029] In one embodiment, if the three-channel values ​​are grayed proportionally in the RGB space, it is global graying and the graying weight of each pixel is consistent, which makes it difficult to distinguish the tape and the background in the grayed image. Even if the graying is combined with the important factors, it is difficult to determine the specific distribution value of the proportional coefficient. Therefore, an autoencoder network is introduced to learn the features of ordinary grayscale images and reconstruct them through the model, and a loss function is designed to guide the learning process of the autoencoder. When designing the loss function, the important factors of the i-th pixel are used to obtain the , so that the autoencoder can learn the characteristics of the tape area and reconstruct the output grayscale image with the characteristics of the tape, so as to achieve the effect of segmenting the tape and the background, which helps to improve the accuracy of tape edge detection.

[0030] However, since light hitting the tape will produce some bright parts, resulting in a low important factor value, when the autoencoder network is used to generate a new grayscale image, tape holes will be generated, and redundant edges will appear, interfering with subsequent calculations and increasing the amount of calculations. The loss function of the autoencoder network is further optimized to achieve better grayscale image enhancement.

[0031] S103: Calculate the weight of the grayscale value of the pixel during reconstruction.

[0032] In one embodiment, Pixels as the center, create a window of preset size. For example: the window size is 3 3 or 5 5, etc. The window size can be adjusted by the implementer according to the specific implementation scenario. Then obtain the important factors of all pixels in the window, and use the binary classification algorithm to classify the important factors of all pixels in the window to obtain the classification results and the classification boundary. The binary classification method can be adjusted by the implementer according to the specific implementation scenario. This scheme uses the k-means, k=2 method for binary classification. Then use the edge detection algorithm to obtain the classification boundary for the classification result. Use the least squares method to fit the data to the pixel coordinates of the classification boundary to obtain the fitting function, calculate the second-order derivative of each data point in the fitting function, take the absolute value of the second-order derivative of all data points and then find the average to obtain the degree of curvature. , represents the curvature of the current boundary, where The larger the value, the closer the boundary is to curvature. When it is 0, it means that the boundary is close to a straight line.

[0033] Specifically, if the bending degree If the value is large, it means that the current pixel is more likely to belong to the tape. Even if the difference between the current pixel and the pixel with the largest category is large, the grayscale value of the pixel after enhancement should be closer to the grayscale value of the pixel at the tape. If it is small or close to 0, it means that the possibility that the current pixel belongs to the tape is relatively small. Even if the difference between the important factor of the current pixel and the pixel with the largest category is small, the grayscale value of the pixel at the tape should be less similar after enhancement.

[0034] However, when the When a pixel is on the tape and at the border between the tape and the background, it may also have a smaller degree of curvature. If we only rely on the degree of bending If the enhancement is performed, the enhancement effect will be poor, and then the calculation of the The difference between the pixel point and the largest important factor among all pixels in the neighborhood. If the difference is small and the important factor is large, it means that the current pixel point The pixel is on the tape and at the boundary between the tape and the background. The pixel point should be closer to the grayscale of the tape after reconstruction, that is, the grayscale value of the pixel point with the largest importance factor in the neighborhood is selected as the first The grayscale value of each pixel needs to be approximated during reconstruction.

[0035] In one embodiment, based on the above characteristics, the weight of the gray value of each pixel during reconstruction can be calculated using the following formula: ; In the formula, Indicates the curvature of the binary classification boundary of the pixel point. Indicates The important factor of pixels is represents the largest important factor in the neighborhood of the i-th pixel, Represents the normalization function.

[0036] in, The larger the value, the The closer the binary classification boundary of each pixel is to a straight line. Indicates The largest important factor in the neighborhood of a pixel, The smaller, and The larger the value, the The pixel is on the tape and at the boundary between the tape and the background. After reconstruction, the pixel should be closer to the gray value of the tape, Use the exp(-x) function for negative correlation mapping.

[0037] Then Pixels The larger the When reconstructing the gray value of a pixel, the closer it is to the gray value of the pixel with the largest important factor in the neighborhood, if the Pixels The smaller it is, the original grayscale should not be retained during reconstruction, but the grayscale should be as small as possible to ensure a distance from the taught grayscale.

[0038] S104: Optimize the loss function of the autoencoder network.

[0039] In one embodiment, the above features are used to optimize the loss function of the autoencoder network, and the expression of the optimized loss function is: ; In the formula, represents the optimized loss function, The first The gray value corresponding to the largest important factor in the neighborhood of each pixel. is the number of autoencoder networks in the current training process. The weight of the pixel gray value during reconstruction, For the The gray value of the pixel after reconstruction, is the number of all pixels.

[0040] In one embodiment, the method of using the loss function to guide the learning of the autoencoder is: input the first grayscale image into the autoencoder network after optimizing the loss function, and reconstruct the second grayscale image with the characteristics of the tape. Since the purpose of the autoencoder model training is to segment the tape and the background, in the process of finding the minimum value of the overall loss function, for the pixels belonging to the tape part, since their influence factor value is large, we hope to generate the second grayscale image with the characteristics of the tape. Gray value of pixel Should be the same as the original pixel value As close as possible, even This ensures that the characteristics of the tape are accurately preserved. For pixels with smaller influencing factors, we hope that the generated pixel value The minimum value can be reached, so that the background part can be effectively suppressed, thereby achieving effective segmentation of the tape and the background.

[0041] S105: Calculate the deviation angle of the tape and determine whether the tape is deviated.

[0042] In one embodiment, calculating the offset angle of the tape includes: Perform straight line detection to obtain the second grayscale image The first straight line parameters in ,in, is the perpendicular distance from the origin to the line, is the angle between the vertical line and the X axis. Then get the next moment Second grayscale image The second straight line parameter in . According to the second grayscale image at the adjacent moment The offset degree of the middle straight line is used to determine whether the tape is offset. Specifically, the offset degree of the tape is calculated according to the distance between the first straight line parameter and the second straight line parameter.

[0043] In one embodiment, the degree of tape deflection is calculated. , the calculation formula is: ; Among them, the second straight line parameter is , the first straight line parameter is .

[0044] In one embodiment, if the degree of deviation of the tape is less than a threshold, the tape continues to be transported; if the degree of deviation of the tape is greater than the threshold, the tape is determined to be deviated. At this time, the staff can discover the problem as early as possible to avoid serious consequences.

[0045] In another embodiment, when it is found that the adhesive tape is offset, the conveyor belt can be stopped and the adhesive tape can be removed from the conveyor belt. Alternatively, the conveyor belt can be stopped first, the position of the adhesive tape can be adjusted, and then the adhesive tape can be transported again.

[0046] The present invention also provides a tape quality monitoring system based on image processing. Figure 2 As shown, the system includes a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a method for monitoring tape quality based on image processing according to the first aspect of the present invention is implemented.

[0047] The system also includes other components familiar to those skilled in the art, such as a communication bus and a communication interface, whose configuration and functions are known in the art and thus will not be described in detail here.

[0048] In the present invention, the aforementioned memory may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus or device. For example, a computer-readable storage medium may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc., or any other medium that can be used to store the required information and can be accessed by an application, a module, or both. Any such computer storage medium may be part of a device or accessible or connectable to a device. Any application or module described in the present invention may be implemented using computer-readable / executable instructions that may be stored or otherwise maintained by such a computer-readable medium.

[0049] In the description of this specification, "plurality" or "several" means at least two, such as two, three or more, etc., unless otherwise clearly and specifically defined.

[0050] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0051] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent application. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A tape quality monitoring method based on image processing, characterized in that: include: Acquiring a target image of the tape, including: arranging a camera device just above the central axis of the tape, photographing the tape using the camera device, and making the tape occupy the largest proportion in the photographed picture, thereby obtaining the target image of the tape; graying the target image to obtain a first grayscale image; Input the first grayscale image into the autoencoder network after optimizing the loss function, reconstruct the second grayscale image with the characteristics of the tape, and calculate the offset angle of the tape, so as to determine whether the tape is offset; Among them, the expression of the optimization loss function is: ； In the formula, represents the optimized loss function, The first The gray value corresponding to the largest important factor in the neighborhood of each pixel. is the number of autoencoder networks in the current training process. The weight of the gray value of each pixel during reconstruction, For the The gray value of the pixel after reconstruction is is the number of all pixels; the importance factor is used to reflect the importance of the pixel in the image; the calculation formula of the importance factor of the pixel is: ; In the formula, Indicates The important factor of pixels is Indicates The hue value of a pixel, Expressed as the base hue value, Expressed as The saturation value of each pixel, For the The brightness value of each pixel, is a hyperparameter; The important factor is inversely correlated with the difference between the hue value of the pixel point and the reference hue value in the HSV space of the target image, is positively correlated with the saturation of the pixel point, and is inversely correlated with the brightness of the pixel point; The weight of the pixel gray value during reconstruction is positively correlated with the curvature of the binary classification boundary of the pixel, and is negatively correlated with the difference between the important factor of the pixel and the maximum important factor in the neighborhood.

2. The tape quality monitoring method based on image processing according to claim 1 is characterized in that: Calculate the offset angle of the tape, and then determine whether the tape is offset, including: Perform straight line detection on the second grayscale image to obtain the first straight line parameter in the second grayscale image ,in, is the perpendicular distance from the origin to the line, is the angle between the vertical line and the X-axis; Whether the adhesive tape is offset is determined based on the extent of the offset of the straight lines in the second grayscale image at adjacent moments.

3. The tape quality monitoring method based on image processing according to claim 2 is characterized in that: Judging whether the tape is offset according to the offset degree of the straight line in the second grayscale image at adjacent moments includes: Get the next moment The second straight line parameters in the second grayscale image ; Calculate the degree of deviation of the tape according to the distance between the first straight line parameter and the second straight line parameter; In response to the degree of deviation being greater than a threshold, it is determined that the tape is deviated.

4. The tape quality monitoring method based on image processing according to claim 3 is characterized in that: Calculate the offset of the tape based on the distance between the first straight line parameter and the second straight line parameter , the calculation formula is: ； The second straight line parameters are , the first straight line parameter is .

5. The tape quality monitoring method based on image processing according to claim 1 is characterized in that: The method for obtaining the reference hue value includes: The target image is converted into HSV space, and a frequency histogram of the hue is constructed with the hue as the horizontal coordinate and the frequency of occurrence of the hue as the vertical coordinate, and the hue corresponding to the maximum frequency in the frequency histogram is selected as the reference hue value.

6. The tape quality monitoring method based on image processing according to claim 1 is characterized in that: Calculate the weight of the gray value of the i-th pixel during reconstruction. The calculation formula is: ； In the formula, Indicates the curvature of the binary classification boundary of the pixel point. Indicates The important factor of pixels is Indicates The largest importance factor in the neighborhood of a pixel.

7. The tape quality monitoring method based on image processing according to claim 6 is characterized in that: The method for calculating the curvature degree of the binary classification boundary of the pixel point includes: Create a window of preset size centered on any pixel; Performing binary classification on the important factors corresponding to the pixels in the window to obtain a classification result; Performing edge detection on the classification result to obtain a classification boundary, and fitting the coordinate data of the pixel points on the classification boundary to obtain a fitting function; Calculate the mean of the absolute values ​​of the second-order derivatives of all coordinate data, and use the mean as the curvature of the binary classification boundary of the pixel point.

8. A tape quality monitoring system based on image processing, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the tape quality monitoring method based on image processing as described in any one of claims 1 to 7.

Images