Automatic deviation correction method for steel strip based on machine vision

Abstract

The present invention relates to the field of steel strip monitoring, and in particular to a method for automatic deviation correction of steel strips based on machine vision. The method comprises: obtaining an image of the steel strip when it is working, calculating an adaptive standard deviation of each pixel point in the image in a bilateral filtering algorithm, and performing denoising on the image using the bilateral filtering algorithm to obtain an optimal image; performing edge detection on the optimal image to obtain an edge of the steel strip, calculating a distance between the edge of the steel strip and a standard position, and adjusting a guide roller according to the obtained distance so that the steel strip is located within the standard position range. The present invention can improve the accuracy of steel strip position recognition and facilitate deviation correction of the steel strip.

Description

Technical Field

[0001] The invention relates to the field of steel strip monitoring, and in particular to a steel strip automatic deviation correction method based on machine vision. Background Art

[0002] As an important material handling equipment, steel belt conveyor plays an important role in grain processing, warehousing, transportation and other links. During the operation of steel belt, deviation of steel belt is a common and unavoidable problem. After the steel belt is deviated, it needs to be adjusted in time, otherwise it may affect production efficiency, product quality, and even cause equipment failure. Based on machine vision technology, the steel belt operation status image monitored by the camera in real time, combined with image processing algorithms and control methods, can be used to monitor and adjust the position of the steel belt in real time, greatly improving the conveying efficiency and safety. However, when collecting real-time images of steel belts, certain electronic noise will be generated, which is usually manifested as some meaningless random bright spots or granular interference in the image, especially in low light environments or high ISO settings. If the real-time image of the steel belt with noise interference is analyzed, the automatic deviation correction process of the steel belt will be greatly affected.

[0003] A Chinese patent application document with publication number CN117522728A discloses a Bayer image denoising method based on bilateral filtering, the method comprising: obtaining a Bayer image, and denoising the pixel points of each channel in the image by bilateral filtering; adjusting the value domain filtering weights in the bilateral filtering according to the position of the pixel points in the image; and adjusting the spatial domain filtering weights and the value domain filtering weights in the bilateral filtering according to the image gain and / or temperature changes based on the aforementioned adjusted value domain filtering weights.

[0004] When bilateral filtering is used to denoise the real-time image of steel strips, the same spatial standard deviation is usually used for all pixels. However, in the steel strip image, the edge and detail information are crucial for the subsequent steel strip correction. If the bilateral filtering uses the same spatial standard deviation in all areas, the edge information of the steel strip will be blurred during denoising, especially in places where the surface texture of the steel strip is more complex or there are more details, which affects the accuracy of steel strip position recognition and thus affects the overall correction effect. Summary of the invention

[0005] In order to improve the accuracy of steel strip position recognition, the present invention provides a steel strip automatic deviation correction method based on machine vision.

[0006] The present invention provides a steel strip automatic deviation correction method based on machine vision, which adopts the following technical solutions:

[0007] Obtain an image of the steel strip in operation, calculate the adaptive standard deviation of each pixel in the image in the bilateral filtering algorithm, and use the bilateral filtering algorithm to denoise the image to obtain the best image;

[0008] Perform edge detection on the best image to obtain the edge of the steel strip, calculate the distance between the edge of the steel strip and the standard position, and adjust the guide roller according to the obtained distance so that the steel strip is within the standard position range;

[0009] The adaptive standard deviation is calculated as:

[0010] A window is constructed with any pixel in the image as the center, and the suspected noise level of the corresponding pixel is calculated. The expression of the suspected noise level is: ; In the formula, Indicates the suspected noise level of the m-th pixel, Represents the gray value of the mth pixel, Represents the average gray value of the pixels in the window. is a hyperparameter, Indicates the number of pixels corresponding to the majority of the grayscale values ​​of pixels in the window except the mth pixel. Indicates the number of pixels in the window except the m-th pixel;

[0011] The filtering requirement of each pixel is calculated, and the filtering requirement is positively correlated with the suspected noise level; the product of the filtering requirement and the preset spatial standard deviation is used as the adaptive standard deviation.

[0012] The effect is that by adaptively adjusting the spatial standard deviation for each pixel, the filter will reduce the spatial standard deviation in areas with complex textures or edges, thereby retaining more details and edge information and avoiding blurring the structure and surface texture of the steel strip due to excessive smoothing. In areas where noise is more concentrated, the filter will increase the spatial standard deviation to effectively remove noise, ensuring the denoising effect while reducing the impact on important details.

[0013] Preferably, the method further comprises:

[0014] A window is constructed with the mth pixel as the center, and the pixel in the window with the same grayscale value as the same position in the historical image is taken as the first pixel, and the number of the obtained first pixel is taken as the first data; the pixel in the window with different grayscale values ​​from the same position in the historical image is taken as the second pixel, and the number of the obtained second pixel is taken as the second data.

[0015] The effect is that by dividing the pixel points into the first pixel points and the second pixel points, it is easy to understand the possibility that the surrounding area belongs to the steel strip area.

[0016] Preferably, the expression of filtering requirement is:

[0017] , ;

[0018] , ;

[0019] In the formula, Indicates the filtering requirement of the mth pixel, Indicates the suspected noise level of the m-th pixel, Indicates the number of pixels with gradient changes in the window centered at the mth pixel. It represents the variance of the sine value of the angle between the gradient direction and the horizontal direction of all pixels with gradient changes in the window centered on the mth pixel. Indicates the maximum value of the first data and the second data, represents the minimum value between the first data and the second data, and β is a hyperparameter.

[0020] The effect is that by calculating the filtering requirement with reference to multiple dimensions, the accuracy of the filtering requirement calculation result is improved.

[0021] Preferably, the expression of filtering requirement is:

[0022] , ;

[0023] , ;

[0024] In the formula, Indicates the filtering requirement of the mth pixel, Indicates the suspected noise level of the m-th pixel, Indicates the number of pixels with gradient changes in the window centered at the mth pixel. It represents the variance of the gradient magnitude of all pixels with gradient changes in the window centered on the mth pixel. Indicates the maximum value of the first data and the second data, represents the minimum value between the first data and the second data, and β is a hyperparameter.

[0025] The effect is that the accuracy of the calculation result of the filtering requirement is improved by calculating the filtering requirement by referring to multiple dimensions such as the suspected noise level, the variance of the gradient size of the pixel point, the first data and the second data.

[0026] Preferably, the expression of filtering requirement is:

[0027] , ;

[0028] , ;

[0029] In the formula, Indicates the filtering requirement of the mth pixel, Indicates the suspected noise level of the m-th pixel, Indicates the number of pixels with gradient changes in the window centered at the mth pixel. Indicates the maximum value of the first data and the second data, represents the minimum value between the first data and the second data, and β is a hyperparameter.

[0030] Preferably, the Sobel operator is used to detect the image to obtain pixel points with gradient changes.

[0031] Preferably, the step of obtaining the image of the steel belt when it is working is: collecting real-time images of the conveying device when it is working, graying the real-time images to obtain grayscale images, and using the SegNet semantic segmentation model to segment the grayscale images to obtain images of the steel belt when it is working.

[0032] The effect is that the image of the steel strip in operation is obtained by segmenting the grayscale image, which reduces the amount of data analysis and improves calculation efficiency.

[0033] Preferably, the Canny algorithm is used to perform edge detection on the optimal image to obtain the edge of the steel strip.

[0034] The present invention has the following technical effects:

[0035] 1. By adaptively adjusting the spatial standard deviation for each pixel point, denoising is flexibly performed according to the characteristics of different regions in the steel strip image. In areas with complex textures or edges, the filter will reduce the spatial standard deviation, thereby retaining more details and edge information, avoiding blurring the structure and surface texture of the steel strip due to excessive smoothing. In areas where noise is more concentrated, the filter will increase the spatial standard deviation, effectively remove noise, ensure the denoising effect while reducing the impact on important details, improve the accuracy of steel strip position recognition, and facilitate the correction of the steel strip.

[0036] 2. By denoising the steel strip image, the steel strip image can be made clearer and key information such as edges and textures can be retained, providing more accurate data for correcting the position of the steel strip. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-restrictive manner, and the same or corresponding reference numerals represent the same or corresponding parts.

[0038] Figure 1The present invention is a flowchart of a method for automatically correcting a steel strip based on machine vision according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0040] It should be understood that when the terms "first", "second", etc. are used in the claims, descriptions, and drawings of the present invention, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the description and claims of the present invention indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections.

[0041] The embodiment of the present invention discloses a method for automatically correcting the deviation of a steel strip based on machine vision, referring to Figure 1 , including the following steps, as follows:

[0042] S1: Acquire the image of the steel belt when it is working.

[0043] A high-definition camera is used to collect real-time images of the conveying device when it is working, and the real-time image is grayed to obtain a grayscale image. The semantic segmentation model segments the grayscale image to obtain the image of the steel strip when it is working.

[0044] S2: Calculate the suspected noise level of the corresponding pixel.

[0045] Construct a window centered on any pixel in the image, with a window size of , calculate the suspected noise level of the corresponding pixel point, and the expression of the suspected noise level is:

[0046] ;

[0047] In the formula, Indicates the suspected noise level of the m-th pixel, Represents the gray value of the mth pixel, Represents the average gray value of the pixels in the window, γ is a hyperparameter, γ=0.001, the existence of the hyperparameter is to prevent the gray value of the mth pixel from being equal to the average gray value in the window area, making the multiplier 0. Indicates the number of pixels corresponding to the majority of the grayscale values ​​of pixels in the window except the mth pixel. Indicates the number of pixels in the window except the mth pixel. The window area of ​​the corresponding pixel is defined as the surrounding area of ​​the pixel.

[0048] in the formula It represents the absolute value of the difference between the gray value of the mth pixel and the gray value mean of the pixels in the surrounding area. The larger the value, the greater the difference between the gray values ​​of the mth pixel and the surrounding pixels, which means that the mth pixel is more likely to be a potential noise data point, and the corresponding suspected noise level will also be greater. It represents the ratio of the number of pixels corresponding to the grayscale mode of the mth pixel in the area around the mth pixel to the number of pixels in the surrounding area excluding the mth pixel. The larger the value, the more consistent the grayscale values ​​of the remaining pixels in the surrounding area after removing the mth pixel. The greater the difference between the grayscale value of the mth pixel and the grayscale values ​​of the surrounding pixels, the greater the possibility that the mth pixel is a noise pixel, and the corresponding suspected noise level will also be greater. The suspected noise level of each pixel is calculated to obtain the possibility that the pixel is a potential noise pixel.

[0049] S3: Calculate the filtering requirement of each pixel point. The filtering requirement is positively correlated with the suspected noise level.

[0050] If a noise pixel is located at the edge of the steel strip in the image, in order to improve the accuracy of the steel strip correction, it is necessary to ensure the denoising effect while reducing the impact on important details, and the pixel should not be filtered too much. Therefore, it is necessary to analyze the grayscale change characteristics of the pixel and the surrounding area in the current image and the grayscale change characteristics of the pixel and the surrounding area in the historical image for the pixel with gradient changes, and combine the suspected noise level of each pixel to obtain the filtering requirement of each pixel.

[0051] S31: Construct a window with the mth pixel as the center, take the pixel in the window with the same grayscale value as the same position in the historical image as the first pixel, and use the number of the obtained first pixel as the first data; take the pixel in the window with different grayscale values ​​as the same position in the historical image as the second pixel, and use the number of the obtained second pixel as the second data.

[0052] For example, at the current moment The window is constructed with the mth pixel in the image as the center, with the historical moment The window is constructed with the mth pixel in the corresponding image as the center, with the historical moment The window is constructed with the mth pixel in the corresponding image as the center, ..., with the historical moment The window is constructed with the mth pixel in the corresponding image as the center. It should be noted that the mth pixel is the pixel at the same position in the image. Compare the current moment image with the five images at the historical moments. If the nth pixel in the window of the current moment image is the same as the pixel at the historical moment, , , , , The grayscale values ​​of the nth pixels in the corresponding 5 image windows are all the same, and the nth pixel is taken as the first pixel. Conversely, if the grayscale values ​​of the nth pixels in the corresponding 5 image windows are the same, the nth pixel is taken as the first pixel. , , , , If the grayscale value of the nth pixel in any image window is different, the nth pixel is used as the second pixel.

[0053] S32: Use the Sobel operator to detect the image to obtain pixel points with gradient changes, where the gradient change refers to a change in gradient direction or gradient amplitude.

[0054] S33: Calculate the filtering requirement of the pixel point.

[0055] In one embodiment, the expression of filtering requirement is:

[0056] , ;

[0057] , ;

[0058] In the formula, Indicates the filtering requirement of the mth pixel, Indicates the suspected noise level of the m-th pixel, Indicates the number of pixels with gradient changes in the window centered at the mth pixel. It represents the variance of the sine value of the angle between the gradient direction and the horizontal direction of all pixels with gradient changes in the window centered on the mth pixel. Indicates the maximum value of the first data and the second data, represents the minimum value of the first data and the second data, β is a hyperparameter, , the hyperparameter is set to prevent the denominator from being 0.

[0059] in the formula Indicates the difference between the number of pixels with grayscale changes and pixels without grayscale changes in the area around the mth pixel at continuous moments. The smaller the value, the smaller the difference between the two, indicating that the surrounding area is more likely to belong to the steel belt area, and the corresponding pixel points with gradient changes in the surrounding area are pixels on the edge of the steel belt. The higher the credibility, the lower the spatial standard deviation should be given to the mth pixel to retain more detail information, so the filtering requirement is smaller. Conversely, the larger the value, the greater the difference between the two, indicating that the surrounding area is less likely to belong to the steel belt area, and the corresponding pixel points with gradient changes in the surrounding area are pixels on the edge of the steel belt. The credibility is also lower, and the filtering requirement is greater.

[0060] in the formula It indicates the consistency of the gradient direction of all pixels with gradient changes in the area around the m-th pixel. The smaller the value, the stronger the consistency. It can be said that the arrangement of all pixels with gradient changes in the area around the m-th pixel is closer to a straight line. It can be further explained that the possibility that the area around this pixel belongs to the edge area of ​​the steel strip is greater. The lower the spatial standard deviation should be given to the m-th pixel to retain more detail information, so the filtering requirement is smaller.

[0061] When the steel belt conveyor transports grain, the steel belt is moving, while the steel belt conveyor body is stationary. Then, in the surrounding area of ​​the pixel points in the edge area of ​​the steel belt, there will be some pixel points whose grayscale values ​​change in the continuous moment image, while some pixel points whose grayscale values ​​do not change in the continuous moment image. Therefore, the image area without gradient change does not belong to the edge area of ​​the steel belt in the real-time image of the steel belt, and the corresponding suspected noise level value for the pixel points in this area will be used as the filtering requirement of the pixel points.

[0062] When calculating the filtering requirement of a pixel, the closer the arrangement of the pixels with gradient changes in the area around a pixel is to a straight line, the more likely the pixel is to be located at the edge of the steel strip. The lower the spatial standard deviation should be given to the pixels in this area to retain more detail information and the smaller the filtering requirement. However, in the real-time image of the steel strip, there may be some objects that are not at the edge of the steel strip but also have straight line features, resulting in low accuracy in quantifying the filtering requirement of only one image pixel. Therefore, in the area around a pixel, the closer the number of pixels with grayscale changes and pixels without grayscale changes at multiple consecutive moments, the more likely it is that this pixel is located at the edge of the steel strip, and the smaller the filtering requirement corresponding to this pixel will be.

[0063] In one embodiment, the expression of filtering requirement is:

[0064] , ;

[0065] , ;

[0066] In the formula, Indicates the filtering requirement of the mth pixel, Indicates the suspected noise level of the m-th pixel, Indicates the number of pixels with gradient changes in the window centered at the mth pixel. Indicates the maximum value of the first data and the second data, represents the minimum value of the first data and the second data, β is a hyperparameter, , the hyperparameter is set to prevent the denominator from being 0.

[0067] In one embodiment, the expression of filtering requirement is:

[0068] , ;

[0069] , ;

[0070] In the formula, Indicates the filtering requirement of the mth pixel, Indicates the suspected noise level of the m-th pixel, Indicates the number of pixels with gradient changes in the window centered at the mth pixel. It represents the variance of the gradient magnitude of all pixels with gradient changes in the window centered on the mth pixel. Indicates the maximum value of the first data and the second data, represents the minimum value between the first data and the second data, and β is a hyperparameter.

[0071] S4: taking the product of the filtering requirement and the preset spatial standard deviation as the adaptive standard deviation.

[0072] The expression of adaptive standard deviation is:

[0073] ;

[0074] In the formula, represents the adaptive standard deviation of the m-th pixel, Represents the preset spatial standard deviation value in the bilateral filtering algorithm, exemplarily, , Indicates the filtering requirement of the m-th pixel.

[0075] S5: Perform edge detection on the best image to obtain the edge of the steel strip, calculate the distance between the edge of the steel strip and the standard position, and adjust the guide roller according to the obtained distance so that the steel strip is located within the standard position range.

[0076] The grayscale standard deviation in bilateral filtering is set. In this embodiment, the grayscale standard deviation is preset to an empirical value of 5. The image is denoised using the bilateral filtering algorithm to obtain the best image. The Canny edge detection algorithm is used to obtain the edge of the steel strip in the real-time image of the steel strip. The distance between the edge of the steel strip and the standard position is calculated. The calculation of the distance between the edge of the steel strip and the standard position is a prior art, and its specific steps are not repeated here. The position or angle of the guide roller is adjusted according to the obtained distance so that the steel strip is within the standard position range.

[0077] By adaptively adjusting the spatial standard deviation for each pixel, denoising can be flexibly performed according to the characteristics of different regions in the steel strip image. In areas with complex textures or edges, the filter will reduce the spatial standard deviation to retain more details and edge information, avoiding blurring the structure and surface texture of the steel strip due to oversmoothing, while in areas where noise is more concentrated, the filter will increase the spatial standard deviation to effectively remove noise, ensuring the denoising effect while reducing the impact on important details.

[0078] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

[0079] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. The automatic deviation correction method of steel strip based on machine vision is characterized by: Includes steps: Obtain an image of the steel strip in operation, calculate the adaptive standard deviation of each pixel in the image in the bilateral filtering algorithm, and use the bilateral filtering algorithm to denoise the image to obtain the best image; Perform edge detection on the best image to obtain the edge of the steel strip, calculate the distance between the edge of the steel strip and the standard position, and adjust the guide roller according to the obtained distance so that the steel strip is within the standard position range; The adaptive standard deviation is calculated as: A window is constructed with any pixel in the image as the center, and the suspected noise level of the corresponding pixel is calculated. The expression of the suspected noise level is: ; In the formula, Indicates the suspected noise level of the m-th pixel, Represents the gray value of the mth pixel, Represents the average gray value of the pixels in the window. is a hyperparameter, Indicates the number of pixels corresponding to the majority of the grayscale values ​​of pixels in the window except the mth pixel. Indicates the number of pixels in the window except the m-th pixel; A window is constructed with the mth pixel as the center, and the pixel in the window with the same gray value as the same position in the historical image is taken as the first pixel, and the number of the obtained first pixel is taken as the first data; the pixel in the window with a different gray value from the same position in the historical image is taken as the second pixel, and the number of the obtained second pixel is taken as the second data; According to the suspected noise level of any pixel point and the possibility that the pixel point belongs to the edge area of ​​the steel strip, the filtering requirement of the pixel point is calculated, and the expression is: ， ； ， ； In the formula, Indicates the filtering requirement of the mth pixel, Indicates the suspected noise level of the m-th pixel, Indicates the number of pixels with gradient changes in the window centered at the mth pixel. It represents the variance of the gradient magnitude of all pixels with gradient changes in the window centered on the mth pixel. Indicates the maximum value of the first data and the second data, represents the minimum value between the first data and the second data, and β is a hyperparameter; The product of the filtering requirement and the preset spatial standard deviation is used as the adaptive standard deviation.

2. The automatic deviation correction method for steel strip based on machine vision according to claim 1 is characterized in that: The expression of filtering requirement is: ， ； ， ； In the formula, Indicates the filtering requirement of the mth pixel, Indicates the suspected noise level of the m-th pixel, Indicates the number of pixels with gradient changes in the window centered at the mth pixel. It represents the variance of the sine value of the angle between the gradient direction and the horizontal direction of all pixels with gradient changes in the window centered on the mth pixel. Indicates the maximum value of the first data and the second data, represents the minimum value between the first data and the second data, and β is a hyperparameter.

3. The automatic deviation correction method for steel strip based on machine vision according to claim 1 is characterized in that: The expression of filtering requirement is: ， ； ， ； In the formula, Indicates the filtering requirement of the mth pixel, Indicates the suspected noise level of the m-th pixel, Indicates the number of pixels with gradient changes in the window centered at the mth pixel. Indicates the maximum value of the first data and the second data, represents the minimum value between the first data and the second data, and β is a hyperparameter.

4. The automatic deviation correction method for steel strip based on machine vision according to claim 2 is characterized in that: The Sobel operator is used to detect the image and obtain the pixels with gradient changes.

5. The automatic deviation correction method for steel strip based on machine vision according to claim 1 is characterized in that: The steps of obtaining the image of the steel belt when it is working are as follows: collecting the real-time image of the conveying device when it is working, graying the real-time image to obtain a grayscale image, and using the SegNet semantic segmentation model to segment the grayscale image to obtain the image of the steel belt when it is working.

6. The automatic deviation correction method for steel strip based on machine vision according to claim 1 is characterized in that: The Canny algorithm is used to detect the edge of the optimal image to obtain the edge of the steel strip.

Images