Steel belt quality detection method and device based on machine vision and storage medium

Abstract

The invention relates to the technical field of steel belt defect detection, and discloses a steel belt quality detection method and device based on machine vision and a storage medium. The method comprises the following steps: acquiring a real-time steel strip image; inputting the real-time steel strip image into a Markov model to obtain an output result; wherein the Markov model is a model obtained by pre-training according to a steel strip sample image; and according to the output results, taking the defect with the maximum probability corresponding to the defect type in the output results as a defect judgment result, and sending the defect judgment result to an operation and maintenance end, so that the operation and maintenance end alarms and solves the defect. According to the method, the Markov model can be used for carrying out classification and state transition modeling on the steel belt features in the feature image, and accurate detection of defect types and positions is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of steel strip defect detection, and in particular to a steel strip quality detection method, device and storage medium based on machine vision. Background Art

[0002] As a material, steel is widely used in various fields such as national industry, agriculture and national defense because of its advantages of high strength, low cost and good machinability. As an industrial product, steel strip is widely used in stamping, hardware, tools, electronics and electrical engineering, metal products and cookers, agricultural machinery, automobiles, aviation, precision instruments, etc. Common steel strip defects include scratches, indentations, pinholes, peeling, etc. According to the causes of formation, they can be divided into: scratches formed by friction between the steel strip and the roller during transportation; indentations formed by the workpiece temperature being too high during the processing of the steel strip and adhering to the mold; pinholes caused by cracks during steel rolling. According to the nature of the defects, they can be divided into: defects on the surface of the steel strip, and the degradation and scrapping of the steel strip caused by them.

[0003] Existing steel strip inspection technology usually uses manual inspection and relies on the observation and experience of technicians to judge steel strip defects. During the production process, workers will observe the surface of the steel strip along the steel strip production line and identify defects on the steel strip surface with the naked eye, such as scratches, indentations, pinholes, peeling, etc. Sometimes they will also use observation tools for inspection, using well-lit lights or magnifying glasses to assist in inspection.

[0004] However, since the steel strips on the steel strip production line usually move at high speed and their state changes constantly during the movement, the traditional manual inspection method has great interference to the production line, low inspection accuracy, certain safety hazards, low efficiency, and cannot meet the inspection requirements. Summary of the invention

[0005] The present invention provides a steel strip quality detection method, device and storage medium based on machine vision, which can segment the steel strip sample image and input it into different layers of a neural network to obtain a feature image, and then use a Markov model to classify the steel strip features in the feature image and perform state transition modeling, so as to achieve accurate detection of defect types and locations.

[0006] In a first aspect, in order to solve the above technical problems, the present invention provides a steel strip quality detection method based on machine vision, comprising: Get real-time steel strip images; Input the real-time steel strip image into a Markov model to obtain an output result; wherein the Markov model is a model obtained after pre-training based on the steel strip sample image; According to the output results, the output result with the highest probability of corresponding defect type is taken as the defect determination result, and the defect determination result is sent to the operation and maintenance end so that the operation and maintenance end can alarm and solve the defect.

[0007] Preferably, the training method of the Markov model comprises: Acquire a steel strip sample image, and preprocess the steel strip sample image to obtain a denoised sample image; wherein the steel strip sample image format is RGB format; Segmenting the denoised sample image to obtain a segmented image; Inputting the segmented image into different neural network layers of a neural network model to obtain a feature image; wherein the neural network layer is composed of a convolution layer and a downsampling layer; Classifying the feature images to obtain defect types; wherein the defect types include normal, scratches, oil stains, scratches, pits and peeling; A Markov model is established using the defect type to obtain a trained Markov model.

[0008] Preferably, the step of acquiring the steel strip sample image and preprocessing the steel strip sample image to obtain a denoised sample image includes: Performing median filtering on the steel strip sample image to obtain a first processed image; Performing image binarization on the first processed image to obtain a second processed image; wherein the threshold of the image binarization is the median of the grayscale values ​​of each pixel of the steel strip sample image and the pixels in the corresponding 3*3 neighborhood; Performing connected domain analysis on the second processed image to obtain a foreground connected region, and calculating the area of ​​the foreground connected region; Based on a preset threshold, noise points whose foreground connected region area is smaller than the preset threshold are removed to obtain a denoised sample image.

[0009] Preferably, segmenting the denoised sample image to obtain a segmented image includes: Based on the segmentation algorithm of region growing, the denoised sample image is identified and extracted to obtain the steel strip region image; Segment the steel strip region image into red, green and blue channels to obtain a three-channel image; According to the three-channel image, the distance between the value of the pixel point in each channel and the maximum value and the minimum value of the channel where the pixel point is located is calculated, and the mean of the distance is calculated to obtain the mean pixel point distance; According to the pixel point distance mean values, the pixel point distance mean values ​​are sorted by size to obtain a sorting result; According to the sorting result, the channel with the largest mean value of the pixel distance is used as the segmentation channel, and the three-channel image is segmented to obtain a segmented image.

[0010] Preferably, the step of inputting the segmented image into different neural network layers of a neural network model to obtain a characteristic image of the steel strip comprises: Input the segmented image into the convolution layer, perform a convolution operation, and use an activation function to perform a nonlinear transformation on the convolution result to obtain a post-convolution feature map; A pooling operation is performed on the convolutional feature map to obtain the feature image.

[0011] Preferably, the use of the defect type to establish a Markov model to obtain a trained Markov model is specifically: The defect type is used as a state in the Markov model, and the transition probability between the defect types is calculated to obtain a state transition matrix; wherein each element Aij in the state transition matrix represents the probability of transitioning from state i to state j; The Markov model is trained to obtain a trained Markov model.

[0012] Preferably, the Markov model is trained to obtain a trained Markov model, specifically: The weights of the Markov model are updated using the gradient descent method; wherein the calculation formula of the gradient descent method is: In the formula, W represents the weight of the defect feature in the Markov model, W′ represents the updated weight, and α represents the weight update step. Indicates the actual values ​​of different defect types, represents the predicted value of the algorithm for different defect types, m represents the number of states in the Markov model, and i represents the defect type.

[0013] In a second aspect, the present invention provides a steel strip quality detection device based on machine vision, comprising: Image acquisition module, used to obtain real-time steel strip images; A defect prediction module, used for inputting the real-time steel strip image into a Markov model to obtain an output result; The defect determination module, based on the output results, takes the output result with the highest probability of corresponding defect type as the defect determination result, and sends the defect determination result to the operation and maintenance end so that the operation and maintenance end can alarm and solve the defect.

[0014] In a third aspect, the present invention further provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements any one of the above-mentioned machine vision-based steel strip quality detection methods.

[0015] In a fourth aspect, the present invention further provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned machine vision-based steel strip quality detection methods.

[0016] Compared with the prior art, the present invention has the following beneficial effects: the embodiment of the present invention provides a steel strip quality detection method, device and storage medium based on machine vision. The method includes: obtaining a real-time steel strip image; inputting the real-time steel strip image into a Markov model to obtain an output result; wherein the Markov model is a model obtained after pre-training based on a steel strip sample image; according to the output result, the output result with the highest probability of corresponding defect type is taken as the defect determination result, and the defect determination result is sent to the operation and maintenance end so that the operation and maintenance end can alarm and solve the defect.

[0017] In the present invention, the method can obtain real-time steel strip images, denoise the real-time steel strip images, segment the denoised steel strip sample images, extract feature images, and classify features through a neural network model, and then use a Markov model to classify and model state transitions of steel strip features in the feature images, and use a gradient descent method to update the weights, so that the results output by the algorithm are more accurate. The Markov model is used to output the probability distribution of defect types, and the result with the highest probability in the probability distribution is used as the defect type on the steel strip surface. The output result of this method directly reflects the defect type on the steel strip surface, and achieves accurate detection of defect types and positions. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 1 is a flow chart of a steel strip quality detection method based on machine vision provided by a first embodiment of the present invention; Figure 2 It is a structural schematic diagram of a steel strip quality detection device based on machine vision provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Reference Figure 1 The first embodiment of the present invention provides a steel strip quality detection method based on machine vision, comprising the following steps: S11, obtaining real-time steel strip image.

[0021] S12, inputting the real-time steel strip image into a Markov model to obtain an output result; wherein the Markov model is a model obtained after pre-training based on steel strip sample images.

[0022] S13, according to the output results, taking the output results with the highest probability of corresponding defect type as the defect determination result, and sending the defect determination result to the operation and maintenance end, so that the operation and maintenance end can alarm and solve the defect.

[0023] It is worth mentioning that common steel strip defects include scratches, indentations, pinholes, peeling, etc. According to the causes, they can be divided into: scratches caused by friction between the steel strip and the roller during transportation; indentations caused by the workpiece being too hot during steel strip processing and adhering to the mold; pinholes caused by cracks during steel rolling. According to the nature of the defects, they can be divided into: defects on the surface of the steel strip, and the resulting downgrading and scrapping of the steel strip.

[0024] In order to facilitate the understanding of the present invention, some preferred embodiments of the present invention are further described below.

[0025] In step S11, a real-time steel strip image is first acquired. Preferably, the method for acquiring the image may be: relying on a machine vision system, using two camera modes - a line array camera and an area array camera. Specifically, a line array camera is one of the commonly used image acquisition devices in industrial inspection, and it acquires an image by shooting a line on the surface of the steel strip each time. The characteristic of a line array camera is that it can form a complete two-dimensional image through multiple scans, and is particularly suitable for high-speed moving object detection, such as steel strips on a production line. In the present invention, a line array camera can complete real-time monitoring of the surface condition of the steel strip by continuously taking pictures. This camera is suitable for use in a high-speed production line environment, and can continuously shoot during the movement of the steel strip to ensure that every part is covered; an area array camera captures a two-dimensional image by one shot, and is suitable for use on a low-speed or medium-speed production line. Each frame of the area array camera is a complete regional image, and can be shot multiple times in a short period of time, and then the defects on the surface of the steel strip are identified by comparing multiple images. In the present invention, the area array camera shoots multiple images at a high frame rate within a certain period of time, and is suitable for a low-speed production line. The advantage of the area array camera is that it can provide a higher image resolution and identify surface defects by comparing multiple images. In addition, the available implementation methods for obtaining real-time steel strip images include: multispectral imaging, which generates images by collecting light information of different wavelengths, usually selecting different bands between visible light and infrared spectra for detection. For surface defects of steel strips, such as oil stains, scratches, oxidation, etc., multispectral imaging can provide more surface information than traditional imaging methods; infrared imaging, which is mainly used to detect temperature differences and identify defects by temperature differences between different parts of the steel strip surface. This technology is particularly suitable for steel strips just out of the furnace, and can detect surface defects caused by temperature changes; laser triangulation, a technology that performs three-dimensional morphology detection based on the principle of laser emission and reflection. It can identify surface defects such as scratches and dents by detecting height differences on the surface of the steel strip; deep learning combined with machine vision, through neural network feature extraction and classification of images, can more intelligently identify surface defects of steel strips and reduce manual intervention.

[0026] In step S12, the real-time steel strip image is input into the Markov model to obtain an output result; wherein the Markov model is a model obtained after pre-training based on the steel strip sample image. Specifically, the training method of the Markov model includes: obtaining a steel strip sample image, pre-processing the steel strip sample image, and obtaining a denoised sample image; wherein the steel strip sample image format is RGB format; segmenting the denoised sample image to obtain a segmented image; inputting the segmented image into different neural network layers of the neural network model to obtain a feature image; wherein the neural network layer is composed of a convolution layer and a downsampling layer; classifying the feature image to obtain a defect type; wherein the defect type includes normal, scratches, oil stains, scratches, pits and peeling; using the defect type to establish a Markov model to obtain a trained Markov model.

[0027] Preferably, the acquiring of the steel strip sample image and the preprocessing of the steel strip sample image to obtain the denoised sample image include: performing median filtering on the steel strip sample image to obtain a first processed image; performing image binarization on the first processed image to obtain a second processed image; wherein the threshold of the image binarization is the median of the grayscale values ​​of each pixel of the steel strip sample image and the pixels in the corresponding 3*3 neighborhood; performing connected domain analysis on the second processed image to obtain a foreground connected region, and calculating the area of ​​the foreground connected region; based on a preset threshold, removing noise points whose area of ​​the foreground connected region is smaller than the preset threshold to obtain a denoised sample image.

[0028] Specifically, median filtering is a common image processing technique, which is mainly used to remove noise in images, especially to remove isolated high-frequency interference such as salt-and-pepper noise, while retaining the edge details of the image. This method effectively reduces the noise in the image while maintaining the image details. Generally, median filtering is implemented by the following steps: first, define a window (usually 3x3, 5x5 or larger), which will move around the target pixel; at each target pixel position, extract all pixel values ​​in the window, usually including the target pixel itself and its neighboring pixels; sort these pixel values ​​from small to large; select the sorted middle value (i.e., median) to replace the original value of the target pixel; replace the value of the target pixel with the calculated median; slide the window to the next pixel position, and repeat the above steps until all pixels are processed. In the present invention, the steel strip sample image after median filtering is used as the first processed image. In the present invention, median filtering is used to perform denoising on the steel strip image to ensure the accuracy of subsequent defect recognition. By removing unnecessary noise information and retaining the key features of the steel strip surface, subsequent image segmentation and defect classification are more accurate.

[0029] Specifically, image binarization is a common image processing technology that converts the pixel values ​​in a grayscale image or a color image into a binary image with only two values ​​(usually 0 and 255, representing black and white, respectively). This conversion is achieved by setting a threshold value, and each pixel value in the image will be compared with the threshold value. Pixels below the threshold value are set to black (0), and pixels above the threshold value are set to white (255). The goal of binarization is to simplify image data and facilitate further image analysis and processing, such as object recognition, edge detection, etc. In general, image binarization is achieved by the following steps: first determine a threshold value (in the present invention, the threshold value of binarization is the median of the grayscale values ​​of each pixel of the steel strip sample image and the corresponding 3*3 neighborhood pixels) to distinguish between the foreground and the background; traverse each pixel of the image and compare its grayscale value with the threshold value; if the pixel value is greater than or equal to the threshold value, the pixel is set to 255 (white); if the pixel value is less than the threshold value, the pixel is set to 0 (black); according to the comparison result, the image is converted into a binary image. In the present invention, binarization is used to convert the pre-processed steel strip image into a binary image containing only the foreground (steel strip) and the background. Through this process, the defective part of the steel strip can be separated more clearly, laying the foundation for subsequent image segmentation, feature extraction and defect detection.

[0030] It is worth noting that in the binarization process, the setting of the threshold is the key factor in determining which parts of the image are identified as foreground (white) or background (black). In the binarization process of steel strip detection, the "median of the grayscale values ​​of each pixel and the pixels in its 3x3 neighborhood" is used as the threshold of binarization. The significance of this method is that it can better cope with the changes in illumination in different areas of the image, so that the surface defects of the steel strip can be correctly segmented even in the case of uneven illumination. At the same time, by calculating the median of the local 3x3 neighborhood, the threshold will be closer to the actual grayscale distribution of the image, thereby better retaining the edge information. The median calculation will not smooth out the edge features in the image like the mean, so it can retain important details in the image while denoising, especially those areas with small grayscale changes.

[0031] Specifically, connected component analysis (CCA) is a technique used in image processing to detect and mark adjacent pixel regions in an image. Connected component analysis is particularly commonly used in binary images, and different connected regions are determined by analyzing which pixels are connected to each other. Each connected region usually represents an independent object or part in the image, such as a character, an object outline, etc. In general, the implementation steps of connected component analysis include: First, the original grayscale image or color image needs to be converted into a binary image; starting from the upper left corner of the image, the image is scanned pixel by pixel to check the neighborhood of each pixel (4-connected or 8-connected); if a pixel is a foreground pixel (value 1) and is connected to a marked pixel, it is assigned the same label. If there are no adjacent marked pixels, a new label is created, indicating that a new connected domain has been found; during the initial scan, some independent connected domains may be found to be actually part of the same object, so these equivalent labels need to be merged after scanning to ensure that each object has only one unique label; after labeling, the properties of each connected domain can be further analyzed, such as the area involved in the present invention. In the present invention, connected domain analysis is used to process the binary steel strip image. By identifying the connected regions in the image, the detection system can determine the location and area of ​​each defect. In particular, for smaller defects, the connected domain analysis can filter out noise points by the area of ​​the connected regions, and only retain defective regions with practical significance. This can improve the accuracy of defect detection and avoid false positives or negative negatives.

[0032] Preferably, the segmenting of the denoised sample image to obtain a segmented image includes: identifying and extracting the denoised sample image based on a segmentation algorithm of region growing to obtain a steel strip region image; segmenting the steel strip region image into three channels of red, green and blue to obtain a three-channel image; according to the three-channel image, calculating the distance between the value of the pixel point in each channel and the maximum value and the minimum value of the channel where the pixel point is located, and calculating the mean of the distance to obtain the mean pixel point distance; according to the mean pixel point distance, sorting the mean pixel point distance by size to obtain a sorting result; according to the sorting result, taking the channel with the largest mean pixel point distance as the segmentation channel, segmenting the three-channel image to obtain a segmented image.

[0033] Specifically, the segmentation algorithm based on region growing is a classic image segmentation method, which aims to divide an image into regions with similar features (such as color, grayscale, texture, etc.). The region growing algorithm selects one or more seed pixels and forms regions by gradually expanding the pixels around the seeds until certain preset similarity criteria are met. This method is suitable for images with significant differences between the target object and the background, and can generate connected regions. The specific steps are: select initial seeds for certain pixels in the image; compare the similarity of the seed pixels with the surrounding pixels, usually using grayscale differences, color differences or texture differences as similarity criteria; add similar pixels to the region and continue to expand to more distant neighborhoods; stop the expansion of the region when no more adjacent pixels meet the similarity criteria; for unassigned pixels, select new seed points and repeat the above process until all pixels are divided into regions. In the present invention, the successfully segmented steel strip region image can be used for subsequent defect detection and analysis through the segmentation algorithm based on region growing.

[0034] Specifically, RGB (red, green, blue) is one of the most common color models and is widely used in display and image processing. Any color image can be regarded as composed of different combinations of three basic colors (red, green and blue), and each pixel can be represented by a triplet (R, G, B). In the present invention, the steel strip area image is segmented by RGB three channels, that is, the original image is decomposed into three separate grayscale images, each of which contains only the information of a certain color channel in the original image. In the quality inspection of steel strips, the purposes of RGB three-channel segmentation of steel strip images are as follows: independently analyze the characteristics of different color channels. After RGB three-channel segmentation, the image characteristics of the red, green and blue channels can be analyzed separately, because some surface defects of steel strips (such as scratches, oil stains, pits) may show different characteristics in different color channels; reduce information interference. The original color image contains all color information, and sometimes this information is redundant or irrelevant. By segmenting the image into three separate channels of red, green and blue, the interference caused by other colors can be eliminated, and the information of specific channels can be analyzed intensively; improve the accuracy of subsequent processing. After the image is segmented into independent channels, each channel can be processed separately. The independent processing of each channel can improve the accuracy of subsequent processing.

[0035] Specifically, after the steel strip area image is split into three channels of red, green and blue, each channel will correspond to a grayscale image. In these images, each pixel has a grayscale value between 0 and 255, which represents the intensity in the color channel. In the present invention, for each channel (red, green, blue), the maximum grayscale value and the minimum grayscale value in the channel will be determined. These maximum and minimum values ​​are used for subsequent calculations to represent the brightest and darkest pixels in the channel. For each pixel, it is necessary to calculate the distance between the grayscale value of the pixel and the maximum and minimum values ​​of the channel in which it is located, and the two distances between the pixel and the maximum and minimum values ​​are averaged. The main purpose of calculating the distance mean is to further extract the features of the steel strip image. Next, the calculated pixel distance mean values ​​are sorted by size. The significance of this step is that the sorting result directly indicates which channel is most suitable for segmentation, because the difference between the pixels in the channel and the extreme values ​​is large, which means that the channel can better distinguish between the foreground and background or defects and normal areas. According to the sorting results, the channel with the largest mean distance is selected as the segmentation channel. Different defects of the steel strip may be more obvious in different channels. For example, some scratches may be clearer in the red channel, while oil stains or other defects may be more easily detected in the green or blue channel. By selecting the channel with the largest mean distance, it can be ensured that the channel that best represents the defect information is selected for segmentation, and the defect characteristics are retained to the maximum extent. After selecting the best channel, the subsequent image segmentation processing can be more accurate, because the segmentation operation is performed on the channel where the defect is most obvious, thereby reducing false positives and false negatives.

[0036] Preferably, the step of inputting the segmented image into different neural network layers of a neural network model to obtain a feature image of the steel strip comprises: inputting the segmented image into a convolution layer, performing a convolution operation, and performing a nonlinear transformation on the convolution result using an activation function to obtain a post-convolution feature map; and performing a pooling operation on the post-convolution feature map to obtain the feature image.

[0037] Specifically, after the previous processing steps (such as the selection of segmentation channels and image segmentation), the segmented image will be input into different layers of the neural network. The neural network here is usually a convolutional neural network (CNN), which is good at processing image data, especially for extracting local features in the image. Through the neural network, the surface features of the steel strip can be automatically extracted, especially the defect features contained in the image (such as scratches, dents, oil stains, etc.). These features will be used in subsequent classification or detection steps.

[0038] In CNN, convolution operation is one of the key steps, which is used to detect local features in the image, such as edges, textures, shapes, etc. It is worth mentioning that the convolution layer contains multiple convolution kernels (also called filters), which are small matrices, usually 3x3 or 5x5 in size. They slide on the image (convolution operation), multiplying and adding a small area in the image each time to extract local features; the convolution operation is performed by performing a dot product operation on the convolution kernel and the pixels of the image. When the convolution kernel slides to each area of ​​the image, the features of the area (such as edges, texture changes, etc.) are extracted. The result of the convolution operation is a feature map, which is a representation of the local features extracted from the input image.

[0039] It is worth noting that after the convolution operation, the convolution result will pass through an activation function. This step is to introduce nonlinear characteristics so that the neural network can process more complex features. Among them, common activation functions include ReLU (Rectified Linear Unit). By introducing nonlinearity through the activation function, the neural network can learn and represent more complex features, especially detecting nonlinear features in images (such as edges, textures, shape changes, etc.).

[0040] Specifically, pooling is another important operation in neural networks. Its function is to reduce the dimension and summarize the feature map, thereby reducing the amount of data while retaining important features. In the present invention, the pooling layer takes the maximum value of a small area (such as a 2x2 area) in the feature map after convolution to represent the most significant feature of the area. The purpose of the pooling operation is to reduce the size of the feature map, thereby reducing the amount of calculation. Through maximum pooling, the pooling operation retains the most significant feature information (such as the strongest changes in edges or textures) while ignoring unimportant details. In the present invention, the feature map after the pooling operation is used as the feature image.

[0041] Preferably, the use of the defect type to establish a Markov model to obtain a trained Markov model is specifically as follows: taking the defect type as a state in the Markov model, calculating the transition probability between defect types, and obtaining a state transfer matrix; wherein each element Aij in the state transfer matrix represents the probability of transitioning from state i to state j; and training the Markov model to obtain a trained Markov model.

[0042] Specifically, the Markov model is a mathematical model used to describe random processes, and is particularly suitable for state changes in sequence data. The Markov model is based on the Markov property, that is, the future state of the system depends only on the current state, and has nothing to do with the previous state. This "memorylessness" is the core feature of the Markov model. In the present invention, the Markov model is used to detect defects on the surface of the steel strip. Specifically, it is used to predict and classify different defect types based on feature changes in the image. The steps of applying the Markov model to detect steel strip defects in the invention are: in the steel strip detection using the Markov model, each state can represent a certain defect type of the steel strip, such as normal, scratches, oil stains, etc. The state set of the Markov model includes these different defect categories; the Markov model defines the transition probability between different defect types through the state transition matrix. For example, the probability of the steel strip transferring from the "normal" state to the "scratch" state can be a specific value. Similarly, the probability of the steel strip transferring from the "scratch" state to the "dent" state is also defined in the transfer matrix; in order for the model to correctly classify defects, the system will first train the Markov model and build the transition relationship between states based on the known steel strip sample images. The purpose of the training is to determine the transition probability matrix between different defects through actual data (features extracted from the image); once the model training is completed, the Markov model can be used to predict and classify the defect types of the steel strip. By inferring the state changes of the feature image, the model can identify which defect state the current steel strip surface belongs to, and predict the subsequent possible states based on the transition probability. It is worth noting that each element Aij in the state transition matrix represents the probability of transitioning from state i to state j, which is determined in the following ways: Based on observation data update, the transition probability matrix of the Markov model will be further optimized by continuously inputting new observation data. The model updates the transition matrix by comparing the difference between the observed value and the actual transfer result using some optimization methods (such as the gradient descent method mentioned in the subsequent steps); Based on expert knowledge or domain experience, if there is a lack of sufficient data or the data is difficult to obtain, the state transition probability can also be manually set based on the experience and knowledge of domain experts. For example, based on material properties and production conditions, it may be considered that the probability of the steel strip surface transitioning from the "normal state" to the "scratched state" is higher, while the probability of transitioning to the "dented state" is lower; Based on reinforcement learning, the state transition probability is optimized through reinforcement learning algorithms (such as Q-learning or deep Q-learning). Reinforcement learning gradually learns the optimal state transition strategy and transition probability through repeated interactions with the environment.

[0043] Preferably, the training of the Markov model to obtain the trained Markov model is specifically: updating the weights of the Markov model using a gradient descent method; wherein the calculation formula of the gradient descent method is: ,W represents the weight of the defect feature in the Markov model, W′ represents the updated weight, and α represents the weight update step. Indicates the actual values ​​of different defect types, represents the predicted value of the algorithm for different defect types, m represents the number of states in the Markov model, and i represents the defect type.

[0044] Specifically, the weights of the Markov model are updated by the gradient descent method. The gradient descent method is an optimization algorithm used to minimize the value of the loss function or error function. By calculating the gradient of the error relative to the weight, the weight is gradually adjusted so that the prediction results of the model are closer and closer to the true value. With the update of the weight, the Markov model can more accurately describe the transition relationship between different states and effectively classify the defect types on the surface of the steel strip. Through this process, the final Markov model can accurately reflect the transfer law of the surface defects of the steel strip, and through continuous adjustment and optimization, the model is made more robust and adaptable to different defect characteristics.

[0045] In step S13, according to the output results, the output result with the highest probability of corresponding defect type is taken as the defect determination result, and the defect determination result is sent to the operation and maintenance end so that the operation and maintenance end can alarm and solve the defect.

[0046] Specifically, after the Markov model training is completed in step S13, the real-time steel strip image is input into the Markov model to obtain an output result. The output result is usually the prediction result of the system in different states, and the transition probability between each state. Specifically, the output result of the Markov model mainly includes the following aspects: prediction of state transition, based on the current state, the model can infer the future evolution trend of the surface defects of the steel strip, for example, how likely it is to transfer from the current state (such as normal state) to other states (such as scratches or dents); prediction of defect type, the Markov model outputs the current state of the steel strip surface according to the characteristics of the input image, combined with the state transfer matrix, that is, predicts which defect type the steel strip belongs to, and this prediction result is based on the combination of the current state and the transition probability. In general, the model will output the probabilities of different defect types, and judge which defect state the current steel strip surface is in based on these probabilities. For example, the model will calculate the probability that the steel strip surface belongs to defects such as "scratches" or "dents". Finally, the model will take the defect type with the highest probability in the output result as the final judgment result, and use the judgment result to alarm and process. The output result is used for real-time monitoring and alarming, and does not directly output each element Aij in the state transfer matrix. Instead, the probability distribution of defects is inferred through the transfer matrix to make defect judgments.

[0047] The following describes the working process of the present invention by taking a relatively common scenario as an example. The working process is as follows: In the steel strip production line, the machine vision system is used to capture the surface image of the steel strip through a line scan camera or an area scan camera. The line scan camera forms a complete two-dimensional image through continuous shooting and is suitable for high-speed production lines; the area scan camera captures the regional image of the steel strip surface on low-speed or medium-speed production lines.

[0048] After the steel strip image is input into the system, it is first preprocessed: median filtering is performed to remove noise in the image, especially high-frequency noise, while maintaining the edge features of the image; image binarization is performed to convert the steel strip image into a binary image to distinguish the foreground (steel strip) and background; connected domain analysis is performed to detect defective areas on the surface of the steel strip by analyzing adjacent pixels, and noise points are removed.

[0049] The preprocessed image is segmented using the region growing algorithm to extract the steel strip region image. Then, the system performs RGB three-channel segmentation on the image and calculates the distance between the pixel points in each channel and the maximum and minimum values, and selects the channel that best represents the defect information for segmentation.

[0050] The segmented image is input into a convolutional neural network (CNN), where the convolution layer extracts features such as the edge and texture of the steel strip surface, the activation function introduces nonlinear transformation, and the pooling layer further reduces the size of the feature map.

[0051] The characteristic images are classified to obtain defect types; wherein the defect types include normal, scratches, oil stains, scratches, pits and peeling.

[0052] The Markov model maps the information in the feature image to different states. Each state represents a possible state of the steel strip surface, such as "normal", "scratches", "oil stains" and other defect types. The model uses the state transition matrix to calculate the transition probability from the current state to other defect states. By comparing the transition probabilities of different states, the model will select the state with the highest probability as the final defect type output. After the judgment is completed, the system will take corresponding measures according to the output defect type, such as triggering an alarm mechanism to notify the operator, or recording defect information for subsequent quality control.

[0053] In summary, the present invention provides an operation control method of an inverter, comprising: acquiring a real-time steel strip image; inputting the real-time steel strip image into a Markov model to obtain an output result; wherein the Markov model is a model obtained after pre-training based on a steel strip sample image; based on the output result, taking the output result with the highest probability of corresponding defect type as a defect judgment result, and sending the defect judgment result to an operation and maintenance end, so that the operation and maintenance end can alarm and solve the defect.

[0054] In the present invention, the method can obtain real-time steel strip images, denoise the real-time steel strip images, segment the denoised steel strip sample images, extract feature images, and classify features through a neural network model, and then use a Markov model to classify and model state transitions of steel strip features in the feature images, and use a gradient descent method to update the weights, so that the results output by the algorithm are more accurate. The Markov model is used to output the probability distribution of defect types, and the result with the highest probability in the probability distribution is used as the defect type on the steel strip surface. The output result of this method directly reflects the defect type on the steel strip surface, and achieves accurate detection of defect types and positions.

[0055] Reference Figure 2 The second embodiment of the present invention provides a steel strip quality detection device based on machine vision, comprising: Image acquisition module, used to obtain real-time steel strip images; A defect prediction module, used for inputting the real-time steel strip image into a Markov model to obtain an output result; The defect determination module, based on the output results, takes the output result with the highest probability of corresponding defect type as the defect determination result, and sends the defect determination result to the operation and maintenance end so that the operation and maintenance end can alarm and solve the defect.

[0056] In an optional embodiment, the defect prediction module is specifically used to: obtain a steel strip sample image, pre-process the steel strip sample image, and obtain a denoised sample image; wherein the steel strip sample image format is RGB format; Segmenting the denoised sample image to obtain a segmented image; Inputting the segmented image into different neural network layers of a neural network model to obtain a feature image; wherein the neural network layer is composed of a convolution layer and a downsampling layer; Classifying the feature images to obtain defect types; wherein the defect types include normal, scratches, oil stains, scratches, pits and peeling; A Markov model is established using the defect type to obtain a trained Markov model.

[0057] In an optional implementation, the defect prediction module is specifically used to: Performing median filtering on the steel strip sample image to obtain a first processed image; Performing image binarization on the first processed image to obtain a second processed image; wherein the threshold of the image binarization is the median of the grayscale values ​​of each pixel of the steel strip sample image and the pixels in the corresponding 3*3 neighborhood; Performing connected domain analysis on the second processed image to obtain a foreground connected region, and calculating the area of ​​the foreground connected region; Based on a preset threshold, noise points whose foreground connected region area is smaller than the preset threshold are removed to obtain a denoised sample image.

[0058] In an optional implementation, the defect prediction module is specifically used to: Based on the segmentation algorithm of region growing, the denoised sample image is identified and extracted to obtain the steel strip region image; Segment the steel strip region image into red, green and blue channels to obtain a three-channel image; According to the three-channel image, the distance between the value of the pixel point in each channel and the maximum value and the minimum value of the channel where the pixel point is located is calculated, and the mean of the distance is calculated to obtain the mean pixel point distance; According to the pixel point distance mean values, the pixel point distance mean values ​​are sorted by size to obtain a sorting result; According to the sorting result, the channel with the largest mean value of the pixel distance is used as the segmentation channel, and the three-channel image is segmented to obtain a segmented image.

[0059] In an optional implementation, the defect prediction module is specifically used to: Input the segmented image into the convolution layer, perform a convolution operation, and use an activation function to perform a nonlinear transformation on the convolution result to obtain a post-convolution feature map; A pooling operation is performed on the convolutional feature map to obtain the feature image.

[0060] In an optional implementation, the defect prediction module is specifically used to: The defect type is used as a state in the Markov model, and the transition probability between the defect types is calculated to obtain a state transition matrix; wherein each element Aij in the state transition matrix represents the probability of transitioning from state i to state j; The Markov model is trained to obtain a trained Markov model.

[0061] In an optional implementation, the defect prediction module is specifically used to: The weights of the Markov model are updated using the gradient descent method; wherein the calculation formula of the gradient descent method is: Where W represents the weight of the defect feature in the Markov model, W′ represents the updated weight, α represents the weight update step, yi represents the actual value of different defect types, xi represents the predicted value of the algorithm for different defect types, m represents the number of states in the Markov model, and i represents the defect type.

[0062] It should be noted that a steel strip quality inspection device based on machine vision provided in an embodiment of the present invention is used to execute all process steps of a steel strip quality inspection method based on machine vision in the above embodiment. The working principles and beneficial effects of the two correspond one to one, and thus will not be elaborated on.

[0063] The embodiment of the present invention further provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a steel strip quality detection method program based on machine vision. When the processor executes the computer program, the steps in the above-mentioned steel strip quality detection method embodiments based on machine vision are implemented, such as Figure 1 Alternatively, when the processor executes the computer program, the functions of the modules / units in the above-mentioned device embodiments are implemented, such as the defect prediction module.

[0064] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the electronic device.

[0065] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, and a smart tablet. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. The electronic device may include more or fewer components than the above components, or may combine certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0066] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device.

[0067] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the electronic device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0068] Wherein, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0069] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0070] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A steel strip quality detection method based on machine vision, characterized in that: include: Get real-time steel strip images; Input the real-time steel strip image into a Markov model to obtain an output result; wherein the Markov model is a model obtained after pre-training based on the steel strip sample image; According to the output results, the output result with the highest probability of corresponding defect type is taken as the defect determination result, and the defect determination result is sent to the operation and maintenance end so that the operation and maintenance end can alarm and solve the defect.

2. The steel strip quality detection method based on machine vision according to claim 1 is characterized in that: The training process of the Markov model includes: Acquire a steel strip sample image, and preprocess the steel strip sample image to obtain a denoised sample image; wherein the steel strip sample image format is RGB format; Segmenting the denoised sample image to obtain a segmented image; Inputting the segmented image into different neural network layers of a neural network model to obtain a feature image; wherein the neural network layer is composed of a convolution layer and a downsampling layer; Classifying the feature images to obtain defect types; wherein the defect types include normal, scratches, oil stains, scratches, pits and peeling; A Markov model is established using the defect type to obtain a trained Markov model.

3. The steel strip quality detection method based on machine vision according to claim 2 is characterized in that: The step of acquiring a steel strip sample image and preprocessing the steel strip sample image to obtain a denoised sample image includes: Performing median filtering on the steel strip sample image to obtain a first processed image; Performing image binarization on the first processed image to obtain a second processed image; wherein the threshold of the image binarization is the median of the grayscale values ​​of each pixel of the steel strip sample image and the pixels in the corresponding 3*3 neighborhood; Performing a connected domain analysis on the second processed image to obtain a foreground connected region, and calculating the area of ​​the foreground connected region; Based on a preset threshold, noise points whose foreground connected region area is smaller than the preset threshold are removed to obtain a denoised sample image.

4. The steel strip quality detection method based on machine vision according to claim 2 is characterized in that: The step of segmenting the denoised sample image to obtain a segmented image includes: Based on the segmentation algorithm of region growing, the denoised sample image is identified and extracted to obtain the steel strip region image; Segment the steel strip region image into red, green and blue channels to obtain a three-channel image; According to the three-channel image, the distance between the value of the pixel point in each channel and the maximum value and the minimum value of the channel where the pixel point is located is calculated, and the mean of the distance is calculated to obtain the mean pixel point distance; According to the pixel point distance mean values, the pixel point distance mean values ​​are sorted by size to obtain a sorting result; According to the sorting result, the channel with the largest mean value of the pixel distance is used as the segmentation channel, and the three-channel image is segmented to obtain a segmented image.

5. The steel strip quality detection method based on machine vision according to claim 2 is characterized in that: The step of inputting the segmented image into different neural network layers of a neural network model to obtain a characteristic image of the steel strip comprises: Input the segmented image into the convolution layer, perform a convolution operation, and use an activation function to perform a nonlinear transformation on the convolution result to obtain a post-convolution feature map; A pooling operation is performed on the convolutional feature map to obtain the feature image.

6. The method for detecting steel strip quality based on machine vision according to claim 2, characterized in that: The Markov model is established by using the defect type to obtain a trained Markov model, specifically: The defect type is used as a state in the Markov model, and the transition probability between defect types is calculated to obtain a state transition matrix; wherein each element Aij in the state transition matrix represents the probability of transitioning from state i to state j; The Markov model is trained to obtain a trained Markov model.

7. The steel strip quality detection method based on machine vision according to claim 2 is characterized in that: The Markov model is trained to obtain a trained Markov model, specifically: The weights of the Markov model are updated using the gradient descent method; wherein the calculation formula of the gradient descent method is: In the formula, W represents the weight of the defect feature in the Markov model, W′ represents the updated weight, and α represents the weight update step. Indicates the actual values ​​of different defect types, represents the predicted value of the algorithm for different defect types, m represents the number of states in the Markov model, and i represents the defect type.

8. A steel strip quality detection device based on machine vision, characterized in that: include: Image acquisition module, used to obtain real-time steel strip images; A defect prediction module, used for inputting the real-time steel strip image into a Markov model to obtain an output result; The defect determination module, based on the output results, takes the output result with the highest probability of corresponding defect type as the defect determination result, and sends the defect determination result to the operation and maintenance end so that the operation and maintenance end can alarm and solve the defect.

9. An electronic device, characterized in that: It comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the machine vision-based steel strip quality detection method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the machine vision-based steel strip quality detection method according to any one of claims 1 to 7.

Images