A rust detection method, a detection device and a storage medium

Abstract

The embodiment of the application discloses a rust detection method, a detection device and a storage medium, and is used in the field of image processing. The method comprises the following steps: acquiring a to-be-detected image, and extracting a feature vector of each pixel point in the to-be-detected image; inputting the feature vector into a pre-trained mixed Gaussian model to obtain a first rust probability of each pixel point in the to-be-detected image; inputting the to-be-detected image into a preset clustering algorithm, clustering pixel points with similar color features in the to-be-detected image into superpixels based on the preset clustering algorithm, and acquiring a second rust probability of each superpixel in the to-be-detected image; obtaining a target threshold of the to-be-detected image based on a preset threshold algorithm and the first rust probability; and determining whether the superpixel is a rust area according to the second rust probability of the superpixel and the target threshold. By clustering pixel points with similar color features into superpixels and determining whether the superpixel is a rust area, the rust area can be accurately determined in the to-be-detected image.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, in particular to a rust detection method, a detection device and a storage medium. BACKGROUND

[0002] Steel materials are widely used in infrastructure structures such as power line towers and bridges due to their good supportability. However, steel is prone to react with active oxidants such as oxygen to produce rust, which shortens the service life of the infrastructure and may lead to disastrous consequences. Power line towers are all erected in the wild and are extremely susceptible to external environmental influences to cause a decline in material performance. If the tower bolts are eroded by rainwater with high salt content for a long time, the bolts will gradually be sanded, and rainwater can seep along the joint between the protective cap and the tower material, which is easy to cause bolt rust. If the rust continues to intensify, the bolts will break through the critical limit of the force of the tower under the influence of external forces such as continuous strong winds and icing, causing accidents such as tower tilting, tower collapse and broken lines, which seriously affect the safe and reliable operation of the power transmission line.

[0003] Existing rust detection often relies on manual detection, which is prone to judgment errors and is difficult to provide objective and fair feedback for subsequent detection and maintenance. The current mainstream rust detection methods include digital image processing, traditional machine learning and deep learning. Digital image processing and traditional machine learning methods are easy to implement and have small computational complexity, and rust features are applied for detection to label the rust area. However, rusts of different degrees have obvious feature differences, and it is difficult to use the former two methods to achieve robust feature representation. At the same time, the boundary of the rust area is close to the non-rust area, and it is difficult to accurately demarcate, and a large amount of training data is required to produce a high-quality data set to train a deep learning model, and it is difficult to label an accurate rust area. SUMMARY

[0004] Embodiments of the present application provide a rust detection method, a detection device and a storage medium, which can accurately determine the rust area in the to-be-detected image.

[0005] Embodiments of the present application provide a rust detection method, which comprises:

[0006] An to-be-detected image is acquired, and a feature vector of each pixel point in the to-be-detected image is extracted;

[0007] The feature vector is input into a pre-trained mixed Gaussian model to obtain a first rust probability of each pixel point in the to-be-detected image;

[0008] The to-be-detected image is input into a preset clustering algorithm, and pixel points with similar color features in the to-be-detected image are clustered into superpixels based on the preset clustering algorithm, and a second rust probability of each superpixel in the to-be-detected image is acquired;

[0009] obtaining a target threshold of the to-be-tested image based on a preset threshold algorithm and the first corrosion probability;

[0010] determining whether the superpixel is a corrosion region according to a second corrosion probability of the superpixel and the target threshold.

[0011] Further, the extracting the feature vector of each pixel point of the to-be-tested image comprises:

[0012] converting the to-be-tested image from an RGB color space to an HSV color space;

[0013] transforming a hue value of a target pixel point in the to-be-tested image according to a formula: H(i,j) = (H(i,j) + 180) mod 360, wherein i and j represent coordinate positions of the target pixel point in the to-be-tested image;

[0014] performing normalization processing on the transformed hue value, a saturation value and a lightness value corresponding to the target pixel point to obtain the feature vector of the target pixel point.

[0015] Further, before the inputting the feature vector into the pre-trained mixed Gaussian model, the method further comprises:

[0016] obtaining a corrosion image training set and an initial mixed Gaussian model, and extracting a first feature vector of a pixel point in the corrosion image training set;

[0017] inputting the first feature vector into the initial mixed Gaussian model, and outputting a third corrosion probability by the initial mixed Gaussian model;

[0018] constructing a loss value according to the third corrosion probability and a preset corrosion probability;

[0019] adjusting a mean value and a covariance of the initial mixed Gaussian model based on the loss value until the initial mixed Gaussian model converges, so as to obtain a trained mixed Gaussian model.

[0020] Further, the inputting the feature vector into the pre-trained mixed Gaussian model to obtain the first corrosion probability of each pixel point in the to-be-tested image comprises:

[0021] inputting the feature vector into the pre-trained mixed Gaussian model:

[0022]

[0023]

[0024] wherein x i represents the i-th feature vector, μ kdenotes a mean value of the kth Gaussian model in the pre-trained Gaussian mixture model, Σ k and w k denote a covariance and a weight of the kth Gaussian model respectively, z i is a latent variable, z i = 1, 2, 3, …, p, p Gaussian models are provided in the pre-trained Gaussian mixture model, and d is a dimension of a feature vector, denotes an initial rust probability of the ith feature vector, P denotes a rust probability of the ith feature vector;

[0025] The pre-trained Gaussian mixture model outputs p rust probabilities corresponding to the ith feature vector, and the maximum value in the p rust probabilities is selected as a first rust probability of a pixel point corresponding to the ith feature vector.

[0026] Further, the preset clustering algorithm includes a superpixel segmentation algorithm, and the clustering of the pixel points with similar color features in the to-be-tested image into superpixels based on the preset clustering algorithm includes:

[0027] According to a total number of the superpixels in the to-be-tested image, an initial clustering center of each superpixel is determined;

[0028] According to a formula: G(x, y) = ||L(x+1, y)-L(x-1, y)| 2 + ||L(x, y+1)-L(x, y-1)| 2 , a gradient value of a pixel point in a neighborhood superpixel corresponding to the current superpixel is obtained, wherein G(x, y) is the gradient value of the pixel point, L(x, y) is a lab vector of the pixel point located at a coordinate (x, y) of the to-be-tested image, wherein l represents color brightness, a and b represent color opposite dimensions, and ||·|| is a Euclidean norm;

[0029] The initial clustering center in the target neighborhood superpixel is moved to a position corresponding to a pixel point with the minimum gradient value in the target neighborhood superpixel as a clustering center of the target neighborhood superpixel;

[0030] Through the superpixel segmentation algorithm:

[0031]

[0032] wherein d lab is a color distance of a pixel point to a clustering center, l i and l j represent values of the l in a rotating coordinate system, a i and a j represent values of the a in the rotating coordinate system, and bi and b j denotes the value in the b rotating coordinate system, d xy is the spatial distance from the pixel point to the cluster center, x i and x j , y i and y j denotes the value in the rotating coordinate system of the coordinates (x, y), m is a variable for adjusting the compactness of the superpixel, and S is a grid interval;

[0033] For the cluster center of each superpixel in the to-be-tested image, the distance from the pixel point in the preset range to the cluster center is measured, and the cluster center with the minimum distance is taken as the target cluster center of the superpixel.

[0034] Further, the preset threshold algorithm includes an adaptive threshold segmentation algorithm, and the target threshold of the to-be-tested image is obtained based on the adaptive threshold segmentation algorithm and the first rust probability, including:

[0035] The rust grayscale image corresponding to the to-be-tested image is generated based on the first rust probability;

[0036] The rust grayscale image is input into the adaptive threshold segmentation algorithm to obtain a binary threshold value corresponding to the to-be-tested image, and the binary threshold value is taken as the target threshold of the to-be-tested image;

[0037] The adaptive threshold segmentation algorithm includes:

[0038]

[0039]

[0040] wherein k * is a binary threshold value, is the maximum inter-class variance, k is the level of the threshold value, L is the gray level of the rust grayscale image, P0(k) and P1(k) are the cumulative probabilities of the k, μ0(k) and μ1(k) are the means of the k, and μ is the mean of the Gaussian model.

[0041] Further, according to the second rust probability of the superpixel and the target threshold, it is determined whether the superpixel is a rust area, including:

[0042] If the second rust probability of the superpixel is greater than the target threshold, it is determined that the superpixel is a rust area;

[0043] If the second rust probability of the superpixel is less than the target threshold, it is determined that the superpixel is a non-rust area.

[0044] The embodiment of the present application further provides a rust detection device, comprising:

[0045] An acquisition unit is configured to acquire a to-be-tested image and extract a feature vector of each pixel point in the to-be-tested image;

[0046] An input unit is configured to input the feature vector into a pre-trained mixed Gaussian model to obtain a first rust probability of each pixel point in the to-be-tested image;

[0047] A clustering unit is configured to input the to-be-tested image into a preset clustering algorithm, cluster pixel points with similar color features in the to-be-tested image into superpixels based on the preset clustering algorithm, and acquire a second rust probability of each superpixel in the to-be-tested image;

[0048] An execution unit is configured to obtain a target threshold of the to-be-tested image based on a preset threshold algorithm and the first rust probability;

[0049] A determination unit is configured to determine whether the superpixel is a rust area according to the second rust probability of the superpixel and the target threshold.

[0050] The embodiment of the present application further provides a rust detection device, comprising:

[0051] A central processing unit, a memory and an input-output interface;

[0052] The memory is a transitory storage memory or a persistent storage memory;

[0053] The central processing unit is configured to communicate with the memory and execute instruction operations in the memory to perform the above method.

[0054] The embodiment of the present application further provides a computer readable storage medium, comprising instructions, when the instructions run on a computer, the computer executes the above method.

[0055] From the above technical solutions, the embodiment of the present application has the following advantages:

[0056] The method of the embodiment of the present application comprises: acquiring a to-be-tested image, and extracting a feature vector of each pixel point in the to-be-tested image; inputting the feature vector into a pre-trained mixed Gaussian model to obtain a first rust probability of each pixel point in the to-be-tested image; inputting the to-be-tested image into a preset clustering algorithm, clustering pixel points with similar color features in the to-be-tested image into superpixels based on the preset clustering algorithm, and acquiring a second rust probability of each superpixel in the to-be-tested image; obtaining a target threshold of the to-be-tested image based on a preset threshold algorithm and the first rust probability; and determining whether the superpixel is a rust area according to the second rust probability of the superpixel and the target threshold. By clustering pixel points with similar color features into superpixels, and determining whether the superpixel is a rust area according to the second rust probability of the superpixel and the target threshold, it can be determined that the superpixel is a rust area accurately, and the rust area can be accurately determined in the to-be-tested image. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art according to these drawings.

[0058] Figure 1 A rust detection flowchart disclosed by the embodiment of the present application;

[0059] Figure 2 Another rust detection flowchart disclosed by the embodiment of the present application;

[0060] Figure 3 A mixed Gaussian model flowchart disclosed by the embodiment of the present application;

[0061] Figure 4 A superpixel segmentation and threshold segmentation flowchart disclosed by the embodiment of the present application;

[0062] Figure 5 An effect diagram of rust detection disclosed by the embodiment of the present application;

[0063] Figure 6 A rust detection device diagram disclosed by the embodiment of the present application;

[0064] Figure 7 Another rust detection device diagram disclosed by the embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to make the purposes, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the scope of protection of the present application.

[0066] In the description of the embodiments of the present application, it should be noted that the orientations or positional relationships indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the embodiments of the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the embodiments of the present application.

[0067] In the description of the embodiments of the present application, it should be noted that, unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connection" should be understood broadly, for example, can be fixed connection, can also be detachable connection, or integrally connected; can be mechanical connection, can also be electrical connection; can be directly connected, can also be indirectly connected through an intermediate medium, can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0068] The existing rust detection often relies on manual detection, which is prone to judgment errors and difficult to provide objective and fair feedback for subsequent detection and maintenance. The current mainstream rust detection methods include digital image processing, traditional machine learning and deep learning. Digital image processing and traditional machine learning method is easy to realize and has small amount of calculation, and rust features are applied for detection, and rust area is labeled, but rust of different degrees has obvious feature difference, and it is difficult to realize robust feature representation by the former two. At the same time, the boundary of the rust area is close to the non-rust area, which is difficult to accurately demarcate, and a large amount of training data is required to produce high-quality data set to train deep learning model. The existing rust detection technology has the problems of non-robust rust feature extraction and large amount of training data, and it is difficult to label accurate rust area. Therefore, the embodiment of the present application provides a rust detection method, which can accurately determine the rust area in the to-be-detected image, as shown in Figure 1 The specific steps are as follows:

[0069] 101, acquiring a to-be-detected image, and extracting a feature vector of each pixel point of the to-be-detected image.

[0070] In this embodiment, the corrosion detection device can first acquire a test image from which the corrosion area needs to be identified or segmented. This test image can be manually input into the detection device or acquired via a network connection; the specific method is not limited here. The detection device can extract the feature vector of each pixel in the test image. Specifically, the hue, saturation, and brightness values ​​in the HSV color space can be used as the main feature vector for corrosion probability prediction. Alternatively, during the extraction and conversion of the feature vector, the pixel values ​​in the RGB color space can be extracted, or some or all of the values ​​in the RGB color space and the HSV color space can be combined through linear combination and function mapping to form different features, thereby forming a feature vector; the specific method is not limited here. Converting the test image from the RGB color space to the HSV color space and extracting the features of the hue, saturation, and brightness channels as the pixel feature vectors better aligns with the human eye's information perception mechanism.

[0071] The specific process of extracting and transforming feature vectors from pixels in the HSV color space is as follows: The image to be tested is converted from the RGB color space to the HSV color space. This can be done using OpenCV or other digital image processing tools; the specific conversion is not limited here. Next, the hue values ​​of the target pixels in the HSV color space are extracted individually (the hue value range is 0 to 360), and the hue values ​​of the target pixels in the image are transformed according to the formula: (H(i,j)=(H(i,j)+180)mod 360), where i and j represent the coordinates of the target pixel in the image. After the transformation, the hue values ​​of the rust spots (corrosion) will be relatively concentrated and continuous. The transformed hue, saturation, and brightness values ​​of the target pixel are normalized to obtain the feature vector of the target pixel. At this point, a three-dimensional vector represents a pixel.

[0072] 102. Input the feature vector into the pre-trained Gaussian mixture model to obtain the first corrosion probability of each pixel in the image to be tested.

[0073] After extracting the feature vectors of each pixel, these feature vectors can be input into a pre-trained Gaussian mixture model (Gaussian model of mixture) to obtain the first corrosion probability of each pixel in the image under test. This means that the pre-trained Gaussian mixture model is used to predict the corrosion probability at the pixel level. Specifically, this pre-trained Gaussian mixture model can be trained from manually collected and processed corrosion (rust spots) images after feature vector transformation. Figure 3As shown, a rust image training set and an initial Gaussian mixture model are obtained, the rust image training set contains image training data of a rust area, and the rust area is cut out from the image training data; then, a first feature vector of a pixel point in the rust image training set is extracted, that is, the pixel point corresponding to the rust area is converted into the first feature vector; the first feature vector is input into the initial Gaussian mixture model (GMM), and the initial Gaussian mixture model outputs a third rust probability; a loss value is constructed according to the third rust probability and a preset rust probability, wherein the preset rust probability can be determined according to the rust area input into the initial Gaussian mixture model; the mean and the covariance of the initial Gaussian mixture model are adjusted based on the loss value until the initial Gaussian mixture model converges, and then a trained Gaussian mixture model is obtained. It can be understood that the convergence means that the difference between the third rust probability and the preset rust probability is small, and the difference is within a certain range.

[0074] After the Gaussian mixture model is pre-trained, in the probability prediction stage, the feature vector of each pixel point in the extracted to-be-detected image can be predicted by the pre-trained Gaussian mixture model to obtain a first rust probability of each pixel point in the to-be-detected image. The Gaussian mixture model includes a plurality of Gaussian models, each Gaussian model represents a rust type, and there are c Gaussian models in total. According to the size of the weight of the Gaussian model, a Gaussian model with a weight greater than c / 1 can be selected, such as p Gaussian models. The feature vector of the pixel point is input into the pre-trained Gaussian mixture model, and the probability that the feature vector belongs to each rust type can be calculated as the probability that the pixel point belongs to each rust type, that is, p rust probabilities can be calculated for each pixel point. The calculation formula is:

[0075]

[0076]

[0077] wherein x i represents the i-th feature vector, μ k represents the mean of the k-th Gaussian model in the pre-trained Gaussian mixture model, Σ k and w k represent the covariance and the weight of the k-th Gaussian model, respectively, z i is a hidden variable, z i = 1, 2, 3, …, p, and there are p Gaussian models in the pre-trained Gaussian mixture model, and d is the dimension of the feature vector. The first rust probability of the i-th feature vector is represented, and P represents the rust probability of the i-th feature vector. The pre-trained Gaussian mixture model outputs p rust probabilities corresponding to the i-th feature vector, and the maximum value of the p rust probabilities is selected as the first rust probability of the pixel point corresponding to the i-th feature vector. Then, a two-dimensional rust probability matrix about the to-be-detected image can be obtained. Then, the rust probability of each pixel point can be displayed by using a heat map or a gray scale map, wherein the rust probability of the pixel point in the heat map is greater, and the color of the pixel point is deeper, and the rust probability of the pixel point in the gray scale map is greater, and the gray value of the pixel point is greater.

[0078] It can be seen that in the embodiments of the present application, the pre-trained Gaussian mixture model is constructed to perform pixel-level rust probability prediction on the to-be-detected image, and to provide a quantifiable numerical basis for subsequent rust region segmentation. The use of the Gaussian mixture model to perform pixel-level rust probability prediction on the to-be-detected image based on the feature vectors extracted in the HSV color space has better robustness for rust of different scenes and different degrees.

[0079] 103. Cluster the pixel points with similar color features in the to-be-detected image into superpixels based on a preset clustering algorithm, and obtain a second rust probability of each superpixel in the to-be-detected image.

[0080] In order to introduce the correlation of adjacent rust pixel points, improve the consistency of rust segmentation, and improve the effect of rust detection, in the embodiments of the present application, superpixel segmentation is performed on the to-be-detected image in the RGB color space, that is, pixel points with similar color features in the to-be-detected image are clustered into superpixels based on a preset clustering algorithm. The preset clustering algorithm can be a K-means clustering algorithm or a SLIC superpixel segmentation algorithm, and the specific implementation is not limited here. It should be noted that the execution sequence of steps 102 and 103 is not limited here.

[0081] As Figure 4As shown, the superpixel segmentation algorithm is used on the to-be-tested image, and the superpixel segmentation result is projected onto the heat map output by the Gaussian mixture model. The superpixel segmentation algorithm can subdivide the to-be-tested image into a plurality of image sub-regions (a set of pixels, superpixels), and each superpixel is composed of a series of small regions of adjacent positions and similar features such as color, brightness, and texture. These small regions retain effective information for further image segmentation and generally do not damage the boundary information of objects in the image. Superpixel segmentation helps to introduce the correlation of adjacent pixels, increase the consistency of segmentation, and improve the consistency of corrosion segmentation. Specifically, the superpixel segmentation algorithm takes the original image (the to-be-tested image in the RGB color space) as input, clusters the superpixels according to the color similarity of the pixels in the image plane, and outputs the superpixels. Specifically, the cluster centers of the superpixels are initialized, the initial cluster center of each superpixel is determined according to the total number of superpixels in the to-be-tested image, and the cluster centers in the image can be distributed at regular grid intervals S; then, the cluster centers are reselected. In order to reselect the cluster centers in the n*n neighborhood (usually n=3), the gradient values of all the pixels in the neighborhood need to be calculated, and the cluster center is moved to the position with the smallest gradient in the neighborhood. Specifically, according to the formula: 2 2 , the gradient value of the pixel in the neighborhood superpixel corresponding to the current superpixel is obtained, where G(x,y) is the gradient value of the pixel, L(x,y) is the lab vector of the pixel at the coordinate (x,y) of the to-be-tested image, l represents color brightness, a and b represent color opposite dimensions, and ||·|| is the Euclidean norm; the initial cluster center in the target neighborhood superpixel is moved to the position corresponding to the pixel with the smallest gradient value in the target neighborhood superpixel as the cluster center of the target neighborhood superpixel; then, the class label of the pixel is assigned, and a class label (indicating the class to which it belongs) is assigned to each pixel in the neighborhood of each cluster center, where the search range is 2S*2S; distance measurement is performed: for each cluster center, the distance between each searched pixel and it is measured, including color distance and spatial distance. Specifically, through the superpixel segmentation algorithm:

[0082] where d lab is the color distance of the pixel to the cluster center, l i and l j represent the values of l in the rotated coordinate system, a i and a j represent the values of a in the rotated coordinate system, and b i and b j ​represents the value in the b-rotated coordinate system, d xy is the spatial distance from the pixel point to the cluster center, x i and x j , y i and y j represents the value in the coordinate (x, y) rotated coordinate system, m is a variable for adjusting the compactness of the superpixel, S is the grid interval, D S is the distance from the pixel point in the preset range to the cluster center;

[0083] For each superpixel in the test image, the distance from the pixel point in the preset range to the cluster center is measured. Each pixel point will be searched by multiple cluster centers, so each pixel point will obtain multiple distances, and the cluster center with the smallest distance is taken as the target cluster center of the superpixel. Finally, iteration is repeated until the residual error (the distance from the cluster center of the previous iteration to the newly calculated cluster center) converges. After the pixel points of the test image are clustered into superpixels, the rust probability of the superpixel can also be obtained. Specifically, the first rust probability of the pixel points contained in the superpixel can be averaged or taken as the mode to obtain the second rust probability of the superpixel, which is not limited here.

[0084] It can be understood that the SLIC superpixel segmentation clusters the pixel points with similar color features into superpixels, which helps to better identify and refine the edges of the detected rust area. For some rust spot areas with low rust probability due to sunlight reflection, SLIC will cluster them into rust spot superpixels, which can be correctly classified. In this way, some small areas of the GMM model determined as non-rust areas can be corrected as rust areas, and vice versa. Considering the aggregation property of rust, the superpixel segmentation algorithm is introduced, thereby to a certain extent, reducing the "rust spots" caused by the prediction error of the pixel point rust probability and the misjudgment of the rust segmentation. The superpixel segmentation introduces the correlation of the boundary pixels, increases the consistency of the segmentation, and improves the effect of the rust segmentation.

[0085] 104. Obtain the target threshold value of the test image based on the preset threshold algorithm and the first rust probability.

[0086] In order to make the robustness and compatibility of the algorithm stronger, and adapt to rust detection in multiple scenes, a preset threshold algorithm can be used for binary segmentation. Based on the preset threshold value and the first rust probability, the target threshold value of the test image is obtained. The preset threshold algorithm can be an adaptive threshold segmentation algorithm or other threshold segmentation algorithm, which is not limited here. For example Figure 4As shown, the adaptive threshold segmentation algorithm is used to calculate the threshold value suitable for the test image on the gray image (probability of rust) output by the model, and the threshold value is projected to [0, 1]. Among them, the adaptive threshold segmentation algorithm can automatically determine the binary segmentation threshold of the image, and is not affected by the brightness and contrast of the image. Compared with the fixed threshold set manually, it has better robustness and generalization. The adaptive threshold segmentation algorithm includes: OTSU algorithm, which is an image segmentation algorithm that divides a gray image into a binary image and returns a globally adaptive binary threshold. The value of the OTSU threshold is not manually selected, but automatically determined by the algorithm, and has adaptive ability for each specific image. This algorithm uses the maximum inter-class variance to distinguish the foreground and background of the image. Therefore, based on the adaptive threshold segmentation algorithm and the first rust probability, the target threshold of the test image is obtained, including: generating a rust gray image corresponding to the test image based on the first rust probability; wherein the rust gray image is that the larger the first rust probability, the larger the gray value, the smaller the first rust probability, the smaller the gray value, and some pixel points in the test image with a first rust probability of 0 have a corresponding gray value of 0. The rust gray image is input into the adaptive threshold segmentation algorithm to obtain a binary threshold value corresponding to the test image, and the binary threshold value is taken as the target threshold of the test image; wherein the adaptive threshold segmentation algorithm includes:

[0087]

[0088]

[0089] wherein k * is the binary threshold value (optimal threshold) selected by maximizing the inter-class variance, is the maximum inter-class variance, k is the level of the threshold value, L is the gray level of the rust gray image, P0(k) and P1(k) are the cumulative probabilities of k, μ0(k) and μ1(k) are the means of k, and μ is the mean of the Gaussian model. The obtained target threshold is projected to [0, 1].

[0090] It can be seen that the adaptive threshold segmentation algorithm extracts a threshold value with good segmentation effect on the output gray image of the GMM model, which is mapped to [0, 1] and used as the segmentation threshold of the heat map after superpixel segmentation. The threshold value of each test image can be automatically found without manually setting the same threshold value for each test image. In an implementable manner, the target threshold can also be a fixed threshold value, which can be obtained by experiments on rust images.

[0091] 105. Determine whether the superpixel is a rust area according to the second rust probability of the superpixel and the target threshold.

[0092] The rust detection device can utilize the target threshold value and the second rust probability of each superpixel to make a binary segmentation on the superpixel segmentation heat map, and obtain a final rust segmentation result. Specifically, if the second rust probability of the superpixel is greater than the target threshold value, the superpixel is determined as a rust area; if the second rust probability of the superpixel is less than the target threshold value, the superpixel is determined as a non-rust area. If the superpixel is a rust area, the superpixel is segmented from the image to be detected and determined as a rust area.

[0093] The method includes: acquiring an image to be detected, and extracting a feature vector of each pixel point in the image to be detected; inputting the feature vector into a pre-trained mixed Gaussian model to obtain a first rust probability of each pixel point in the image to be detected; inputting the image to be detected into a preset clustering algorithm, clustering pixel points with similar color features in the image to be detected into superpixels based on the preset clustering algorithm, and obtaining a second rust probability of each superpixel in the image to be detected; obtaining a target threshold value of the image to be detected based on a preset threshold algorithm and the first rust probability; and determining whether the superpixel is a rust area according to the second rust probability of the superpixel and the target threshold value. By clustering pixel points with similar color features into superpixels and according to the second rust probability of the superpixel and the target threshold value, whether the superpixel is a rust area can be accurately determined, and a rust area in the image to be detected can be accurately determined.

[0094] Further, as shown in Figure 2 and Figure 5 , the rust detection device can perform rust detection on multiple images to be detected and segment a rust area. An image in an input RGB color space is converted into an image in an HSV color space, a feature vector is extracted, the feature vector is input into a mixed Gaussian model to predict a rust probability of a pixel point, pixel points are clustered using superpixel segmentation to retain the correlation of adjacent pixel points, and based on adaptive threshold segmentation, an image of completed detection is output after rust segmentation, thereby realizing fast and accurate rust area segmentation and embodying good robustness and segmentation consistency.

[0095] The rust detection device can perform rust detection on multiple images to be detected and segment a rust area. An image in an input RGB color space is converted into an image in an HSV color space, a feature vector is extracted, the feature vector is input into a mixed Gaussian model to predict a rust probability of a pixel point, pixel points are clustered using superpixel segmentation to retain the correlation of adjacent pixel points, and based on adaptive threshold segmentation, an image of completed detection is output after rust segmentation, thereby realizing fast and accurate rust area segmentation and embodying good robustness and segmentation consistency. Figure 6

[0096] The acquisition unit 601 is configured to acquire an image to be detected, and extract a feature vector of each pixel point in the image to be detected;

[0097] The input unit 602 is configured to input the feature vector into a pre-trained mixed Gaussian model to obtain a first rust probability of each pixel point in the image to be detected;

[0098] ​The clustering unit 603 is configured to input the to-be-tested image into a preset clustering algorithm, cluster pixel points with similar color features in the to-be-tested image into superpixels based on the preset clustering algorithm, and obtain a second rust probability of each superpixel in the to-be-tested image.

[0099] The execution unit 604 is configured to obtain a target threshold of the to-be-tested image based on a preset threshold algorithm and the first rust probability.

[0100] The determination unit 605 is configured to determine whether the superpixel is a rust area according to the second rust probability of the superpixel and the target threshold.

[0101] The embodiments of the present application further provide a rust detection device 700, as shown in the accompanying drawings, comprising: Figure 7

[0102] A central processing unit 701, a memory 702, and an input and output interface 703.

[0103] The memory 702 is a transitory storage memory or a persistent storage memory.

[0104] The central processing unit 701 is configured to communicate with the memory 702 and execute instruction operations in the memory 702 to perform the detection method described above.

[0105] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the system, device and unit described above can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0106] In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other manners. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0107] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0108] ​In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0109] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or in the form of a contribution to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, read-only memory), a random access memory (RAM, random access memory), a magnetic disk or an optical disk, and various media that can store program codes.

Claims

1. A method for detecting rust, characterized in that, include: Acquire the image to be tested, and extract the feature vector of each pixel in the image to be tested; The feature vector is input into a pre-trained Gaussian mixture model to obtain the first corrosion probability of each pixel in the image to be tested. The image to be tested is input into a preset clustering algorithm. Based on the preset clustering algorithm, pixels with similar color features in the image to be tested are clustered into superpixels, and the second corrosion probability of each superpixel in the image to be tested is obtained. Based on the preset threshold algorithm and the first corrosion probability, the target threshold of the image to be tested is obtained; Based on the second corrosion probability of the superpixel and the target threshold, it is determined whether the superpixel is a corrosion region.

2. The detection method according to claim 1, characterized in that, The extraction of the feature vector of each pixel in the image to be tested includes: Convert the image to be tested from the RGB color space to the HSV color space; The to-be-tested image in the HSV color space is converted to the hue value of the target pixel in the to-be-tested image according to the formula: H(i,j)=(H(i,j)+180)mod 360, where i,j represent the coordinate position of the target pixel in the to-be-tested image; The transformed hue value, saturation value, and brightness value corresponding to the target pixel are normalized to obtain the feature vector of the target pixel.

3. The detection method according to claim 1, characterized in that, Before inputting the feature vector into the pre-trained Gaussian mixture model, the method further includes: Obtain a training set of rust images and an initial Gaussian mixture model, and extract the first feature vector of the pixels in the training set of rust images; The first feature vector is input into the initial Gaussian mixture model, and the initial Gaussian mixture model outputs the third corrosion probability; A loss value is constructed based on the third corrosion probability and the preset corrosion probability; The mean and covariance of the initial Gaussian mixture model are adjusted based on the loss value until the initial Gaussian mixture model converges, thus obtaining the trained Gaussian mixture model.

4. The detection method according to claim 1, characterized in that, The step of inputting the feature vector into a pre-trained Gaussian mixture model to obtain the first corrosion probability of each pixel in the image to be tested includes: The feature vector is then input into a pre-trained Gaussian mixture model: Where, x i Let μ represent the i-th eigenvector. k Σ represents the mean of the k-th Gaussian model in the pre-trained Gaussian mixture model. k and w k Let z represent the covariance and weights of the k-th Gaussian model, respectively. i z is a latent variable i =1,2,3,…,p, where the pre-trained Gaussian mixture model has p Gaussian models, and d is the dimension of the feature vectors. Let represent the initial corrosion probability of the i-th feature vector, and p represent the corrosion probability of the i-th feature vector; The pre-trained Gaussian mixture model outputs p corrosion probabilities corresponding to the i-th feature vector, and the maximum value among the p corrosion probabilities is selected as the first corrosion probability of the pixel corresponding to the i-th feature vector.

5. The detection method according to claim 1, characterized in that, The preset clustering algorithm includes a superpixel segmentation algorithm, wherein clustering pixels with similar color features in the image to be tested into superpixels based on the preset clustering algorithm includes: Based on the total number of superpixels in the image to be tested, determine the initial cluster center for each superpixel; According to the formula: G(x,y)=||L(x+1,y)-L(x-1,y)|| 2 +||L(x,y+1)-L(x,y-1)|| 2 The gradient values ​​of the pixels in the neighborhood superpixels corresponding to the current superpixel are obtained, where G(x,y) is the gradient value of the pixel, L(x,y) is the lab vector of the pixel located at coordinates (x,y) in the image to be tested, where l represents color brightness, a and b represent color opposite dimensions, and ||·|| is the Euclidean norm. The initial cluster center in the target neighborhood superpixel is moved to the position corresponding to the pixel with the smallest gradient value in the target neighborhood superpixel and used as the cluster center of the target neighborhood superpixel; Using the aforementioned superpixel segmentation algorithm: Where, d lab l is the color distance from a pixel to the cluster center. i and l j Indicates the value of l in the rotating coordinate system, a i and a j Indicates the value of a in the rotating coordinate system, b i and b j The value of b in the rotating coordinate system is represented by d. xy x is the spatial distance from a pixel to the cluster center. i and x j y i and y j D represents the values ​​of coordinates (x, y) in a rotated coordinate system. S is the distance from the pixel point to the cluster center within the preset range, m is the variable for adjusting the superpixel compactness, and S is the grid interval; For each superpixel in the image to be tested, the distance from the cluster center to the pixel within a preset range is measured, and the cluster center with the smallest distance is taken as the target cluster center of the superpixel.

6. The detection method according to claim 1, characterized in that, The preset threshold algorithm includes: an adaptive threshold segmentation algorithm, wherein obtaining the target threshold of the image to be tested based on the preset threshold algorithm and the first corrosion probability includes: Based on the first corrosion probability, a corrosion grayscale image corresponding to the image to be tested is generated; The rust grayscale image is input into the adaptive threshold segmentation algorithm to obtain the binarized threshold corresponding to the image under test, and the binarized threshold is used as the target threshold of the image under test. The adaptive threshold segmentation algorithm includes: Where, k * For binarization threshold, To maximize the inter-class variance, k is the threshold level, L is the gray level of the rust grayscale image, P0(k) and P1(k) are the cumulative probabilities of k, μ0(k) and μ1(k) are the mean of k, and μ is the mean of the Gaussian model.

7. The detection method according to claim 1, characterized in that, Determining whether a superpixel is a rusted region based on the second corrosion probability of the superpixel and the target threshold includes: If the second corrosion probability of the superpixel is greater than the target threshold, then the superpixel is determined to be a corrosion region; If the second corrosion probability of the superpixel is less than the target threshold, then the superpixel is determined to be a non-corrosion region.

8. A rust detection device, characterized in that, include: An acquisition unit is used to acquire the image to be tested and extract the feature vector of each pixel in the image to be tested. The input unit is used to input the feature vector into a pre-trained Gaussian mixture model to obtain the first corrosion probability of each pixel in the image to be tested. The clustering unit is used to input the image to be tested into a preset clustering algorithm, and based on the preset clustering algorithm, cluster pixels with similar color features in the image to be tested into superpixels, and obtain the second corrosion probability of each superpixel in the image to be tested. An execution unit is configured to obtain a target threshold for the image under test based on a preset threshold algorithm and the first corrosion probability. The determining unit is configured to determine whether the superpixel is a rusted region based on the second rust probability of the superpixel and the target threshold.

9. A rust detection device, characterized in that, include: Central processing unit, memory, and input / output interfaces; The memory is either a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 7.

Images