Metal surface rust spot detection method integrating multi-directional and multi-universe local segmentation

Abstract

The present invention relates to a method for detecting rust spots on a metal surface by fusing multi-directional and multi-universe local segmentation. First, an image of the rust spots on the metal surface to be detected is collected, and the image is preprocessed by median filtering and homomorphic filtering. Secondly, global multi-directional gray-scale scanning analysis is performed on the image to determine the main gray-scale distribution curve and search for and locate peak and valley points. Then, the multi-universe optimization algorithm is used to optimize the fluctuation amplitude threshold and floating threshold parameters for peak and valley interval segmentation, and the optimal local segmentation in each direction is obtained. Finally, the principal component analysis method is used to compress and fuse the optimal local segmentation images in multiple directions to obtain the final rust spot detection result. This method is beneficial to improving the detection efficiency and accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of metal rust detection, and particularly relates to a method for detecting rust spots on a metal surface by fusing multi-directional and multi-universe local segmentation. Background Art

[0002] Rust spots on a metal surface refer to the physical volume and chemical property loss and damage formed under the action of surrounding water, air, chemical solvents, etc. on the metal material. In engineering, rust spots on a metal surface will directly cause rapid corrosion of the metal surface, resulting in a decrease in the bearing capacity, stiffness and stability of the metal. After rust spots appear on the metal surface, its surface presents characteristic morphologies such as unevenness and rust skin swelling, and these characteristics are important data sources for rust grade evaluation. At present, the detection of rust spots on a metal surface in the engineering field mainly relies on manual visual inspection. However, due to the weak characteristics of the metal surface in the initial stage of rust spot formation and the irregular distribution of rust spots during the dynamic development of corrosion, it is difficult to accurately and quantitatively describe characteristic information such as the corrosion area and corrosion degree by manual visual inspection, and the reliability of the detection results is poor.

[0003] With the rapid development of machine vision and image processing technologies, images, as a carrier for recording and describing information, have shown great application potential in the field of metal rust detection. The detection of rust spots on a metal surface can be understood as an image foreground segmentation problem. By using an image acquisition device to obtain the corrosion image of the metal surface and using image processing algorithms to extract the surface rust spots, the quantitative evaluation of the metal surface corrosion grade can be further realized. However, in actual engineering applications, metal surface images often have interference problems such as uneven brightness, uneven metal surface reflection, and high-brightness noise points, and traditional digital image segmentation methods are difficult to objectively and accurately detect all rust spot areas.

[0004] In recent years, deep learning technologies have been applied to corrosion area segmentation. It has the potential to accurately and reliably detect and extract corrosion characteristics in complex backgrounds, but it can only qualitatively analyze the corrosion condition and cannot accurately identify and extract the corrosion area. It has achieved certain segmentation effects for typical metal surface rust spot detection, but for small rust spots with relatively light corrosion or wide distribution, its segmentation accuracy often deviates from the actual corrosion scenario. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for detecting rust spots on a metal surface by fusing multi-directional and multi-universe local segmentation, which is beneficial to improving the detection efficiency and accuracy.

[0006] To achieve the above object, the technical solution adopted by the present invention is: a method for detecting rust spots on the metal surface by fusing multi-directional and multi-universe local segmentation. First, collect the rust spot image on the metal surface to be detected, and perform median filtering and homomorphic filtering preprocessing on the image. Secondly, perform global multi-directional gray-scale scanning analysis on the image to determine the main gray-scale distribution curve and search for and locate the peak and valley points. Then, use the multi-universe optimization algorithm to optimize the fluctuation amplitude threshold and floating threshold parameters for local segmentation of the peak and valley intervals, and obtain the optimal local segmentation in each direction. Finally, use the principal component analysis method to compress and fuse the optimal local segmentation images in multiple directions to obtain the final rust spot detection result.

[0007] Further, the method includes the following steps:

[0008] Step S1: First, collect the rust spot image on the metal surface to be detected, obtain the industrial image in the offline state, and then perform median filtering on the image. At the same time, set the parameters of homomorphic filtering according to the characteristics of the image and perform homomorphic filtering. Then, perform global multi-directional gray-scale change scanning on the image, calculate the mean square error K of the gray-scale change curve of each row of the image in each direction respectively, and select the gray-scale change curve with the largest mean square error K value under the gray-scale scanning in each direction as the main gray-scale distribution curve in each direction for subsequent analysis;

[0009] Step S2: Initialize the parameters of the multi-universe optimization algorithm: the number of universes n, the number of objects d in the universe, the maximum number of iterations L, the probability of the existence of the wormhole WEP, the travel distance value TDR, and the exploitation degree p; regard the fluctuation amplitude threshold parameter H and the floating threshold parameter ξ as two objects in a single universe, and randomly initialize a multi-universe population U = [U1, U2,..., U n T ;

[0010] Step S3: According to the fluctuation amplitude threshold parameter H and the floating threshold parameter ξ in each universe individual in the multi-universe population U, use the local threshold segmentation method for the peak and valley intervals of the gray-scale fluctuation curve to search for and locate the peak and valley points on the main gray-scale distribution curve obtained in Step S1, calculate the local threshold of the peak and valley intervals, and then perform local threshold segmentation of the peak and valley intervals to obtain the segmentation results corresponding to each universe individual;

[0011] Step S4: Evaluate the fitness of the segmentation results corresponding to each universe individual in the multi-universe population, calculate the fitness of the universe individual corresponding to the segmentation result, and obtain the universe individual corresponding to the minimum fitness as the current optimal universe by comparing the fitness values;

[0012] Step S5: Enter the main loop, and update the probability of the existence of the wormhole WEP and the travel distance value TDR;

[0013] ​Step S6: Implement the roulette wheel mechanism to transfer the objects in the cosmic individuals, then calculate the fitness values of the updated objects in each cosmic individual, select the cosmic with the minimum fitness value as the updated optimal cosmic, and record the values of objects H and ξ and their corresponding fitness values in this optimal cosmic; if it is better than the current optimal cosmic, replace it, otherwise, retain the current optimal cosmic;

[0014] Step S7: Judge the termination condition. If the number of iterations reaches the preset maximum number of iterations, exit the main loop and output the optimal cosmic and its fitness value. Otherwise, return to Step S4;

[0015] Step S8: Output the local threshold segmentation results of the peak-valley intervals corresponding to the two objects H and ξ in the optimal cosmic when the maximum number of iterations is reached;

[0016] Step S9: Repeat Steps S2 - S8 for each main gray distribution curve obtained in Step S1 in sequence, output the optimal cosmics and their fitness values corresponding to each main gray distribution curve, and the local threshold segmentation results of the peak-valley intervals corresponding to the optimal cosmics when the maximum number of iterations is reached in each direction;

[0017] Step S10: Perform PCA method image compression on the segmentation results I0, I 45 , I 90 , I 135 , I 180 , I 225 , I 270 , I 315 obtained in Step S9, and output the final segmentation results after compression of the segmentation results in each direction, which is the rust spot detection result.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The present invention can effectively overcome the influence of uneven brightness of the metal surface image, uneven reflection of the metal surface, and wire drawing texture of the metal surface, adaptively optimize the segmentation hyperparameters, and can effectively segment rust spot images of different sizes and different distributions. The rust spot detection accuracy is high, reducing the large amount of time consumed by parameter adjustment, and improving the actual application efficiency. By using this method to detect rust spots on the metal surface, the detection efficiency and accuracy can be greatly improved, and the accurate and quantitative description of the rust spot area and corrosion degree can be realized. Description of the Drawings

[0019] Figure 1 is the flowchart of the method implementation in the embodiment of the present invention;

[0020] Figure 2 is the flowchart of the implementation of the multi - universe optimization (MVO) algorithm in the embodiment of the present invention;

[0021] Figure 3 is the rust spot image of Image2 in the embodiment of the present invention;

[0022] Figure 4 is the unidirectional parameter random segmentation result in the embodiment of the present invention;

[0023] Figure 5 is the segmentation result of the corrosion area on the multi-directional metal spray-painted surface and the fusion result by the PCA method in the embodiment of the present invention;

[0024] Figure 6 is the comparison of the experimental results of each algorithm in the embodiment of the present invention. Detailed implementation manners

[0025] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0026] It should be noted that the following detailed description is exemplary and is intended to provide further illustration of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs.

[0027] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary implementation manners according to the present application. As used herein, unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0028] This embodiment provides a method for detecting rust spots on a metal surface by fusing multi-directional multiverse local segmentation. First, an image of rust spots on the metal surface to be detected is collected, and the image is preprocessed by median filtering and homomorphic filtering. Secondly, a global multi-directional gray-scale scan analysis is performed on the image to determine the main gray-scale distribution curve and search for and locate the peak and valley points. Then, the multi-universe optimization algorithm is used to optimize the fluctuation amplitude threshold and floating threshold parameters for peak and valley interval segmentation to obtain the optimal local segmentation in each direction. Finally, the principal component analysis method is used to compress and fuse the optimal local segmentation images in multiple directions to obtain the final rust spot detection result.

[0029] As Figure 1 shown, the method specifically includes the following steps:

[0030] Step S1: First, collect the rust spot images on the surface of the metal to be detected to obtain industrial images in the offline state. Then, perform median filtering on the images. At the same time, set the parameters of homomorphic filtering according to the characteristics of the images and perform homomorphic filtering. After that, perform a global multi-directional gray-level change scan on the images, calculate the mean square error K of the gray-level change curve of each row of the images in each direction respectively, and select the gray-level change curve with the largest mean square error K value under the gray-level scan in each direction as the main gray-level distribution curve in each direction for subsequent analysis.

[0031] Step S2: Initialize the parameters of the Multiverse Optimization (MVO) algorithm: the number of universes n, the number of objects (dimensions) d in the universe, the maximum number of iterations L, the probability of the existence of the wormhole WEP, the travel distance value TDR, and the exploitation degree p; regard the fluctuation amplitude threshold parameter H and the floating threshold parameter ξ as two objects in a single universe, and randomly initialize a multiverse population U = [U1, U2, …, U n T 。

[0032] Step S3: According to the fluctuation amplitude threshold parameter H and the floating threshold parameter ξ in each universe individual in the multiverse population U, use the local threshold segmentation method for the peak-valley interval of the gray-level fluctuation curve to search for and locate the peak-valley points and calculate the local threshold of the peak-valley interval on the main gray-level distribution curve obtained in Step S1, and then perform local threshold segmentation of the peak-valley interval to obtain the segmentation results corresponding to each universe individual.

[0033] Step S4: Evaluate the fitness of the segmentation results corresponding to each universe individual in the multiverse population, calculate the fitness of the universe individual corresponding to the segmentation result, and obtain the universe individual corresponding to the minimum fitness as the current optimal universe by comparing the fitness values. Among them, the selected fitness function is:

[0034]

[0035] contrast(s,t) = |f(s) - f(t)|

[0036] where n is the class label value of the connected regions in the segmentation result image and is also the number of connected regions; A i is the total number of pixels in region i, R i is the set of all pixels in region i, t is the intersection pixel of the neighborhood pixel set W(s) of pixel s and the set R i of all pixels in region i, and f(s) and f(t) are the gray-level values of pixel s and pixel t respectively.

[0037] Step S5: Enter the main loop to update the probability of the existence of the wormhole WEP and the travel distance value TDR.

[0038] ​Step S6: Execute the roulette wheel mechanism to transfer the objects in the cosmic individuals according to formula (9), then calculate the fitness values of the updated objects in each cosmic individual according to formula (10), select the cosmic with the minimum fitness value as the updated optimal cosmic, and record the values of objects H and ξ and their corresponding fitness values in this optimal cosmic; if it is better than the current optimal cosmic, replace it, otherwise, retain the current optimal cosmic.

[0039] Step S7: Judge the termination condition. If the number of iterations reaches the preset maximum number of iterations, exit the main loop and output the optimal cosmic and its fitness value. Otherwise, return to Step S4.

[0040] Step S8: Output the local threshold segmentation results of the peak-valley intervals corresponding to the two objects H and ξ in the optimal cosmic when the maximum number of iterations is reached.

[0041] Step S9: Repeat Steps S2 - S8 for each main gray distribution curve obtained in Step S-1 in turn, output the optimal cosmic and its fitness value corresponding to each main gray distribution curve, and the local threshold segmentation results of the peak-valley intervals corresponding to the optimal cosmic when the maximum number of iterations is reached in each direction.

[0042] Step S10: Perform PCA method image compression on the segmentation results I0, I 45 , I 90 , I 135 , I 180 , I 225 , I 270 , I 315 obtained in Step S9, and output the final segmentation results after compression of the segmentation results in each direction, which are the rust spot detection results.

[0043] The following further explains the relevant content involved in the above method steps.

[0044] 1 Local Threshold Segmentation of Multi-Directional Gray Fluctuation Curve Analysis

[0045] 1.1 Median Filtering and Homomorphic Filtering

[0046] Affected by the complex industrial shooting environment, image transmission, and imaging equipment, there are usually various noises in the acquired target images, which results in the final acquired target images often showing uneven illumination phenomena such as discrete, light and dark alternating, and irregular. Such characteristics of uneven illumination images and a large amount of noise in the images greatly affect the subsequent processing of the images. To solve such problems and eliminate noise as much as possible and reduce the influence brought by uneven illumination, before image segmentation, the present invention first performs median filtering using a median filtering template with a kernel size of 3*3 to filter out salt-and-pepper noise in the image. Secondly, the homomorphic filtering method is adopted. By setting reasonable parameters, multiplicative noise and a large amount of useless low-frequency information in the target image are removed, the contrast of rust spot targets and the high-frequency components of edge details are enhanced, and the accuracy of subsequent segmentation is further improved.

[0047] 1.2 Local threshold segmentation method based on peak-valley intervals of multi-directional gray-scale fluctuation curves

[0048] Each image can be regarded as a two-dimensional gray-scale function composed of pixel point coordinates and pixel point gray-scale values. In a three-dimensional coordinate system, it can be intuitively represented as a three-dimensional topographic map. By analogy, the continuous mountains and valleys in the topographic map are compared with the three-dimensional map of the image. The mountains correspond to the peaks in the three-dimensional map of the image, that is, the areas with higher gray-scale values in the image, and the valleys correspond to the troughs in the three-dimensional map of the image, that is, the areas with lower gray-scale values in the image. In the actual rust spot images on the metal surface, the rust spots, as the areas with lower gray-scale values in the image and also the target areas to be segmented and extracted, form a sharp contrast with the background area. They are represented as troughs in the three-dimensional map of the image, while the background area is represented as peaks, and the two together form the gray-scale extreme value area. Based on this characteristic, by selecting a suitable threshold segmentation point between the peak and valley points of each selected neighborhood, the detailed information of the rust spot targets in the image can be completely segmented and extracted. Therefore, the present invention proposes a local threshold segmentation method based on the analysis of multi-directional gray-scale curves of image peaks and valleys. This method first performs a global gray-scale scan and analysis of the rust spot gray-scale images on the metal surface from eight directions: 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°, that is, the horizontal, vertical, and two diagonal directions of the image. Then, according to the analysis results, the peak and valley points of the main gray-scale distribution curves in each direction are searched and screened. Finally, a suitable threshold is selected in the gray-scale peak-valley interval for local threshold segmentation, and finally, the crack threshold segmentation images in eight directions are obtained.

[0049] 1) Global multi-directional gray-scale scan and analysis

[0050] Perform a global multi-directional gray-scale change scan on a gray-scale image. First, represent the image in the form of a two-dimensional array. Let the specific number of rows in the array be y = y0, and the specific number of columns be x, where x = 1, 2, 3,..., m, and m is the total number of pixel points in that row of the image, i.e., the total number of columns of the image. The image gray-scale value is f, then f0 = f(x, y0) represents a one-dimensional function with the number of columns x of the image as the independent variable and the image gray-scale value f as the dependent variable for a specific row y0 of the image. In the rectangular coordinate system, it is reflected as the gray-scale distribution curve of a specific row of the image. Taking the horizontal direction as an example, perform a horizontal gray-scale change scan on the gray-scale image. By setting different rows y, the gray-scale distribution curves of each row of the gray-scale image can be obtained. That is, multiple groups of one-dimensional gray-scale distribution curves can be obtained by scanning row by row in the horizontal direction, and the same is true for other directions. However, due to the requirement for real-time performance in the actual industrial background, to reduce the amount of calculation, only one group of gray-scale distribution curves is selected as the main gray-scale distribution curve in that direction to participate in subsequent analysis and processing. To determine the main gray-scale distribution curve, first introduce the gray-scale change mean square deviation that measures the change amplitude of the curve. By calculating the mean square deviation of each group of gray-scale distribution curves in the horizontal direction, select the curve with the largest change amplitude, that is, the curve with the largest gray-scale mean square deviation, as the main gray-scale distribution curve in that direction. The definition of the gray-scale change mean square deviation K is:

[0051]

[0052] where K i represents the gray-scale change mean square deviation of the gray-scale distribution curve of the i-th row in the horizontal direction, and f i (x, y i ) represents the gray-scale value of the pixel point in the x-th column of the gray-scale distribution curve of a specific row y i , and i = 1, 2, 3,..., n, where n is the total number of rows of the image. f i (k) is the gray-scale average value of the gray-scale distribution curve of the i-th row of the image, and m is the number of pixel points in the i-th row of the gray-scale distribution.

[0053] By performing gray-scale scans on the image in eight global directions, calculating and comparing the gray-scale change mean square deviations K of each group of gray-scale change curves in each direction, the gray-scale distribution curve with the largest gray-scale change mean square deviation K value in each direction can be obtained, that is, the main gray-scale distribution curve in each direction. The characteristic that the gray-scale amplitude change of the main gray-scale distribution curve selected by this method is relatively large makes the subsequent local threshold segmentation process more accurate for each local block, and the calculation of its local threshold is also more accurate, which is conducive to retaining more details of the target rust spot.

[0054] 2) Search and positioning of peak and valley points of the main gray-scale distribution curve

[0055] Taking the main gray-level distribution curve in the horizontal direction as an example, the search and positioning of peak and valley points are carried out. As analyzed above, there are generally several large-scale or small-scale peaks and valleys in the actual gray-level distribution curve. Some of these peaks and valleys may not correspond to the peaks and valleys of the crack target. Therefore, before positioning the peak and valley points, it is necessary to search out all these large-scale and small-scale peaks and valleys, and then screen and eliminate them through the subsequent fluctuation amplitude threshold H, and finally locate the peaks and valleys corresponding to the crack target.

[0056] First, the search for potential peak and valley points is carried out. In the definition of mathematical expressions, for a continuous one-dimensional function curve, the peak and valley points are the extreme points of the curve. Among them, the minimum value point (i.e., the valley point) satisfies the first derivative being 0 and the second derivative being greater than 0; the maximum value point (i.e., the peak point) satisfies the first derivative being 0 and the second derivative being less than 0. However, since the gray-level distribution curve is not a continuous curve but a discrete curve, the forward and backward difference operations are adopted, and whether it is a potential peak and valley point is judged by the sign of the result obtained by the gray-level value difference calculation of a certain pixel point. The judgment conditions are as follows:

[0057]

[0058] where k is the specific serial number of the pixel point on the gray-level distribution curve, f(k) is the gray-level value of the gray-level distribution curve at the specific pixel point serial number k, f(k i-1 ) and f(k i+1 ) are the gray-level values of the previous pixel point and the next pixel point corresponding to the pixel point with the specific pixel point serial number k i .

[0059] By searching for potential peak and valley points on the main gray-level distribution curve, the positions and corresponding gray-level values of all potential peak and valley points on the curve are obtained; assuming E top(i) (i = 1, 2,..., m) are all the maximum value points on the curve, and E top(i) [k i , f(k i )] represents the i-th maximum value point on the curve. Among them, k i is the pixel serial number of the i-th maximum value point, and f(k i ) is the gray-level value corresponding to the pixel serial number k i of the i-th maximum value point, and m is the number of all maximum value points; E bottom(j) (j = 1, 2,..., n) are all the minimum value points on the curve, and E bottom(j) [k j , f(k j )] represents the j-th minimum value point on the curve. Among them, k j is the pixel serial number of the j-th minimum value point, and f(k j ) is the gray-level value corresponding to the pixel serial number k jThe corresponding gray value, and n is the total number of minimum points.

[0060] After completing the search for potential peak and valley points, before locating the large-scale peak and valley points among the potential peak and valley points, it is necessary to first determine the first peak and valley point as the initial iterative search condition. To determine the first peak and valley point, the first extreme point on the gray-scale distribution curve can be obtained by comparing the pixel sequence numbers of the first maximum point and the first minimum point, and then select it as the first peak and valley point. If the first extreme point is a maximum point, select it as the first peak point; if the first extreme point is a minimum point, select it as the first valley point. Using the first peak and valley point as the initial iterative condition, iteratively search for the next peak and valley point in sequence, and the positioning of all peak and valley points on the gray-scale distribution curve can be completed.

[0061] Assume that the previous point is the first peak point or the previous peak point T that meets the iterative search condition k = E top(m) [k m , f(k m )], then the next valley point B k = E bottom(j) [k j , f(k j )] should meet the following iterative search conditions:

[0062]

[0063] Among them, H ∈ (0, 255) is the gray-scale fluctuation amplitude threshold condition. However, to avoid rounding errors during image calculation and ensure calculation accuracy, it is mapped to the interval (0, 1), that is, H ∈ (0, 1), E bottom(m) [k m , f(k m )] is the peak point E top(m) [k m , f(k m )] the next minimum point after, E top(i) [k i , f(k i )] is the valley point E bottom(j) [k j , f(k j )] the next maximum point after, then E bottom(i-1) [k i-1 , f(k i-1 )] is the minimum point before this maximum point; f bottom (k j ) is the gray value of the valley point, f top (k j+1 ), f top (k j) are the grayscale values of the maximum point after the valley point and the grayscale value of the previous maximum point respectively.

[0064] Assume that the previous point is the first valley point or the last valley point B that meets the iterative search conditions k =E bottom(n) [k n ,f(k n )], then the next peak point T k =E top(i) [k i ,f(k i )] The iterative search conditions satisfied are as follows:

[0065]

[0066] Among them, E top(n) [k n ,f(k n )] is the valley point E bottom(n) [k n ,f(k n )] after a maximum point, E bottom(j) [k j ,f(k j )] is the peak point E top(i) [k i ,f(k i )] is a minimum point after E top(j-1) [k j-1 ,f(k j-1 )] is the maximum point before the minimum point; f top (k i ) is the peak gray value, f bottom (k i+1 ),f bottom (k i ) are the grayscale values of the minimum point after the peak point and the grayscale value of the previous minimum point respectively; in the above formula, f top (k i ) and f bottom (k i ) i The values of are not the same, f top (k i ) in k i represents the pixel number corresponding to the i-th maximum point, and f bottom (k i ) in k i Indicates the pixel number corresponding to the i-th minimum point.

[0067] The search and positioning mechanism ensures the peak and valley points with the maximum amplitude by setting the fluctuation amplitude threshold H, and to a certain extent filters out some small-scale peak and valley points that do not belong to the target. Through a large number of experimental tests, the optimal value of H is between 0.01 and 0.06. The first extreme point is selected as the first peak and valley point, and as the initial condition for iterative search, the main gray distribution curve in the horizontal direction is continuously searched and positioned, and finally all the peak and valley points on this curve can be obtained. The peak point and valley point sequences obtained by search and positioning can be used for the next calculation of the local threshold.

[0068] 3) Determination of the threshold for the peak and valley intervals of the main gray distribution curve

[0069] Based on the peak and valley point sequence obtained by search and positioning, the peak and valley sequence is successively divided into several peak-valley and valley-peak intervals in the order of peak and valley points, that is, there are several peak and valley intervals. Let [E bottom(m) [k m ,f(k m )],E top(m) [k m ,f(k m )]] be one of the valley-peak intervals, and the local threshold calculation for this interval is:

[0070] threshold=ξ×[f bottom (k m )-f top (k m )]+f top (k m )×(1-ξ) (6)

[0071] Among them, f top (k m ) is the gray value of the peak point in the interval, f bottom (k m ) is the gray value of the valley point in the interval, ξ is the floating threshold parameter, and by setting the floating parameter, the thickness of the target is controlled; assuming that the pixel point serial number of the peak point is k top , and the pixel point serial number of the valley point is k bottom , and the pixel point (k,f(k)) is a certain pixel in this interval, then k∈(k top ,k bottom ), and the following determination method is used for binary segmentation of the crack target in this interval:

[0072]

[0073] When f(k) > threshold, the pixel point (k, f(k)) belongs to the target; when f(k) < threshold, the pixel point (k, f(k)) belongs to the background. Therefore, by traversing several peak-valley and valley-peak intervals to calculate the local threshold and performing local threshold segmentation on the corresponding intervals, the local threshold segmentation of the peak-valley interval in the horizontal direction gray-scale scanning can be achieved. Note that for the intervals before the first extreme point and after the last extreme point, the local threshold takes the threshold of its adjacent interval. The same applies to other directions.

[0074] 2 Multi-universe optimization algorithm for optimizing local threshold segmentation

[0075] Regarding the uncertainty in the selection of the fluctuation amplitude threshold H and the floating threshold parameter ξ, which have a great impact on the segmentation effect in the above segmentation method, the present invention introduces an unsupervised evaluation index for image segmentation as the fitness function, and then uses the multi-universe optimization algorithm (MVO) to optimize these two parameters H and ξ to achieve adaptive segmentation. Although it can be known from a large number of experimental tests that the optimal value of the fluctuation amplitude threshold H is between 0.01 - 0.06, and the optimal value of the floating threshold parameter ξ is between 0.1 - 0.6, and the flexible selection of these two parameters gives the segmentation algorithm great controllability of segmentation sensitivity, for target images in different environments, it takes a long time to manually set the two parameters for segmentation testing and obtain a set of optimal parameters with good segmentation effects, and it is extremely difficult to measure the optimal segmentation result. To optimize the parameters H and ξ and achieve the optimal segmentation effect, the present invention introduces the multi-universe optimization algorithm (MVO) to optimize the fluctuation amplitude threshold H and the floating threshold parameter ξ. Taking the horizontal direction as an example, first, select the fitness function, initialize a multi-universe population according to the upper and lower bounds of the variables H and ξ, and set the number of universes n = 60 and the number of individuals in the universe d = 2, which means the objects in these two universes of H and ξ, and d is the dimension, also known as the number of variables. Secondly, set the remaining parameters and related variables in the algorithm, calculate the fitness values of the universe individuals, and compare to obtain the current optimal universe. Then, enter the main loop, update the WEP and TDP and execute the roulette mechanism to obtain the updated optimal universe, compare it with the current optimal universe, and retain the better one. Finally, complete the preset number of iterations and exit the main loop, output the optimal universe, that is, H and ξ and the optimal fitness value, and use the optimized H and ξ for local threshold segmentation of the gray-scale fluctuation curve analysis to obtain the binary image of the crack segmentation in the horizontal direction. The same applies to other directions.

[0076] 2.1 Multi-universe optimization algorithm (MVO)

[0077] The multi-universe optimization algorithm (Multi-Verse Optimizer, MVO) is a new type of swarm intelligence optimization algorithm based on the multi-universe theory in physics. The implementation process is as Figure 2As shown, this algorithm simulates the multi-universe population movement behavior under the combined action of white holes, black holes, and wormholes. It has the characteristics of high precision, easy to understand, and strong search ability. In addition, during the application process, the algorithm requires fewer parameters and also has good performance in low-dimensional numerical experiments. Like other swarm intelligence optimization algorithms, the optimization execution process of MVO is divided into two stages, namely exploration and exploitation. In the constructed mathematical model, a universe is the solution to the optimization problem, and each object in a single universe is regarded as the position vector of the corresponding solution. The expansion rate of each universe is proportional to the fitness of the corresponding solution.

[0078] (1) Initialization of the multiverse

[0079] The initialization of the universe as the solution is used to start the optimization process. Each update of each object in the universe, that is, each set of solutions, is executed through a progressive iteration rule. The random initialization U of the multi-universe population is defined by the following formula:

[0080]

[0081] Among them, n represents the number of universes, corresponding to the population size; d represents the number of individuals in the universe, corresponding to the dimension of the search space; represents the i-th universe, represents the j-th object in the i-th universe.

[0082] (2) Object selection following the roulette wheel mechanism

[0083] Due to the different expansion rates of each universe individual, that is, different fitness values, the objects in the universe are selected and updated through the roulette wheel mechanism and then transferred through the white hole / black hole orbit. The formula is as follows:

[0084]

[0085] Among them, NI(U i ) represents the normalized expansion rate of the i-th universe, r1 is a random number in the range of [0,1], represents the j-th object of the k-th universe selected by the roulette wheel mechanism.

[0086] (3) Movement update of the optimal universe

[0087] Without considering the size of the expansion rate, in order to achieve local changes and improve its own expansion rate, the universe individual will stimulate the internal objects to move towards the current optimal universe; this process is executed according to the following formula:

[0088]

[0089] Among them, x jDenote the j-th object of the current optimal universe, where lb and ub represent the lower and upper bounds of x respectively, r2, r3, r4 are random numbers within the range of [0, 1], WEP represents the probability of the existence of a wormhole, and TDR represents the travel distance value. These two important parameters respectively represent the probability of the existence of a wormhole in the multiverse space and the step length for an object to move towards the current optimal universe; its update principle follows the formula as below:

[0090]

[0091] where t is the current iteration number and T is the maximum iteration number. In this embodiment, WEP min and WEP max take values 0.2 and 1, that is, WEP min = 0.2, WEP max = 1, p represents the exploitation degree and takes the value of 6.

[0092] 2.2 Construction of the fitness function

[0093] To optimize the fluctuation amplitude threshold H and the floating threshold parameter ξ through the multiverse optimization algorithm to obtain the best segmentation result, the present invention adopts the unsupervised segmentation quality evaluation index Zeb as the fitness function of this algorithm. Zeb is an unsupervised segmentation quality evaluation index based on contrast, which fully considers the internal contrast of the region measured near each pixel. The smaller the internal contrast of the region, the higher the segmentation quality. Assume that W(s) is the neighborhood pixels of pixel s, f(s) is the gray value of pixel s, and L is the maximum gray value of the pixels, then the internal contrast I i The calculation formula is as follows:

[0094]

[0095] contrast(s,t) = |f(s)-f(t)| (13)

[0096] where I i represents the internal contrast of region i, A i is the total number of pixels in region i, R i is the set of all pixels in region i, t is the intersection pixel of the neighborhood pixel set W(s) of pixel s and the set R i of all pixels in region i, and f(s), f(t) are the gray values of pixel s and pixel t respectively. Therefore, to obtain the best segmentation result with higher segmentation quality, the following fitness function is constructed:

[0097]

[0098] where n is the class label value of the connected regions in the segmentation result image, that is, the number of connected regions.

[0099] 3 Multi-directional Optimal Local Threshold Segmentation Image Fusion

[0100] Using the multi-directional gray-scale fluctuation curve peak-valley interval local threshold segmentation method with optimized parameters by combining the multi-universe optimization algorithm proposed above, sub-images in eight directions of 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315° of the target image are selected and local threshold segmentation is performed respectively, and the crack segmentation binary images I0, I 45 , I 90 , I 135 , I 180 , I 225 , I 270 , I 315 can be obtained. Since the local threshold segmentation method based on gray-scale fluctuation analysis has different sensitivities to the segmentation of small rust spots at different angles in different directions, therefore, the binary images in 8 directions may each retain the information of small rust spots or other detailed information with relatively high sensitivity in that direction, and this part of the information may not overlap in the 8 images. If the I0, I 45 , I 90 , I 135 , I 180 , I 225 , I 270 , I 315 images are simply synthesized, for example, performing an AND operation on the images may lose the detailed information retained by the segmented images in different directions, while performing an OR operation is likely to retain the noise of the segmented images in each direction. To simplify the calculation amount, retain the rust spot detail information as much as possible and ensure the segmentation effect to the greatest extent, the present invention uses the PCA algorithm to compress and fuse the 8 binary images, and compresses the 8-dimensional vector I = {I0, I 45 , I 90 , I 135 , I 180 , I 225 , I 270 , I 315} composed of 8 images to 1 dimension {I'}. The PCA algorithm is a common mathematical transformation method. It converts a set of variable data that may be correlated through a linear transformation into a set of linearly uncorrelated variables. It is the projection of high-dimensional data in a low-dimensional space where the maximum data volume, that is, the maximum variance, is contained in the dimension expected to be projected, so as to achieve feature dimensionality reduction. This method first combines the 8 binary images I0, I 45 , I 90 , I 135 , I 180 , I 225 , I 270 , I 315Convert them into one-dimensional vectors respectively, and use these one-dimensional vectors as samples to form a sample matrix Z = {z1, z2, …, z n}, where n is the total number of pixels in the binarized image. Then calculate the mean vector μ of Z and centralize each row of Z:

[0101]

[0102] After obtaining the centralized matrix , then calculate the eigen covariance matrix C of the matrix . Perform SVD transformation on the covariance matrix C to calculate the eigenvector matrix U {ξ0, ξ1, ξ2, ξ3, ξ4, ξ5, ξ6, ξ7} and eigenvalues {λ0, λ1, λ2, λ3, λ4, λ5, λ6, λ7} of the covariance matrix, where λ0 > λ1 > λ2 > λ3 > λ4 > λ5 > λ6 > λ7. Sort the eigenvalues from smallest to largest, and select the eigenvector corresponding to the largest eigenvalue for data dimensionality reduction:

[0103] Z' = Z T ξ0 (17)

[0104] where Z' is the sample matrix of the one-dimensional vectors after compression of 8 images. Re-converting it to the original image space can obtain the grayscale matrix I' of the compressed image, that is, the compressed image. Using the PCA algorithm for image compression largely preserves the non-overlapping detailed information of each of the 8 binarized images, and at the same time compresses the image from 8 dimensions to 1 dimension. Among them, the segmentation results in each direction are the local threshold segmentation results obtained by optimizing the parameters H and ξ using the multi-universe optimization algorithm and then performing gray-scale fluctuation analysis.

[0105] 4 Specific embodiments and descriptions

[0106] To verify the effectiveness and advantages of the method proposed in the present invention, take the rust spot image on the metal surface in the offline state as the experimental object, and conduct simulation tests on this method. Through a large number of experiments, the method is verified, and the results of each experimental step are given. The method of the present invention is compared and analyzed with the classical edge detection, image segmentation methods and the segmentation methods improved based on these methods in recent years. The hardware platform for the experiment is Intel(R) Core(TM) i5-8250U CPU@1.60GHz, 12.0GB (RAM), and the software running environment is Windows 10 Matlab 2018b.

[0107] Take the rust spot image on the metal surface Figure 3For example, first, image preprocessing and grayscale change scanning in the horizontal direction are performed to obtain the main grayscale distribution curve in the horizontal direction. Then, within the parameter range obtained through a large number of experiments, the grayscale fluctuation amplitude threshold parameter H = 0.02 and the floating threshold parameter ξ = 0.17 are randomly selected for local threshold segmentation of grayscale fluctuation analysis, and the segmentation result in the horizontal direction is obtained as Figure 4 shown. From the segmentation result Figure 4 it can be seen that although this method can roughly separate the target and the background and is not affected by uneven illumination, salt-and-pepper noise, and small-scale fluctuations, since the main grayscale distribution curve under single-direction scanning cannot cover the global fluctuation information of the completely metal surface corrosion grayscale image, there is a phenomenon of losing some targets in the segmentation result in the horizontal direction. In addition, since the grayscale fluctuation amplitude threshold parameter H and the floating threshold parameter ξ have a great influence on the segmentation effect, the optimal segmentation effect depends on H and ξ. Randomly and manually setting these two parameters makes the segmentation effect have a large contingency, and it is extremely difficult and time-consuming to obtain the best segmentation effect.

[0108] To verify the advantages of the method proposed in the present invention compared with the above method, 3 metal surface rust images, namely Image1, Image2, and Image3, with a size of 300 pixel × 300 pixel are selected as test images. Taking the image Image1 as an example, first, image preprocessing is performed, and global multi-directional grayscale scanning of the image is carried out. Secondly, the grayscale mean square deviation of all grayscale change curves in each direction is calculated, and the grayscale change curve with the largest grayscale mean square deviation is selected as the main grayscale distribution curve in that direction. Then, the multi-universe optimization algorithm is initialized, and the fitness minimum optimization is performed on the grayscale fluctuation amplitude threshold parameter H and the floating threshold parameter ξ to adaptively obtain the optimal H and ξ required for searching and positioning the peak and valley points of the main grayscale distribution curve in each direction. Furthermore, according to the optimized parameters, local threshold segmentation of the peak and valley intervals of the main grayscale distribution curve in each direction is performed. Finally, the segmentation results in each direction are fused through a multi-directional segmentation image fusion method based on the principal component analysis method to obtain the final segmentation result. As shown in the figure, Figure 5They are the segmentation result images in eight directions and the final segmentation result image obtained by using the method proposed in the present invention for three metal surface corrosion images. Among them, Groudtruth is the true value image manually annotated, which is convenient for more intuitive comparative analysis and participating in the subsequent segmentation performance evaluation. Taking Image2 as an example, affected by the wire-drawing texture on the metal surface and the brightness of the metal surface, the segmentation sensitivities of the segmentation methods in each direction proposed in the present invention for the smaller corrosion areas on the metal surface are different. Therefore, the degrees of detail retention in the segmentation results in each direction are different. In the segmentation result image in direction 5 and the segmentation result image in direction 7 of Image2, the corrosion points above the large corrosion area are lost in the segmentation result image in direction 5, showing under-segmentation. While in the segmentation result image in direction 7, the segmentation effect of this part of the corrosion area is better. Compared with the true value image, the degree of detail retention in the segmentation effect of this part is higher. The final segmentation result image obtained by using the PCA method for multi-directional segmentation image fusion well retains the parts with a relatively high degree of detail retention in each direction segmentation image. Compared with the single-direction segmentation image, the segmentation effect is better. Table 1 shows the optimal parameters H and ξ and the minimum fitness F obtained by optimizing through the multi-universe optimization algorithm during the optimized segmentation process in eight directions for three images. Zeb . By introducing the multi-universe optimization algorithm and the fitness function F Zeb to optimize the parameters, it not only reduces the time consumed by manually setting parameters to find the optimal segmentation effect but also ensures the final segmentation quality.

[0109] Table 1 Optimal parameters and optimal fitness in each direction of metal spray-painted surface corrosion images

[0110]

[0111] For the comparison of segmentation effects, in the selection of the comparison algorithm, the two-dimensional Otsu method is selected as the representative of the global threshold segmentation algorithm. Since the Sauvola method is a currently widely used classical neighborhood method, it can be used as the representative of the local threshold segmentation algorithm. In addition, the KittlerMe method, the improved two-dimensional Tsallis entropy method, the Niblack method and the method proposed in the present invention are also selected for experimental result comparative analysis.

[0112] Figure 6The three rows of images shown respectively represent three metal surface corrosion images and the segmentation effect images of different algorithms. Among them, in the Image1 image, there are several punctate corrosion areas of different sizes on the metal surface. At the same time, the metal surface has metal wire drawing texture and uneven illumination caused by the influence of light. In the comparative experiment, the KittlerMe method can only complete the segmentation of some larger corrosion areas. And due to the influence of uneven illumination, there is an under-segmentation phenomenon of corrosion areas in the upper half of the image. The two-dimensional Otsu method and the improved two-dimensional Tsallis entropy method are greatly affected by uneven illumination, and there is an over-segmentation phenomenon in the lower right part of the image, and the segmentation effect is poor. Although the Niblack method can completely segment the corrosion area, it does not eliminate the influence of the metal wire drawing texture and mis-segments the metal wire drawing texture as the corrosion area. And there are more noises in the segmentation result of the Sauvola method. The algorithm of this patent can reduce the influence of uneven illumination and the metal surface wire drawing texture phenomenon, complete the segmentation of the metal surface corrosion area, and the degree of detail retention is relatively high. In the Image2 image, there is a large area of corrosion on the metal surface. At the same time, there are also non-uniform illumination and metal wire drawing texture phenomena. In the comparative experiment, the two-dimensional Otsu method, the Niblack method and the Sauvola method are greatly affected by metal reflection and wire drawing texture, and there are a large amount of noises and textures left in the segmentation result, and the effective extraction of the corrosion area cannot be completed. Although the KittlerMe method better overcomes the influence of non-uniform illumination and wire drawing texture, it is difficult to segment small punctate corrosion areas, and the segmentation effect of this method is better. In the Image3 image, there are many large round punctate corrosion areas and wire drawing texture on the metal surface. At the same time, there is an obvious dark area in the lower part of the image. Limited by the global characteristics, the two-dimensional Otsu method and the two-dimensional Tsallis entropy method lose most of the corrosion point targets in the dark area in the segmentation result, as shown in (c) and (d) of the third row of the figure. Although the KittlerMe method greatly reduces the influence of uneven illumination and wire drawing texture, the edge details of the segmented target area are not complete enough, and the overall segmentation effect is not accurate enough. As for the Niblack method and the Sauvola method, although the segmentation effect has been improved and the details of the target area can be completely segmented, there are a large amount of noises and mis-segmentations of the metal wire drawing texture in the background area, as shown in (e) and (f) of the third row of the figure. The algorithm of this patent is more accurate in segmenting such corrosion areas, greatly reducing the mis-segmentation of noises and at the same time reducing the influence of uneven illumination on the segmentation result, and the processing of the target edge and details is better, as shown in (g) of the third row of the figure.

[0113] To quantitatively evaluate the detection performance of this patent compared with other algorithms, two common image segmentation quality evaluation metrics, the Intersection over Union (IOU) and the False Positive Rate (Error), are used as evaluation criteria. Among them, IOU is the intersection over union of the predicted target region and the target region in the standard binary mask image, that is, the ground truth map. In fact, it is also the ratio of the intersection of the segmentation result and the Groundtruth to their union, and the value range is within [0,1]. The larger the IOU, the higher the segmentation quality. Its calculation formula is:

[0114]

[0115] Error is the proportion of all negative examples mis-segmented as positive examples in the segmentation result, that is, the false detection rate. The value range is also within [0,1]. The smaller the Error, the higher the segmentation quality. Its calculation formula is:

[0116]

[0117] Table 2 Segmentation Quality Evaluation Metrics of Each Algorithm

[0118]

[0119]

[0120] According to the results in Table 2, by comprehensively comparing and analyzing the IOU intersection ratio and the Error false positive rate of the segmentation results of the method of the present invention and the segmentation results of the remaining algorithms, the segmentation performance of this method is better than that of the remaining algorithms. For example, in Image1, although neither the IOU nor the Error of this method is the optimal value, both indicators are only slightly inferior to the optimal values of all algorithms and are far better than the indicators of the remaining algorithms except the optimal values. Therefore, the method of this patent can effectively segment rust spot images of different sizes and different distributions, and has the highest detection accuracy.

[0121] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0122] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a means for implementing the functions specified in the flow Figure 1 one or more flows and / or blocks Figure 1 or means for implementing the functions specified in a block or multiple blocks.

[0123] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means for implementing the functions specified in the flow Figure 1 one or more flows and / or blocks Figure 1 or means for implementing the functions specified in a block or multiple blocks.

[0124] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the flow Figure 1 one or more flows and / or blocks Figure 1 or means for implementing the functions specified in a block or multiple blocks.

[0125] As described above, it is only a preferred embodiment of the present invention, and it is not a limitation of the present invention in other forms. Any person skilled in the art may use the disclosed technical content to make changes or modifications into equivalent embodiments with equivalent changes. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the technical solution content of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for detecting rust spots on a metal surface that integrates multi-directional and multi-universe local segmentation, characterized in that, First, collect the rust spot images on the surface of the metal to be detected, and perform median filtering and homomorphic filtering preprocessing on the images. Secondly, perform global multi-directional gray-scale scanning analysis on the images to determine the main gray-scale distribution curve and search for and locate the peak and valley points. Then, use the multi-universe optimization algorithm to optimize the fluctuation amplitude threshold and floating threshold parameters for peak-valley interval segmentation, and obtain the optimal local segmentation in each direction. Finally, use the principal component analysis method to compress and fuse the optimal local segmentation images in multiple directions to obtain the final rust spot detection result; The rust spot detection method for the metal surface includes the following steps: Step S1: First, collect the rust spot images on the surface of the metal to be detected, obtain the industrial images in the offline state, and then perform median filtering on the images. At the same time, set the parameters of homomorphic filtering according to the characteristics of the images and perform homomorphic filtering. Then, perform global multi-directional gray-scale change scanning on the images, calculate the mean square error K of the gray-scale change curve of each row of the images in each direction respectively, and select the gray-scale change curve with the largest mean square error K value under the gray-scale scanning in each direction as the main gray-scale distribution curve in each direction for subsequent analysis; Step S2, Initialize the parameters of the multi-universe optimization algorithm: the number of universes n, the number of objects in the universe d, the maximum number of iterations L, the probability of the existence of the wormhole WEP, the travel distance value TDR, and the exploitation degree p; regard the fluctuation amplitude threshold parameter H and the floating threshold parameter ξ as two objects in a single universe, and randomly initialize a multi-universe population U = [U1, U2, …, U n T ;​ Step S3: According to the fluctuation amplitude threshold parameter H and the floating threshold parameter ξ in each cosmic individual in the multi-universe population U, use the local threshold segmentation method for the peak-valley interval of the gray-scale fluctuation curve to search for and locate the peak and valley points and calculate the local threshold of the peak-valley interval on the main gray-scale distribution curve obtained in Step S1, and then perform local threshold segmentation of the peak-valley interval to obtain the segmentation results corresponding to each cosmic individual; Step S4: Evaluate the fitness of the segmentation results corresponding to each cosmic individual in the multi-universe population, calculate the fitness of the cosmic individual corresponding to the segmentation result, and obtain the cosmic individual corresponding to the minimum fitness as the current optimal universe by comparing the fitness values; Step S5: Enter the main loop, and update the probability of the existence of the wormhole WEP and the travel distance value TDR; Step S6: Execute the roulette mechanism to transfer the objects in the cosmic individual, then calculate the updated fitness value of the objects in each cosmic individual, select the cosmic individual with the minimum fitness value as the updated optimal universe and record the values of H and ξ of the objects in this optimal universe and their corresponding fitness values; if it is better than the current optimal universe, replace it, otherwise, retain the current optimal universe; Step S7: Judge the termination condition. If the number of iterations reaches the preset maximum number of iterations, exit the main loop and output the optimal universe and its fitness value. Otherwise, return to Step S4; Step S8: Output the local threshold segmentation results of the peak-valley interval corresponding to the two objects H and ξ in the optimal universe when the maximum number of iterations is reached; Step S9: Repeat Steps S2 - S8 for the main gray-scale distribution curves in each direction obtained in Step S1 in turn, output the optimal universe and its fitness value corresponding to the main gray-scale distribution curve in each direction, and the local threshold segmentation results of the peak-valley interval corresponding to the optimal universe when the maximum number of iterations is reached in each direction; Step S10: Perform PCA-based image compression on each of the segmentation results I0, I 45 , I 90 , I 135 , I 180 , I 225 , I 270 , I 315 , and output the final segmentation result after compression of each directional segmentation result, which is the rust spot detection result.

2. The method for detecting rust spots on a metal surface by fusing multi-directional and multi-universe local segmentation according to claim 1, wherein, Before image segmentation, first perform median filtering using a median filter template with a kernel size of 3*3 to filter out salt-and-pepper noise in the image. Secondly, use homomorphic filtering. By setting reasonable parameters, multiplicative noise and a large amount of useless low-frequency information in the target image are removed, the contrast of the rust target and the high-frequency components of the edge details are enhanced, and the accuracy of subsequent segmentation is further improved.

3. The method for detecting rust spots on a metal surface by fusing multi-directional and multi-universe local segmentation according to claim 1, wherein Each image is regarded as a two-dimensional grayscale function composed of pixel point coordinates and pixel point grayscale values, and in a three-dimensional coordinate system, it is intuitively represented as a three-dimensional topographic map. The continuous mountains and valleys in the topographic map are compared with the three-dimensional map of the image. The mountains correspond to the peaks in the three-dimensional map of the image, that is, the regions with higher grayscale values in the image, and the valleys correspond to the troughs in the three-dimensional map of the image, that is, the regions with lower grayscale values in the image. In the rust image on the metal surface, the rust, as a region with low grayscale values in the image and also the target region to be segmented and extracted, forms a sharp contrast with the background region. It is represented as a trough in the three-dimensional map of the image, while the background region is represented as a peak, and the two together form a grayscale extreme value region. Based on this characteristic, through the local threshold segmentation method of the peak-valley interval of the multi-directional grayscale fluctuation curve of the image peak and valley, a threshold segmentation point is selected between the peak and valley points of each neighborhood screened out to completely segment and extract the rust target information details in the image. This method first performs a global multi-directional grayscale scan analysis on the rust grayscale image on the metal surface from eight directions: 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°, that is, the horizontal, vertical, and two diagonal directions of the image. Then, according to the analysis results, the peak and valley points of the main grayscale distribution curves in each direction are searched and screened. Finally, a threshold is selected in the grayscale peak-valley interval for local threshold segmentation, and finally, the crack threshold segmentation images in eight directions are obtained. For the global multi-directional grayscale change scan of the grayscale image, first represent the image in the form of a two-dimensional array. Let the specific number of rows in the array be y = y0, and the specific number of columns be x, where x = 1, 2, 3,..., m, and m is the total number of pixel points in this row of the image, that is, the total number of columns of the image. The grayscale value of the image is f, then f0 = f(x, y0) represents a one-dimensional function with the number of columns x of the image as the independent variable and the grayscale value f as the dependent variable for a specific row y0 of the image, which is reflected as the grayscale distribution curve of a specific row of the image in the rectangular coordinate system. To reduce the amount of calculation, only one group of grayscale distribution curves is selected as the main grayscale distribution curve in this direction to participate in subsequent analysis and processing. To determine the main grayscale distribution curve, first introduce the mean square deviation of grayscale change that measures the change amplitude of the curve. By calculating the mean square deviation of each group of grayscale distribution curves in the horizontal direction, the curve with the largest change amplitude, that is, the curve with the largest mean square deviation of grayscale, is selected as the main grayscale distribution curve in this direction. The definition of the mean square deviation K of grayscale change is: Among them, K i represents the mean square deviation of the gray level change of the gray level distribution curve of the i-th row in the horizontal direction, f i (x, y i ) represents the gray level value of the pixel at the x-th column of the gray level distribution curve of the specific row y i , where i = 1, 2, 3,..., n, and n is the total number of rows of the image; f i (k) is the average gray level of the gray level distribution curve of the i-th row of the image, and m is the number of gray level distribution pixel points of the i-th row; By performing a global eight-direction grayscale scan of the image, calculating and comparing the mean square deviation K of grayscale change of each group of grayscale change curves in each direction, the grayscale distribution curve with the largest mean square deviation K value of grayscale change in each direction is obtained, that is, the main grayscale distribution curve in each direction. The method for searching and locating peak and valley points on the main gray-scale distribution curve is as follows: First, search for potential peak and valley points. For a continuous one-dimensional function curve, its peak and valley points are the extreme points of the curve. Among them, the points that satisfy the first derivative being 0 and the second derivative being greater than 0 are the minimum points, i.e., valley points; the points that satisfy the first derivative being 0 and the second derivative being less than 0 are the maximum points, i.e., peak points. However, since the gray-scale distribution curve is not a continuous curve but a discrete curve, forward and backward difference operations are used. Whether a point is a potential peak or valley point is judged by the sign of the result obtained from the differential calculation of the gray value of a certain pixel point. The judgment conditions are as follows: Among them, k is the specific serial number of the pixel point on the gray-scale distribution curve, f(k) is the gray-scale value of the gray-scale distribution curve at the specific pixel point serial number k, f(k i-1 ) and f(k i+1 ) are the gray-scale values of the pixel point before and after the pixel point corresponding to the specific pixel point serial number k i ; By searching for potential peak and valley points on the main gray-level distribution curve, the positions and corresponding gray-level values of all potential peak and valley points on the curve are obtained; assume that E top(i) (i = 1, 2, ..., m) are all the maximum points on the curve, and there is E top(i) [k i , f(k i )] represents the i-th maximum point on the curve, where k i is the pixel serial number of the i-th maximum point, and f(k i ) is the gray-level value corresponding to the pixel serial number k i of the i-th maximum point, and m is the number of all maximum points; E bottom(j) (j = 1, 2, ..., n) are all the minimum points on the curve, and there is E bottom(j) [k j , f(k j )] represents the j-th minimum point on the curve, where k j is the pixel serial number of the j-th minimum point, and f(k j ) is the gray-level value corresponding to the pixel serial number k j of the j-th minimum point, and n is the number of all minimum points; After completing the search for potential peak and valley points, before locating the large-scale peak and valley points among the potential peak and valley points, first determine the first peak and valley point as the initial iterative search condition. To determine the first peak and valley point, compare the pixel serial numbers of the first maximum point and the first minimum point to obtain the first extreme point on the gray-scale distribution curve, and then select it as the first peak and valley point. If the first extreme point is a maximum point, select it as the first peak point; if the first extreme point is a minimum point, select it as the first valley point. Using the first peak and valley point as the initial iterative condition, iteratively search for the next peak and valley point in turn, and the positioning of all peak and valley points on the gray-scale distribution curve can be completed. Assume that the previous point is the first peak point or the previous peak point T that satisfies the iterative search condition k = E top(m) [k m , f(k m )], then the next valley point B k = E bottom(j) [k j , f(k j )] should satisfy the following iterative search conditions: Among them, H ∈ (0, 255) is the threshold condition for the gray-scale fluctuation amplitude. However, to avoid rounding errors during image calculation and ensure calculation accuracy, it is mapped to the interval (0, 1), that is, H ∈ (0, 1), E bottom(m) [k m , f(k m )] is the peak point E top(m) [k m , f(k m )] is a minimum point after the peak point E top(i) [k i , f(k i )] is the valley point E bottom(j) [k j , f(k j )] is a maximum point after the valley point, then E bottom(i-1) [k i-1 , f(k i-1 )] is the minimum point before the maximum point; f bottom (k j ) is the gray-scale value of the valley point, f top (k j+1 ), f top (k j ) are respectively the gray-scale values of the maximum point after the valley point and the maximum point before the valley point; Assume that the previous point is the first valley point or the previous valley point B that satisfies the iterative search condition k = E bottom(n) [k n , f(k n )], then the next peak point T k = E top(i) [k i , f(k i )] satisfies the following iterative search conditions: Among them, E top(n) [k n ,f(k n )] is the valley point E bottom(n) [k n ,f(k n )] after a maximum point, E bottom(j) [k j ,f(k j )] is the peak point E top(i) [k i ,f(k i )] is a minimum point after E top(j-1) [k j-1 ,f(k j-1 )] is the maximum point before the minimum point; f top (k i ) is the peak gray value, f bottom (k i+1 ), f bottom (k i ) are the grayscale values of the minimum point after the peak point and the grayscale value of the previous minimum point respectively; in the above formula, f top (k i ) and f bottom (k i ) i The values of are not the same, f top (k i ) in k i represents the pixel number corresponding to the i-th maximum point, and f bottom (k i ) in k i Indicates the pixel number corresponding to the i-th minimum point; The method for calculating the local threshold in the peak-valley interval on the main gray-scale distribution curve is as follows: Based on the peak-valley point sequence obtained by search and positioning, the peak-valley sequence is sequentially divided into several peak-valley and valley-peak intervals in the order of peak and valley points, that is, there are several peak-valley intervals; Let bottom(m) [k m , f(k m )], E top(m) [k m , f(k m )]] be one of the valley-peak intervals, and the local threshold calculation for this interval is as follows: threshold = ξ × [f bottom (k m ) - f top (k m )] + f top (k m ) × (1 - ξ) (6) Among them, f top (k m ) is the gray value of the peak point in the interval, f bottom (k m ) is the gray value of the valley point in the interval, ξ is the floating threshold parameter, and by setting the floating parameter, the thickness of the target is controlled; assume that the pixel point number of the peak point is k top , the pixel point number of the valley point is k bottom , and the pixel point (k, f(k)) is a certain pixel in this interval, then k ∈ (k top , k bottom ). The following determination method is used to perform binary segmentation of the crack target in this interval: When f(k) > threshold, the pixel point (k, f(k)) belongs to the target, and when f(k) < threshold, the pixel point (k, f(k)) belongs to the background. Therefore, by traversing several peak-valley and valley-peak intervals to calculate the local threshold and performing local threshold segmentation on the corresponding intervals, local threshold segmentation of the image in the peak-valley interval during horizontal gray-scale scanning can be achieved. The local thresholds for the intervals before the first extreme point and after the last extreme point take the thresholds of their adjacent intervals.

4. The method for detecting rust spots on a metal surface by fusing multi-directional and multi-universe local segmentation according to claim 3, wherein In the multi-universe optimization algorithm, a universe is a solution to the optimization problem. Each object in a single universe is regarded as the position vector of the corresponding solution, and the expansion rate of each universe is proportional to the fitness of the corresponding solution. The initialization of the universe as a solution is used to start the optimization process. Each update of each object in the universe, that is, each set of solutions, is executed through progressive iterative rules. The random initialization U of the multi-universe population is defined by the following formula: Among them, n represents the number of universes, corresponding to the population size; d represents the number of individuals in the universe, corresponding to the dimension of the search space; represents the i-th universe, represents the j-th object in the i-th universe; Due to the different expansion rates of each universe individual, that is, different fitness values, the objects in the universe are selected and updated through the roulette wheel mechanism and then transferred through the white hole / black hole orbit. The formula is as follows: where NI(U i ) represents the normalized expansion rate of the i-th universe, r1 is a random number in the range of [0,1], represents the j-th object of the k-th universe selected by the roulette mechanism; Without considering the magnitude of the expansion rate, in order for a universe individual to achieve local changes and improve its own expansion rate, it will stimulate the internal objects to move towards the current optimal universe. This process is executed according to the following formula: where x j represents the j-th object of the current optimal universe, lb and ub respectively refer to the lower and upper bounds of x, r2, r3, and r4 are random numbers in the range of [0, 1], WEP represents the wormhole existence probability, and TDR represents the travel distance value. These two important parameters respectively represent the probability of the existence of a wormhole in the multiverse space and the step length for an object to move towards the current optimal universe; its update principle follows the formula as follows: where t is the current iteration number and T is the maximum iteration number.

5. The method for detecting rust spots on a metal surface by fusing multi-directional and multi-universe local segmentation according to claim 4, characterized in that, To optimize the fluctuation amplitude threshold H and the floating threshold parameter ξ through the multiverse optimization algorithm to obtain the best segmentation result, the unsupervised segmentation quality evaluation index Zeb is used as the fitness function of the algorithm; assuming that W(s) is the neighborhood pixel of pixel s, f(s) is the gray value of pixel s, and L is the maximum gray value of the pixel, then the intra-region contrast I i The calculation formula is as follows: contrast(s,t)=|f(s)-f(t)| (13) Among them, I i represents the internal contrast of region i, A i is the total number of pixels in region i, R i is the set of all pixels in region i, t is the intersection pixel of the neighborhood pixel set W(s) of pixel s and the set R i of all pixels in region i, and f(s) and f(t) are the gray values of pixel s and pixel t respectively; to obtain the best segmentation result with higher segmentation quality, the following fitness function is constructed: where n is the class label value of the connected region in the segmented result image, that is, the number of connected regions.

6. The method for detecting rust spots on a metal surface by fusing multi-directional and multi-universe local segmentation according to claim 5, characterized in that, A method for local threshold segmentation of the peak-valley interval of a multi-directional grayscale fluctuation curve that optimizes parameters using a combined multiverse optimization algorithm. Sub-images in eight directions of 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315° of the target image are selected and local threshold segmentation is performed respectively to obtain crack segmentation binary images I0, I 45 , I 90 , I 135 , I 180 , I 225 , I 270 , I 315 ; The PCA algorithm is used to compress and fuse the 8 binary images. The 8-dimensional vector I = {I0, I 45 , I 90 , I 135 , I 180 , I 225 , I 270 , I 315} composed of 8 images is compressed to 1 dimension {I'}; First, the 8 binary images I0, I 45 , I 90 , I 135 , I 180 , I 225 , I 270 , I 315 are respectively converted into one-dimensional vectors, and these one-dimensional vectors are used as samples to form a sample matrix Z = {z1, z2,..., z n} of an 8-dimensional vector, where n is the total number of pixels in the binary image. Then, the mean vector μ of Z is calculated, and each row of Z is centered: The calculated centralized matrix After that, calculate the matrix The eigen covariance matrix C of, perform SVD transformation on the covariance matrix C to calculate the eigenvector matrix U{ξ0, ξ1, ξ2, ξ3, ξ4, ξ5, ξ6, ξ7} and eigenvalues {λ0, λ1, λ2, λ3, λ4, λ5, λ6, λ7} of the covariance matrix, where λ0 > λ1 > λ2 > λ3 > λ4 > λ5 > λ6 > λ7; sort the eigenvalues from smallest to largest, and select the eigenvector corresponding to the largest eigenvalue for data dimensionality reduction: Z' = Z T ξ0 (17) Among them, Z' is the sample matrix of the one-dimensional vectors after compressing 8 images. By re-converting it to the original image space, the gray-scale matrix I' of the compressed image, that is, the compressed image, can be obtained. Using the PCA algorithm for image compression largely preserves the non-overlapping detailed information of each of the 8 binary images, and at the same time compresses the image from 8 dimensions to 1 dimension. The segmentation results in each direction are all local threshold segmentation results obtained by performing gray-scale fluctuation analysis after optimizing the parameters H and ξ using the multi-universe optimization algorithm.

Images