Superpixel clustering method and device for infrared image

Abstract

The invention discloses a superpixel clustering method and device for an infrared image, and the method comprises the steps: collecting an original infrared image, converting the original infrared image into a Lab color space, and obtaining a Lab color mode image; extracting a region of interest from the Lab color mode image; calculating the gradient of each feature element of each pixel of the Lab color mode image by adopting a Sobel operator to obtain a gradient value of each pixel; setting an initial clustering number, and performing initial clustering on the Lab color mode image by adopting a non-iterative clustering method according to the gradient value and the initial clustering number to obtain an initial super-pixel clustering result; updating the clustering number according to the initial super-pixel clustering result; performing secondary clustering on the region of interest by adopting an iterative clustering method according to the gradient value of the pixels in the region of interest and the updated clustering number to obtain a fine super-pixel clustering result of the region of interest; and finally, fragmentary superpixel combination is carried out. According to the method, the accuracy of super-pixel clustering can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a superpixel clustering method and device for infrared images. Background Art

[0002] The stable operation of substation equipment is crucial to ensure the safety and reliability of public power supply. However, due to long-term live operation and material aging, substation equipment may generate heat and high-temperature failures, which may lead to circuit breakage and equipment damage. Therefore, using infrared thermal imagers to collect infrared images for regular inspections is a key step to ensure the safe operation of the power system.

[0003] The infrared images collected by thermal infrared instruments have low resolution and are greatly affected by the environment. The data of the same data point may fluctuate, which will cause inaccuracies when the infrared images are registered and fused with other data. To address this problem, superpixel clustering is generally used in the prior art to solve this problem.

[0004] Superpixel clustering is a key technology in image processing. It clusters images into regions with similar features, which is of great significance for subsequent image analysis tasks. Existing superpixel clustering can be mainly divided into traditional superpixel clustering methods and superpixel clustering methods based on deep learning. Traditional superpixel clustering methods can also be simply divided into computer graphics calculation methods and gradient-based clustering methods. The first type mainly uses computer graphics methods such as corrosion, expansion, opening and closing operations to iteratively calculate pixels in the image. This method usually uses the neighboring relationship of pixels as the edge retrieval condition, converting superpixel clustering into an optimization problem of graph connectivity partitioning. Generally, a large number of iterative operations are required and the weight parameters of some operators are extremely sensitive. The second method is mainly derived from the idea of ​​Kmeans clustering. Its representative method is Simple Linear Iterative Clustering (SLIC). After selecting the initial number of clusters, it updates the cluster center according to the distance metric function and optimizes the clustering results until the number of iterations is met. This method is simple in idea, low in complexity and only requires a constant number of iterations.

[0005] Superpixel clustering based on deep learning is mostly realized by relying on the powerful feature extraction effect of CNN for pictures. First, CNN is used to extract features, and then a clustering method is used to perform superpixel clustering according to the features. In essence, after CNN expands the data feature dimension, the traditional clustering method is used to further complete the superpixel clustering. In addition, there is also a method in the prior art that combines deep learning with the idea of ​​SLIC. For example, the patent text (CN107316289B) discloses a field rice ear clustering method based on deep learning and superpixel clustering. The method uses a simple linear iterative clustering method in superpixel clustering technology to form image blocks, i.e., superpixels, with adjacent pixels having similar features. On the basis of automatic annotation and selection of large-scale training samples, the convolutional neural network in the deep learning technology discriminates the superpixel category to realize the preliminary clustering of rice ears, and the preliminary clustering results are optimized by the superpixel clustering method based on the entropy rate. It also retains the idea of ​​SLIC to iteratively update the cluster center for many times, but requires a large amount of manual annotation, and is not significantly optimized in terms of time. However, these deep learning methods do not constrain the connectivity of superpixels, so separated and fragmented superpixels may be generated. In order to ensure the connectivity of superpixels, additional post-processing steps need to be added to merge the fragmented superpixels, and this post-processing step is often independent of the deep learning framework. Therefore, in order to save the post-processing step of merging superpixels, a superpixel interpolation network (SIN) is proposed in the prior art, which generates superpixel results of initial resolution according to the number of superpixels, and generates superpixel clustering results with the same resolution as the original image by alternating interpolation in the horizontal and vertical directions. This method can avoid the generation of fragmented superpixels and ensure timeliness, but the results of superpixel clustering are not ideal. The above methods are all supervised learning methods, which require a large amount of data processing and annotation in advance, and require a large amount of data for training to achieve the generalization of the model.

[0006] In addition, there is also the unsupervised generation of superpixels using CNN in the prior art. This method regards the image superpixel clustering task as a classification task, performs classification tasks of different superpixel modules for each pixel, and constructs a loss function using image reconstruction loss and cross entropy loss to remove the dependence on labels. The LNS (Learning Superpixel in a Non-Iterative and Lifelong Manner) method also implements an unsupervised, non-iterative, lightweight superpixel clustering network based on CNN. However, the above unsupervised methods strongly rely on a large amount of data training. If the input image has large differences in features from the training set, the final performance of the model will be greatly challenged.

[0007] In summary, due to the low resolution, blurry images, and high noise of infrared images collected by infrared thermal imagers, as well as the complex structure of substation equipment, traditional superpixel clustering methods often have difficulty accurately capturing edge information, resulting in distorted clustering results and low clustering accuracy. Deep learning methods do not constrain the connectivity of superpixels, so they may produce separated and fragmented superpixels. In addition, both supervised and unsupervised learning methods require large amounts of data for training to achieve model generalization, and the calculation speed is slow. Summary of the invention

[0008] The present invention provides a superpixel clustering method and device for infrared images, which can ensure the accuracy of superpixel clustering of infrared images.

[0009] A superpixel clustering method for infrared images, comprising:

[0010] Collecting an original infrared image, and converting the original infrared image into a Lab color space to obtain a Lab color mode image;

[0011] Extracting a region of interest from the Lab color mode image;

[0012] Using the Sobel operator to calculate the gradient of each characteristic element of each pixel of the Lab color mode image to obtain the gradient value of each pixel;

[0013] Setting an initial number of clusters, and performing initial clustering on the Lab color mode image using a non-iterative clustering method according to the gradient value and the initial number of clusters to obtain an initial superpixel clustering result;

[0014] Updating the number of clusters according to the initial superpixel clustering result;

[0015] According to the gradient value of the pixels in the region of interest and the updated number of clusters, the region of interest is clustered again using an iterative clustering method to obtain a fine superpixel clustering result of the region of interest;

[0016] Based on the initial superpixel clustering results and the refined superpixel clustering results of the region of interest, fragmented superpixels are merged to obtain the final clustering result.

[0017] Further, extracting a region of interest from the Lab color mode image includes:

[0018] According to the preset a channel hue threshold and b channel hue threshold, each pixel of the Lab color mode image is traversed, and pixels whose a channel value and b channel value are greater than the a channel hue threshold and the b channel hue threshold are selected as the region of interest.

[0019] Furthermore, the gradient of each characteristic element of each pixel of the Lab color mode image is calculated using a Sobel operator, including:

[0020] Using the Sobel operator to perform gradient calculation on the brightness, a channel and b channel of each pixel of the Lab color mode image to obtain a brightness gradient value and a hue gradient value;

[0021] According to the brightness space range and the hue space range, the brightness gradient value and the hue gradient value are normalized and merged to obtain the gradient value of the pixel.

[0022] Further, performing initial clustering on the Lab color mode image using a non-iterative clustering method according to the gradient value and the initial cluster number to obtain an initial superpixel clustering result includes:

[0023] Set K initial first cluster centers and establish K queues, where K is the number of initial clusters;

[0024] Search for pixels in the neighborhood of the initial first cluster center, and calculate the distance between the pixel and the initial first cluster center according to the gradient value of the searched pixel, combine the pixel information and the distance information into elements, and add the elements to the corresponding queue in order from small to large distances;

[0025] The initial first cluster center is used as the initial superpixel;

[0026] Extract the pixel corresponding to the first element from each queue and add it to the corresponding superpixel, release the remaining elements in the queue, and mark the pixels added to the superpixel;

[0027] The following first clustering operation is performed in a loop until all pixels are assigned to corresponding superpixels: the average value of the distance between each pixel in the superpixel and the first cluster center is calculated as the updated first cluster center, unmarked pixels in the neighborhood of the updated first cluster center are searched, the distance between the pixel and the updated first cluster center is calculated according to the gradient value of the searched pixel, pixel information and distance information are combined into elements, and the elements are added to corresponding queues in ascending order of distance, the pixel corresponding to the first element is extracted from each queue and added to the corresponding superpixel, and the remaining elements in the queue are released, and the pixels added to the superpixel are marked.

[0028] Further, the obtained initial superpixel clustering results include the initial superpixel clustering results of the region of interest and the initial superpixel clustering results of the region of no interest; the updated number of clusters is the number of initial superpixels in the region of interest;

[0029] According to the gradient value of the pixels in the region of interest and the updated number of clusters, the region of interest is clustered again using an iterative clustering method to obtain the fine superpixel clustering results of the region of interest, including:

[0030] Set K' initial second cluster centers, where K' is the number of initial superpixels in the region of interest;

[0031] According to the size of the region of interest, determining the distance between adjacent initial second cluster centers;

[0032] Obtain the gradient values ​​of the pixels in the neighborhood of the initial second cluster center in the region of interest, and select the pixel with the smallest gradient value as the updated second cluster center:

[0033] The updated second cluster center is used as the initial superpixel of the iteration;

[0034] The following second clustering operation is cyclically performed in the region of interest until all pixels are assigned to corresponding superpixels: searching for unlabeled pixels in the neighborhood of the updated second cluster center, calculating the distance between the pixel and the updated second cluster center according to the gradient value of the searched pixel, selecting the pixel with the smallest distance to add to the superpixel and labeling the pixel, and calculating the average value of the distance between each pixel in the superpixel and the second cluster center as the updated second cluster center;

[0035] After each round of the second clustering operation, it is determined whether a stop condition is met. If the stop condition is met, the calculation is stopped. If the stop condition is not met, the second clustering operation is executed again in a loop.

[0036] The stopping condition is that the distance between each current second cluster center and the corresponding second cluster center in the previous round of calculation results is less than a preset value.

[0037] Further, calculating the distance between the pixel and the first cluster center according to the gradient value of the searched pixel includes:

[0038] Calculate a first brightness and hue difference based on the brightness, a channel, and b channel of the searched pixel and the first cluster center;

[0039] Calculate the first Euclidean distance based on the pixel coordinates of the searched pixel and the first cluster center;

[0040] Adding the square value of the first brightness and tone difference and the square value of the gradient value of the searched pixel to obtain a first sum value;

[0041] The distance between the searched pixel and the first cluster center is calculated based on the first sum value, the first Euclidean distance, the relevant balance coefficient, and the distance between adjacent first cluster centers.

[0042] Further, the distance between the pixel and the second cluster center is calculated according to the gradient value of the searched pixel, including:

[0043] Calculate the second brightness and hue difference according to the brightness, a channel and b channel of the searched pixel and the second cluster center;

[0044] Calculate the second Euclidean distance based on the pixel coordinates of the searched pixel and the second cluster center;

[0045] Adding the square value of the second brightness and tone difference and the square value of the gradient value of the searched pixel to obtain a second sum value;

[0046] The distance between the searched pixel and the second cluster center is calculated according to the second sum value, the second Euclidean distance, the relevant balance coefficient, and the distance between adjacent second cluster centers.

[0047] Furthermore, fragmented superpixels are merged based on the initial superpixel clustering results and the refined superpixel clustering results of the region of interest to obtain the final clustering results, including:

[0048] Merging the initial superpixel clustering result of the non-interest area and the refined superpixel clustering result of the interest area to obtain a merged clustering result;

[0049] Setting an adjustable parameter, and calculating a superpixel threshold according to the adjustable parameter, the size of the original infrared image, and the number of initial clusters;

[0050] Comparing the number of pixels in each superpixel in the merged clustering result with the superpixel threshold to determine fragmented superpixels;

[0051] Traversing all superpixels in the merged clustering result, and calculating the average brightness value of all superpixels;

[0052] Traversing the fragmented superpixels, and calculating the similarity between the fragmented superpixels and the superpixels in their neighborhood according to the average brightness value;

[0053] If the similarity between the fragmented superpixels and the superpixels in the neighborhood unit is greater than a preset threshold, they are merged;

[0054] The final clustering result is obtained after merging all fragmented superpixels.

[0055] Furthermore, the similarity between a fragmented superpixel and the superpixels in its neighborhood is calculated using the following formula:

[0056]

[0057] Among them, DLightn Indicates the similarity between a fragmented superpixel and the nth superpixel in its neighborhood, μ n represents the average brightness value of the nth superpixel within the neighborhood of the fragmented superpixel, and μ represents the average brightness value of the fragmented superpixel.

[0058] A superpixel clustering device for infrared images, comprising:

[0059] An acquisition module is used to acquire an original infrared image and convert the original infrared image into a Lab color space to obtain a Lab color mode image;

[0060] An extraction module, used for extracting a region of interest from the Lab color mode image;

[0061] A gradient calculation module, used to calculate the gradient of each characteristic element of each pixel of the Lab color mode image using a Sobel operator to obtain a gradient value of each pixel;

[0062] An initial clustering module, used to set the initial number of clusters, and perform initial clustering on the Lab color mode image using a non-iterative clustering method according to the gradient value and the initial number of clusters to obtain an initial superpixel clustering result;

[0063] An updating module, used for updating the number of clusters according to the initial superpixel clustering result;

[0064] A secondary clustering module is used to perform secondary clustering on the region of interest using an iterative clustering method according to the gradient values ​​of the pixels in the region of interest and the updated number of clusters, so as to obtain a fine superpixel clustering result of the region of interest;

[0065] The merging module is used to merge fragmented superpixels based on the initial superpixel clustering results and the refined superpixel clustering results of the region of interest to obtain the final clustering results.

[0066] Furthermore, the extraction module extracts a region of interest from the Lab color mode image, comprising:

[0067] According to the preset a channel hue threshold and b channel hue threshold, each pixel of the Lab color mode image is traversed, and pixels whose a channel value and b channel value are greater than the a channel hue threshold and the b channel hue threshold are selected as the region of interest.

[0068] Furthermore, the gradient calculation module calculates the gradient of each characteristic element of each pixel of the Lab color mode image using a Sobel operator, including:

[0069] Using the Sobel operator to perform gradient calculation on the brightness, a channel and b channel of each pixel of the Lab color mode image to obtain a brightness gradient value and a hue gradient value;

[0070] According to the brightness space range and the hue space range, the brightness gradient value and the hue gradient value are normalized and merged to obtain the gradient value of the pixel.

[0071] Furthermore, the initial clustering module performs initial clustering on the Lab color mode image using a non-iterative clustering method according to the gradient value and the initial cluster number to obtain an initial superpixel clustering result, including:

[0072] Set K initial first cluster centers and establish K queues, where K is the number of initial clusters;

[0073] Search for pixels in the neighborhood of the initial first cluster center, and calculate the distance between the pixel and the initial first cluster center according to the gradient value of the searched pixel, combine the pixel information and the distance information into elements, and add the elements to the corresponding queue in order from small to large distances;

[0074] The initial first cluster center is used as the initial superpixel;

[0075] Extract the pixel corresponding to the first element from each queue and add it to the corresponding superpixel, release the remaining elements in the queue, and mark the pixels added to the superpixel;

[0076] The following first clustering operation is performed in a loop until all pixels are assigned to corresponding superpixels: the average value of the distance between each pixel in the superpixel and the first cluster center is calculated as the updated first cluster center, unmarked pixels in the neighborhood of the updated first cluster center are searched, the distance between the pixel and the updated first cluster center is calculated according to the gradient value of the searched pixel, pixel information and distance information are combined into elements, and the elements are added to corresponding queues in ascending order of distance, the pixel corresponding to the first element is extracted from each queue and added to the corresponding superpixel, and the remaining elements in the queue are released, and the pixels added to the superpixel are marked.

[0077] Further, the obtained initial superpixel clustering results include the initial superpixel clustering results of the region of interest and the initial superpixel clustering results of the region of no interest; the updated number of clusters is the number of initial superpixels in the region of interest;

[0078] The secondary clustering module performs secondary clustering on the region of interest using an iterative clustering method according to the gradient value of the pixels in the region of interest and the updated number of clusters, to obtain a fine superpixel clustering result of the region of interest, including:

[0079] Set K' initial second cluster centers, where K' is the number of initial superpixels in the region of interest;

[0080] According to the size of the region of interest, determining the distance between adjacent initial second cluster centers;

[0081] Obtain the gradient values ​​of the pixels in the neighborhood of the initial second cluster center in the region of interest, and select the pixel with the smallest gradient value as the updated second cluster center:

[0082] The updated second cluster center is used as the initial superpixel of the iteration;

[0083] The following second clustering operation is cyclically performed in the region of interest until all pixels are assigned to corresponding superpixels: searching for unlabeled pixels in the neighborhood of the updated second cluster center, calculating the distance between the pixel and the updated second cluster center according to the gradient value of the searched pixel, selecting the pixel with the smallest distance to add to the superpixel and labeling the pixel, and calculating the average value of the distance between each pixel in the superpixel and the second cluster center as the updated second cluster center;

[0084] After each round of the second clustering operation, it is determined whether a stop condition is met. If the stop condition is met, the calculation is stopped. If the stop condition is not met, the second clustering operation is executed again in a loop.

[0085] The stopping condition is that the distance between each current second cluster center and the corresponding second cluster center in the previous round of calculation results is less than a preset value.

[0086] Furthermore, the initial clustering module calculates the distance between the pixel and the first cluster center according to the gradient value of the searched pixel, including:

[0087] Calculate a first brightness and hue difference based on the brightness, a channel, and b channel of the searched pixel and the first cluster center;

[0088] Calculate the first Euclidean distance based on the pixel coordinates of the searched pixel and the first cluster center;

[0089] Adding the square value of the first brightness and tone difference and the square value of the gradient value of the searched pixel to obtain a first sum value;

[0090] The distance between the searched pixel and the first cluster center is calculated based on the first sum value, the first Euclidean distance, the relevant balance coefficient, and the distance between adjacent first cluster centers.

[0091] Furthermore, the secondary clustering module calculates the distance between the pixel and the second cluster center according to the gradient value of the searched pixel, including:

[0092] Calculate the second brightness and hue difference according to the brightness, a channel and b channel of the searched pixel and the second cluster center;

[0093] Calculate the second Euclidean distance based on the pixel coordinates of the searched pixel and the second cluster center;

[0094] Adding the square value of the second brightness and tone difference and the square value of the gradient value of the searched pixel to obtain a second sum value;

[0095] The distance between the searched pixel and the second cluster center is calculated according to the second sum value, the second Euclidean distance, the relevant balance coefficient, and the distance between adjacent second cluster centers.

[0096] Furthermore, the merging module performs fragmented superpixel merging based on the initial superpixel clustering result and the refined superpixel clustering result of the region of interest to obtain a final clustering result, including:

[0097] Merging the initial superpixel clustering result of the non-interest area and the refined superpixel clustering result of the interest area to obtain a merged clustering result;

[0098] Setting an adjustable parameter, and calculating a superpixel threshold according to the adjustable parameter, the size of the original infrared image, and the number of initial clusters;

[0099] Comparing the number of pixels in each superpixel in the merged clustering result with the superpixel threshold to determine fragmented superpixels;

[0100] Traversing all superpixels in the merged clustering result, and calculating the average brightness value of all superpixels;

[0101] Traversing the fragmented superpixels, and calculating the similarity between the fragmented superpixels and the superpixels in their neighborhood according to the average brightness value;

[0102] If the similarity between the fragmented superpixels and the superpixels in the neighborhood unit is greater than a preset threshold, they are merged;

[0103] The final clustering result is obtained after merging all fragmented superpixels.

[0104] Furthermore, the similarity between a fragmented superpixel and the superpixels in its neighborhood is calculated using the following formula:

[0105]

[0106] Among them, DLight n Indicates the similarity between a fragmented superpixel and the nth superpixel in its neighborhood, μ nrepresents the average brightness value of the nth superpixel within the neighborhood of the fragmented superpixel, and μ represents the average brightness value of the fragmented superpixel.

[0107] The superpixel clustering method and device for infrared images provided by the present invention have at least the following beneficial effects:

[0108] (1) Based on the non-iterative clustering method, the infrared image is initially clustered to obtain the initial superpixel division result, and then the region of interest is secondary clustered through an iterative method to perform a more refined superpixel division of the region of interest, while ensuring the accuracy of the superpixel clustering of the region of interest and improving the overall calculation speed;

[0109] (2) In the initial clustering and secondary clustering process, the gradient value of the pixel is introduced to calculate the cluster distance, so as to distinguish "adjacent" pixels with "large gradient changes" as much as possible, and avoid dividing the above situation into one superpixel, which can effectively enhance the edge information and thus improve the accuracy of superpixel clustering; that is, this method can distinguish the detail information of the image to a certain extent, meeting the requirements for detail division of infrared temperature images of electrical equipment;

[0110] (3) The fragmented superpixels are merged based on similarity to further improve the accuracy of superpixel clustering. BRIEF DESCRIPTION OF THE DRAWINGS

[0111] Figure 1 The present invention is a flowchart of an embodiment of a superpixel clustering method for infrared images provided by the present invention.

[0112] Figure 2 A schematic structural diagram of an embodiment of a superpixel clustering device for infrared images provided by the present invention. DETAILED DESCRIPTION

[0113] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0114] refer to Figure 1 In some embodiments, a superpixel clustering method for infrared images is provided, comprising:

[0115] S1, collecting an original infrared image, and converting the original infrared image into a Lab color space to obtain a Lab color mode image;

[0116] S2, extracting a region of interest from the Lab color mode image;

[0117] S3, using the Sobel operator to calculate the gradient of each characteristic element of each pixel of the Lab color mode image to obtain the gradient value of each pixel;

[0118] S4, setting the initial number of clusters, and performing initial clustering on the Lab color mode image using a non-iterative clustering method according to the gradient value and the initial number of clusters to obtain an initial superpixel clustering result;

[0119] S5. updating the number of clusters according to the initial superpixel clustering result;

[0120] S6, performing secondary clustering on the region of interest using an iterative clustering method according to the gradient values ​​of the pixels in the region of interest and the updated number of clusters, to obtain a fine superpixel clustering result of the region of interest;

[0121] S7. Based on the initial superpixel clustering result and the refined superpixel clustering result of the region of interest, fragmented superpixels are merged to obtain the final clustering result.

[0122] Specifically, in step S1, the original infrared image of the RGB channel is converted to the Lab color space to obtain a Lab color mode image, and subsequent algorithms are all performed in the Lab color space.

[0123] Furthermore, in step S2, extracting a region of interest from the Lab color mode image includes:

[0124] According to the preset a channel hue threshold and b channel hue threshold, each pixel of the Lab color mode image is traversed, and pixels whose a channel value and b channel value are greater than the a channel hue threshold and the b channel hue threshold are selected as the region of interest.

[0125] Specifically, in the actual superpixel clustering process, it is often necessary to focus on retaining the characteristic performance of our area of ​​interest. Taking power equipment as an example, we focus on the heating of power equipment, but not on its background and other pillars. For example, infrared temperature data pays more attention to the heating situation, so it pays more attention to the red and yellow tones. The a channel hue threshold and the b channel hue threshold can be set to 0, that is, pixels with a channel value and b channel value greater than 0 are selected as the area of ​​interest.

[0126] Furthermore, in step S3, the gradient of each characteristic element of each pixel of the Lab color mode image is calculated using a Sobel operator, including:

[0127] S31, using the Sobel operator to perform gradient calculation on the brightness, a channel and b channel of each pixel of the Lab color mode image to obtain a brightness gradient value and a hue gradient value;

[0128] S32. Normalize and merge the brightness gradient value and the hue gradient value according to the brightness spatial range and the hue spatial range to obtain a gradient value of the pixel.

[0129] Specifically, in step S31, the Sobel operators in the x direction and the y direction are as follows:

[0130]

[0131] Wherein, Gx represents the Sobel operator in the x direction, and Gy represents the Sobel operator in the y direction.

[0132] Each pixel of the Lab color mode image includes three characteristic elements, namely brightness (L), a channel (indicating the range from red to green), and b channel (indicating the range from blue to yellow). The brightness of each pixel is calculated to obtain the brightness gradient value, which is:

[0133]

[0134] Among them, I LGrad Represents the brightness gradient value of pixel I, I L Represents the brightness value of pixel I.

[0135] Furthermore, the a channel of each pixel is calculated to obtain the a channel gradient value, which is specifically:

[0136]

[0137] Among them, I aGrad Represents the gradient value of the a channel of pixel I, I a Represents the alpha channel value of pixel I.

[0138] Furthermore, the b channel of each pixel is calculated to obtain the b channel gradient value, which is specifically:

[0139]

[0140] Among them, I bGrad Represents the b channel gradient value of pixel I, I b Represents the b channel value of pixel I.

[0141] Furthermore, the hue gradient value is

[0142] Furthermore, in step S32, since the range of the brightness space is (0, 100) and the range of the hue space is (-128, 127), the above values ​​need to be normalized. The specific calculation formula is as follows:

[0143]

[0144] Among them, I LabGrad Represents the normalized gradient value.

[0145] Therefore, for the pixel in the i-th row and j-th column, its gradient value is:

[0146] d grad =I LabGrad (i,j); (6)

[0147] Among them, d grad Represents the gradient value of the pixel in the i-th row and j-th column.

[0148] Furthermore, in step S4, the Lab color mode image is initially clustered using a non-iterative clustering method according to the gradient value and the initial number of clusters to obtain an initial superpixel clustering result, including:

[0149] S41, setting K initial first cluster centers and establishing K queues, where K is the number of initial clusters;

[0150] S42, searching for pixels in the neighborhood of the initial first cluster center, and calculating the distance between the pixel and the initial first cluster center according to the gradient value of the searched pixel, combining the pixel information and the distance information into elements, and sequentially adding the elements to corresponding queues in ascending order of distance;

[0151] S43, taking the initial first cluster center as the initial superpixel;

[0152] S44, extracting pixels corresponding to the first elements from each queue and adding them to the corresponding superpixel, releasing the remaining elements in the queue, and marking the pixels added to the superpixel;

[0153] S45. Loop through the following first clustering operation until all pixels are assigned to corresponding superpixels: calculate the average value of the distance between each pixel in the superpixel and the first cluster center as the updated first cluster center, search for unmarked pixels in the neighborhood of the updated first cluster center, calculate the distance between the pixel and the updated first cluster center according to the gradient value of the searched pixel, compose elements with pixel information and distance information, add the elements to corresponding queues in ascending order of distance, extract the pixel corresponding to the first element from each queue and add it to the corresponding superpixel, release the remaining elements in the queue, and mark the pixels added to the superpixel.

[0154] Specifically, in step S41, the number of initial clusters is set, that is, the number of superpixels in the initial superpixel clustering result obtained after the initial clustering is completed.

[0155] Further, in step S42, pixels in the neighborhood of the initial first cluster center are searched, and the distance between the pixel and the initial first cluster center is calculated according to the gradient value of the searched pixel, specifically:

[0156] S421, calculating a first brightness and hue difference according to the brightness, a channel, and b channel of the searched pixel and the first cluster center;

[0157] S422, calculating a first Euclidean distance according to the pixel coordinates of the searched pixel and the first cluster center;

[0158] S423, adding the square value of the first brightness and hue difference and the square value of the gradient value of the searched pixel to obtain a first sum value;

[0159] S424: Calculate the distance between the searched pixel and the first cluster center according to the first sum value, the first Euclidean distance, the relevant balance coefficient, and the distance between adjacent first cluster centers.

[0160] Specifically, in step S421, the searched pixel is k, the first cluster center is i, and the first brightness and hue difference is:

[0161]

[0162] Among them, d lab1 represents the first brightness hue difference, l k Indicates the brightness value of the searched pixel, l i represents the brightness value of the first cluster center, a k Indicates the a channel value of the searched pixel, a i a represents the channel value of the first cluster center, b k Indicates the b channel value of the searched pixel, b i Indicates the b channel value of the first cluster center.

[0163] Further, in step S422, the first Euclidean distance is calculated by the following formula:

[0164]

[0165] Among them, d xy1 represents the first Euclidean distance, x k Indicates the x coordinate of the searched pixel, x i represents the x-coordinate of the first cluster center, y k Indicates the y coordinate of the searched pixel, y i Indicates the y coordinate of the first cluster center.

[0166] S423, adding the square value of the first brightness and hue difference and the square value of the gradient value of the searched pixel to obtain a first sum value, that is, Among them, d grad，k Represents the gradient value of the searched pixel.

[0167] Furthermore, in step S424, the distance between the searched pixel and the first cluster center is calculated by the following formula:

[0168]

[0169] Among them, D 1 represents the distance between the searched pixel and the first cluster center, m represents the balance coefficient between the brightness and hue difference and the Euclidean distance, and its value affects the clustering result of the superpixel, S represents the distance between adjacent first cluster centers, and d grad，k Represents the gradient value of the searched pixel, d lab1 represents the first brightness hue difference, d xy1 represents the first Euclidean distance.

[0170] In the superpixel clustering process, the image edge features are crucial to maintaining clustering accuracy. In particular, for substation equipment, such as insulator strings and other structures, the temperature distribution in infrared images is prone to dispersion, making it difficult for traditional clustering methods to accurately capture it. To this end, in the method proposed in this embodiment, image gradient changes are introduced in clustering distance calculation to enhance edge information.

[0171] Furthermore, in step S42, after obtaining the distance between the searched pixel and the initial first cluster center, the coordinates of the searched pixel and the calculated distance are composed into elements, and arranged from small to large according to the size of the distance, and the elements are added to the corresponding queue in order from small to large.

[0172] Furthermore, in step S43 and step S44, the initial first cluster center is used as the initial superpixel, and then the pixel corresponding to the first element is extracted from each queue and added to the corresponding superpixel. According to the first-in-first-out principle, the first element is the pixel with the smallest clustering distance, which is added to the corresponding superpixel, and the remaining elements in the queue are released, and the pixels added to the superpixel are marked to avoid being searched next time.

[0173] Further, in step S45, the first clustering operation is performed in a loop until all pixels are assigned to corresponding superpixels, thereby completing the initial clustering and obtaining an initial superpixel clustering result, which includes the initial superpixels and their number of the entire Lab color mode image, where the number is K. According to the division of the region of interest and the region of non-interest, the initial superpixel clustering result of the region of interest and the initial superpixel clustering result of the region of non-interest are included. The initial superpixel clustering result of the region of interest includes the initial superpixels and their number thereon, and the initial superpixel clustering result of the region of non-interest includes the initial superpixels and their number thereon.

[0174] In step S4, a queue is used to dynamically update the cluster centers in real time, realizing non-iterative operations in the clustering process. This has an obvious speed advantage and can quickly obtain superpixel clustering results. However, the non-iterative clustering method is sensitive to the selection of the initial cluster centers, and the generated superpixel shapes are relatively irregular. Therefore, in this embodiment, a method for secondary clustering is also proposed.

[0175] Furthermore, in step S5, the number of clusters is updated according to the initial superpixel clustering result, and the updated number of clusters is the number of superpixels in the region of interest.

[0176] Further, in step S6, the region of interest is clustered again using an iterative clustering method according to the gradient values ​​of the pixels in the region of interest and the updated number of clusters to obtain a fine superpixel clustering result of the region of interest, including:

[0177] S61, setting K' initial second cluster centers, where K' is the number of initial superpixels in the region of interest;

[0178] S62, determining the distance between adjacent initial second cluster centers according to the size of the region of interest;

[0179] S63, obtaining the gradient values ​​of the pixels in the neighborhood of the initial second cluster center in the region of interest, and selecting the pixel with the smallest gradient value as the updated second cluster center:

[0180] S64, using the updated second cluster center as the initial superpixel of iteration;

[0181] S65, cyclically performing the following second clustering operation in the region of interest until all pixels are assigned to corresponding superpixels: searching for unlabeled pixels in the neighborhood of the updated second cluster center, calculating the distance between the pixel and the updated second cluster center according to the gradient value of the searched pixel, selecting the pixel with the smallest distance to add to the superpixel and labeling the pixel, and calculating the average value of the distance between each pixel in the superpixel and the second cluster center as the updated second cluster center;

[0182] S66, after each round of the second clustering operation, determine whether a stop condition is met, if the stop condition is met, stop the calculation, if not, loop through the second clustering operation again;

[0183] The stopping condition is that the distance between each current second cluster center and the corresponding second cluster center in the previous round of calculation results is less than a preset value.

[0184] Specifically, in step S61, the number of clusters for secondary clustering is set, that is, the number of superpixels that need to be obtained in the end. In this embodiment, the number of superpixels in the region of interest obtained by initial clustering is used as the number of clusters for secondary clustering.

[0185] In the actual process of superpixel clustering, it is often necessary to focus on retaining the characteristic performance of the region of interest. Taking power equipment as an example, the focus is on the heating of the power equipment, while the background and other pillar data are not concerned. If the initialization clustering points are set uniformly, if too many clustering points are set, it may cause too much iteration time overhead, the clustering effect of the background will not be used for other downstream tasks, and a large number of fragmented superpixel areas may be generated, which require subsequent iterative merging; if too few clustering points are set, the background area may take away most of the clustering pixel results, and the clustering effect of the region of interest cannot guarantee the restoration of data characteristics as much as possible, which is not convenient for our downstream tasks. Therefore, in this embodiment, the initial superpixel division result of the region of interest is first obtained through rapid initial clustering, and then the number of initial superpixels on it is used as the number of clusters for secondary clustering of the region of interest for further division.

[0186] Further, in step S62, the distance between adjacent initial second cluster centers is determined according to the size of the region of interest; assuming that the size of the region of interest is M*M, the distance s between adjacent initial second cluster centers is set to s=sqrt(M 2 / K').

[0187] In step S63, the gradient values ​​of the pixels in the neighborhood of the initial second cluster center are obtained in the region of interest, that is, the gradient values ​​calculated by formula (6), and the pixel with the smallest gradient value is selected as the updated second cluster center.

[0188] Furthermore, in step S65, the distance between the pixel and the second cluster center is calculated according to the gradient value of the searched pixel, including:

[0189] S651, calculating a second brightness and hue difference according to the brightness, a channel, and b channel of the searched pixel and the second cluster center;

[0190] S652, calculating a second Euclidean distance according to the pixel coordinates of the searched pixel and the second cluster center;

[0191] S653, adding the square value of the second brightness and tone difference and the square value of the gradient value of the searched pixel to obtain a second sum value;

[0192] S654: Calculate the distance between the searched pixel and the second cluster center according to the second sum value, the second Euclidean distance, the relevant balance coefficient, and the distance between adjacent second cluster centers.

[0193] Specifically, in step S651, the searched pixel is p, the second cluster center is j, and the second brightness and hue difference is:

[0194]

[0195] Among them, d lab2 Indicates the second brightness hue difference, l p Indicates the brightness value of the searched pixel, l j represents the brightness value of the second cluster center, a p Indicates the a channel value of the searched pixel, a j a represents the channel value of the second cluster center, b p Indicates the b channel value of the searched pixel, b j Indicates the b channel value of the second cluster center.

[0196] Furthermore, in step S652, the second Euclidean distance is calculated by the following formula:

[0197]

[0198] Among them, d xy2 represents the second Euclidean distance, x p Indicates the x coordinate of the searched pixel, x j represents the x-coordinate of the second cluster center, y p Indicates the y coordinate of the searched pixel, y j Indicates the y-coordinate of the second cluster center.

[0199] S653: Add the square value of the second brightness and tone difference and the square value of the gradient value of the searched pixel to obtain a second sum value, that is, Among them, d grad，p Represents the gradient value of the searched pixel.

[0200] Furthermore, in step S654, the distance between the searched pixel and the second cluster center is calculated by the following formula:

[0201]

[0202] Among them, D 2 represents the distance between the searched pixel and the second cluster center, m represents the balance coefficient between the brightness and hue difference and the Euclidean distance, and its value affects the clustering result of the superpixel, S' represents the distance between adjacent second cluster centers, and d grad，p Represents the gradient value of the searched pixel, d lab2 Indicates the second brightness hue difference, d xy2 Represents the second Euclidean distance.

[0203] Furthermore, step S65 is a round of second clustering operation. After each round of second clustering operation, a clustering result can be obtained. Therefore, in step S66, after each round of second clustering operation, it is determined whether the stop condition is met. If the stop condition is met, the calculation is stopped. If the stop condition is not met, the second clustering operation is executed again in a loop. The stop condition is that the distance between each current second cluster center and the corresponding second cluster center in the previous round of calculation results is less than a preset value. That is, when the second cluster center basically does not change, the iteration is stopped.

[0204] In the secondary clustering, image gradient changes are still introduced to enhance edge information.

[0205] Through secondary clustering and iterative clustering methods, the initial clustering results can be further optimized, and more refined superpixel division results can be obtained for the region of interest.

[0206] Furthermore, in step S, fragmented superpixels are merged based on the initial superpixel clustering result and the refined superpixel clustering result of the region of interest to obtain the final clustering result, including:

[0207] S71, merging the initial superpixel clustering result of the non-interest region and the fine superpixel clustering result of the interest region to obtain a merged clustering result;

[0208] S72, setting adjustable parameters, and calculating superpixel thresholds according to the adjustable parameters, the size of the original infrared image, and the number of initial clusters;

[0209] S73, comparing the number of pixels in each superpixel in the merged clustering result with the superpixel threshold to determine fragmented superpixels;

[0210] S74, traversing all superpixels in the merged clustering result, and calculating the average brightness value of all superpixels;

[0211] S75, traversing the fragmented superpixels, and calculating the similarity between the fragmented superpixels and the superpixels in their neighborhood according to the average brightness value;

[0212] S76, if the similarity between the fragmentary superpixels and the superpixels in the neighborhood unit is greater than a preset threshold, merge them;

[0213] S77, after merging all fragmented superpixels, obtain the final clustering result.

[0214] Specifically, in step S71, the initial superpixel clustering result of the non-interest area and the fine superpixel clustering result of the interest area are merged to obtain a merged clustering result, in which the superpixel division of the interest area is finer, and the superpixel division of the non-interest area is relatively unplanned.

[0215] Furthermore, in step S32, a pixel threshold tol is set, which means that when the number of pixels in a superpixel is less than the threshold, it is considered to be a fragmented superpixel. It should be noted that the pixel threshold determines the number of pixels in the superpixel, which to some extent affects the representation of image details by superpixel clustering, so a reasonable superpixel threshold needs to be set. Define the original image size as N*N, the number of initial cluster centers as K, and the superpixel threshold can be calculated as follows:

[0216]

[0217] Among them, λ is represented as an adjustable parameter, which is used to adjust the superpixel threshold size according to clustering requirements. In order to characterize the image features as much as possible, it is generally 0.25 here.

[0218] Furthermore, in step S73, when the number of pixels of a superpixel is less than the superpixel threshold, it is considered to be a fragmented superpixel.

[0219] Furthermore, in step S74, all superpixels in the merged clustering result are traversed to calculate the average brightness value of all superpixels, that is, the average value of the L channel of the pixels in each superpixel.

[0220] Furthermore, in step S75, the similarity between the fragmentary superpixel and the superpixel in its neighborhood is calculated by the following formula:

[0221]

[0222] Among them, DLight n Indicates the similarity between a fragmented superpixel and the nth superpixel in its neighborhood, μ n represents the average brightness value of the nth superpixel within the neighborhood of the fragmented superpixel, and μ represents the average brightness value of the fragmented superpixel.

[0223] Furthermore, in step S76 and step S77, if the degree of similarity between the fragmentary superpixels and the superpixels in the neighborhood unit is greater than a preset degree threshold, they are merged, otherwise no operation is performed, and the final clustering result is obtained after all the fragmentary superpixels are merged.

[0224] refer to Figure 2 In some embodiments, a superpixel clustering device for infrared images is provided, comprising:

[0225] The acquisition module 201 is used to acquire the original infrared image and convert the original infrared image into the Lab color space to obtain a Lab color mode image;

[0226] An extraction module 202, used for extracting a region of interest from the Lab color mode image;

[0227] A gradient calculation module 203, used to calculate the gradient of each characteristic element of each pixel of the Lab color mode image using a Sobel operator to obtain a gradient value of each pixel;

[0228] An initial clustering module 204 is used to set the initial number of clusters, and to perform initial clustering on the Lab color mode image using a non-iterative clustering method according to the gradient value and the initial number of clusters to obtain an initial superpixel clustering result;

[0229] An updating module 205, configured to update the number of clusters according to the initial superpixel clustering result;

[0230] A secondary clustering module 206 is used to perform secondary clustering on the region of interest using an iterative clustering method according to the gradient values ​​of the pixels in the region of interest and the updated number of clusters, to obtain a fine superpixel clustering result of the region of interest;

[0231] The merging module 207 is used to merge the fragmented superpixels based on the initial superpixel clustering result and the refined superpixel clustering result of the region of interest to obtain a final clustering result.

[0232] Furthermore, the extraction module 202 extracts a region of interest from the Lab color mode image, including:

[0233] According to the preset a channel hue threshold and b channel hue threshold, each pixel of the Lab color mode image is traversed, and pixels whose a channel value and b channel value are greater than the a channel hue threshold and the b channel hue threshold are selected as the region of interest.

[0234] Furthermore, the gradient calculation module 203 calculates the gradient of each characteristic element of each pixel of the Lab color mode image using the Sobel operator, including:

[0235] Using the Sobel operator to perform gradient calculation on the brightness, a channel and b channel of each pixel of the Lab color mode image to obtain a brightness gradient value and a hue gradient value;

[0236] According to the brightness space range and the hue space range, the brightness gradient value and the hue gradient value are normalized and merged to obtain the gradient value of the pixel.

[0237] Furthermore, the initial clustering module 204 performs initial clustering on the Lab color mode image using a non-iterative clustering method according to the gradient value and the initial cluster number to obtain an initial superpixel clustering result, including:

[0238] Set K initial first cluster centers and establish K queues, where K is the number of initial clusters;

[0239] Search for pixels in the neighborhood of the initial first cluster center, and calculate the distance between the pixel and the initial first cluster center according to the gradient value of the searched pixel, combine the pixel information and the distance information into elements, and add the elements to the corresponding queue in order from small to large distances;

[0240] The initial first cluster center is used as the initial superpixel;

[0241] Extract the pixel corresponding to the first element from each queue and add it to the corresponding superpixel, release the remaining elements in the queue, and mark the pixels added to the superpixel;

[0242] The following first clustering operation is performed in a loop until all pixels are assigned to corresponding superpixels: the average value of the distance between each pixel in the superpixel and the first cluster center is calculated as the updated first cluster center, unmarked pixels in the neighborhood of the updated first cluster center are searched, the distance between the pixel and the updated first cluster center is calculated according to the gradient value of the searched pixel, pixel information and distance information are combined into elements, and the elements are added to corresponding queues in ascending order of distance, the pixel corresponding to the first element is extracted from each queue and added to the corresponding superpixel, and the remaining elements in the queue are released, and the pixels added to the superpixel are marked.

[0243] Further, the obtained initial superpixel clustering results include the initial superpixel clustering results of the region of interest and the initial superpixel clustering results of the region of no interest; the updated number of clusters is the number of superpixels in the region of interest;

[0244] The secondary clustering module 206 performs secondary clustering on the region of interest using an iterative clustering method according to the gradient values ​​of the pixels in the region of interest and the updated number of clusters, to obtain a fine superpixel clustering result of the region of interest, including:

[0245] Set K' initial second cluster centers, where K' is the number of initial superpixels in the region of interest;

[0246] According to the size of the region of interest, determining the distance between adjacent initial second cluster centers;

[0247] Obtain the gradient values ​​of the pixels in the neighborhood of the initial second cluster center in the region of interest, and select the pixel with the smallest gradient value as the updated second cluster center:

[0248] The updated second cluster center is used as the initial superpixel of the iteration;

[0249] The following second clustering operation is cyclically performed in the region of interest until all pixels are assigned to corresponding superpixels: searching for unlabeled pixels in the neighborhood of the updated second cluster center, calculating the distance between the pixel and the updated second cluster center according to the gradient value of the searched pixel, selecting the pixel with the smallest distance to add to the superpixel and labeling the pixel, and calculating the average value of the distance between each pixel in the superpixel and the second cluster center as the updated second cluster center;

[0250] After each round of the second clustering operation, it is determined whether a stop condition is met. If the stop condition is met, the calculation is stopped. If the stop condition is not met, the second clustering operation is executed again in a loop.

[0251] The stopping condition is that the distance between each current second cluster center and the corresponding second cluster center in the previous round of calculation results is less than a preset value.

[0252] Furthermore, the initial clustering module 204 calculates the distance between the pixel and the first cluster center according to the gradient value of the searched pixel, including:

[0253] Calculate a first brightness and hue difference based on the brightness, a channel, and b channel of the searched pixel and the first cluster center;

[0254] Calculate the first Euclidean distance based on the pixel coordinates of the searched pixel and the first cluster center;

[0255] Adding the square value of the first brightness and tone difference and the square value of the gradient value of the searched pixel to obtain a first sum value;

[0256] The distance between the searched pixel and the first cluster center is calculated based on the first sum value, the first Euclidean distance, the relevant balance coefficient, and the distance between adjacent first cluster centers.

[0257] Furthermore, the secondary clustering module 206 calculates the distance between the pixel and the second cluster center according to the gradient value of the searched pixel, including:

[0258] Calculate the second brightness and hue difference according to the brightness, a channel and b channel of the searched pixel and the second cluster center;

[0259] Calculate the second Euclidean distance based on the pixel coordinates of the searched pixel and the second cluster center;

[0260] Adding the square value of the second brightness and tone difference and the square value of the gradient value of the searched pixel to obtain a second sum value;

[0261] The distance between the searched pixel and the second cluster center is calculated according to the second sum value, the second Euclidean distance, the relevant balance coefficient, and the distance between adjacent second cluster centers.

[0262] Furthermore, the merging module 207 performs fragmented superpixel merging based on the initial superpixel clustering result and the refined superpixel clustering result of the region of interest to obtain a final clustering result, including:

[0263] Merging the initial superpixel clustering result of the non-interest area and the refined superpixel clustering result of the interest area to obtain a merged clustering result;

[0264] Setting an adjustable parameter, and calculating a superpixel threshold according to the adjustable parameter, the size of the original infrared image, and the number of initial clusters;

[0265] Comparing the number of pixels in each superpixel in the merged clustering result with the superpixel threshold to determine fragmented superpixels;

[0266] Traversing all superpixels in the merged clustering result, and calculating the average brightness value of all superpixels;

[0267] Traversing the fragmented superpixels, and calculating the similarity between the fragmented superpixels and the superpixels in their neighborhood according to the average brightness value;

[0268] If the similarity between the fragmented superpixels and the superpixels in the neighborhood unit is greater than a preset threshold, they are merged;

[0269] The final clustering result is obtained after merging all fragmented superpixels.

[0270] Furthermore, the similarity between a fragmented superpixel and the superpixels in its neighborhood is calculated using the following formula:

[0271]

[0272] Among them, DLight n Indicates the similarity between a fragmented superpixel and the nth superpixel in its neighborhood, μ n represents the average brightness value of the nth superpixel within the neighborhood of the fragmented superpixel, and μ represents the average brightness value of the fragmented superpixel.

[0273] The superpixel clustering method and device for infrared images provided in the above embodiments have at least the following beneficial effects:

[0274] (1) Based on the non-iterative clustering method, the infrared image is initially clustered to obtain the initial superpixel division result, and then the region of interest is secondary clustered through an iterative method to perform a more refined superpixel division of the region of interest, while ensuring the accuracy of the superpixel clustering of the region of interest and improving the overall calculation speed;

[0275] (2) In the initial clustering and secondary clustering process, the gradient value of the pixel is introduced to calculate the cluster distance, so as to distinguish "adjacent" pixels with "large gradient changes" as much as possible, and avoid dividing the above situation into one superpixel, which can effectively enhance the edge information and thus improve the accuracy of superpixel clustering; that is, this method can distinguish the detail information of the image to a certain extent, meeting the requirements for detail division of infrared temperature images of electrical equipment;

[0276] (3) The fragmented superpixels are merged based on similarity to further improve the accuracy of superpixel clustering.

[0277] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A superpixel clustering method for infrared images, characterized in that: include: Collecting an original infrared image, and converting the original infrared image into a Lab color space to obtain a Lab color mode image; Extracting a region of interest from the Lab color mode image; Using the Sobel operator to calculate the gradient of each characteristic element of each pixel of the Lab color mode image to obtain the gradient value of each pixel; Setting an initial number of clusters, and performing initial clustering on the Lab color mode image using a non-iterative clustering method according to the gradient value and the initial number of clusters to obtain an initial superpixel clustering result; Updating the number of clusters according to the initial superpixel clustering result; According to the gradient value of the pixels in the region of interest and the updated number of clusters, the region of interest is clustered again using an iterative clustering method to obtain a fine superpixel clustering result of the region of interest; Based on the initial superpixel clustering results and the refined superpixel clustering results of the region of interest, fragmented superpixels are merged to obtain the final clustering result.

2. The method according to claim 1, characterized in that Extracting a region of interest from the Lab color mode image includes: According to the preset a channel hue threshold and b channel hue threshold, each pixel of the Lab color mode image is traversed, and pixels whose a channel value is greater than the a channel hue threshold or whose b channel value is greater than the b channel hue threshold are selected as the region of interest.

3. The method according to claim 1, characterized in that The gradient of each characteristic element of each pixel of the Lab color mode image is calculated using the Sobel operator, including: Using the Sobel operator to perform gradient calculation on the brightness, a channel and b channel of each pixel of the Lab color mode image to obtain a brightness gradient value and a hue gradient value; According to the brightness space range and the hue space range, the brightness gradient value and the hue gradient value are normalized and merged to obtain the gradient value of the pixel.

4. The method according to claim 1, characterized in that Performing initial clustering on the Lab color mode image using a non-iterative clustering method according to the gradient value and the initial cluster number to obtain an initial superpixel clustering result, including: Set K initial first cluster centers and establish K queues, where K is the number of initial clusters; Search for pixels in the neighborhood of the initial first cluster center, and calculate the distance between the pixel and the initial first cluster center according to the gradient value of the searched pixel, combine the pixel information and the distance information into elements, and add the elements to the corresponding queue in order from small to large distances; The initial first cluster center is used as the initial superpixel; Extract the pixel corresponding to the first element from each queue and add it to the corresponding superpixel, release the remaining elements in the queue, and mark the pixels added to the superpixel; The following first clustering operation is performed in a loop until all pixels are assigned to corresponding superpixels: the average value of the distance between each pixel in the superpixel and the first cluster center is calculated as the updated first cluster center, unmarked pixels in the neighborhood of the updated first cluster center are searched, the distance between the pixel and the updated first cluster center is calculated according to the gradient value of the searched pixel, pixel information and distance information are combined into elements, and the elements are added to corresponding queues in ascending order of distance, the pixel corresponding to the first element is extracted from each queue and added to the corresponding superpixel, and the remaining elements in the queue are released, and the pixels added to the superpixel are marked.

5. The method according to claim 1, characterized in that: The obtained initial superpixel clustering results include the initial superpixel clustering results of the region of interest and the initial superpixel clustering results of the region of no interest; the updated number of clusters is the number of initial superpixels of the region of interest; According to the gradient value of the pixels in the region of interest and the updated number of clusters, the region of interest is clustered again using an iterative clustering method to obtain the fine superpixel clustering results of the region of interest, including: Set K' initial second cluster centers, where K' is the number of initial superpixels in the region of interest; According to the size of the region of interest, determining the distance between adjacent initial second cluster centers; Obtain the gradient values ​​of the pixels in the neighborhood of the initial second cluster center in the region of interest, and select the pixel with the smallest gradient value as the updated second cluster center: The updated second cluster center is used as the initial superpixel of the iteration; The following second clustering operation is cyclically performed in the region of interest until all pixels are assigned to corresponding superpixels: searching for unlabeled pixels in the neighborhood of the updated second cluster center, calculating the distance between the pixel and the updated second cluster center according to the gradient value of the searched pixel, selecting the pixel with the smallest distance to add to the superpixel and labeling the pixel, and calculating the average value of the distance between each pixel in the superpixel and the second cluster center as the updated second cluster center; After each round of the second clustering operation, it is determined whether a stop condition is met. If the stop condition is met, the calculation is stopped. If the stop condition is not met, the second clustering operation is executed again in a loop. The stopping condition is that the distance between each current second cluster center and the corresponding second cluster center in the previous round of calculation results is less than a preset value.

6. The method according to claim 4, characterized in that The distance between the pixel and the first cluster center is calculated according to the gradient value of the searched pixel, including: Calculate a first brightness and hue difference based on the brightness, a channel, and b channel of the searched pixel and the first cluster center; Calculate the first Euclidean distance based on the pixel coordinates of the searched pixel and the first cluster center; Adding the square value of the first brightness and tone difference and the square value of the gradient value of the searched pixel to obtain a first sum value; The distance between the searched pixel and the first cluster center is calculated based on the first sum value, the first Euclidean distance, the relevant balance coefficient, and the distance between adjacent first cluster centers.

7. The method according to claim 5, characterized in that The distance between the pixel and the second cluster center is calculated according to the gradient value of the searched pixel, including: Calculate the second brightness and hue difference according to the brightness, a channel and b channel of the searched pixel and the second cluster center; Calculate the second Euclidean distance based on the pixel coordinates of the searched pixel and the second cluster center; Adding the square value of the second brightness and tone difference and the square value of the gradient value of the searched pixel to obtain a second sum value; The distance between the searched pixel and the second cluster center is calculated according to the second sum value, the second Euclidean distance, the relevant balance coefficient, and the distance between adjacent second cluster centers.

8. The method according to claim 1, characterized in that Based on the initial superpixel clustering results and the refined superpixel clustering results of the region of interest, the fragmented superpixels are merged to obtain the final clustering results, including: Merging the initial superpixel clustering result of the non-interest area and the refined superpixel clustering result of the interest area to obtain a merged clustering result; Setting an adjustable parameter, and calculating a superpixel threshold according to the adjustable parameter, the size of the original infrared image, and the number of initial clusters; Comparing the number of pixels in each superpixel in the merged clustering result with the superpixel threshold to determine fragmented superpixels; Traversing all superpixels in the merged clustering result, and calculating the average brightness value of all superpixels; Traversing the fragmented superpixels, and calculating the similarity between the fragmented superpixels and the superpixels in their neighborhood according to the average brightness value; If the similarity between the fragmented superpixels and the superpixels in the neighborhood unit is greater than a preset threshold, they are merged; The final clustering result is obtained after merging all fragmented superpixels.

9. The method according to claim 7, characterized in that: The similarity between a fragmented superpixel and its neighboring superpixels is calculated using the following formula: DLight n =e -μ-μn ; Among them, DLight n Indicates the similarity between a fragmented superpixel and the nth superpixel in its neighborhood, μ n represents the average brightness value of the nth superpixel within the neighborhood of the fragmented superpixel, and μ represents the average brightness value of the fragmented superpixel.

10. A superpixel clustering device for infrared images, characterized in that: include: An acquisition module is used to acquire an original infrared image and convert the original infrared image into a Lab color space to obtain a Lab color mode image; An extraction module, used for extracting a region of interest from the Lab color mode image; A gradient calculation module, used to calculate the gradient of each characteristic element of each pixel of the Lab color mode image using a Sobel operator to obtain a gradient value of each pixel; An initial clustering module, used to set the initial number of clusters, and perform initial clustering on the Lab color mode image using a non-iterative clustering method according to the gradient value and the initial number of clusters to obtain an initial superpixel clustering result; An updating module, used for updating the number of clusters according to the initial superpixel clustering result; A secondary clustering module is used to perform secondary clustering on the region of interest using an iterative clustering method according to the gradient values ​​of the pixels in the region of interest and the updated number of clusters, so as to obtain a fine superpixel clustering result of the region of interest; The merging module is used to merge fragmented superpixels based on the initial superpixel clustering results and the refined superpixel clustering results of the region of interest to obtain the final clustering results.

Images