A hyperspectral image segmentation method based on semi-supervised and superpixel progressive growing

Abstract

The application discloses a hyperspectral image segmentation method based on semi-supervision and superpixel gradual growth, and steps are as follows: normalizing a hyperspectral image to be segmented; taking the normalized hyperspectral image as input, constructing a full convolution network comprising an input layer, a convolution layer, a batch normalization layer and an output layer, and the network output being a semantic segmentation result corresponding to the hyperspectral image; performing superpixel segmentation on the normalized hyperspectral image, and giving each initial superpixel region an initial pseudo label; merging similar initial superpixel regions and updating the pseudo label corresponding to the superpixel region after merging; minimizing a loss function to reversely optimize feature extraction of a feature extractor, fine-tuning the full convolution network by using a small amount of labeled samples, continuously optimizing the superpixel region, making the model converge, and obtaining a final segmentation result.

Description

Technical Field

[0001] This invention belongs to the field of pattern recognition and image processing technology, specifically relating to a hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth. Background Technology

[0002] In the industrial production sector, with the rapid development of technology, production scale is constantly expanding, and production lines are becoming increasingly complex. Against this backdrop, traditional industrial inspection methods, such as manual sampling and offline batch analysis, are no longer sufficient to meet the stringent requirements of modern industry for ensuring product quality and production safety due to their limitations, including low efficiency, susceptibility to subjective factors, and inability to detect problems in a timely manner during production.

[0003] Hyperspectral images possess both spatial and spectral information, exhibiting a "combined image and spectrum" characteristic. Due to the specificity of the spectral characteristics of different substances, hyperspectral imaging technology can accurately capture subtle features of products, detecting issues such as foreign matter contamination and missing components. This enables real-time, non-destructive, and non-contact detection, allowing for the timely discovery of potential quality hazards. With its rich spectral information, hyperspectral imaging technology has demonstrated significant application value in numerous fields and is gradually becoming a focus of industrial inspection.

[0004] Hyperspectral image segmentation, a key technology for processing and analyzing hyperspectral images, is playing an increasingly important role in industrial inspection. This technology, leveraging advanced algorithm models, deeply analyzes the spectral characteristics of each pixel in an image and the spatial relationships between pixels, dividing the hyperspectral image into different regions with similar spectral and spatial features. This lays the foundation for subsequent inspection tasks, accurately capturing subtle product features and promptly identifying potential quality issues.

[0005] Current hyperspectral image segmentation largely relies on deep learning models. However, the performance of these models is significantly limited by the quantity and quality of the labeled data. For hyperspectral images, high-quality labeled data is often scarce, and the annotation process is complex and costly. This not only restricts the widespread application of deep learning models in hyperspectral image segmentation but also hinders performance improvement. Therefore, improving the training effectiveness and generalization ability of deep learning models under limited manual annotation conditions has become a crucial research challenge.

[0006] Furthermore, industrial inspection environments are complex and variable; factors such as lighting conditions and interference from production equipment can affect the acquisition and analysis results of hyperspectral images. Moreover, developing suitable hyperspectral image analysis algorithms for objects inspected in industrial environments requires significant time and effort, and the adaptability and robustness of these algorithms need improvement. Researching and solving these problems will drive the further development of industrial inspection technology and improve the quality and efficiency of industrial production. Summary of the Invention

[0007] To address the aforementioned problems in existing technologies, this invention proposes a hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth. This method is rationally designed, overcomes the shortcomings of existing technologies, and achieves good results.

[0008] A hyperspectral image segmentation method based on semi-supervised and progressive superpixel growth includes the following steps:

[0009] S1. Normalize the hyperspectral image to be segmented.

[0010] S2. Using the normalized hyperspectral image as input, construct a fully convolutional network containing an input layer, a convolutional layer, a batch normalization layer, and an output layer. The network output is the semantic segmentation result of the corresponding hyperspectral image.

[0011] S3. Based on the features of each pixel extracted by the fully convolutional network in S2, the SLIC algorithm is used to perform superpixel segmentation on the normalized hyperspectral image to obtain the initial superpixel region, and each initial superpixel region is assigned an initial pseudo-label.

[0012] S4. Construct a superpixel progressive growth model, use the K-means clustering algorithm to merge similar initial superpixel regions, gradually expand the superpixel scale, realize the progressive growth of superpixels, and update the pseudo-labels corresponding to the grown superpixel regions simultaneously.

[0013] S5. Using dynamic pseudo-labels of superpixel regions as a guiding principle, the feature extraction of the fully convolutional network is optimized in reverse by minimizing the loss function, and the fully convolutional network is fine-tuned using a small number of labeled samples, so that the fully convolutional network can learn the correspondence between the spatial-spectral feature vectors of pixels and the categories more accurately.

[0014] S6. By continuously optimizing the superpixel region, the superpixel progressive growth model converges, and the final segmentation result is obtained.

[0015] Further, in S2, the spatial dimensions of the normalized hyperspectral image are H×W×C, where H and W represent the spatial dimensions of the input hyperspectral image, and C represents its number of spectral channels; the fully convolutional network sequentially includes one input layer, four convolutional layers, four batch normalization layers, and one output layer, wherein the convolutional layers use three-dimensional convolution, each convolutional kernel has a size of 3×3×C, a stride of 1, and the activation function is the modified linear unit ReLU;

[0016] The output of the fully convolutional network is a feature map F of size H×W×d, where d represents the feature channel dimension of the feature map. The d-dimensional vector corresponding to each spatial location (i,j) is the spatial-spectral feature vector f(i,j) of the pixel.

[0017] Further, step S3 includes the following sub-steps:

[0018] S31. The SLIC algorithm is used to perform superpixel segmentation on the normalized hyperspectral image to obtain the initial superpixel region Ω. k , k = 1, ..., K, the initial number of superpixel regions K is:

[0019]

[0020] Where S∈[5,10] represents the superpixel size parameter;

[0021] S32. Based on the feature map output by the fully convolutional network in S2, traverse all superpixel regions Ω. k Specifically:

[0022] (a) For the initial superpixel region Ω k For all pixels within the initial superpixel region, based on the feature map output by the fully convolutional network, extract the spatial-spectral feature vector f(i,j) of the corresponding pixel within each initial superpixel region;

[0023] (b) Calculate the average feature vector v for each superpixel region. k :

[0024]

[0025] S33. Use the K-means clustering algorithm to analyze v k Perform clustering to generate superpixel-level pseudo-labels L k In the process of calculating the distance between samples using the K-means clustering algorithm, the average feature vector v of the initial superpixel region containing the true labeled samples is calculated. k This gives it a greater weight in distance calculations;

[0026] S34. For each clustered initial superpixel region Ω k∧If the superpixel region contains labeled samples, then the pseudo-labels of all pixels within that initial superpixel region are assigned the true class of the labeled samples; otherwise, they are assigned the pseudo-labels L generated by clustering. k .

[0027] Further, step S4 includes the following sub-steps:

[0028] S41. Among multiple initial superpixel regions, select a superpixel pair (Ω). e ,Ω l ), e, l∈[1,K] and e≠l, if the superpixel pair (Ω) e ,Ω l If the superpixel region contains labeled samples, then the feature vector of the initial superpixel region containing the labeled samples is weighted.

[0029] S42. Calculate the feature similarity matrix D between superpixels, and the e-th initial superpixel region Ω. e With the l-th initial superpixel region Ω l Similarity D between el for:

[0030]

[0031] Set a threshold β, for conditions satisfying D el Superpixel pairs ≥β (Ω) e ,Ω l During the merging process, if a superpixel pair contains labeled samples and these labeled samples belong to the same category, then the superpixel pair will be merged first to form a new superpixel region Ω'. m ;

[0032] S43. Calculate the average feature vector for the merged superpixel regions, and use the K-means clustering algorithm to perform secondary clustering, generating and updating superpixel pseudo-labels.

[0033] S44, the process of superpixel semantic enhancement and gradual scaling is as follows:

[0034] (a) Define a superpixel size growth function and dynamically adjust the superpixel size by changing the loss function:

[0035]

[0036] Where the rate of change of the loss function Let K be the value of the mixed loss function in the t-th iteration. (t) Let be the total number of superpixel regions merged in the t-th iteration. It is a floor function;

[0037] (b) Merge superpixels using the similarity matrix D from step S42;

[0038] (c) If the merged superpixel region contains labeled samples, its pseudo-label is forcibly assigned the real category of the labeled sample to avoid conflict between the pseudo-label and the real label.

[0039] Further, step S5 includes the following sub-steps:

[0040] S51. Constructing a hybrid loss function

[0041]

[0042] Cross-entropy loss and regularization loss It is a mixed loss function An important component; among which, Y true To label the true labels of the samples, Y pseudo The pseudo-labels are for unlabeled samples, α and λ are the weights for the proportion of labeled samples and the pseudo-label regularization weights, respectively, both initialized to 0.1; B is the current batch sample set, containing either labeled or unlabeled samples, and N... B y is the number of samples (i) ∈[0,1] is the pseudo-label of the i-th superpixel region, f (i) K is the feature vector of the i-th superpixel region; h This represents the total number of superpixel regions merged in this iteration. For pseudo-label y (i) Corresponding class center;

[0043] S52. Backpropagation optimization and parameter tuning specifically include:

[0044] (a) Update network parameters using backpropagation algorithm combined with momentum stochastic gradient descent algorithm;

[0045] (b) If the current loss If the loss decreases more than the threshold γ = 0.05 compared to the previous cycle, then increase λ; if the loss decreases less than γ, then stop increasing λ.

[0046] S53, Through This process causes similar superpixel features to cluster towards the class center. Meanwhile, updated superpixel features are merged again using the similarity metric of S42 to generate more accurate pseudo-labels.

[0047] Further, step S6 includes the following sub-steps:

[0048] S61. Establish a dual-mode convergence criterion, calculate the pixel ratio ε1 of pseudo-label change and the relative rate of change ε2 of loss function. The model is considered convergent when both ε1 < 0.01 and ε2 < 0.005 are satisfied. The formula is as follows:

[0049]

[0050] Where H×W is the total number of pixels in the image, M (t) (i,j) is the pseudo-label of position (i,j) at the t-th iteration. Indicates an indicator function;

[0051] S62. When the model converges, select the position of the maximum value of the feature vector corresponding to the pixel as its category label, and generate the final segmentation result image.

[0052] The beneficial technical effects of this invention are as follows:

[0053] This invention achieves semi-supervised hyperspectral image segmentation by combining a small number of labeled samples with a progressive superpixel growth mechanism. By using both pseudo-labels and real labels to guide the progressive superpixel growth process, effective segmentation of hyperspectral targets is achieved even in the absence of labeled samples. Attached Figure Description

[0054] Figure 1 This is a flowchart of the hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth according to the present invention;

[0055] Figure 2 This is a schematic diagram of a hyperspectral image segmentation method based on semi-supervised and progressive superpixel growth in the embodiment. Detailed Implementation

[0056] The specific embodiments of the present invention will be further described below with reference to specific examples:

[0057] A hyperspectral image segmentation method based on semi-supervised and progressive superpixel growth, such as Figure 1 As shown, a semi-supervised learning strategy is adopted, using a small number of labeled samples and a large number of unlabeled samples to train the model to achieve high-precision hyperspectral image segmentation, including the following steps:

[0058] S1. Normalize the hyperspectral image to be segmented.

[0059] S2. Using the normalized hyperspectral image as input, construct a fully convolutional network containing an input layer, a convolutional layer, a batch normalization layer, and an output layer. The network output is the semantic segmentation result of the corresponding hyperspectral image.

[0060] Specifically, the spatial dimensions of the normalized hyperspectral image are H×W×C, where H and W represent the spatial dimensions of the input hyperspectral image, and C represents the number of its spectral channels; the fully convolutional network consists of one input layer, four convolutional layers, four batch normalization layers, and one output layer, where the convolutional layers use three-dimensional convolution, each convolutional kernel has a size of 3×3×C, a stride of 1, and the activation function is the modified linear unit ReLU;

[0061] The output of a fully convolutional network is a feature map F of size H×W×d, where d represents the feature channel dimension of the feature map. The d-dimensional vector corresponding to each spatial location (i,j) is the spatial-spectral feature vector f(i,j) of the pixel.

[0062] S3. Based on the features of each pixel extracted by the fully convolutional network in S2, the SLIC algorithm is used to perform superpixel segmentation on the normalized hyperspectral image to obtain the initial superpixel region, and each initial superpixel region is assigned an initial pseudo-label.

[0063] Specifically, S3 includes the following sub-steps:

[0064] S31. The SLIC algorithm is used to perform superpixel segmentation on the normalized hyperspectral image to obtain the initial superpixel region Ω. k , k = 1, ..., K, the initial number of superpixel regions K is:

[0065]

[0066] Where S∈[5,10] represents the superpixel size parameter;

[0067] S32. Based on the feature map output by the fully convolutional network in S2, traverse all superpixel regions Ω. k Specifically:

[0068] (a) For the initial superpixel region Ω k For all pixels within the initial superpixel region, based on the feature map output by the fully convolutional network, extract the spatial-spectral feature vector f(i,j) of the corresponding pixel within each initial superpixel region;

[0069] (b) Calculate the average feature vector v for each superpixel region. k :

[0070]

[0071] S33. Use the K-means clustering algorithm to analyze v k Perform clustering to generate superpixel-level pseudo-labels L k In the process of calculating the distance between samples using the K-means clustering algorithm, the average feature vector v of the initial superpixel region containing the true labeled samples is calculated. kThis assigns greater weight to the distance calculation, giving it a greater influence in clustering decisions. In this way, the clustering results better match the category features exhibited by the labeled samples, improving the accuracy of the generated superpixel-level pseudo-labels.

[0072] S34. For each clustered initial superpixel region Ω k∧ If the superpixel region contains labeled samples, then the pseudo-labels of all pixels within that initial superpixel region are assigned the true class of the labeled samples; otherwise, they are assigned the pseudo-labels L generated by clustering. k .

[0073] S4. Construct a superpixel progressive growth model, use the K-means clustering algorithm to merge similar initial superpixel regions, gradually expand the superpixel scale, realize the progressive growth of superpixels, and update the pseudo-labels corresponding to the grown superpixel regions simultaneously.

[0074] Specifically, S4 includes the following sub-steps:

[0075] S41. Merge similar superpixels using feature cosine similarity to achieve progressive superpixel growth: Select a pair of superpixels (Ω) from multiple initial superpixel regions. e ,Ω l ), e, l∈[1,K] and e≠l, if the superpixel pair (Ω) e ,Ω l If the superpixel region contains labeled samples, then the feature vector of the initial superpixel region where the labeled samples are located is weighted to give it a more important role in the similarity calculation.

[0076] S42. Calculate the feature similarity matrix D between superpixels, and the e-th initial superpixel region Ω. e With the l-th initial superpixel region Ω l The similarity between them is:

[0077]

[0078] Set a threshold β, for conditions satisfying D el Superpixel pairs ≥β (Ω) e ,Ω l During the merging process, if a superpixel pair contains labeled samples and these labeled samples belong to the same category, then the superpixel pair will be merged first to form a new superpixel region Ω'. m ;

[0079] S43. Calculate the average feature vector for the merged superpixel regions, and use the K-means clustering algorithm to perform secondary clustering, generating and updating superpixel pseudo-labels.

[0080] S44, the process of superpixel semantic enhancement and gradual scaling is as follows:

[0081] (a) Dynamic superpixel scaling: Define a superpixel scaling growth function and dynamically adjust the superpixel scaling by changing the loss function.

[0082]

[0083] Where the rate of change of the loss function Let K be the value of the mixed loss function in the t-th iteration. (t) Let be the total number of superpixel regions merged in the t-th iteration. It is a floor function;

[0084] (b) Semi-supervised guided similarity merging: Merge superpixels using the similarity matrix D from step S42;

[0085] (c) Label confidence constraint: If the merged superpixel region contains labeled samples, its pseudo label is forced to be assigned the true category of the labeled sample to avoid conflict between pseudo label and true label.

[0086] S5. Using dynamic pseudo-labels of superpixel regions as a guiding principle, the feature extraction of the fully convolutional network is optimized in reverse by minimizing the loss function, and the fully convolutional network is fine-tuned using a small number of labeled samples, so that the fully convolutional network can learn the correspondence between the spatial-spectral feature vectors of pixels and the categories more accurately.

[0087] Specifically, S5 includes the following sub-steps:

[0088] S51. Constructing a hybrid loss function

[0089]

[0090] Cross-entropy loss and regularization loss It is a mixed loss function An important component; among which, Y true To label the true labels of the samples, Y pseudo The pseudo-labels are for unlabeled samples, α and λ are the weights for the proportion of labeled samples and the pseudo-label regularization weights, respectively, both initialized to 0.1; B is the current batch sample set, containing either labeled or unlabeled samples, and N... B y is the number of samples (i) ∈[0,1] is the pseudo-label of the i-th superpixel region, f (i) K is the feature vector of the i-th superpixel region; h This represents the total number of superpixel regions merged in this iteration. For pseudo-label y (i) Corresponding class center;

[0091] S52. Backpropagation optimization and parameter tuning specifically include:

[0092] (a) Update network parameters using backpropagation algorithm combined with momentum stochastic gradient descent algorithm;

[0093] (b) Dynamically adjust pseudo-label weights λ: if the current loss... If the loss decreases more than the threshold γ = 0.05 compared to the previous period, then λ is increased; if the loss decreases less than γ, then the increase of λ is paused to prevent overfitting.

[0094] S53. Alternating optimization of feature extraction and labeling, improving segmentation performance through closed-loop iteration: [This is achieved through...] This process causes similar superpixel features to cluster towards the class center. Meanwhile, updated superpixel features are merged again using the similarity metric of S42 to generate more accurate pseudo-labels.

[0095] S6. By continuously optimizing the superpixel region, the superpixel progressive growth model converges, and the final segmentation result is obtained.

[0096] Specifically, S6 includes the following sub-steps:

[0097] S61. Establish a dual-mode convergence criterion, calculate the pixel ratio ε1 of pseudo-label change and the relative rate of change ε2 of loss function. The model is considered convergent when both ε1 < 0.01 and ε2 < 0.005 are satisfied. The formula is as follows:

[0098]

[0099] Where H×W is the total number of pixels in the image, M (t) (i,j) is the pseudo-label of position (i,j) at the t-th iteration. Indicates an indicator function;

[0100] S62. Output optimization results: When the model converges, select the position of the maximum value of the feature vector corresponding to the pixel as its category label, and generate the final segmentation result map.

[0101] Example 1

[0102] This embodiment uses Figure 2 The hyperspectral image shown (denoted as HYDICE) is used to illustrate experimental data. This image contains 316×216 pixels, with each pixel corresponding to 148 bands. Using this hyperspectral image as input, a fully convolutional network with four convolutional layers was created; the following steps were followed:

[0103] S1. Normalize the HYDICE image by normalizing the pixel values ​​to between 0 and 1.

[0104] S2. Using the normalized HYDICE image Using the input as input, a fully convolutional network is constructed, consisting of an input layer, a convolutional layer, a batch normalization layer, and an output layer. The network output is the semantic segmentation result of the corresponding hyperspectral image.

[0105] The fully convolutional network consists of an input layer, four convolutional layers, a batch normalization layer, and an output layer. Each convolutional layer is followed by a batch normalization layer and an activation function. The convolutional layers use three-dimensional convolution, with each convolutional kernel having a size of 3×3×148 and a stride of 1. The activation function used is the Modified Linear Unit (ReLU). The batch normalization algorithm is used to process the data to improve the model's generalization ability and stability.

[0106] The output of the fully convolutional network is a feature map F with a size of 316×216×d, where d represents the feature channel dimension of the feature map. The d-dimensional vector corresponding to each spatial location (i,j) is the spatial-spectral feature vector f(i,j) of that pixel.

[0107] S3. Based on the features of each pixel extracted by the network in S2, the SLIC algorithm is used to perform superpixel segmentation on the normalized HYDICE image to obtain initial superpixels, and each initial superpixel is assigned an initial pseudo-label; including the following sub-steps:

[0108] S31. The SLIC algorithm is used to perform superpixel segmentation on the normalized image to obtain the initial superpixel region Ω. k The initial number of superpixel regions K is:

[0109]

[0110] Where S∈[5,10] represents the superpixel size parameter.

[0111] S32. Based on the feature map output by S2, traverse all superpixel regions Ω. k :

[0112] (a) For the superpixel region Ω k For all pixels within a superpixel, based on the feature map output in step S2, extract the spatial-spectral feature vector f(i,j) of the corresponding pixel within that superpixel;

[0113] (b) Calculate the average feature vector v for each superpixel region. k :

[0114]

[0115] S33. Use the K-means clustering algorithm to analyze v k Perform clustering to generate superpixel-level pseudo-labels L kIn the process of calculating the distance between samples using the K-means algorithm, for the average feature vector v of the superpixel containing the real labeled samples... k This assigns greater weight to the distance calculation, giving it a larger influence in clustering decisions. In this way, the clustering results better match the category characteristics of the labeled samples, improving the accuracy of the generated superpixel-level pseudo-labels.

[0116] S34. For each clustered superpixel region Ω k∧ If the superpixel contains labeled samples, then the pseudo-labels of all pixels within that superpixel are assigned the true category of the labeled samples; otherwise, they are assigned the pseudo-labels L generated by clustering. k .

[0117] S4. Merge similar superpixels using the K-means clustering algorithm, gradually expanding the superpixel scale to achieve incremental superpixel growth, and simultaneously update the pseudo-labels corresponding to the grown superpixels; including the following sub-steps:

[0118] S41. Among multiple initial superpixel regions, select a superpixel pair (Ω). e ,Ω l ), e, l∈[1,K] and e≠l, if the superpixel pair (Ω) e ,Ω l If the superpixel region contains labeled samples, then the feature vector of the initial superpixel region containing the labeled samples is weighted.

[0119] S42. Merge similar superpixels using feature cosine similarity to achieve incremental superpixel growth: Calculate the feature similarity matrix D between superpixels, where the similarity between the e-th superpixel and the l-th superpixel is:

[0120]

[0121] Set a threshold β, for conditions satisfying D el Superpixel pairs ≥β (Ω) ke ,Ω kl During the merging process, if a superpixel pair contains labeled samples and these labeled samples belong to the same category, then the superpixel pair will be merged first to form a new superpixel set {Ω'}. m}

[0122] S43. Calculate the average feature vector of the merged superpixels, and use the K-means clustering algorithm to perform secondary clustering, generating and updating the superpixel pseudo-labels.

[0123] S44. Superpixel semantic enhancement and the process of incremental scaling:

[0124] (a) Dynamic superpixel scaling: Define a superpixel scaling growth function and dynamically adjust the superpixel scaling by changing the loss function.

[0125]

[0126] Where the rate of change of the loss function Let K be the value of the mixed loss function in the t-th iteration. (t) Let be the total number of superpixels after merging in the t-th iteration. It is a floor function.

[0127] (b) Semi-supervised guided similarity merging: Merge superpixels using the similarity matrix D from step S41.

[0128] (c) Label confidence constraint: If the merged superpixel region contains labeled samples, its pseudo label is forced to be assigned the true category of the labeled sample to avoid conflict between pseudo label and true label.

[0129] S5. Using dynamic pseudo-labels of superpixels as a guiding principle, the feature extraction of the fully convolutional network is optimized by minimizing the loss function, and the network is fine-tuned using a small number of labeled samples, so that the network can learn the correspondence between the spatial-spectral feature vectors of pixels and the categories more accurately.

[0130] Step S5 includes the following sub-steps:

[0131] S51. Constructing a hybrid loss function

[0132]

[0133] Cross-entropy loss and regularization loss It is a mixed loss function An important component. Among them, Y true To label the true labels of the samples, Y pseudo The pseudo-labels are for unlabeled samples. α and λ are the weights for the proportion of labeled samples and the pseudo-label regularization weights, respectively, both initialized to 0.1. B is the current batch sample set, containing either labeled or unlabeled samples, and N... B y is the number of samples (i) ∈[0,1] is the pseudo-label of the i-th superpixel, f (i) Let K be the feature vector of the i-th superpixel; h This represents the total number of superpixels merged in this iteration. For pseudo-label y (i) Corresponding class center.

[0134] S52. Backpropagation optimization and parameter tuning:

[0135] (a) Update network parameters using backpropagation algorithm combined with momentum stochastic gradient descent algorithm.

[0136] (b) Dynamically adjust pseudo-label weights λ: if the current loss... If the loss decreases by more than the threshold γ = 0.05 compared to the previous period, then λ is increased. If the loss decreases by less than γ, then the increase of λ is paused to prevent overfitting.

[0137] S53. Alternating optimization of feature extraction and labeling, improving segmentation performance through closed-loop iteration:

[0138] pass This process causes similar superpixel features to cluster towards the class center. Meanwhile, updated superpixel features are merged again using the similarity metric of S41 to generate more accurate pseudo-labels.

[0139] S6. By continuously optimizing the superpixels, the progressive growth model of the superpixels converges, and the final segmentation result is obtained.

[0140] Step S6 includes the following sub-steps:

[0141] S61. Establish a dual-mode convergence criterion, calculate the pixel ratio ε1 of pseudo-label change and the relative rate of change ε2 of the loss function. The model is considered convergent when both ε1 < 0.01 and ε2 < 0.005 are satisfied. The formula is as follows:

[0142]

[0143] Where H×W is the total number of pixels in the image, M (t) (i,j) is the pseudo-label of position (i,j) at the t-th iteration. Indicates an indicator function.

[0144] S62. Output optimization results: When the model converges, select the position of the maximum value of the feature vector corresponding to the pixel as its category label, and generate the final segmentation result map.

[0145] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A hyperspectral image segmentation method based on semi-supervised and progressive superpixel growth, characterized in that, Includes the following steps: S1. Normalize the hyperspectral image to be segmented. S2. Using the normalized hyperspectral image as input, construct a fully convolutional network containing an input layer, a convolutional layer, a batch normalization layer, and an output layer. The network output is the semantic segmentation result of the corresponding hyperspectral image. S3. Based on the features of each pixel extracted by the fully convolutional network in S2, the SLIC algorithm is used to perform superpixel segmentation on the normalized hyperspectral image to obtain the initial superpixel region, and each initial superpixel region is assigned an initial pseudo-label. S4. Construct a superpixel progressive growth model, use the K-means clustering algorithm to merge similar initial superpixel regions, gradually expand the superpixel scale, realize the progressive growth of superpixels, and update the pseudo-labels corresponding to the grown superpixel regions simultaneously. S4 includes the following sub-steps: S41. Select a superpixel pair from multiple initial superpixel regions. , and If the superpixel pair If the superpixel region contains labeled samples, then the feature vector of the initial superpixel region containing the labeled samples is weighted. S42. Calculate the feature similarity matrix between superpixels. , No. Initial superpixel region With the Initial superpixel region similarity between for: ， ； Set threshold , for satisfying superpixel pairs During the merging process, if a superpixel pair contains labeled samples and these labeled samples belong to the same category, then the superpixel pair will be merged first to form a new superpixel region. ; S43. Calculate the average feature vector for the merged superpixel regions, and use the K-means clustering algorithm to perform secondary clustering, generating and updating superpixel pseudo-labels. S44, the process of superpixel semantic enhancement and gradual scaling is as follows: (a) Define a superpixel size growth function and dynamically adjust the superpixel size by changing the loss function: ； Where the rate of change of the loss function , For the first The mixed loss function value of the next iteration. For the first The total number of superpixel regions merged in the next iteration. It is a floor function; (b) Using the similarity matrix from step S42 Merge superpixels; (c) If the merged superpixel region contains labeled samples, its pseudo-label is forcibly assigned the real category of the labeled sample to avoid conflict between the pseudo-label and the real label. S5. Using dynamic pseudo-labels of superpixel regions as a guiding principle, the feature extraction of the fully convolutional network is optimized in reverse by minimizing the loss function, and the fully convolutional network is fine-tuned using a small number of labeled samples, so that the fully convolutional network can learn the correspondence between the spatial-spectral feature vectors of pixels and the categories more accurately. S6. By continuously optimizing the superpixel region, the superpixel progressive growth model converges, and the final segmentation result is obtained.

2. The hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth according to claim 1, characterized in that, In S2, the spatial size of the normalized hyperspectral image ,in, and Indicates the spatial dimensions of the input hyperspectral image. Its spectral channel count; the fully convolutional network sequentially includes one input layer, four convolutional layers, four batch normalization layers, and one output layer, wherein the convolutional layers employ three-dimensional convolution, and the size of each convolutional kernel is [size missing]. The step size is set to 1, and the activation function is the modified linear unit ReLU; The output of the fully convolutional network is of size [size missing]. Feature map , This represents the feature map feature channel dimension, and each spatial location corresponding The dimensional vector is the spatial-spectral feature vector of a pixel. .

3. The hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth according to claim 2, characterized in that, S3 includes the following sub-steps: S31. The SLIC algorithm is used to perform superpixel segmentation on the normalized hyperspectral image to obtain the initial superpixel region. , Initial number of superpixel regions for: ； in, Indicates the superpixel size parameter; S32. Based on the feature map output by the fully convolutional network in S2, traverse all superpixel regions. Specifically: (a) For the initial superpixel region For all pixels within the initial superpixel region, based on the feature map output by the fully convolutional network, extract the spatial-spectral feature vector of the corresponding pixel within each superpixel region. ; (b) Calculate the average feature vector for each superpixel region. : ； S33. Use the K-means clustering algorithm to... Perform clustering to generate superpixel-level pseudo-labels In the process of calculating the distance between samples using the K-means clustering algorithm, the average feature vector of the initial superpixel region containing the true labeled samples is used. This gives it a greater weight in distance calculations; S34. For each clustered initial superpixel region If the superpixel region contains labeled samples, then the pseudo-labels of all pixels within that initial superpixel region are assigned the true class of the labeled samples; otherwise, they are assigned the pseudo-labels generated by clustering. .

4. The hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth according to claim 3, characterized in that, S5 includes the following sub-steps: S51. Constructing a hybrid loss function : ； ； ； Cross-entropy loss and regularization loss It is a mixed loss function An important component; among which, To label the true labels of the samples, These are pseudo-labels for unlabeled samples. and These are the weights for the proportion of labeled samples and the regularization weights for pseudo-labels, respectively, both initially set to 0.1; This refers to the current batch of sample sets, which may contain labeled or unlabeled samples. For the number of samples, For the first Pseudo-labels for superpixel regions For the first Feature vectors of each superpixel region; This represents the total number of superpixel regions merged in this iteration. Pseudo-label Corresponding class center; S52. Backpropagation optimization and parameter tuning specifically include: (a) Update network parameters using backpropagation algorithm combined with momentum stochastic gradient descent algorithm; (b) If the current loss The decline compared to the previous cycle exceeded the threshold. Then increase If the loss decreases insufficient Then stop adding. ; S53, Through This process causes similar superpixel features to cluster towards the class center. Meanwhile, updated superpixel features are merged again using the similarity metric of S42 to generate more accurate pseudo-labels.

5. The hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth according to claim 4, characterized in that, S6 includes the following sub-steps: S61. Establish a dual-mode convergence criterion and calculate the pixel ratio of pseudo-label changes. and the relative rate of change of the loss function When both conditions are met , Assuming the model converges, the formula is as follows: ； ； in, The total number of pixels in the image. For the first Position at the next iteration pseudo-tags Indicates an indicator function; S62. When the model converges, select the position of the maximum value of the feature vector corresponding to the pixel as its category label, and generate the final segmentation result image.

Images