Hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth

Abstract

The invention discloses a hyperspectral image segmentation method based on semi-supervision and super-pixel progressive growth. The method comprises the following steps: carrying out normalization processing on a hyperspectral image to be segmented; taking the normalized hyperspectral image as input, constructing a full convolutional network comprising an input layer, a convolutional layer, a batch normalization layer and an output layer, and taking the network output as a semantic segmentation result corresponding to the hyperspectral image; performing superpixel segmentation on the normalized hyperspectral image, and endowing each initial superpixel area with an initial pseudo label; combining the similar initial superpixel areas, and updating the pseudo labels corresponding to the combined superpixel areas; the method comprises the following steps: minimizing a loss function, reversely optimizing feature extraction of a feature extractor, performing fine adjustment on a full convolutional network by using a small number of labeled samples, and continuously optimizing a superpixel region to converge a model to obtain a final segmentation result.

Description

Technical Field

[0001] The present invention belongs to the technical field of pattern recognition and image processing, and particularly relates to a hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth. Background Art

[0002] In the field of industrial production, with the rapid development of technology, the production scale has been continuously expanding, and the production line has become increasingly complex. Against this background, traditional industrial detection methods, such as manual sampling inspection, offline batch analysis, etc., have limitations such as low efficiency, being easily affected by subjective factors, and being unable to detect problems in a timely manner during the production process, and it has been difficult to meet the strict requirements of modern industry for ensuring product quality and production safety.

[0003] Hyperspectral images have both spatial and spectral information, with the characteristic of "combining spectrum and image". Due to the specificity of the spectral characteristics of different substances, hyperspectral imaging technology can accurately capture the subtle features of products, detect problems such as foreign matter mixing and component loss in products, achieve real-time, non-destructive, and non-contact detection, and timely discover potential quality hazards. With its rich spectral information, hyperspectral imaging technology has shown significant application value in many fields and has gradually become the focus of industrial detection.

[0004] Hyperspectral image segmentation, as a key technology for processing and analyzing hyperspectral images, is playing an increasingly important role in the field of industrial detection. This technology uses advanced algorithm models to deeply analyze the spectral characteristics of each pixel point in the image and the spatial position relationship between pixels, divides the hyperspectral image into different regions with similar spectral and spatial characteristics, can lay a foundation for subsequent detection tasks, accurately capture the subtle features of products, and timely discover quality hazards.

[0005] Most current hyperspectral image segmentations are achieved by relying on deep learning models. However, the performance of deep learning models is largely limited by the quantity and quality of labeled data. For hyperspectral images, high-quality labeled data is often very limited, and the labeling process is complex and costly. This not only restricts the wide application of deep learning models in the field of hyperspectral image segmentation but also poses a constraint on the improvement of their performance. Therefore, how to improve the training effect and generalization ability of deep learning models under limited manual labeling conditions has become an important issue in current research.

[0006] In addition, the industrial inspection environment is complex and variable, and factors such as lighting conditions and interference from production equipment will affect the acquisition and analysis results of hyperspectral images. Moreover, for the inspection objects in the industrial environment, developing suitable hyperspectral image analysis algorithms requires a large amount of time and effort, and the adaptability and robustness of the algorithms also need to be improved. The research and solution of these problems will promote the further development of industrial inspection technology and improve the quality and efficiency of industrial production. Summary of the Invention

[0007] In view of the above problems existing in the prior art, the present invention proposes a hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth, with reasonable design, which solves the deficiencies of the prior art and has good effects.

[0008] A hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth includes the following steps:

[0009] S1. Normalize the hyperspectral image to be segmented;

[0010] S2. Taking the normalized hyperspectral image as input, construct a fully convolutional network including an input layer, a convolutional layer, a batch normalization layer and an output layer, and the network output is the semantic segmentation result of the corresponding hyperspectral image;

[0011] S3. According to the features of each pixel point extracted by the fully convolutional network in S2, use the SLIC algorithm to perform superpixel segmentation on the normalized hyperspectral image to obtain initial superpixel regions, and assign an initial pseudo-label to each initial superpixel region;

[0012] S4. Construct a superpixel progressive growth model, use the K-means clustering algorithm to merge similar initial superpixel regions, gradually expand the superpixel scale, realize superpixel progressive growth, and synchronously update the pseudo-labels corresponding to the superpixel regions after growth;

[0013] S5. Using the dynamic pseudo-labels of the superpixel regions as guiding principles, reverse-optimize the feature extraction of the fully convolutional network by minimizing the loss function, and fine-tune the fully convolutional network with a small number of labeled samples to make the fully convolutional network more accurately learn the correspondence between the spatial-spectral feature vectors of pixel points and categories;

[0014] S6. By continuously optimizing the superpixel regions, make the superpixel progressive growth model converge to obtain the final segmentation result.

[0015] Further, in S2, the spatial size of the normalized hyperspectral image is H×W×C, where H and W represent the spatial size of the input hyperspectral image, and C is the number of its spectral channels; the fully convolutional network sequentially includes 1 input layer, 4 convolutional layers, 4 batch normalization layers, and 1 output layer. Among them, the convolutional layer uses three-dimensional convolution, the size of each convolutional kernel is 3×3×C, the stride is set to 1, and the activation function selects the rectified linear unit ReLU;

[0016] The output of the fully convolutional network is a feature map F with a size of H×W×d. d represents the feature channel dimension of the feature map, and the d-dimensional vector corresponding to each spatial position (i, j) is the spatial-spectral feature vector f(i, j) of the pixel.

[0017] Further, S3 includes the following sub-steps:

[0018] S31. Use the SLIC algorithm to perform superpixel segmentation on the normalized hyperspectral image to obtain the initial superpixel region Ω k , k = 1, …, K. The number of initial superpixel regions K is:

[0019]

[0020] where S∈[5, 10] represents the superpixel size parameter;

[0021] S32. Based on the feature map output by the fully convolutional network in S2, traverse all superpixel regions Ω k , specifically:

[0022] (a) For all pixels within the initial superpixel region Ω k , according to the feature map output by the fully convolutional network, extract the spatial-spectral feature vector f(i, j) of the corresponding pixels within each initial superpixel region;

[0023] (b) Calculate the average feature vector v k of each superpixel region:

[0024]

[0025] S33. Use the K-means clustering algorithm to cluster v k to generate the superpixel-level pseudo-label L k ; in the process of calculating the distance between samples by the K-means clustering algorithm, for the average feature vector v k of the initial superpixel region containing the true annotation samples, assign it a greater distance calculation weight;

[0026] S34. For each clustered initial superpixel region Ω k∧, if it contains labeled samples, assign the pseudo - labels of all pixels in the initial super - pixel region to the true class of the labeled samples; otherwise, assign the pseudo - label L generated by clustering k .

[0027] Further, the S4 includes the following sub - steps:

[0028] S41. Among multiple initial super - pixel regions, select a super - pixel pair (Ω e , Ω l ), e, l ∈ [1, K] and e ≠ l. If the super - pixel pair (Ω e , Ω l ) contains labeled samples, then perform weighted processing on the feature vectors of the initial super - pixel region where the labeled samples are located;

[0029] S42. Calculate the feature similarity matrix D between super - pixels. The similarity D e between the e - th initial super - pixel region Ω l and the l - th initial super - pixel region Ω el is:

[0030]

[0031] Set a threshold β. For super - pixel pairs (Ω el ≥ β), merge them. During the merging process, if the super - pixel pair contains labeled samples and these labeled samples belong to the same class, then this super - pixel pair will be preferentially merged to form a new super - pixel region Ω' e , Ω l ; m ;

[0032] S43. Calculate the average feature vector for the merged super - pixel region and use the K - means clustering algorithm for secondary clustering to generate and update the super - pixel pseudo - labels;

[0033] S44. The super - pixel semantic enhancement and scale progressive growth process is specifically as follows:

[0034] (a) Define the super - pixel scale growth function and dynamically adjust the super - pixel scale through the change of the loss function:

[0035]

[0036] where the loss function change rate is the value of the mixed loss function at the t - th iteration, K (t) is the total number of merged super - pixel regions at the t - th iteration, is the floor function;

[0037] (b) Merge super - pixels through the similarity matrix D in step S42;

[0038] (c) If the merged superpixel region contains labeled samples, its pseudo-label is forced to be assigned the true category of the labeled samples to avoid conflicts between pseudo-labels and true labels.

[0039] Furthermore, S5 includes the following sub-steps:

[0040] S51. Construct a hybrid loss function

[0041]

[0042] Cross-entropy loss and regularization loss are important components of the hybrid loss function; where, Y is the true label of the labeled sample, Y true is the true label of the labeled sample, Y pseudo is the pseudo-label of the unlabeled sample, α and λ are the proportion weights of the labeled samples and the pseudo-label regularization weights respectively, and the initial values are both set to 0.1; B is the current batch of sample sets, including labeled samples or unlabeled samples, N B is the number of samples, y (i) ∈[0, 1] is the pseudo-label of the i-th superpixel region, f (i) is the feature vector of the i-th superpixel region; K h is the total number of merged superpixel regions in this round of iteration, is the class center corresponding to the pseudo-label y (i) ;

[0043] S52. Backpropagation optimization and parameter adjustment specifically include:

[0044] (a) Update the network parameters through the backpropagation algorithm combined with the momentum stochastic gradient descent algorithm;

[0045] (b) If the current loss decreases by more than the threshold γ = 0.05 compared to the previous cycle, then increase λ. If the loss decrease is less than γ, then suspend increasing λ;

[0046] S53. Through make the superpixel features of the same class gather towards the class center. At the same time, the updated superpixel features are used for superpixel merging again through the similarity metric in S42 to generate more accurate pseudo-labels.

[0047] Furthermore, S6 includes the following sub-steps:

[0048] S61. Establish a dual-mode convergence criterion, calculate the pixel ratio ε1 of the change in the pseudo-label and the relative change rate ε2 of the loss function. When both ε1 < 0.01 and ε2 < 0.005 are satisfied, it is considered that the model converges. The formula is as follows:

[0049]

[0050] Among them, H×W is the total number of image pixels, M (t) (i,j) is the pseudo-label at position (i,j) in the t-th iteration, denotes the indicator function;

[0051] S62. When the model converges, select the position where the maximum value of the eigenvector corresponding to the pixel is located as its class label, and generate the final segmentation result map.

[0052] The beneficial technical effects brought by the present invention:

[0053] The present invention realizes the segmentation of semi-supervised hyperspectral images by combining a small number of labeled samples with the superpixel progressive growth mechanism. Through the guidance of the pseudo-label and the true label on the superpixel scale progressive growth process, the effective segmentation of hyperspectral targets is realized in the absence of labeled samples. Description of the Drawings

[0054] Figure 1 is the flow chart of the hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth of the present invention;

[0055] Figure 2 is the schematic diagram of the hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth in the embodiment; Detailed Embodiment

[0056] The following further describes the specific embodiments of the present invention in conjunction with specific embodiments:

[0057] A hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth, as Figure 1 shown, adopts a semi-supervised learning strategy, uses a small number of labeled samples and a large number of unlabeled samples to train the model to achieve high-precision hyperspectral image segmentation, including the following steps:

[0058] S1. Normalize the hyperspectral image to be segmented;

[0059] S2. Taking the normalized hyperspectral image as the input, construct a fully convolutional network including an input layer, a convolutional layer, a batch normalization layer and an output layer, and the network output is the semantic segmentation result of the corresponding hyperspectral image;

[0060] Specifically, the spatial dimensions of the normalized hyperspectral image are H×W×C, where H and W represent the spatial dimensions of the input hyperspectral image, and C is the number of its spectral channels; the fully convolutional network sequentially includes 1 input layer, 4 convolutional layers, 4 batch normalization layers, and 1 output layer. Among them, the convolutional layer uses three-dimensional convolution, the size of each convolutional kernel is 3×3×C, the stride is set to 1, and the rectified linear unit ReLU is selected as the activation function;

[0061] The output of the fully convolutional network is a feature map F with dimensions H×W×d. d represents the feature channel dimension of the feature map, and the d-dimensional vector corresponding to each spatial position (i,j) is the spatial-spectral feature vector f(i,j) of the pixel.

[0062] S3. According to the features of each pixel point extracted by the fully convolutional network in S2, use the SLIC algorithm to perform superpixel segmentation on the normalized hyperspectral image to obtain initial superpixel regions, and assign an initial pseudo-label to each initial superpixel region;

[0063] Specifically, S3 includes the following sub-steps:

[0064] S31. Use the SLIC algorithm to perform superpixel segmentation on the normalized hyperspectral image to obtain initial superpixel regions Ω k , k = 1,…,K. The number of initial superpixel regions K is:

[0065]

[0066] where S∈[5,10] represents the superpixel size parameter;

[0067] S32. Based on the feature map output by the fully convolutional network in S2, traverse all superpixel regions Ω k , specifically:

[0068] (a) For all pixels within the initial superpixel region Ω k , according to the feature map output by the fully convolutional network, extract the spatial-spectral feature vector f(i,j) of the corresponding pixels within each initial superpixel region;

[0069] (b) Calculate the average feature vector v k of each superpixel region:

[0070]

[0071] S33. Use the K-means clustering algorithm to cluster v k to generate superpixel-level pseudo-labels L k ; in the process of calculating the distance between samples by the K-means clustering algorithm, for the average feature vector v k, assign a greater weight to the distance calculation, making it have a greater influence in the clustering decision. In this way, the clustering result can better conform to the category characteristics reflected by the labeled samples, improving the accuracy of the generated superpixel-level pseudo labels;

[0072] S34. For each initially clustered superpixel region Ω k∧ , if it contains labeled samples, assign the pseudo labels of all pixels within this initially clustered superpixel region to the true category of the labeled samples; otherwise, assign the pseudo label L generated by clustering k .

[0073] S4. Construct a superpixel progressive growth model, use the K-means clustering algorithm to merge similar initially clustered superpixel regions, gradually expand the superpixel scale, achieve superpixel progressive growth, and synchronously update the pseudo labels corresponding to the superpixel regions after growth;

[0074] Specifically, S4 includes the following sub-steps:

[0075] S41. Use the feature cosine similarity to merge similar superpixels to achieve superpixel progressive growth: Among multiple initially clustered superpixel regions, select a pair of superpixels (Ω e , Ω l ), e, l ∈ [1, K] and e ≠ l. If this pair of superpixels (Ω e , Ω l ) contains labeled samples, then for the feature vector of the initially clustered superpixel region where the labeled samples are located, perform weighted processing to make it play a more important role in the similarity calculation;

[0076] S42. Calculate the feature similarity matrix D between superpixels. The similarity between the e-th initially clustered superpixel region Ω e and the l-th initially clustered superpixel region Ω l is:

[0077]

[0078] Set a threshold β. For superpixel pairs (Ω el ≥β) that meet the condition, merge them. During the merging process, if the superpixel pair contains labeled samples and these labeled samples belong to the same category, then this superpixel pair will be preferentially merged to form a new superpixel region Ω' e , Ω l ; m ;

[0079] S43. Calculate the average feature vector for the merged superpixel region and use the K-means clustering algorithm for secondary clustering to generate and update the superpixel pseudo labels;

[0080] S44. The specific process of superpixel semantic enhancement and scale progressive growth is as follows:

[0081] (a) Dynamic superpixel scale adjustment: Define the superpixel scale growth function and dynamically adjust the superpixel scale according to the change of the loss function:

[0082]

[0083] where the change rate of the loss function is the value of the mixed loss function at the t-th iteration, and K (t) is the total number of superpixel regions merged in the t-th iteration, is the floor function;

[0084] (b) Semi-supervised guided similarity merging: Merge superpixels through the similarity matrix D in step S42;

[0085] (c) Label confidence constraint: If the merged superpixel region contains labeled samples, its pseudo-label is forced to be assigned the true category of the labeled samples to avoid conflicts between pseudo-labels and true labels.

[0086] S5. Use the dynamic pseudo-labels of superpixel regions as guiding principles, reverse-optimize the feature extraction of the fully convolutional network by minimizing the loss function, and fine-tune the fully convolutional network using a small number of labeled samples to enable the fully convolutional network to more accurately learn the correspondence between the spatial-spectral feature vectors of pixel points and categories;

[0087] Specifically, S5 includes the following sub-steps:

[0088] S51. Construct the mixed loss function

[0089]

[0090] Cross-entropy loss and regularization loss are important components of the mixed loss function ; where Y true is the true label of the labeled sample, Y pseudo is the pseudo-label of the unlabeled sample, α and λ are the proportion weights of the labeled samples and the pseudo-label regularization weights respectively, and their initial values are both set to 0.1; B is the current batch of sample sets, including labeled samples or unlabeled samples, N B is the number of samples, y (i) ∈[0, 1] is the pseudo-label of the i-th superpixel region, f (i) is the feature vector of the i-th superpixel region; K h is the total number of superpixel regions merged in this iteration, is the class center corresponding to the pseudo-label y (i) ;

[0091] S52. Backpropagation optimization and parameter adjustment specifically include:

[0092] (a) Update network parameters through the backpropagation algorithm combined with the momentum stochastic gradient descent algorithm;

[0093] (b) Dynamically adjust the pseudo-label weight λ: If the current loss decreases by more than the threshold γ = 0.05 compared to the previous cycle, increase λ. If the loss decrease is less than γ, suspend increasing λ to prevent overfitting;

[0094] S53. Feature extraction and label alternating optimization to improve the segmentation performance through closed-loop iteration: By aggregating the features of homogeneous superpixels towards the class center. Meanwhile, the updated superpixel features are used for superpixel merging again through the similarity measurement in S42 to generate more accurate pseudo-labels.

[0095] S6. By continuously optimizing the superpixel region, the superpixel progressive growth model converges to obtain the final segmentation result.

[0096] Specifically, S6 includes the following sub-steps:

[0097] S61. Establish a dual-mode convergence criterion, calculate the pixel ratio ε1 of the pseudo-label change and the relative change rate ε2 of the loss function. When both ε1 < 0.01 and ε2 < 0.005 are satisfied, it is considered that the model converges. The formula is as follows:

[0098]

[0099] where H×W is the total number of image pixels, M (t) (i,j) is the pseudo-label at position (i,j) in the t-th iteration, represents the indicator function;

[0100] S62. Output the optimization result: When the model converges, select the position where the maximum value of the feature vector corresponding to the pixel is located as its class label to generate the final segmentation result map.

[0101] Example 1

[0102] This example is illustrated with the hyperspectral image (denoted as HYDICE) shown in Figure 2 as the experimental data. This image contains 316×216 pixel points, and the number of bands corresponding to each pixel point is 148. Using this hyperspectral image as the input, a fully convolutional network with 4 convolutional layers is created; the following steps are adopted:

[0103] S1. Normalize the HYDICE image to normalize the pixel values of the image to the range of 0 to 1;

[0104] S2. Using the normalized HYDICE image as the input, construct a fully convolutional network that includes an input layer, a convolutional layer, a batch normalization layer, and an output layer. The network output is the semantic segmentation result of the corresponding hyperspectral image;

[0105] The fully convolutional network sequentially includes an input layer, four convolutional layers, a batch normalization layer, and an output layer. Each convolutional layer is followed by a batch normalization layer and an activation function. Among them, the convolutional layer uses three-dimensional convolution, the size of each convolutional kernel is 3×3×148, the stride is set to 1, the activation function selects the rectified linear unit ReLU, and the batch normalization algorithm is used to process the data to improve the generalization ability and stability of the model.

[0106] The output of the fully convolutional network is a feature map F with a size of 316×216×d, where d represents the feature channel dimension of the feature map. The d-dimensional vector corresponding to each spatial position (i,j) is the spatial-spectral feature vector f(i,j) of the pixel.

[0107] S3. Based on the features of each pixel point extracted by the network in S2, use the SLIC algorithm to perform superpixel segmentation on the normalized HYDICE image to obtain initial superpixels, and assign an initial pseudo-label to each initial superpixel. It includes the following sub-steps:

[0108] S31. Use the SLIC algorithm to perform superpixel segmentation on the normalized image to obtain the initial superpixel region Ω k , and the number K of the initial superpixel regions is:

[0109]

[0110] where S∈[5,10] represents the superpixel size parameter.

[0111] S32. Based on the feature map output in S2, traverse all the superpixel regions Ω k :

[0112] (a) For all the pixels in the superpixel region Ω k , according to the feature map output in step S2, extract the spatial-spectral feature vector f(i,j) of the corresponding pixels in this superpixel;

[0113] (b) Calculate the average feature vector v of each superpixel region k :

[0114]

[0115] S33. Use the K-means clustering algorithm to cluster v k to generate the superpixel-level pseudo-label L k; During the process of calculating the distance between samples by the K-means algorithm, for the average feature vector v of the superpixels containing the truly labeled samples k , a greater distance calculation weight is assigned to it, making it have a greater influence in the clustering decision. In this way, the clustering result can better fit the class features reflected by the labeled samples, improving the accuracy of the generated superpixel-level pseudo-labels.

[0116] S34. For each superpixel region Ω k∧ after clustering, if it contains labeled samples, the pseudo-labels of all pixels within this superpixel are assigned the true class of the labeled samples; otherwise, they are assigned the pseudo-label L k .

[0117] S4. Use the K-means clustering algorithm to merge similar superpixels, gradually expand the scale of the superpixels, achieve the progressive growth of the superpixels, and synchronously update the pseudo-labels corresponding to the superpixels after growth; including the following sub-steps:

[0118] S41. Among multiple initial superpixel regions, select a pair of superpixels (Ω e , Ω l ), e, l ∈ [1, K] and e ≠ l. If this pair of superpixels (Ω e , Ω l ) contains labeled samples, then the feature vector of the initial superpixel region where the labeled samples are located is weighted.

[0119] S42. Use the feature cosine similarity to merge similar superpixels to achieve the progressive growth of the superpixels: Calculate the feature similarity matrix D between superpixels. The similarity between the e-th superpixel and the l-th superpixel is:

[0120]

[0121] Set a threshold β. For the pair of superpixels (Ω el ≥ β) (Ω ke , Ω kl ) that meet the condition, they are merged. During the merging process, if the pair of superpixels contains labeled samples and these labeled samples belong to the same class, then this pair of superpixels will be preferentially merged to form a new set of superpixels {Ω' m}.

[0122] S43. Calculate the average feature vector of the merged superpixels and use the K-means clustering algorithm for secondary clustering to generate and update the superpixel pseudo-labels.

[0123] S44. The process of superpixel semantic enhancement and scale progressive growth:

[0124] (a) Dynamic superpixel scale adjustment: Define the superpixel scale growth function and dynamically adjust the superpixel scale according to the change of the loss function:

[0125]

[0126] where the change rate of the loss function is the value of the mixed loss function at the t-th iteration, K (t) is the total number of merged superpixels in the t-th iteration, is the floor function.

[0127] (b) Semi-supervised guided similarity merging: Merge superpixels through the similarity matrix D in step S41.

[0128] (c) Label confidence constraint: If the merged superpixel region contains labeled samples, its pseudo-label is forced to be assigned the true category of the labeled samples to avoid conflicts between pseudo-labels and true labels.

[0129] S5. Use the dynamic pseudo-labels of superpixels as guiding principles, reverse-optimize the feature extraction of the fully convolutional network by minimizing the loss function, and fine-tune the network using a small number of labeled samples to enable the network to more accurately learn the correspondence between the spatial-spectral feature vectors of pixel points and categories;

[0130] Step S5 includes the following sub-steps:

[0131] S51. Construct the mixed loss function

[0132]

[0133] Cross-entropy loss and regularization loss are important components of the mixed loss function . Among them, Y true is the true label of the labeled sample, Y pseudo is the pseudo-label of the unlabeled sample, α and λ are the weights of the labeled sample ratio and the pseudo-label regularization weight respectively, and their initial values are both set to 0.1. B is the current batch of sample sets, including labeled samples or unlabeled samples, N B is the number of samples, y (i) ∈[0, 1] is the pseudo-label of the i-th superpixel, f (i) is the feature vector of the i-th superpixel,; K h is the total number of merged superpixels in this iteration, is the class center corresponding to the pseudo-label y (i) .

[0134] S52. Backpropagation optimization and parameter adjustment:

[0135] (a) Update the network parameters by the backpropagation algorithm combined with the momentum stochastic gradient descent algorithm.

[0136] (b) Dynamically adjust the pseudo-label weight λ: If the current loss decreases by more than the threshold γ = 0.05 compared to the previous cycle, increase λ. If the loss decrease is less than γ, pause increasing λ to prevent overfitting.

[0137] S53. Feature extraction and label alternating optimization to improve the segmentation performance through closed-loop iteration:

[0138] By cluster the superpixel features of the same class towards the class center. Meanwhile, the updated superpixel features are used for superpixel merging again through the similarity metric in S41 to generate more accurate pseudo-labels.

[0139] S6. Through continuous optimization of the superpixels, make the superpixel progressive growth model converge to obtain the final segmentation result.

[0140] Step S6 includes the following sub-steps:

[0141] S61. Establish a dual-mode convergence criterion, calculate the pixel ratio ε1 of the change in the pseudo-label and the relative change rate ε2 of the loss function. When both ε1 < 0.01 and ε2 < 0.005 are satisfied, it is considered that the model converges. The formula is as follows:

[0142]

[0143] where H×W is the total number of pixels in the image, and M (t) (i,j) is the pseudo-label at position (i,j) in the t-th iteration. represents the indicator function.

[0144] S62. Output the optimization result: When the model converges, select the position where the maximum value of the feature vector corresponding to the pixel is located as its class label to generate the final segmentation result map.

[0145] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions, or substitutions made by those skilled in the art within the scope of the essence of the present invention should also fall within the protection scope of the present invention.

Claims

1. A hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth, characterized in that, It includes the following steps: S1. Normalize the hyperspectral image to be segmented; S2. Taking the normalized hyperspectral image as the input, construct a fully convolutional network including an input layer, convolutional layers, batch normalization layers and an output layer, and the network output is the semantic segmentation result of the corresponding hyperspectral image; S3. According to the features of each pixel point extracted by the fully convolutional network in S2, use the SLIC algorithm to perform superpixel segmentation on the normalized hyperspectral image to obtain initial superpixel regions, and assign an initial pseudo-label to each initial superpixel region; S4. Construct a superpixel progressive growth model, use the K-means clustering algorithm to merge similar initial superpixel regions, gradually expand the superpixel scale to achieve superpixel progressive growth, and synchronously update the pseudo-labels corresponding to the superpixel regions after growth; S5. Using the dynamic pseudo-labels of the superpixel regions as guiding principles, reverse-optimize the feature extraction of the fully convolutional network by minimizing the loss function, and fine-tune the fully convolutional network with a small number of labeled samples to make the fully convolutional network more accurately learn the corresponding relationship between the spatial-spectral feature vectors of pixel points and categories; S6. By continuously optimizing the superpixel regions, make the superpixel progressive growth model converge to obtain the final segmentation result.

2. A hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth according to claim 1, characterized in that In S2, the spatial size of the normalized hyperspectral image is H×W×C, where H and W represent the spatial size of the input hyperspectral image, and C is the number of its spectral channels; the fully convolutional network sequentially includes 1 input layer, 4 convolutional layers, 4 batch normalization layers and 1 output layer. Among them, the convolutional layers use three-dimensional convolutions, the size of each convolutional kernel is 3×3×C, the stride is set to 1, and the activation function selects the rectified linear unit ReLU; The output of the fully convolutional network is a feature map F with a size of H×W×d, where d represents the feature channel dimension of the feature map, and the d-dimensional vector corresponding to each spatial position (i,j) is the spatial-spectral feature vector f(i,j) of the pixel.

3. A hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth according to claim 2, characterized in that, S3 includes the following sub-steps: S31. Use the SLIC algorithm to perform superpixel segmentation on the normalized hyperspectral image to obtain the initial superpixel region Ω k , k = 1, …, K, where the number of initial superpixel regions K is: Among them, S∈[5,10] represents the superpixel size parameter; S32. Traverse all superpixel regions Ω based on the feature map output by the fully convolutional network in S2 k , specifically as follows: (a) For the initial superpixel region Ω k For all pixels within it, according to the feature map output by the fully convolutional network, extract the spatial-spectral feature vector f(i, j) of the corresponding pixels within each initial superpixel region; (b) Calculate the average feature vector v of each superpixel region k : S33. Use the K-means clustering algorithm to cluster v k to generate superpixel-level pseudo-labels L k ; during the process of calculating the distance between samples in the K-means clustering algorithm, for the average feature vector v k of the initial superpixel region containing the truly labeled samples, assign it a greater distance calculation weight; S34. For each initial superpixel region Ω after clustering k∧ If it contains labeled samples, assign the pseudo - labels of all pixels in this initial superpixel region to the true class of the labeled samples; otherwise, assign the pseudo - label L generated by clustering k .

4. A hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth according to claim 3, characterized in that, S4 includes the following sub-steps: S41. Among multiple initial superpixel regions, select a superpixel pair (Ω e , Ω l ), where e, l ∈ [1, K] and e ≠ l. If the superpixel pair (Ω e , Ω l ) contains labeled samples, then perform weighted processing on the feature vector of the initial superpixel region where the labeled samples are located; S42. Calculate the feature similarity matrix D between superpixels. The similarity D between the e-th initial superpixel region Ω e and the l-th initial superpixel region Ω l is as follows: el For: Set a threshold β, for the superpixel pairs (Ω el ≥β that satisfy D e ,Ω l ), perform merging. During the merging process, if the superpixel pair contains labeled samples and these labeled samples belong to the same category, then this superpixel pair will be preferentially merged to form a new superpixel region Ω' m ; S43. Calculate the average feature vector of the merged superpixel regions, and use the K-means clustering algorithm for secondary clustering to generate and update the superpixel pseudo-labels; S44. The superpixel semantic enhancement and scale progressive growth process is specifically as follows: (a) Define a superpixel scale growth function, and dynamically adjust the superpixel scale through the change of the loss function: where the change rate of the loss function is the value of the mixed loss function at the t-th iteration, and K (t) is the total number of merged superpixel regions in the t-th iteration, is the floor function; (b) Merge the superpixels through the similarity matrix D in step S42; (c) If the merged superpixel region contains labeled samples, its pseudo-label is forcibly assigned to the true category of the labeled sample to avoid conflicts between the pseudo-label and the true label.

5. A hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth according to claim 4, characterized in that, S5 includes the following sub-steps: S51. Construct a hybrid loss function Cross-entropy loss and regularization loss are important components of the mixed loss function; where Y is the true label of the labeled sample, Y true is the pseudo-label of the unlabeled sample, α and λ are the proportion weight of the labeled sample and the pseudo-label regularization weight respectively, and the initial values are both set to 0.1; B is the current batch of sample sets, including labeled samples or unlabeled samples, N pseudo is the number of samples, y B ∈[0, 1] is the pseudo-label of the i-th superpixel region, f (i) is the feature vector of the i-th superpixel region; K (i) is the total number of merged superpixel regions in this round of iteration, h is the class center corresponding to the pseudo-label y (i) ;​ S52. The backpropagation optimization and parameter adjustment specifically include: (a) Update the network parameters through the backpropagation algorithm combined with the momentum stochastic gradient descent algorithm; (b) If the current loss decreases by more than the threshold γ = 0.05 compared to the previous cycle, then increase λ. If the loss decrease is less than γ, then suspend increasing λ; S53. By aggregating homogeneous superpixel features towards the class center, and at the same time, the updated superpixel features are superpixel merged again through the similarity metric in S42 to generate more accurate pseudo-labels.

6. A hyperspectral image segmentation method based on semi-supervised and superpixel progressive growth according to claim 5, characterized in that S6 includes the following sub-steps: S61. Establish a dual-mode convergence criterion, calculate the pixel ratio ε1 of the change of the pseudo-label and the relative change rate ε2 of the loss function. When both ε1<0.01 and ε2<0.005 are satisfied, it is considered that the model converges. The formula is as follows: where H×W is the total number of image pixels, and M (t) (i, j) is the pseudo-label at position (i, j) at the t-th iteration, denotes the indicator function; S62. When the model converges, select the position where the maximum value of the eigenvector corresponding to the pixel is located as its class label, and generate the final segmentation result map.

Images