Short-time-sequence landslide image recognition method based on adaptive superpixel segmentation fusion

Abstract

The invention provides a short-time-sequence landslide image recognition method based on adaptive superpixel segmentation fusion, and relates to the technical field of image recognition, and the method specifically comprises the following steps: calculating a plurality of evaluation indexes for different superpixel segmentation methods and segmentation numbers; performing normalization processing on the evaluation indexes, and calculating the weight of each evaluation index based on principal component analysis (PCA); constructing a superpixel segmentation comprehensive evaluation calculation method, selecting an optimal superpixel segmentation method and segmentation number according to the comprehensive score, and performing preliminary superpixel segmentation on the landslide image; constructing an improved superpixel region adjacency graph RAG to describe the similarity between superpixels, and carrying out superpixel merging and segmentation on the landslide image by adopting a normalized cutting method; and denoising the landslide image. According to the technical scheme, the problem that the short-time-sequence landslide deformation area under the rainstorm effect cannot be accurately recognized and monitored under the condition of data scarcity in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a short-time series landslide image recognition method based on adaptive superpixel segmentation and fusion. Background Art

[0002] In recent years, extreme weather has occurred frequently, especially landslides caused by heavy rains, which are sudden and short-lived, resulting in extremely limited relevant image data that can be collected. How to accurately identify and monitor short-term landslide deformation areas under the condition of scarce data has become a key problem that urgently needs to be solved.

[0003] The identification of landslide deformation areas generally goes through three stages:

[0004] In the first stage, the displacement of certain points of the landslide was monitored mainly by burying sensors. However, this method requires drilling holes to bury monitoring equipment, which destroys the original structure of the slope, and the monitoring data is for a single point, which cannot accurately reflect the overall condition of the landslide.

[0005] In the second stage, with the development of computer vision technology, the use of two-dimensional or three-dimensional images to identify landslide deformation areas has gradually been applied. However, most of the current two-dimensional landslide image recognition methods are only applicable to small areas with relatively simple colors and textures, and cannot accurately identify some more complex landslide scenes. At the same time, although the use of three-dimensional image data to identify landslide deformation areas has improved the recognition accuracy to a certain extent, it still has certain limitations, such as limited monitoring distance and difficulty in point cloud data analysis. In addition, the presence of rain affects the image clarity of landslide images, further increasing the difficulty of landslide image recognition.

[0006] In the third stage, in order to further improve the accuracy of landslide deformation area recognition, the researchers collected a large amount of data and used neural network training. However, the setting of image labels and the training of neural network models require the collection of a large amount of preliminary data, and the recognition accuracy is overly dependent on the amount of data, which is not applicable to the short-term landslide scene of heavy rain.

[0007] Therefore, a method is needed to accurately identify and monitor short-term landslide deformation areas under the condition of data scarcity. Summary of the invention

[0008] The main purpose of the present invention is to provide a short-time landslide image recognition method based on adaptive superpixel segmentation and fusion to solve the problem that the prior art cannot accurately identify and monitor the short-time landslide deformation area under the condition of data scarcity.

[0009] To achieve the above object, the present invention provides a short-time series landslide image recognition method based on adaptive superpixel segmentation and fusion, which specifically includes the following steps: S1, for different superpixel segmentation methods and segmentation numbers, multiple evaluation indicators are calculated; the evaluation indicators include: regional consistency, shape regularity, superpixel variance and segmentation time.

[0010] S2, normalizing the evaluation indicators in step S1, and calculating the weight of each evaluation indicator based on principal component analysis PCA.

[0011] S3, construct a comprehensive evaluation calculation method for superpixel segmentation, select the optimal superpixel segmentation method and number of segmentations according to the comprehensive score, and perform preliminary superpixel segmentation on the landslide image.

[0012] S4, an improved superpixel region adjacency graph RAG is constructed to describe the similarity between superpixels, and the normalized cut method is used to merge and segment the landslide image superpixels.

[0013] S5, denoising the landslide image processed in step S4.

[0014] Furthermore, step S1 specifically includes the following steps: S1.1, Calculation of regional consistency : ; in, represents the number of superpixels, is the color variance, are the three channels in the LAB color space, For the channel, is a super pixel.

[0015] S1.2, Calculation of shape regularity : ; in, is the perimeter of the superpixel, is the area of ​​the superpixel.

[0016] S1.3, calculate the variance between superpixels : ; in, For the The average color of superpixels, Superpixel and The color distance between them.

[0017] S1.4, calculate the computation time of different superpixel segmentation methods.

[0018] Furthermore, the calculation of the weights of each evaluation index based on principal component analysis PCA in step S2 specifically includes the following steps: S2.1, extract the normalized evaluation indicator matrix.

[0019] S2.2, perform principal component analysis on the evaluation indicators to obtain the variance explanation rate of each principal component.

[0020] S2.3, calculate the load coefficient of each evaluation index on different principal components, that is, the principal component matrix.

[0021] S2.4, calculate the total contribution of each evaluation indicator based on the load coefficient and variance explanation rate, and standardize the total contribution to the weight: ; in, For the The final weight of the evaluation indicators; For the The eigenvalues ​​of the principal components; For the The evaluation index is The loading coefficients on the principal components; is the number of principal components retained; is the total number of indicators.

[0022] Furthermore, the comprehensive score in step S3 The calculation formula is: ; in, , , , are the weights of regional consistency, shape regularity, superpixel variance and segmentation time, respectively. is the normalized value of the execution time.

[0023] Furthermore, in step S4, when constructing the region adjacency graph, the similarity weights between superpixels are dynamically adjusted. The calculation formula is: ; in, , Superpixel and The color mean of , Superpixel and The mean brightness of , are the standard deviation adjustment parameters for color and brightness, is an exponential function.

[0024] Furthermore, the optimization goal of the normalized cut in step S4 is The formula is: ; in, and are the two sub-graphs of the segmentation. is the edge weight between subgraphs, For subgraph With Figure The sum of the connection weights of all nodes in .

[0025] Furthermore, step S5 specifically performs binarization processing on the grayscale image, and then removes the small area therein, and only retains the most significant area, which is expressed as follows: ; in, is the input grayscale image; The optimal threshold calculated by Otsu method; is the pixel in the binary image The connected area to which it belongs; is the area, For connected areas area; is the proportionality coefficient, To obtain the maximum value function, is a binary function.

[0026] The present invention has the following beneficial effects: (1) Dynamic weight allocation in multi-index fusion model:

[0027] Based on the existing indicators (regional consistency, shape regularity, etc.), a data-driven principal component analysis (PCA) dynamic weight calculation method is proposed. Through the improved principal component analysis steps, the contribution of each indicator to the comprehensive score is automatically determined instead of relying on manually set fixed weights.

[0028] The problem of strong subjectivity of weights in traditional multi-index evaluation is solved, making the selection of the optimal segmentation method more adaptable to different image qualities (such as blurred images caused by heavy rain).

[0029] (2) Scene-adaptive optimization of normalized cutting algorithm: When constructing the regional adjacency graph (RAG), a similarity correction factor under heavy rain conditions is introduced, and the suppression term for light reflection interference is added to the edge weight formula, for example: , Superpixel and The mean brightness of , They are the standard deviation adjustment parameters for color and brightness respectively. Significantly reduced to suppress the similarity of areas with sudden brightness changes.

[0030] In rainstorm images, the accuracy of superpixel merging is improved by suppressing the similarity of high-brightness noise points.

[0031] (3) Noise suppression by dynamic area threshold: Propose a ratio threshold based on the area of ​​the largest connected region , rather than a fixed threshold. The scope of the reserved area is dynamically adjusted to adapt to different landslide scales and environmental noise intensities.

[0032] In short-time landslide scenarios where data is scarce, avoid over-segmentation or under-segmentation problems caused by fixed thresholds. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the specific implementation of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the specific implementation or the prior art description. Obviously, the drawings described below are some implementations of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings: Figure 1 The flowchart of the method for short-time series landslide image recognition based on adaptive superpixel segmentation and fusion of the present invention is shown.

[0034] Figure 2 An image of landslide identification performed using the method provided by the present invention at a certain moment is shown.

[0035] Figure 3 An image of landslide identification performed using the method provided by the present invention at another time is shown.

[0036] Figure 4 A comparison chart of the accuracy of the method provided by the present invention and the U-Net neural network recognition method is shown. DETAILED DESCRIPTION

[0037] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] like Figure 1 The short-time series landslide image recognition method based on adaptive superpixel segmentation and fusion shown in the figure specifically includes the following steps: S1, for different superpixel segmentation methods and segmentation numbers, multiple evaluation indicators are calculated; the evaluation indicators include: regional consistency, shape regularity, superpixel variance and segmentation time.

[0039] S2, normalizing the evaluation indicators in step S1, and calculating the weight of each evaluation indicator based on principal component analysis PCA.

[0040] S3, construct a comprehensive evaluation calculation method for superpixel segmentation, dynamically balance the segmentation quality and segmentation time, select the optimal superpixel segmentation method and number of segmentations based on the comprehensive score, and perform preliminary superpixel segmentation on the landslide image.

[0041] S4, an improved superpixel region adjacency graph RAG is constructed to describe the similarity between superpixels, and the normalized cut method is used to merge and segment the landslide image superpixels.

[0042] S5, denoising the landslide image processed in step S4.

[0043] Specifically, step S1 includes the following steps: S1.1, Calculation of regional consistency : ; in, represents the number of superpixels, is the color variance, are the three channels in the LAB color space, For the channel, is a super pixel.

[0044] Regional consistency reflects the color consistency within the superpixel. First, the image is converted to the LAB color space, and the color variance is calculated for each superpixel region. The smaller the variance, the more consistent the color in the region, and the higher the quality of superpixel segmentation.

[0045] S1.2, Calculation of shape regularity : ; in, is the perimeter of the superpixel, is the area of ​​the superpixel.

[0046] Shape regularity measures whether the shape of a superpixel is close to a square or a circle. The smaller the value, the more regular the shape and the higher the quality of superpixel segmentation. The principle of shape regularity is to evaluate the regularity of the superpixel shape by calculating the ratio of the perimeter to the area of ​​each superpixel.

[0047] S1.3, calculate the variance between superpixels : ; in, For the The average color of superpixels, Superpixel and The color distance between them.

[0048] The variance between superpixels is used to measure the color difference between different superpixels. The larger the variance, the more obvious the color difference between superpixels. Therefore, the larger the variance between superpixels, the higher the segmentation quality.

[0049] S1.4, calculate the computational time of different superpixel segmentation methods. In the landslide monitoring application scenario, the computational efficiency of the superpixel segmentation algorithm is crucial. Although some algorithms may perform well in segmentation quality, if their computational time is too long, it is difficult to meet the real-time and high-efficiency requirements of landslide monitoring. Therefore, in the landslide monitoring task, the balance between the algorithm's processing speed and segmentation accuracy is particularly critical. Excessive computational time will directly affect the system's response speed and the timeliness of monitoring.

[0050] Specifically, the calculation of the weights of each evaluation index based on principal component analysis PCA in step S2 specifically includes the following steps: S2.1, extract the normalized evaluation indicator matrix.

[0051] Since the dimensions of each evaluation indicator are different, the evaluation indicators need to be normalized. Normalization uses the 0-1 standardization method, and the calculation formula is: ; in, is the original indicator value, is the normalized value. After normalization, the data of all indicators are scaled to the same range (0-1), so that different indicators can be compared and weighted.

[0052] S2.2, perform principal component analysis on the evaluation indicators to obtain the variance explanation rate of each principal component.

[0053] S2.3, calculate the load coefficient of each evaluation index on different principal components, that is, the principal component matrix.

[0054] S2.4, calculate the total contribution of each evaluation indicator based on the load coefficient and variance explanation rate, and standardize the total contribution to the weight: ; in, For the The final weight of the evaluation indicators; For the The eigenvalues ​​of the principal components; For the The evaluation index is The loading coefficients on the principal components; is the number of principal components retained; is the total number of indicators (such as regional consistency, shape regularity, etc.).

[0055] Based on the existing evaluation indicators (regional consistency, shape regularity, etc.), this paper proposes a data-driven principal component analysis (PCA) dynamic weight calculation method, rather than relying on manual experience or fixed weights. PCA analyzes the contribution of each indicator to the overall segmentation effect and automatically assigns weights to ensure that the evaluation model adapts to different image qualities, such as blurred images caused by heavy rain.

[0056] Specifically, the comprehensive score in step S3 The calculation formula is: ; in, , , , are the weights of regional consistency, shape regularity, superpixel variance and segmentation time, respectively. is the normalized value of execution time. Execution time is a negative indicator, so its impact needs to be subtracted.

[0057] Through the above process, the performance of each superpixel segmentation method under different numbers of superpixels is obtained, and based on normalization, principal component analysis and comprehensive score sorting, the optimal superpixel segmentation method and number of segmentations are finally selected.

[0058] The goal of superpixel segmentation is to divide adjacent pixels into several uniform regions according to features such as color and texture, thereby reducing the computational complexity of subsequent processing. According to the selected superpixel segmentation method (SLIC, SEEDS or LSC) and the number of segmentations, the present invention performs different preprocessing on the landslide image.

[0059] Specifically, in step S4, when constructing the region adjacency graph, a brightness difference suppression factor is introduced to dynamically adjust the similarity weights between superpixels. The calculation formula is: ; in, , Superpixel and The color mean of , Superpixel and The mean brightness of , are the standard deviation adjustment parameters for color and brightness, is an exponential function. (In heavy rain scenario is significantly reduced to suppress the similarity of brightness mutation areas).

[0060] When constructing the region adjacency graph, traditional methods only consider color and spatial similarity and do not optimize for high-brightness noise caused by heavy rain. Formula, the present invention introduces a brightness difference suppression factor, improves the region adjacency graph construction method, and dynamically adjusts the similarity weights between superpixels, thereby reducing the probability of mismerging reflective areas.

[0061] Specifically, after constructing the superpixel region adjacency graph, the normalized cuts method is used to further merge and segment the superpixels. The normalized cuts algorithm divides the image by minimizing the cut edges in the graph, so that the similarity within the segmented sub-images is as large as possible, while the similarity between sub-images is as small as possible.

[0062] The optimization goal of normalized cutting in step S4 is The formula is: ; in, and are the two sub-graphs of the segmentation. is the edge weight between subgraphs, For subgraph With Figure The sum of the connection weights of all nodes in .

[0063] Specifically, in order to achieve the purpose of accurately identifying the landslide area, the present invention proposes an automatic denoising method.

[0064] Step S5 specifically performs binarization processing on the grayscale image, then removes the small area therein, and only retains the most significant area, which is expressed as: ; in, is the input grayscale image; The optimal threshold calculated for the OTSU method; is the pixel in the binary image The connected area to which it belongs; is the area, For connected areas area; is the proportionality coefficient, To obtain the maximum value function, is a binary function.

[0065] In the preliminary results after superpixel segmentation, different superpixel areas have different grayscale values. In order to further simplify the image and highlight the target area, the grayscale image needs to be binarized first. This application uses the OTSU method, which is an adaptive threshold segmentation technique that automatically determines the optimal threshold by maximizing the inter-class variance. . After binarizing the image, the next step is to remove small areas and keep only the most significant areas. This process is achieved through connected component analysis. Connected component analysis is used to identify all connected areas in the image and assign a unique label to each connected area. The goal of this process is to retain the largest white connected area and remove other smaller areas. After the above steps, the final image only contains the largest white connected area. The result of this process retains the most significant target area in the image and removes noise and small areas.

[0066] In order to verify the effect of the present invention, the recognition results of the method proposed in this application are compared with the recognition results of the U-Net neural network recognition method: The final recognition result of this application method is shown in the attached Figure 2 and 3 This application produced a total of 47 images and their corresponding label data to train the U-Net model. The final comparison results are shown in Figure 4 As shown in the figure, the similarity coefficient DSC, volume / area overlap error VOE and relative volume difference RVD evaluation indicators of the method of the present application are 0.995, 0.010 and 0.005 respectively. The DSC, VOE and RVD of the U-Net neural network segmentation method are 0.898, 0.182 and 0.144 respectively. All indicators of the method of the present application are better than those of the U-Net neural network recognition method.

[0067] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A short-time series landslide image recognition method based on adaptive superpixel segmentation and fusion, characterized in that: The specific steps include: S1, for different superpixel segmentation methods and number of segmentations, calculate multiple evaluation indicators; the evaluation indicators include: regional consistency, shape regularity, superpixel variance and segmentation time; S2, normalizing the evaluation indicators in step S1, and calculating the weights of each evaluation indicator based on principal component analysis PCA; S3, constructing a comprehensive evaluation calculation method for superpixel segmentation, selecting the optimal superpixel segmentation method and the number of segmentations according to the comprehensive score, and performing preliminary superpixel segmentation on the landslide image; S4, construct an improved superpixel region adjacency graph RAG to describe the similarity between superpixels, and use the normalized cut method to merge and segment the landslide image superpixels; S5, denoising the landslide image processed in step S4.

2. According to the method of claim 1, the method is characterized by: Step S1 specifically includes the following steps: S1.1, Calculation of regional consistency : ； in, represents the number of superpixels, is the color variance, are the three channels in the LAB color space, For the channel, is a superpixel; S1.2, Calculation of shape regularity : ； in, is the perimeter of the superpixel, is the area of ​​the superpixel; S1.3, calculate the variance between superpixels : ； in, For the The average color of superpixels, Superpixel and The color distance between S1.4, calculate the computation time of different superpixel segmentation methods.

3. According to the method of claim 1, the method is characterized by: The calculation of the weights of each evaluation index based on principal component analysis PCA in step S2 specifically includes the following steps: S2.1, extract the normalized evaluation index matrix; S2.2, perform principal component analysis on the evaluation indicators to obtain the variance explanation rate of each principal component; S2.3, calculate the load coefficient of each evaluation index on different principal components, that is, the principal component matrix; S2.4, calculate the total contribution of each evaluation indicator based on the load coefficient and variance explanation rate, and standardize the total contribution to the weight: ； in, For the The final weight of the evaluation indicators; For the The eigenvalues ​​of the principal components; For the The evaluation index is The loading coefficients on the principal components; is the number of principal components retained; is the total number of indicators.

4. The short-time series landslide image recognition method based on adaptive superpixel segmentation and fusion according to claim 1 is characterized in that: Comprehensive score in step S3 The calculation formula is: ； in, , , , are the weights of regional consistency, shape regularity, superpixel variance and segmentation time, respectively. is the normalized value of the execution time.

5. The short-time series landslide image recognition method based on adaptive superpixel segmentation and fusion according to claim 1 is characterized in that: In step S4, when constructing the region adjacency graph, the similarity weights between superpixels are dynamically adjusted. The calculation formula is: ； in, , Superpixel and The color mean of , Superpixel and The mean brightness of , are the standard deviation adjustment parameters for color and brightness, is an exponential function.

6. The short-time series landslide image recognition method based on adaptive superpixel segmentation and fusion according to claim 5 is characterized in that: The optimization goal of normalized cutting in step S4 is The formula is: ； in, and are the two sub-graphs of the segmentation. is the edge weight between subgraphs, For subgraph With Figure The sum of the connection weights of all nodes in .

7. The short-time series landslide image recognition method based on adaptive superpixel segmentation and fusion according to claim 1 is characterized in that: Step S5 specifically performs binarization processing on the grayscale image, then removes the small area therein, and only retains the most significant area, which is expressed as: ； in, is the input grayscale image; The optimal threshold calculated by Otsu method; is the pixel in the binary image The connected area to which it belongs; is the area, For connected areas area; is the proportionality coefficient, To obtain the maximum value function, is a binary function.

Images