Automatic landslide identification method, system and computer equipment based on large visual model

Abstract

The present invention discloses a method, system and computer equipment for automatic identification of landslides based on a large visual model, and relates to the technical field of landslide identification. The method comprises: constructing a landslide semantic segmentation model, training the landslide semantic segmentation model, and obtaining a trained landslide semantic segmentation model; using a trained preset deep learning model, obtaining a detection frame of a landslide target of a landslide image to be identified, and obtaining landslide semantic prompt information of the landslide image to be identified according to the detection frame of the landslide target of the landslide image to be identified; inputting the landslide semantic prompt information of the landslide image to be identified into the trained landslide semantic segmentation model, and obtaining a semantic segmentation mask of the landslide image to be identified. The present invention combines the advantages of remote sensing big data with the large visual model, and effectively improves the problems of traditional landslide identification methods in terms of poor spatial generalization ability and strong sample dependence.

Description

Background Art

[0002] Geological disasters can cause environmental damage, casualties and huge property losses. Therefore, the use of remote sensing data to identify and monitor existing landslides is of great significance for the identification and verification of geological disaster risks.

[0003] At present, although visual interpretation has a high recognition accuracy, the process is time-consuming and laborious. Therefore, in recent years, landslide recognition methods based on artificial intelligence have been highly sought after. Such methods can be roughly divided into two types, including pixel-based methods and object-based methods. Specifically:

[0004] 1) Pixel-based methods: mainly use machine learning methods, such as decision trees, support vector machines, artificial neural networks, and random forests. The disadvantage of this type of method is that it is easy to produce a "salt and pepper effect" and the utilization rate of data space information is insufficient.

[0005] 2) Object-based computing methods: Prior knowledge of the object’s shape, color, texture, etc. can be incorporated into the algorithm. Although this type of method effectively improves the “salt and pepper effect” problem, it is difficult to effectively identify targets with variable scales and low recognizability.

[0006] In recent years, a large number of landslide identification methods based on convolutional neural networks have emerged, which can not only effectively utilize the spatiotemporal joint features of remote sensing data to effectively overcome the "salt and pepper effect", but also significantly improve the accuracy of landslide identification in different scenarios. However, due to the limitation of training data, it is currently difficult for methods based on convolutional neural networks to achieve universal and automatic identification while maintaining high accuracy. In recent years, the emergence of large computer vision models has provided a possible way to solve this problem.

[0007] Large models have set off a pioneering wave in the field of artificial intelligence. Due to the training of large amounts of data, these models have shown significant generalization capabilities in various scenarios. Pre-trained large models such as BERT and GPT first appeared and developed in the field of natural language processing. The emergence of ChatGPT has had a significant impact on various industries. Inspired by the success of NLP basic models, the exploration of large models in the field of computer vision has also made progress. ViT-22G, SwinTransformer and MAE have become the first generation of large models. Their characteristics make full use of big data and Transformer to achieve optimal performance of visual tasks under single modality; CLIP, as the second generation of large models, uses contrastive learning to align the visual and text spaces in noisy image-text pairs, realizes the intercommunication of semantic knowledge under cross-modality, and demonstrates strong zero-shot recognition and generalization capabilities in downstream tasks.

[0008] The introduction of the SAM model marks the birth of the third-generation large model. It not only has the ability of cross-modal and zero-sample recognition, but also has strong interactive capabilities. As the first general computer vision network, it also shows great application potential in the field of remote sensing applications. However, remote sensing data is different from natural images. The diversity of spatial resolution, temporal resolution, spectral resolution and payload types determines the limitations of computer vision models that use natural images as training samples in the application of remote sensing data. Through evaluation, the application of the SAM model to remote sensing data still faces challenges such as the inability to achieve multi-target recognition of similar objects and poor accuracy of downstream task migration. The key to solving the above problems lies in whether the semantic prompt of the target can be accurately obtained. Summary of the invention

[0009] The technical problem to be solved by the present invention is to address the deficiencies of the prior art and specifically provide a landslide automatic identification method, system and computer device based on a large visual model, as follows:

[0010] 1) In the first aspect, the present invention provides a method for automatic identification of landslides based on a large visual model, and the specific technical scheme is as follows:

[0011] Constructing a landslide semantic segmentation model, training the landslide semantic segmentation model, and obtaining a trained landslide semantic segmentation model, wherein the input of the landslide semantic segmentation model is the landslide semantic prompt information, the output of the landslide semantic segmentation model is the semantic segmentation mask of the landslide, and the image encoder of the landslide semantic segmentation model includes a visual large model feature extractor;

[0012] Using the trained preset deep learning model, a detection frame of the landslide target of the landslide image to be identified is obtained, and according to the detection frame of the landslide target of the landslide image to be identified, landslide semantic prompt information of the landslide image to be identified is obtained;

[0013] The landslide semantic prompt information of the landslide image to be identified is input into the trained landslide semantic segmentation model to obtain the semantic segmentation mask of the landslide image to be identified.

[0014] The beneficial effects of the automatic landslide identification method based on a large visual model provided by the present invention are as follows:

[0015] Combining the advantages of remote sensing big data with large visual models has effectively improved the problems of traditional landslide identification methods in terms of poor spatial generalization ability and strong sample dependence. This is of great significance for carrying out wide-area landslide target identification and improving the intelligent level of geological disaster risk identification.

[0016] On the basis of the above scheme, the automatic landslide identification method based on a large visual model of the present invention can also be improved as follows.

[0017] Furthermore, before each isometric global attention module of the visual large model feature extractor, an embeddable fine-tuning module is connected in a skip-layer connection manner, and the embeddable fine-tuning module includes at least one multi-layer perceptron and at least one ReLU activation layer.

[0018] The beneficial effect of adopting the above further solution is: by adding an embeddable fine-tuning module, the adaptability of the large visual model to downstream tasks can be improved, effectively solving the problem that large visual models are difficult to perform transfer learning.

[0019] Furthermore, according to the detection frame of the landslide target of the landslide image to be identified, the landslide semantic prompt information of the landslide image to be identified is obtained, including:

[0020] The detection frame of the landslide target of the landslide image to be identified is feature visualized to obtain an attention heat map of the landslide target of the landslide image to be identified, a plurality of positive sample points are randomly generated within a first probability value range in the attention heat map, a plurality of negative sample points are randomly generated within a second probability value range in the attention heat map, and all the randomly generated positive sample points, negative sample points and the detection frame of the landslide target of the landslide image to be identified constitute landslide semantic prompt information of the landslide image to be identified.

[0021] Furthermore, the preset deep learning model is the YOLOV8 model.

[0022] 2) In the second aspect, the present invention also provides a landslide automatic identification system based on a large visual model, and the specific technical solution is as follows:

[0023] Including building training module, acquisition module and recognition module;

[0024] Constructing a training module for: constructing a landslide semantic segmentation model, training the landslide semantic segmentation model, and obtaining a trained landslide semantic segmentation model, wherein the input of the landslide semantic segmentation model is the landslide semantic prompt information, the output of the landslide semantic segmentation model is the semantic segmentation mask of the landslide, and the image encoder of the landslide semantic segmentation model includes a visual large model feature extractor;

[0025] The acquisition module is used to: obtain the detection frame of the landslide target of the landslide image to be identified by using the trained preset deep learning model, and obtain the landslide semantic prompt information of the landslide image to be identified according to the detection frame of the landslide target of the landslide image to be identified;

[0026] The recognition module is used to input the landslide semantic prompt information of the landslide image to be identified into the trained landslide semantic segmentation model to obtain the semantic segmentation mask of the landslide image to be identified.

[0027] On the basis of the above scheme, the automatic landslide identification system based on visual large model of the present invention can also be improved as follows.

[0028] Furthermore, before each isometric global attention module of the visual large model feature extractor, an embeddable fine-tuning module is connected in a skip-layer connection manner, and the embeddable fine-tuning module includes at least one multi-layer perceptron and at least one ReLU activation layer.

[0029] Furthermore, the acquisition module is specifically used for:

[0030] The detection frame of the landslide target of the landslide image to be identified is feature visualized to obtain an attention heat map of the landslide target of the landslide image to be identified, a plurality of positive sample points are randomly generated within a first probability value range in the attention heat map, a plurality of negative sample points are randomly generated within a second probability value range in the attention heat map, and all the randomly generated positive sample points, negative sample points and the detection frame of the landslide target of the landslide image to be identified constitute landslide semantic prompt information of the landslide image to be identified.

[0031] Furthermore, the preset deep learning model is the YOLOV8 model.

[0032] 3) In a third aspect, the present invention further provides a computer device, comprising a processor coupled to a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor, so that the computer device implements any of the above-mentioned automatic landslide identification systems based on a large visual model.

[0033] 4) In a fourth aspect, the present invention further provides a computer-readable storage medium, in which at least one computer program is stored, and the at least one computer program is loaded and executed by a processor so that the computer implements any of the above-mentioned automatic landslide identification systems based on a large visual model.

[0034] It should be noted that the beneficial effects achieved by the technical solutions of the second to fourth aspects of the present invention and the corresponding possible implementation methods can be found in the above-mentioned technical effects of the first aspect and its corresponding possible implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0036] Figure 1 One of the flow charts of a method for automatic landslide identification based on a large visual model according to an embodiment of the present invention;

[0037] Figure 2 Schematic diagram of the landslide semantic segmentation model;

[0038] Figure 3It is a schematic diagram of the YOLOV8 model;

[0039] Figure 4 It is a structural schematic diagram of a landslide automatic identification system based on a large visual model according to an embodiment of the present invention;

[0040] Figure 5 The figure is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0042] like Figure 1 As shown, a method for automatic landslide identification based on a visual large model according to an embodiment of the present invention comprises the following steps:

[0043] S1. constructing a landslide semantic segmentation model, training the landslide semantic segmentation model, and obtaining a trained landslide semantic segmentation model, wherein the input of the landslide semantic segmentation model is the landslide semantic prompt information, the output of the landslide semantic segmentation model is the semantic segmentation mask of the landslide, and the image encoder of the landslide semantic segmentation model includes a visual large model feature extractor;

[0044] The landslide semantic segmentation model includes an image encoder, a keyword decoder and an image decoder. The image encoder of the landslide semantic segmentation model includes a visual large model feature extractor. The visual large model feature extractor can be specifically a Vision Transformer (ViT). For example, the ViT-H / 16 version of ViT is used. The ViT-H / 16 version of ViT mainly includes an upsampling layer, a Patch feature embedding layer and four isometric global attention modules. The isometric global attention module is Figure 2 The Transformer module in can better capture the long-distance dependencies between image slices. The output of the image encoder is the image feature embedding information, which is used to input the image decoder.

[0045] S2. Using the trained preset deep learning model, a detection frame of the landslide target of the landslide image to be identified is obtained, and based on the detection frame of the landslide target of the landslide image to be identified, landslide semantic prompt information of the landslide image to be identified is obtained;

[0046] S3. Inputting the landslide semantic prompt information of the landslide image to be identified into the trained landslide semantic segmentation model to obtain the semantic segmentation mask of the landslide image to be identified.

[0047] An automatic landslide identification method based on a visual large model in an embodiment of the present invention combines the advantages of remote sensing big data with the visual large model, effectively improving the problems of traditional landslide identification methods in terms of poor spatial generalization ability and strong sample dependence, and is of great significance for carrying out wide-area landslide target identification and improving the intelligent level of geological disaster risk identification.

[0048] Optionally, in the above technical solution, before each isometric global attention module of the visual large model feature extractor, an embeddable fine-tuning module is connected in a skip-layer connection manner, and the embeddable fine-tuning module includes at least one multi-layer perceptron and at least one ReLU activation layer.

[0049] Among them, the number of multi-layer perceptrons and ReLU activation layers can be set according to actual conditions, and the number of multi-layer perceptrons and ReLU activation layers is the same, and the multi-layer perceptrons and ReLU activation layers in the embeddable fine-tuning module are set alternately in sequence, and the alternating setting starts with the multi-layer perceptron. That is to say, the multi-layer perceptron located in the first layer of the embeddable fine-tuning module is used to receive the data output by the upper layer (such as the Patch feature embedding layer or the isometric global attention module, etc.), and the ReLU activation layer located in the last layer of the embeddable fine-tuning module sends the data to the lower layer (such as the isometric global attention module or the image decoder, etc.). By adding the embeddable fine-tuning module, the adaptability of the large visual model to downstream tasks can be improved, which effectively solves the problem that the large visual model is difficult to perform transfer learning.

[0050] Optionally, in the above technical solution, obtaining the landslide semantic prompt information of the landslide image to be identified according to the detection frame of the landslide target of the landslide image to be identified includes:

[0051] The detection frame of the landslide target of the landslide image to be identified is feature visualized to obtain an attention heat map of the landslide target of the landslide image to be identified, a plurality of positive sample points are randomly generated within a first probability value range in the attention heat map, a plurality of negative sample points are randomly generated within a second probability value range in the attention heat map, and all the randomly generated positive sample points, negative sample points and the detection frame of the landslide target of the landslide image to be identified constitute landslide semantic prompt information of the landslide image to be identified.

[0052] Among them, the first probability value range is 0.75 to 1, the second probability value range is 0.05 to 0.25, the number of negative sample points is 5, and the number of positive sample points is 5. It should be noted that the first probability value range, the second probability value range, the number of negative sample points and the number of positive sample points can be set according to time conditions.

[0053] Optionally, in the above technical solution, the preset deep learning model is a YOLOV8 model.

[0054] Another embodiment is shown below, and combined with Figure 2 and Figure 3 , the present invention is described. In this embodiment, the preset deep learning model is the YOLOV8 model, the visual large model feature extractor is the ViT-H / 16 version of ViT, and before each isometric global attention module of the visual large model feature extractor, an embeddable fine-tuning module is connected in a skip-layer connection manner, specifically including the following steps:

[0055] S10. Based on the published open source landslide data, the landslide data includes multiple landslide images and labels of each landslide image, a training data set is constructed, the training data set is preprocessed, and the label of each landslide image is converted into a search frame, specifically including S100 to S101:

[0056] S100. Collect the currently open-source landslide data worldwide and construct a training data set. The training data set includes m landslide images and labels. First, the landslide images are cropped to form a first data set D1 including m slices of 1024×1024×3 size. The first data set D1 is normalized using the maximum and minimum normalization method so that the pixel value in each slice ranges between 0 and 1. The normalized slices constitute a second data set D2.

[0057] S101, cropping m labels to form m cropped labels of 1024×1024×1, binarizing each cropped label, specifically assigning the pixel value representing the landslide target in each cropped label to 1, and assigning the other pixel values ​​to 0, to obtain a binarized label data set L1. For each cropped label in the binarized label data set L1, the number of landslide targets in the binarized label data set L1 is obtained by edge detection algorithm, and the coordinates (x , y ) of the rectangular box surrounding the landslide target are obtained for each landslide target. min ,y min , x max ,y max ), x min The minimum value of the rectangular box representing the landslide target on the x-axis, min The minimum value of the rectangular box representing the landslide target on the y-axis, x max The maximum value of the rectangular box representing the landslide target on the x-axis, max The maximum value of the rectangular box representing the landslide target on the y-axis is the coordinate of the four corners of the detection box. The detection box corresponding to each landslide target is obtained, and the detection box corresponding to each landslide target constitutes a new label data set L2.

[0058] S11, constructing a YOLOV8 model, and training the YOLOV8 model based on the second data set D2 and the new label data set L2 to obtain a trained YOLOV8 model, the output of the trained YOLOV8 model is a detection frame of the landslide target, Figure 2 In the YOLOV8 model, the backbone network, bottleneck structure and detector are all existing technologies and no additional explanation is given.

[0059] S12. Constructing landslide semantic segmentation model:

[0060] In the present invention, the landslide semantic segmentation model includes an image encoder, a keyword decoder and an image decoder, and also includes multiple embeddable fine-tuning modules. Specifically, before each isometric global attention module of the visual large model feature extractor, an embeddable fine-tuning module is connected in a skip-layer connection manner. The embeddable fine-tuning module includes at least one multi-layer perceptron and at least one ReLU activation layer, so as to improve its adaptability to the landslide identification task, and use SAM to initialize the parameters of the visual large model feature extractor.

[0061] Among them, the landslide semantic segmentation model in the present invention is defined as an LSDetctor model. Specifically, the LSDetctor model can be built in the Pytorch environment. The LSDetctor model can better capture the long-distance dependencies between image slices. The embeddable fine-tuning module consists of two groups of multi-layer perceptrons and two layers of ReLU activation layers. The embeddable fine-tuning module allows the model to be more adaptable to downstream tasks while fully utilizing the extraction capabilities of the pre-trained ViT large model, thereby enabling the model to complete transfer learning at a lower computational cost. The input of the image encoder is an image of size 1024×1024×3, and the output of the image encoder is a 16×16 image feature embedding processed by the patch embedding operation and the ViT module. The image decoder consists of a ViT module and two layers of multi-layer perceptrons, which are used to restore the image feature embedding information to label data consistent with the size of the input image. In addition to receiving the embedded image features, the image decoder also accepts the output information of the keyword decoder. The input of the keyword decoder is prompt information such as positive and negative coordinate points and detection boxes, and the output is a feature map of the prompt information in a high-dimensional space. The image encoder of LSDetctor is initialized with parameters from the large SAM model trained on massive natural images, so it has strong general semantic segmentation performance. The adaptability of the model to landslide identification tasks can be improved through the embeddable fine-tuning module.

[0062] For each binarized label in the binarized label dataset L1, 5 positive sample points are randomly generated in the area where the value of each binarized label in L1 is 1. ′POS , randomly generate 5 negative sample points from the area with a value of 0 ′ NAG , composed of m groups [Point ′ POS , Point ′ NAG ], and the m groups of detection frames in L2 constitute m groups of real landslide semantic prompt information corresponding to the second data set D2, and use the second data set D2, the binarized label data set L1 and the m groups of real landslide semantic prompt information corresponding to the second data set D2 to train the landslide semantic segmentation model to obtain the trained landslide semantic segmentation model.

[0063] S13, feature visualization is performed on the detection frame of the landslide target of the landslide image to be identified, an attention heat map of the landslide target of the landslide image to be identified is obtained, a plurality of positive sample points are randomly generated within a first probability value range in the attention heat map, a plurality of negative sample points are randomly generated within a second probability value range in the attention heat map, and all the randomly generated positive sample points, negative sample points and the detection frame of the landslide target of the landslide image to be identified constitute the landslide semantic prompt information of the landslide image to be identified, wherein:

[0064] 1) The first probability value range is 0.75 to 1, the second probability value range is 0.05 to 0.25, the number of negative sample points is 5, and the number of positive sample points is 5;

[0065] 2) The process of calculating the attention heat map corresponding to the detection box of the landslide target in the landslide image to be identified is as follows:

[0066] Assume that the output feature map of the last convolutional layer of the trained YOLOV8 model is A. Then, calculate the gradient of the landslide target C with respect to the feature map A. Next, Perform global average pooling to obtain weight x C , weight x C represents the importance of landslide target C to different location features. Finally, the weight x C The weighted superposition is performed with the feature map A to obtain the visualized attention heat map Sc of the landslide target, so as to realize the feature visualization of the detection frame of the landslide target and obtain the weight x C The calculation process is expressed as follows:

[0067]

[0068] Where Z is the size of the gradient, A ij is the activation value of the (i, j)th position of the feature map A. The calculation process of the visualized attention heat map Sc is as follows:

[0069] S c =ReLU(∑ i ∑ j x C ·A ij ) (2)

[0070] Among them, ReLU(∑ i ∑ j x C ·A ij ) represents the pair (∑ i ∑ j x C ·A ij ) takes a non-negative value.

[0071] 3) From the attention heat map S c Select the part with probability value between 0.75 and 1, and randomly generate 5 positive sample points POS , randomly generate 5 negative sample points from the probability value between 0.05 and 0.25 NAG , all positive sample points Point POS 、All negative sample Point NAG And the detection box Bbox of the landslide target in the landslide image to be identified constitutes the landslide semantic prompt information of the landslide target in the landslide image to be identified.

[0072] Among them, Bbox = Model YOLO (img) (3)

[0073] Model YOLO represents the trained YOLOV8 model, and img represents the landslide image to be identified.

[0074] S14, inputting the landslide semantic prompt information of the landslide image to be identified into the trained landslide semantic segmentation model to obtain the semantic segmentation mask of the landslide image to be identified. Specifically:

[0075] The trained YOLOV8 model is used to predict the unlabeled landslide data, i.e., the landslide image to be identified, and the landslide semantic prompt information of the landslide image to be identified is obtained: [Point POS , Point NAG ] and [Bbox], input the landslide data and semantic hint information into the trained landslide semantic segmentation model Model final Specifically, the landslide semantic prompt information of the landslide image to be identified is input into the keyword prompter, mapped into a feature vector and then passed into the image decoder.

[0076] Among them, by using formula (4) to predict any unlabeled landslide image, that is, the landslide image to be identified, the corresponding landslide semantic segmentation mask result can be obtained, that is, the semantic segmentation mask of the landslide image to be identified can be obtained.

[0077] M=Model final {img, ([Point POS , Point NAG ],[Bbox])}(4)

[0078] Where M is the semantic segmentation mask of the landslide image to be identified.

[0079] The obtained YOLOV8 model predicts the unlabeled landslide data to obtain the semantic hint information of the landslide. At the same time, the landslide data and semantic hint information are input into the final model to obtain the semantic segmentation mask of the landslide, thereby realizing the accurate identification of the landslide boundary. Since the model is fine-tuned based on a large model, it is superior to the traditional semantic segmentation network in recognition accuracy and has good generalization ability.

[0080] The present invention proposes a method for automatic landslide identification based on a large visual model. A spatially generalizable landslide semantic prompt information generation model is trained through massive landslide sample data, and the automatic identification and semantic segmentation of landslide targets are completed in combination with the large visual model. The present invention proposes a method for improving the adaptability of the large visual model to downstream tasks by adding embeddable modules, which effectively solves the problem that the large visual model is difficult to transfer learning. The proposed automatic landslide identification method cleverly combines the advantages of remote sensing big data with the large visual model, effectively improving the problems of traditional landslide identification methods in terms of poor spatial generalization ability and strong sample dependence, and is of great significance for carrying out wide-area landslide target identification and improving the intelligent level of geological disaster risk identification.

[0081] In the above embodiments, although the steps are numbered S1, S2, etc., these are only specific embodiments given by the present invention. Those skilled in the art may adjust the execution order of S1, S2, etc. according to actual conditions, which is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0082] like Figure 4 As shown, a landslide automatic identification system 200 based on a visual large model according to an embodiment of the present invention includes a training module 201, an acquisition module 202 and an identification module 203;

[0083] The training module 201 is used to: construct a landslide semantic segmentation model, train the landslide semantic segmentation model, and obtain a trained landslide semantic segmentation model, wherein the input of the landslide semantic segmentation model is the landslide semantic prompt information, the output of the landslide semantic segmentation model is the semantic segmentation mask of the landslide, and the image encoder of the landslide semantic segmentation model includes a visual large model feature extractor;

[0084] The acquisition module 202 is used to: obtain a detection frame of a landslide target of a landslide image to be identified by using a trained preset deep learning model, and obtain landslide semantic prompt information of the landslide image to be identified according to the detection frame of the landslide target of the landslide image to be identified;

[0085] The recognition module 203 is used to input the landslide semantic prompt information of the landslide image to be recognized into the trained landslide semantic segmentation model to obtain the semantic segmentation mask of the landslide image to be recognized.

[0086] Optionally, in the above technical solution, before each isometric global attention module of the visual large model feature extractor, an embeddable fine-tuning module is connected in a skip-layer connection manner, and the embeddable fine-tuning module includes at least one multi-layer perceptron and at least one ReLU activation layer.

[0087] Optionally, in the above technical solution, the acquisition module 202 is specifically used for:

[0088] The detection frame of the landslide target of the landslide image to be identified is feature visualized to obtain an attention heat map of the landslide target of the landslide image to be identified, a plurality of positive sample points are randomly generated within a first probability value range in the attention heat map, a plurality of negative sample points are randomly generated within a second probability value range in the attention heat map, and all the randomly generated positive sample points, negative sample points and the detection frame of the landslide target of the landslide image to be identified constitute landslide semantic prompt information of the landslide image to be identified.

[0089] Optionally, in the above technical solution, the preset deep learning model is a YOLOV8 model.

[0090] It should be noted that the beneficial effects of the landslide automatic identification system 200 based on a visual large model provided in the above embodiment are the same as the beneficial effects of the landslide automatic identification method based on a visual large model, which will not be repeated here. In addition, when the system provided in the above embodiment realizes its functions, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to actual conditions to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0091] like Figure 5 As shown, a computer device 300 according to an embodiment of the present invention includes a processor 320, the processor 320 is coupled to a memory 310, and the memory 310 stores at least one computer program 330, and the at least one computer program 330 is loaded and executed by the processor 320, so that the computer device 300 implements any of the above-mentioned landslide automatic identification methods based on a visual large model, specifically:

[0092] The computer device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors 320 (Central Processing Units, CPU) and one or more memories 310, wherein at least one computer program 330 is stored in the one or more memories 310, and the at least one computer program 330 is loaded and executed by the one or more processors 320, so that the computer device 300 implements any of the automatic landslide identification methods based on a visual large model provided in the above embodiments. Of course, the computer device 300 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the computer device 300 may also include other components for implementing device functions, which will not be described in detail here.

[0093] A computer-readable storage medium according to an embodiment of the present invention stores at least one computer program, which is loaded and executed by a processor so that a computer implements any of the above-mentioned methods for automatic identification of landslides based on a large visual model.

[0094] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0095] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes any of the above-mentioned methods for automatic identification of landslides based on a large visual model.

[0096] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects and represent the definition of a specific order or sequence. The order of use of similar objects can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0097] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.

[0098] Any combination of one or more computer-readable media can be used. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0099] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A landslide automatic identification method based on a visual large model, characterized in that: include: Constructing a landslide semantic segmentation model, training the landslide semantic segmentation model to obtain a trained landslide semantic segmentation model, wherein the input of the landslide semantic segmentation model is landslide semantic prompt information, the output of the landslide semantic segmentation model is a semantic segmentation mask of the landslide, and the image encoder of the landslide semantic segmentation model includes a visual large model feature extractor; Using the trained preset deep learning model, a detection frame of a landslide target of the landslide image to be identified is obtained, and according to the detection frame of the landslide target of the landslide image to be identified, landslide semantic prompt information of the landslide image to be identified is obtained; The preset deep learning model is the YOLOV8 model; Inputting the landslide semantic prompt information of the landslide image to be identified into the trained landslide semantic segmentation model to obtain a semantic segmentation mask of the landslide image to be identified; Before each isometric global attention module of the visual large model feature extractor, an embeddable fine-tuning module is connected in a skip-layer connection manner, and the embeddable fine-tuning module includes at least one multi-layer perceptron and at least one ReLU activation layer; According to the detection frame of the landslide target of the landslide image to be identified, obtaining the landslide semantic prompt information of the landslide image to be identified, including: Performing feature visualization on the detection frame of the landslide target of the landslide image to be identified, obtaining an attention heat map of the landslide target of the landslide image to be identified, randomly generating a plurality of positive sample points within a first probability value range in the attention heat map, and randomly generating a plurality of negative sample points within a second probability value range in the attention heat map, all randomly generated positive sample points, negative sample points and the detection frame of the landslide target of the landslide image to be identified constitute landslide semantic prompt information of the landslide image to be identified; The method of inputting the landslide semantic prompt information of the landslide image to be identified into the trained landslide semantic segmentation model to obtain the semantic segmentation mask of the landslide image to be identified includes: The trained YOLOV8 model is used to predict the unlabeled landslide data, i.e., the landslide image to be identified, and the landslide semantic prompt information of the landslide image to be identified is obtained: [Point POS , Point NAG ] and [Bbox], input the landslide data and semantic hint information into the trained landslide semantic segmentation model Model final ,Specifically: the landslide semantic prompt information of the landslide image to be identified is input into the ,keyword prompter, mapped into a feature vector and then passed into the ,image decoder; Among them, using the formula M = Model final {img,([Point POS, Point NAG ],[Bbox])} predict any unlabeled landslide image, i.e., the landslide image to be identified, and obtain the corresponding landslide semantic segmentation mask result, i.e., the semantic segmentation mask of the landslide image to be identified, where M is the semantic segmentation mask of the landslide image to be identified, and img represents the landslide image to be identified.

2. A landslide automatic identification system based on a large visual model, characterized in that: Including building training module, acquisition module and recognition module; The construction training module is used to: construct a landslide semantic segmentation model, train the landslide semantic segmentation model, and obtain a trained landslide semantic segmentation model, wherein the input of the landslide semantic segmentation model is landslide semantic prompt information, the output of the landslide semantic segmentation model is a semantic segmentation mask of the landslide, and the image encoder of the landslide semantic segmentation model includes a visual large model feature extractor; The acquisition module is used to: obtain a detection frame of a landslide target of a landslide image to be identified by using a trained preset deep learning model, and obtain landslide semantic prompt information of the landslide image to be identified according to the detection frame of the landslide target of the landslide image to be identified; The preset deep learning model is the YOLOV8 model; The recognition module is used to: input the landslide semantic prompt information of the landslide image to be identified into the trained landslide semantic segmentation model to obtain the semantic segmentation mask of the landslide image to be identified; Before each isometric global attention module of the visual large model feature extractor, an embeddable fine-tuning module is connected in a skip-layer connection manner, and the embeddable fine-tuning module includes at least one multi-layer perceptron and at least one ReLU activation layer; The acquisition module is specifically used for: Performing feature visualization on the detection frame of the landslide target of the landslide image to be identified, obtaining an attention heat map of the landslide target of the landslide image to be identified, randomly generating a plurality of positive sample points within a first probability value range in the attention heat map, and randomly generating a plurality of negative sample points within a second probability value range in the attention heat map, all randomly generated positive sample points, negative sample points and the detection frame of the landslide target of the landslide image to be identified constitute landslide semantic prompt information of the landslide image to be identified; The method of inputting the landslide semantic prompt information of the landslide image to be identified into the trained landslide semantic segmentation model to obtain the semantic segmentation mask of the landslide image to be identified includes: The trained YOLOV8 model is used to predict the unlabeled landslide data, i.e., the landslide image to be identified, and the landslide semantic prompt information of the landslide image to be identified is obtained: [Point POS , Point NAG ] and [Bbox], input the landslide data and semantic hint information into the trained landslide semantic segmentation model Model final ,Specifically: the landslide semantic prompt information of the landslide image to be identified is input into the ,keyword prompter, mapped into a feature vector and then passed into the ,image decoder; Among them, using the formula M = Model final {img,([Point POS, Point NAG ],[Bbox])} predict any unlabeled landslide image, i.e., the landslide image to be identified, and obtain the corresponding landslide semantic segmentation mask result, i.e., the semantic segmentation mask of the landslide image to be identified, where M is the semantic segmentation mask of the landslide image to be identified, and img represents the landslide image to be identified.

3. A computer device, characterized in that: The computer device includes a processor, the processor is coupled to a memory, at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the computer device implements the automatic landslide identification method based on a visual large model as described in claim 1.

4. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the computer implements the automatic landslide identification method based on a visual large model as claimed in claim 1.

Images