Marine disaster risk photo identification method based on deep artificial neural network

Abstract

The invention discloses an ocean disaster risk photo identification method based on a deep artificial neural network, and belongs to the technical field of photo data intelligent processing. The method comprises the following steps: determining a disaster-bearing body plot through a remote sensing image and performing multi-angle shooting to obtain a ground acquisition photo data set; checking the label conformity through visual interpretation, marking a marker surface feature, and constructing a marine disaster risk photo sample library; constructing a scene classification model based on a Transform architecture and a target detection model based on a self-attention mechanism, and training the models by using a sample library; processing the sample library by adopting momentum contrast learning and a ConvNeXt model, and constructing a marine disaster risk knowledge graph; integrating the trained model and the knowledge graph to form a complete recognition model; and inputting the unclassified plot pattern spots into the model to complete automatic identification. The method is mainly used for automatically identifying the pictures of the high-risk disaster-bearing body of the ocean disaster in the coastal region.

Description

Technical Field

[0001] This invention belongs to the field of intelligent identification and classification technology of marine disaster risk photos, specifically involving the application of image feature extraction, model training and knowledge graph integration technology based on deep artificial neural networks in the identification of high-risk disaster-bearing bodies in marine disasters. Background Technology

[0002] As a crucial link in Earth's ecosystem, the ocean is of great significance to global ecological balance and sustainable development of human society. However, it faces frequent marine disasters such as storm surges and tsunamis, posing a serious threat to coastal areas. Therefore, it is necessary to collect and identify ground photographs of high-risk disaster-bearing bodies along the coast. Traditional methods for marine disaster risk photo identification rely on human experience and expert knowledge, resulting in high subjectivity and low efficiency. Furthermore, this field faces the challenge of a scarcity of high-quality, accurately labeled data. Moreover, marine disaster risk photo identification has unique requirements compared to general photo identification, making it a complex problem involving multiple disciplines. While deep learning technology provides an objective and efficient identification path, improving data usability through automatic learning of image features combined with data augmentation preprocessing, and enhancing feature extraction capabilities by introducing attention mechanisms and residual connections, and constructing multi-task learning models to achieve integrated processing of disaster identification and impact assessment, existing technologies still have shortcomings.

[0003] For example, Chinese invention patent CN112949612A proposes a coastal land cover classification method based on high-resolution UAV remote sensing imagery. It utilizes an improved PSPNet algorithm and replaces the backbone feature extraction network with MobileNetV2 to reduce computation and improve classification efficiency. However, it relies on a specifically optimized PSPNet model, which may have insufficient generalization ability and robustness for coastal land cover remote sensing imagery from different regions and time periods. Furthermore, there is limited discussion on the efficient acquisition and processing of large-scale, multi-source remote sensing data. Another Chinese invention patent CN118608952A combines a GEE platform with a U-Net model to achieve coastal land cover remote sensing monitoring. It uses the GEE platform to solve the problems of difficult acquisition and complex preprocessing of traditional remote sensing data and establishes a model evaluation mechanism to ensure classification reliability. However, it relies on the GEE platform and the data sources are mostly medium- and low-resolution satellite imagery. It has limitations in fine land cover classification, especially in depicting the edge details and spatial structure of complex coastal land cover. At the same time, the U-Net model structure is relatively fixed, and its adaptability to specific complex scenarios needs to be improved. Chinese invention patent CN117709580B discloses a vulnerability assessment method for marine disaster-bearing bodies based on SETR and geographic grids. It extracts information about marine disaster-bearing bodies and performs vulnerability assessments using sequence-to-sequence deep learning networks and geographic grid technology. However, its data source relies entirely on remote sensing images from a top-down perspective, lacking detailed information from a ground-level perspective, and it lacks an automated verification mechanism based on scene logic and object relationships, making it prone to logical misjudgments. Chinese invention patent CN120107801B proposes an emergency monitoring method and system for structures based on remote sensing images. It employs a cascaded architecture combining scene classification and target detection for emergency monitoring of structures. However, this technology is still limited to a remote sensing perspective, making it difficult to obtain details of facade damage or specific landmarks. Furthermore, the detection logic lacks high-level semantic logic constraints, and the model's anti-interference ability and logical self-consistency still have room for improvement.

[0004] Overall, existing technologies for marine disaster risk photo recognition still suffer from insufficient integrated processing of "air, space, and ground" data and inadequate precision in refined ground feature classification, making it difficult to fully meet practical application needs. Specifically, insufficient integrated processing of "air, space, and ground" data refers to the failure to organically integrate and fully utilize the three types of data—"air," "space," and "ground"—in marine disaster risk photo recognition and related technical scenarios. "Air" typically refers to aerial remote sensing imagery data (such as high-resolution and medium-resolution remote sensing images), "space" encompasses high-altitude data sources such as satellite remote sensing, and "ground" refers to ground-based photographic data collected on-site (as shown in the image patches). Summary of the Invention

[0005] The purpose of this invention is to provide a marine disaster risk photo recognition model based on deep artificial neural networks, which can solve the problems of insufficient integrated processing of "air, space, and ground" data and insufficient accuracy of refined ground feature classification in the existing identification of coastal risk-bearing bodies, so as to achieve more reliable automatic identification of marine disaster risk photos.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for identifying marine disaster risk from photographs based on deep artificial neural networks, characterized by the following steps:

[0008] The location of the disaster-bearing body is determined by remote sensing imagery, and the contents of the land plot are photographed from multiple angles to obtain a set of ground-collected photo data.

[0009] Based on visual interpretation, the consistency between the patch content of the ground-collected photo data group and the disaster-bearing body type label is verified. At the same time, the landmark features within the ground-collected photo data group are selected and marked to obtain a marine disaster risk photo sample library.

[0010] A scene classification model and an object detection model architecture are constructed, and the scene classification model and the object detection model are trained using the marine disaster risk photo sample library;

[0011] Based on deep learning and self-supervised learning techniques, momentum contrastive learning and ConvNeXt model are used to process the marine disaster risk photo sample library, obtain the relationship between scenes and targets, and construct a marine disaster risk knowledge graph.

[0012] By integrating the trained scene classification model and target detection model with the marine disaster risk knowledge graph, a complete marine disaster risk photo recognition model is formed.

[0013] The unclassified land parcels are input into the marine disaster risk photo recognition model to complete the automatic recognition of marine disaster risk photos.

[0014] In one possible implementation, when determining the location of a disaster-bearing body using remote sensing imagery, high-resolution and medium-resolution remote sensing images of the same area and similar time phases are first selected and downloaded from an open-source remote sensing imagery data website, and areas with less cloud cover are screened. The high-resolution remote sensing images are then visually interpreted to preliminarily confirm areas suspected of being high-risk disaster-bearing bodies. Personnel are then dispatched to conduct on-site investigations of the suspected high-risk disaster-bearing body areas, taking photos of the land plot from multiple angles to obtain a set of ground-collected photo data.

[0015] In one possible implementation, when visually interpreting and verifying the consistency between the patch content and the disaster-bearing body type label of the ground-acquired photo data group, each ground-acquired photo in the data group is interpreted. The overall patch consistency is determined based on the proportion of ground-acquired photos that match the disaster-bearing body type label, and positive and negative samples are divided to obtain a marine disaster risk photo scene classification sample library. Landmark features and their characteristics corresponding to each label in the positive samples are extracted. Several ground-acquired photos are extracted from each label in the positive samples, and the land features in the ground-acquired photos are labeled based on the landmark features to obtain a marine disaster risk photo target detection sample library. The marine disaster risk photo scene classification sample library and the marine disaster risk photo target detection sample library together constitute the marine disaster risk photo sample library.

[0016] In one possible implementation, when constructing the scene classification model architecture, a Transformer architecture is used as the foundation, integrating a hierarchical architecture enhancement, a shift window module, and an EfficientNet network. A large amount of unlabeled data is simply filtered, and the filtered unlabeled data is input into a self-supervised model for training to obtain a preliminary feature extraction model. The data is further filtered and labeled to create a multi-category ground-collected photo dataset of patches. The parameter weights of the preliminary feature extraction model are loaded, and supervised training is performed using the multi-category ground-collected photo dataset of patches to deeply learn the features of different categories. Suitable samples are selected from the marine disaster risk photo sample library, and targeted model training is performed for patches of different marine disaster-bearing body types. After testing on a test set and model integration, a scene classification model is obtained.

[0017] In one possible implementation, when constructing the target detection model architecture, an end-to-end target detection model is built based on the self-attention mechanism, combined with the Swing Transformer network and the window self-attention module; the marine disaster risk photo target detection sample library in the marine disaster risk photo sample library is used as training data for subsequent model training.

[0018] In one possible implementation, when training the scene classification model and the object detection model using the marine disaster risk photo sample library, the training parameters are adjusted, including the number of iterations, batch size, optimizer selection, initial learning rate setting, weight decay rate setting, and loss function selection; a distributed deployment approach is adopted for training, automatic mixed precision is enabled, data is loaded in parallel through subprocesses, and a random erasure strategy is used; gradients are clipped during the training process to limit the gradient norm and prevent gradient explosion; in the training of the object detection model, auxiliary loss and denoising training mechanisms are introduced to stabilize the training process and improve the robustness of the model.

[0019] In one possible implementation, when processing the marine disaster risk photo sample library using momentum contrastive learning and the ConvNeXt model, the relationship between images and entities in the marine disaster risk photo sample library is extracted through supervised learning methods of image classification models; the marine disaster risk photo sample library is trained using machine learning and deep learning techniques to extract features, and the relationship between images and entities is constructed based on the features; the relationship between scenes and targets is obtained using momentum contrastive learning and the ConvNeXt model, the area ratio requirements of landmark features in ground-collected photos of various disaster-bearing bodies are clarified, and a marine disaster risk knowledge graph is constructed.

[0020] In one possible implementation, when integrating the trained scene classification model and target detection model using the marine disaster risk knowledge graph, the scene classification model determines whether the input ground-collected photos and their labels are similar and outputs a Boolean value result; the target detection model identifies each landmark feature in the ground-collected photos and outputs the total area percentage of the landmark features; based on the landmark feature area percentage thresholds corresponding to various labels in the marine disaster risk knowledge graph, the Boolean value result and the total area percentage are inferred and corrected; when the scene classification result matches the label and the landmark feature area percentage meets the threshold requirement, the classification is confirmed to be valid, and a complete marine disaster risk photo recognition model is finally formed.

[0021] In one possible implementation, when unclassified land parcels are input into the marine disaster risk photo recognition model, the unclassified land parcels are first pre-labeled; the marine disaster risk photo recognition model determines whether the content of the pre-labeled land parcels conforms to the label, and outputs the basis for interpreting feature features; when the proportion of ground-collected photos that conform to the label exceeds a preset standard, the land parcel recognition is confirmed to be effective, and automatic recognition is completed.

[0022] In one possible implementation, after obtaining the marine disaster risk photo scene classification sample library and the marine disaster risk photo target detection sample library, the marine disaster risk photo scene classification sample library and the marine disaster risk photo target detection sample library are divided into a training set, a validation set and a test set according to a preset ratio. The training set is used for model training, the validation set is used for parameter adjustment during the model training process, and the test set is used for performance testing after model training.

[0023] Compared with existing technologies, the advantages of this invention are as follows: From the perspective of technical structure, this model first identifies disaster-bearing land parcels by combining high-resolution and medium-resolution remote sensing images of the same area and similar time phases. Then, it acquires ground-collected photo data sets through multi-angle on-site photography, effectively integrating multi-source data from "air, space, and ground," which is different from the limitations of existing technologies that either rely on a single remote sensing data source or lack detailed on-site data. At the same time, surveying professionals visually interpret and verify the matching degree of map patch labels and mark landmark features, dividing scene classifications and target detection sample libraries to ensure the accuracy of sample labeling, thus solving the problem of scarce high-quality labeled data in existing technologies.

[0024] At the model construction and operation level, the scene classification model integrates a hierarchical architecture, a shifted window module, and an EfficientNet network based on the Transformer architecture. It first obtains a preliminary model by mining unlabeled data features through self-supervised learning, and then conducts supervised training and disaster-bearing body-specific training in combination with a sample library. The target detection model builds end-to-end detection capabilities based on a self-attention mechanism and a Swing Transformer network. Both can extract scene and land feature features from marine disaster risk photos more accurately. Compared with the existing technologies that rely on fixed-structure PSPNet or U-Net models and have insufficient generalization ability, the model of this invention is more adaptable to marine disaster risk photos of different regions and time phases. In addition, the marine disaster risk knowledge graph constructed by momentum contrastive learning and the ConvNeXt model can clarify the threshold of the area ratio of landmark land features corresponding to various disaster-bearing bodies. It integrates the results of scene classification and target detection models and performs inference correction. For example, when identifying photovoltaic disaster-bearing bodies in the embodiment, it judges whether the area ratio of landmark land features meets the standard based on the knowledge graph, and finally accurately confirms the validity of the map patch, avoiding the problem of existing technologies that rely on only a single model output and are prone to misjudgment.

[0025] Compared with existing technologies, this invention not only solves the problem of insufficient integrated processing of "space, air, and ground" data, but also improves the accuracy of land cover classification through refined sample annotation, targeted model training, and knowledge graph integration. In practical applications, it can more reliably complete the automatic identification of marine disaster risk photos and meet the actual needs of disaster risk monitoring in coastal areas. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1This is a flowchart of the marine disaster risk photo recognition method based on deep artificial neural networks according to an embodiment of the present invention;

[0028] Figure 2 This is a technical roadmap for constructing a marine disaster risk photo sample library according to an embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of the integrated scene classification model and target detection model using a marine disaster risk knowledge graph, as described in an embodiment of the present invention.

[0030] Figure 4 This is a visualization of the target detection model's recognition results of marine disaster risk photos in an embodiment of the present invention;

[0031] Figure 5 This is a flowchart illustrating the marine disaster risk photo recognition method according to an embodiment of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0033] Example:

[0034] It should be noted that the terms "comprising" and "having" and any variations thereof in the embodiments of the present invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products, or devices.

[0035] See Figure 1 and Figure 5 This embodiment provides a method for marine disaster risk photo recognition based on deep artificial neural networks, including the following steps:

[0036] Step 101: Determine the location of the disaster-bearing body through remote sensing imagery, and take multi-angle photos of the contents of the land plot to obtain a set of ground-collected photo data.

[0037] In determining the location of disaster-bearing bodies using remote sensing imagery, high-resolution and medium-resolution remote sensing images of the same area and similar time phases are first selected and downloaded from open-source remote sensing imagery data websites to screen areas with less cloud cover. The high-resolution remote sensing images are then visually interpreted to preliminarily confirm areas suspected of being high-risk disaster-bearing bodies. Personnel are then dispatched to conduct on-site investigations of the suspected high-risk disaster-bearing body areas, taking photos of the land plots from multiple angles to obtain a set of ground-collected photo data.

[0038] For example, see Figure 2 High-resolution remote sensing imagery data, such as the Landsat series, was selected and downloaded from publicly available data websites like Google Earth. The downloaded high-resolution remote sensing imagery data was visually interpreted to preliminarily identify areas suspected of being high-risk disaster-bearing bodies. High-risk disaster-bearing bodies include 13 categories such as grasslands, roads, port and wharf land, farmland, and photovoltaic power plants. After preliminary identification, personnel were dispatched to conduct on-site investigations based on the geographical coordinates of the suspected high-risk disaster-bearing body areas, taking multi-angle photographs of the land parcels to obtain ground-collected photo data sets (map patches).

[0039] Step 102: Based on visual interpretation, verify the consistency between the patch content of the ground-collected photo data group and the disaster-bearing body type label, and simultaneously select and mark the landmark features within the ground-collected photo data group to obtain the marine disaster risk photo sample library.

[0040] Specifically, when visually interpreting and verifying the consistency between the patch content and the disaster-bearing body type label of the ground-collected photo data group, each ground-collected photo in the ground-collected photo data group is interpreted. The overall patch consistency is determined based on the proportion of ground-collected photo images that match the disaster-bearing body type label. Positive and negative samples are then divided to obtain a marine disaster risk photo scene classification sample library. The landmark features and landmark features corresponding to each label in the positive samples are extracted. Several ground-collected photo images are extracted from each label in the positive samples, and the features in the ground-collected photo images are labeled based on the landmark features to obtain a marine disaster risk photo target detection sample library. The marine disaster risk photo scene classification sample library and the marine disaster risk photo target detection sample library together constitute the marine disaster risk photo sample library.

[0041] For example, surveying professionals visually interpret ground-collected photographic data sets to check whether the labels marked on the patches match the patch content. Specifically, surveying professionals interpret each ground-collected photograph within a patch by considering the overall environment and landmarks. The overall patch conforms to the label if at least 50% of the interpreted ground-collected photographs match the label. Simultaneously, surveying professionals use tools such as labelimg to annotate landmarks as bounding boxes. After completing these operations, the interpretation and annotation results are divided into training, validation, and test sets according to a ratio of 70%, 10%, and 20%, respectively, forming a marine disaster risk photograph scene recognition sample library and a marine disaster risk photograph target detection sample library.

[0042] Step 103: Construct the scene classification model and target detection model architecture, and train the scene classification model and target detection model using the marine disaster risk photo sample library.

[0043] In constructing the scene classification model architecture, the Transformer architecture is used as the foundation, integrating a hierarchical architecture enhancement, a shift window module, and the EfficientNet network. A large amount of unlabeled data is simply filtered, and the filtered unlabeled data is input into a self-supervised model for training to obtain a preliminary feature extraction model. The data is further filtered and labeled to create a multi-category ground-collected photo dataset of patches. The parameter weights of the preliminary feature extraction model are loaded, and supervised training is performed using the multi-category ground-collected photo dataset of patches to deeply learn the features of different categories. Suitable samples are selected from the marine disaster risk photo sample library, and targeted model training is performed for patches of different marine disaster-bearing body types. After testing on a test set and model integration, a scene classification model is obtained.

[0044] In constructing the target detection model architecture, a self-attention mechanism is used as the basis, combined with the Swin Transformer network and the window self-attention module to build an end-to-end target detection model; the marine disaster risk photo target detection sample library in the marine disaster risk photo sample library is used as training data for subsequent model training.

[0045] When training the scene classification model and the object detection model using the marine disaster risk photo sample library, the training parameters are adjusted, including the number of iterations, batch size, optimizer selection, initial learning rate setting, weight decay rate setting, and loss function selection. A distributed deployment approach is adopted for training, automatic mixed precision is enabled, data is loaded in parallel through subprocesses, and a random erasure strategy is used. Gradients are pruned during training to limit the gradient norm and prevent gradient explosion. In the object detection model training, auxiliary loss and denoising training mechanisms are introduced to stabilize the training process and improve model robustness.

[0046] For example, see Figure 3 The scene classification model is based on the Transformer architecture, including a hierarchical architecture enhancement, a shifted window module, and the EfficientNet network. The object detection model is based on a self-attention mechanism, using a Swin Transformer network and a window self-attention module. During training, it first fully mines the inherent features of existing data through self-supervised learning, performing simple filtering on a large amount of unlabeled data before inputting it into the self-supervised model for training, obtaining a preliminary feature extraction model. Further, the preliminary model is reinforced using samples from various disaster-bearing bodies, performing deep learning on features of different categories. The object detection model is mainly based on end-to-end object detection using a self-attention mechanism, directly using a sample library of all landmark features, i.e., the overall marine disaster risk photo object detection database, for training.

[0047] For the scene recognition model, a pre-built sample library of marine disaster risk photos was used. The pre-trained model was trained on four GPUs for binary classification tasks. The entire training process lasted 500 epochs, including five warm-up epochs to help stabilize the model. The optimizer used an initial learning rate of 0.02 and employed cosine annealing for learning rate scheduling. To improve training efficiency, automatic mixed precision was enabled, and data was loaded in parallel using eight subprocesses with a probability of 0.25 for random pixel pattern erasure. The batch size per GPU was set to 4, and gradient norm was limited to 1.0 through gradient clipping to prevent gradient explosion during training. For the object detection model, training was distributed across four GPUs with a batch size of 2 per GPU, resulting in a total effective batch size of 8. The optimizer AdamW was chosen with an initial learning rate of 0.0001, with the backbone network's learning rate set even lower at 0.00001 to preserve pre-trained features. Weight decay was set to 0.0001 for regularization. The entire training process will last for 36 epochs, with the learning rate decreasing according to the StepLR strategy on the 11th epoch. Furthermore, gradient pruning is employed, limiting the maximum gradient norm to 0.1 to prevent gradient explosion during training. For the loss function, the model uses a weighted sum of classification loss (focal loss, focal_alpha=0.25) and bounding box regression loss (L1 loss and GIoU loss) as the overall loss function. These losses are matched between the target and the prediction using a Hungarian Matcher. The matching costs for class, bounding box L1, and GIoU are 2.0, 5.0, and 2.0, respectively. Additionally, aux_loss is enabled, introducing auxiliary losses from the decoder's intermediate layers to further stabilize training. To improve the model's robustness and convergence speed, a denoising training (DN) mechanism is employed, introducing 100 denoised queries and setting the bounding box noise scale to 0.4 and the label noise ratio to 0.5.

[0048] Step 104: Based on deep learning and self-supervised learning techniques, momentum contrastive learning and the ConvNeXt model are used to process the marine disaster risk photo sample library, obtain the relationship between scenes and targets, and construct a marine disaster risk knowledge graph.

[0049] Specifically, when processing the marine disaster risk photo sample library using momentum contrastive learning and the ConvNeXt model, the relationship between images and entities in the marine disaster risk photo sample library is extracted through supervised learning methods of image classification models; machine learning and deep learning technologies are combined to train the marine disaster risk photo sample library to extract features, and the relationship between images and entities is constructed based on the features; momentum contrastive learning and the ConvNeXt model are used to obtain the relationship between scenes and targets, clarify the area ratio requirements of landmark features in ground-collected photos of various disaster-bearing bodies, and construct a marine disaster risk knowledge graph.

[0050] For example, for the knowledge graph, a marine disaster risk photo sample library is used as training samples. Momentum contrastive learning (MoCo) and ConvNeXt models are employed to obtain the relationship between scenes and targets, thus constructing a marine disaster risk knowledge graph. The completed knowledge graph reflects the area proportions of landmark features that should and should not appear in the ground-based photo sample library of various disaster-bearing bodies.

[0051] Step 105: Integrate the trained scene classification model and target detection model using the marine disaster risk knowledge graph to form a complete marine disaster risk photo recognition model;

[0052] Specifically, when integrating the trained scene classification model and target detection model using the marine disaster risk knowledge graph, the scene classification model determines whether the input ground-collected photos and their labels are similar and outputs a Boolean value result; the target detection model identifies each landmark feature in the ground-collected photos and outputs the total area percentage of the landmark features; based on the landmark feature area percentage thresholds corresponding to various labels in the marine disaster risk knowledge graph, the Boolean value result and the total area percentage are inferred and corrected. When the scene classification result matches the label and the landmark feature area percentage meets the threshold requirement, the classification is confirmed to be valid, and a complete marine disaster risk photo recognition model is finally formed.

[0053] For example, for any input ground-collected photo and its label, the scene recognition model will generally determine whether it is similar to the corresponding label type in the sample library and return a Boolean value; the object detection model will identify the ground features in the ground-collected photo and return them as the total area percentage. The outputs of the scene recognition model and the object detection model are further reasoned using a knowledge graph. Specifically, based on the scene recognition model's judgment, the program corrects the judgment by checking whether the area percentage of the ground features returned by the object recognition model reaches the threshold corresponding to the label type in the knowledge graph. For ground-collected photos with a scene recognition result of TRUE, if there are ground features whose area percentage does not reach the threshold, and there are no ground features whose area percentage exceeds the threshold, then the photo is considered to conform to the type indicated by the label; otherwise, it is considered not to conform.

[0054] Step 106: Input the unclassified land parcels into the marine disaster risk photo recognition model to complete the automatic recognition of marine disaster risk photos.

[0055] When unclassified land parcels are input into the marine disaster risk photo recognition model, the unclassified land parcels are first pre-labeled; the marine disaster risk photo recognition model determines whether the content of the pre-labeled land parcels matches the label, and outputs the basis for interpreting feature features; when the proportion of ground-collected photos that match the label exceeds a preset standard, the land parcel recognition is confirmed to be effective, and automatic recognition is completed.

[0056] After obtaining the marine disaster risk photo scene classification sample library and the marine disaster risk photo target detection sample library, the marine disaster risk photo scene classification sample library and the marine disaster risk photo target detection sample library are divided into training set, validation set and test set according to a preset ratio. The training set is used for model training, the validation set is used for parameter adjustment during the model training process, and the test set is used for performance testing after model training.

[0057] For example, ground-collected photographic patches are first pre-labeled. A marine disaster risk photo recognition model then determines whether the patch content matches the label and provides criteria for interpreting feature objects. Specifically, Figure 4 An example of the judgment process is given. Figure 4 The map features seven ground-collected photographs, pre-labeled as photovoltaic (PV) type disaster-bearing structures. All ground-collected photographs underwent scene identification for PV; during the target detection phase, it was found that most ground-collected photographs detected PV structures (such as...). Figure 4 According to the photovoltaic-related content in the knowledge graph, the photovoltaic area in all 7 ground-based photos exceeded the preset threshold, specifically 68.84%, 61.38%, 55.42%, 59.02%, 59.76%, 78.12%, 58.82%, and 70.72%. However, Figure 4 (g) The area of ​​the tree exceeds the threshold of the photovoltaic knowledge graph, which is 4.82%. Ultimately, 6 ground-collected photos passed the review, with a pass rate of more than half, so the review result for this patch is approved.

[0058] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0059] The above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the content of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for identifying marine disaster risk from photographs based on deep artificial neural networks, characterized in that, Includes the following steps: The location of the disaster-bearing body is determined by remote sensing imagery, and the contents of the land plot are photographed from multiple angles to obtain a set of ground-collected photo data. Based on visual interpretation, the consistency between the patch content of the ground-collected photo data group and the disaster-bearing body type label is verified. At the same time, the landmark features within the ground-collected photo data group are selected and marked to obtain a marine disaster risk photo sample library. A scene classification model and an object detection model architecture are constructed, and the scene classification model and the object detection model are trained using the marine disaster risk photo sample library; Based on deep learning and self-supervised learning techniques, momentum contrastive learning and ConvNeXt model are used to process the marine disaster risk photo sample library, obtain the relationship between scenes and targets, and construct a marine disaster risk knowledge graph. By integrating the trained scene classification model and target detection model with the marine disaster risk knowledge graph, a complete marine disaster risk photo recognition model is formed. The unclassified land parcels are input into the marine disaster risk photo recognition model to complete the automatic recognition of marine disaster risk photos.

2. The marine disaster risk photo recognition method based on deep artificial neural networks according to claim 1, characterized in that, When determining the location of disaster-bearing bodies using remote sensing imagery, first select and download high-resolution and medium-resolution remote sensing images of the same area and similar time phases from open-source remote sensing imagery data websites, and filter areas with less cloud cover; perform visual interpretation of the high-resolution remote sensing images to preliminarily confirm suspected high-risk disaster-bearing body areas; dispatch personnel to conduct on-site investigations of the suspected high-risk disaster-bearing body areas, take multi-angle photos of the land plot, and obtain ground-collected photo data sets.

3. The marine disaster risk photo recognition method based on deep artificial neural networks according to claim 1, characterized in that, When visually interpreting and verifying the consistency between the patch content and the disaster-bearing body type label of the ground-collected photo data group, each ground-collected photo in the ground-collected photo data group is interpreted. The overall patch consistency is determined based on the proportion of ground-collected photo images that match the disaster-bearing body type label. Positive and negative samples are then divided to obtain a marine disaster risk photo scene classification sample library. The landmark features and landmark features corresponding to each label in the positive samples are extracted. Several ground-collected photographs are extracted from each label of the positive samples, and the ground features in the ground-collected photographs are labeled based on the landmark features to obtain a marine disaster risk photograph target detection sample library; the marine disaster risk photograph scene classification sample library and the marine disaster risk photograph target detection sample library together constitute the marine disaster risk photograph sample library.

4. The marine disaster risk photo recognition method based on deep artificial neural networks according to claim 1, characterized in that, When constructing the scene classification model architecture, the Transformer architecture is used as the basis, and the hierarchical architecture enhancement, shift window module and EfficientNet network are integrated. A large amount of unlabeled data is simply filtered, and the filtered unlabeled data is input into the self-supervised model for training to obtain a preliminary feature extraction model. The data is further filtered and labeled to create a dataset of ground-collected photos of multi-category patches; the parameter weights of the preliminary feature extraction model are loaded, and supervised training is performed using the dataset of ground-collected photos of multi-category patches to learn the features of different categories in depth; suitable samples are selected from the marine disaster risk photo sample library, and targeted model training is performed for patches of different types of marine disaster-bearing bodies. After testing on the test set and model integration, a scene classification model is obtained.

5. The marine disaster risk photo recognition method based on deep artificial neural networks according to claim 1, characterized in that, When constructing the target detection model architecture, a self-attention mechanism is used as the basis, combined with the Swin Transformer network and the window self-attention module to build an end-to-end target detection model; the marine disaster risk photo target detection sample library in the marine disaster risk photo sample library is used as training data for subsequent model training.

6. The marine disaster risk photo recognition method based on deep artificial neural networks according to claim 1, characterized in that, When training the scene classification model and the object detection model using the marine disaster risk photo sample library, the training parameters are adjusted, including the number of iterations, batch size, optimizer selection, initial learning rate setting, weight decay rate setting, and loss function selection. A distributed deployment approach is adopted for training, automatic mixed precision is enabled, data is loaded in parallel through subprocesses, and a random erasure strategy is used. Gradients are pruned during training to limit the gradient norm and prevent gradient explosion. In the object detection model training, auxiliary loss and denoising training mechanisms are introduced to stabilize the training process and improve model robustness.

7. The marine disaster risk photo recognition method based on deep artificial neural networks according to claim 1, characterized in that, When processing the marine disaster risk photo sample library using momentum contrastive learning and the ConvNeXt model, the relationship between images and entities in the marine disaster risk photo sample library is extracted through supervised learning methods of image classification models; combined with machine learning and deep learning techniques, the marine disaster risk photo sample library is trained to extract features, and the relationship between images and entities is constructed based on the features; Momentum contrastive learning and ConvNeXt model are used to obtain the relationship between the scene and the target, clarify the area ratio requirements of landmark features in ground-collected photos of various disaster-bearing bodies, and construct a knowledge graph of marine disaster risk.

8. The marine disaster risk photo recognition method based on deep artificial neural networks according to claim 1, characterized in that, When integrating the trained scene classification model and target detection model using the marine disaster risk knowledge graph, the scene classification model determines whether the input ground-collected photos and their labels are similar and outputs a Boolean value result; the target detection model identifies each landmark in the ground-collected photos and outputs the total area percentage of the landmarks. Based on the threshold of the area ratio of landmarks corresponding to various labels in the marine disaster risk knowledge graph, the Boolean value results and the total area ratio are inferred and corrected. When the scene classification result matches the label and the area ratio of landmarks meets the threshold requirements, the classification is confirmed to be effective, and a complete marine disaster risk photo recognition model is finally formed.

9. The marine disaster risk photo recognition method based on deep artificial neural networks according to claim 1, characterized in that, When unclassified land parcels are input into the marine disaster risk photo recognition model, the unclassified land parcels are first pre-labeled; the marine disaster risk photo recognition model determines whether the content of the pre-labeled land parcels matches the label, and outputs the basis for interpreting feature features; when the proportion of ground-collected photos that match the label exceeds a preset standard, the land parcel recognition is confirmed to be effective, and automatic recognition is completed.

10. The marine disaster risk photo recognition method based on deep artificial neural networks according to claim 3, characterized in that, After obtaining the marine disaster risk photo scene classification sample library and the marine disaster risk photo target detection sample library, the marine disaster risk photo scene classification sample library and the marine disaster risk photo target detection sample library are divided into training set, validation set and test set according to a preset ratio. The training set is used for model training, the validation set is used for parameter adjustment during the model training process, and the test set is used for performance testing after model training.

Images