Unmanned aerial vehicle disaster image open vocabulary risk assessment method and device, and storage medium

Abstract

The invention discloses an unmanned aerial vehicle disaster image open vocabulary risk assessment method and device, and a storage medium, and the method comprises the steps: collecting multi-view image data of a disaster region through an unmanned aerial vehicle group, dynamically focusing a key region through an adaptive attention mechanism, and extracting disaster scene elements through multi-scale feature fusion; constructing a disaster open vocabulary library capable of being dynamically updated, and performing semantic matching on image features and vocabulary library elements by using a visual language model; a disaster element relationship is modeled based on a space-time diagram network, and the risk of victims is dynamically quantified and a rescue path scheme is generated in combination with a large language model. According to the method, the flexibility and generalization ability of personnel risk assessment in a disaster scene can be effectively improved, the accuracy and timeliness of rescue decision making are remarkably improved, effective rescue priority guidance is provided, and the method has important application value.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of intelligent disaster relief and computer vision, specifically involving a technical solution for disaster scene cognition and dynamic risk quantification assessment based on multimodal spatiotemporal perception. Background Technology

[0002] Unmanned aerial vehicle (UAV) imagery analysis is a crucial technological tool in modern disaster response and emergency management. With the rapid development of UAV technology, UAVs, due to their flexibility, efficiency, and low cost, have been widely applied in various fields such as post-disaster assessment, rescue command, and disaster monitoring. UAVs can rapidly acquire large-scale, high-resolution imagery data at disaster sites, providing valuable first-hand information for disaster assessment and rescue decision-making. However, UAV imagery faces several challenges in disaster scenarios: 1. Wide image coverage, small targets, and incomplete information: Disaster sites typically cover a wide area and have complex structures, and the targets of disaster victims are often small. Images collected by drones may not have enough visual detail, and relying solely on image tags and traditional target detection methods is insufficient to comprehensively and accurately capture the specific situation at the disaster site.

[0003] 2. The variable disaster environment affects the accuracy of assessment: The disaster environment is highly uncertain and dynamically changing, such as weather changes, fires, and floods, which may lead to problems such as scene lighting, object occlusion, and background interference. These changes make it difficult for traditional image analysis techniques to provide stable and accurate assessment results in disaster scenarios.

[0004] 3. Lack of comprehensiveness and accuracy in labeling information: Disaster images often lack high-quality labeled data, and the scarcity and incompleteness of labeling information limits the application of existing image processing algorithms in disaster scenarios. Even when labeled data is available, the labels often focus on macro-level disaster assessment, neglecting in-depth analysis of specific affected individuals or environmental factors.

[0005] To address these issues, multimodal information fusion methods combining image processing, computer vision, and natural language processing have become a research hotspot in recent years. Existing disaster image processing methods mainly focus on the following aspects: 1. Disaster detection methods based on traditional image processing: These methods rely heavily on traditional computer vision techniques such as image segmentation, object detection, and edge extraction. They primarily use manually designed features for preliminary classification and detection of disaster scenes. The advantage of these methods is their high computational efficiency, but in complex disaster scenes, they are easily affected by factors such as lighting and viewing angle, and struggle to cope with dynamic changes and noise interference.

[0006] 2. Disaster Recognition Methods Based on Deep Learning: In recent years, disaster recognition methods based on convolutional neural networks (CNNs) and Transformers have been widely used. Through extensive training data, deep networks can automatically extract features from disaster images and perform classification or segmentation. However, due to the diversity and dynamism of drone images, traditional deep learning methods have poor generalization ability for disaster scenarios and are still limited by the quality and scale of data annotation.

[0007] 3. Disaster Assessment Methods Based on Multimodal Data Fusion: With technological advancements, some studies have begun to explore fusing different types of data (such as images, sensor data, and text) to improve the accuracy and comprehensiveness of disaster assessments. These methods typically employ deep learning models to process image data and combine them with natural language processing techniques for semantic analysis of disaster information to better understand the situation. However, these methods still face challenges such as how to extract effective features in a dynamic environment and how to correlate different modalities.

[0008] Currently, risk assessment for disaster victims remains a significant challenge. Traditional methods often rely on intuitive image analysis, lacking in-depth exploration of multi-dimensional factors, making it difficult to comprehensively and accurately assess the specific risks to individuals. Furthermore, the complexity of disaster environments makes existing assessment models ill-suited to dynamically changing situations, leading to unstable and lagging assessment results.

[0009] Therefore, developing a method that can combine UAV imagery and multi-dimensional data, and employ advanced semantic analysis and risk assessment methods to adapt to complex and ever-changing disaster scenarios, has significant application value for improving disaster relief efficiency and optimizing resource allocation. Summary of the Invention

[0010] To address the shortcomings of existing UAV image analysis methods for disaster scene identification and disaster victim risk assessment, this invention provides an open-vocabulary analysis method for disaster scenes and disaster victim risk assessment based on UAV imagery. This method combines open-vocabulary recognition with an adaptive attention mechanism and large language model prompting engineering to solve the challenges posed by variable environmental factors in disaster scenes to disaster victim risk assessment.

[0011] The technical solution of this invention provides a method for risk assessment of open vocabulary for unmanned aerial vehicle (UAV) disaster images, comprising the following processes: By collecting multi-view image data of the disaster area through drone swarms, adopting an adaptive attention mechanism to dynamically focus on key areas, and extracting disaster scene elements through multi-scale feature fusion; Construct a dynamically updatable open disaster vocabulary and use a visual language model to semantically match image features with vocabulary elements; Based on spatiotemporal graph network modeling of disaster element relationships, combined with large language modeling, the risk of disaster victims is dynamically quantified and rescue route plans are generated.

[0012] Moreover, the adaptive attention mechanism includes dynamically adjusting the key point positions at preset key point locations in the image through an offset learning network, and calculating the attention weights of each region based on the adjusted key point regions.

[0013] Moreover, the multi-scale feature fusion includes simultaneously extracting multi-scale features using convolutional kernels with different receptive fields, integrating features of different scales through a bidirectional weighted fusion mechanism, and balancing the contribution of each scale feature using learnable parameters.

[0014] Moreover, the construction method of the disaster open vocabulary database includes establishing a knowledge graph ontology structure containing disaster entities and semantic relationships, extracting new disaster terms from the emergency database in real time through an online new word discovery function, and periodically expanding the disaster entity set in the vocabulary database.

[0015] Furthermore, when constructing the disaster open vocabulary, text semantic embeddings are generated for each disaster entity, structured embeddings are generated based on the knowledge graph structure, and text embeddings and structured embeddings are spliced ​​together to form a hybrid feature vector.

[0016] Furthermore, when modeling the relationship between disaster elements based on spatiotemporal graph networks, disaster elements are abstracted as graph nodes, and the relationships between elements are abstracted as graph edges. A spatiotemporal graph structure is constructed, and the graph edge weights are dynamically calculated through the relationship strength prediction network. Real-time sensing data is then integrated to update the relationship strength between nodes and edges.

[0017] Moreover, the method for generating the rescue path includes constructing a disaster risk gravitational field based on individual risk distribution and generating the optimal rescue path through the potential field gradient.

[0018] On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the open vocabulary risk assessment method for unmanned aerial vehicle disaster images as described above.

[0019] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the open vocabulary risk assessment method for UAV disaster images as described above.

[0020] On the other hand, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the open vocabulary risk assessment method for unmanned aerial vehicle disaster images as described above.

[0021] This invention specifically addresses three core challenges in UAV imagery: multi-scale target recognition, open-vocabulary semantic association, and dynamic risk field extrapolation. It proposes a comprehensive solution integrating multi-scale adaptive feature extraction, cross-modal contrastive learning, and spatiotemporal graph neural networks. The technical solution conceived in this invention can significantly improve the application effectiveness of UAV imagery in disaster scenarios, overcoming the problems of low accuracy and poor timeliness in disaster identification and personnel risk assessment in existing technologies.

[0022] Compared with the prior art, the present invention has the following beneficial effects: 1. Improve the accuracy and timeliness of personnel risk assessment in disaster scenarios. Through adaptive attention mechanisms and multidimensional feature analysis, this invention can accurately identify key elements in complex disaster environments and dynamically assess personnel risk through large language models, ensuring more accurate and timely assessment results.

[0023] 2. Enhance the model's generalization ability This invention combines an open vocabulary database with a large language model, avoiding the reliance on manual labeling in traditional methods. It can adapt to various disaster scenarios and achieve wider application.

[0024] 3. Enable real-time dynamic updates and feedback. The dynamic update mechanism of this invention enables risk assessment to respond in real time to changes in the disaster environment, adjust assessment results in a timely manner, provide the latest information for rescue decisions, and improve rescue efficiency.

[0025] 4. Optimize rescue decision-making and resource allocation The personnel risk assessment report provided by this invention can effectively guide rescue personnel to identify priority rescue target areas and rationally allocate resources based on real-time assessment results, thereby improving rescue efficiency and resource utilization.

[0026] In summary, this invention provides a method for disaster scene vocabulary analysis and disaster risk assessment based on UAV imagery. It effectively addresses the accuracy, timeliness, and adaptability issues in existing disaster scene identification and personnel risk assessment technologies, demonstrating significant application prospects and practical importance. This technology has significant engineering application value in fields such as disaster emergency response, intelligent rescue decision-making, and urban safety control. Attached Figure Description

[0027] Figure 1 This is an overall flowchart of an embodiment of the present invention.

[0028] Figure 2 This is a diagram of the drone swarm-ground mobile base station data acquisition and feedback system according to an embodiment of the present invention.

[0029] Figure 3This is a flowchart of the feature extraction process of the multi-scale adaptive attention mechanism in an embodiment of the present invention.

[0030] Figure 4 This is an experimental example diagram of an embodiment of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the following detailed description of the method for disaster scene vocabulary analysis and disaster victim risk assessment based on UAV imagery, in conjunction with the accompanying drawings and embodiments, is provided. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0032] This invention discloses an open vocabulary analysis and disaster risk assessment scheme for disaster scenes based on UAV imagery. With the rapid development of UAV technology, UAVs are widely used in disaster relief due to their flexibility, efficiency, and low cost. However, UAV imagery covers a wide range, targets are small, and disaster environments are highly variable. Label information alone cannot accurately and comprehensively reflect the full picture of the disaster site, making it difficult to effectively assess the risk situation of disaster victims. This invention proposes a personnel risk assessment scheme that combines open vocabulary recognition with an adaptive attention mechanism and causal reasoning using a large language model, effectively solving the problem of difficult personnel risk assessment in UAV imagery where multiple environmental factors are coupled. First, multi-view image data of the disaster area is collected using UAV imagery, and an adaptive attention mechanism is introduced to extract key features at multiple scales. Then, a visual language model is used to perform semantic association analysis between multi-dimensional features and an open vocabulary, enabling the identification and mining of various elements. Finally, risk assessment and dynamic analysis are performed using prompting engineering based on a large language model, and semantic reasoning is conducted by integrating elements and their topological relationships to generate a personnel risk assessment report. This invention can effectively improve the flexibility and generalization of personnel risk assessment in disaster scenarios, significantly improve the accuracy and timeliness of rescue decisions, and provide effective guidance on rescue priorities, thus having significant application value.

[0033] like Figure 1 As shown in the embodiments of the present invention, the open vocabulary risk assessment method for UAV disaster images mainly includes the following steps: S1. UAV imagery data acquisition and adaptive attention mechanism feature extraction By using drones to collect multi-view image data of disaster areas, multi-dimensional image data of the disaster scene is obtained. Furthermore, an adaptive attention mechanism is used to process the image data, and by multi-scale localization and enhancement of key features, the accuracy of target recognition in the disaster environment is improved.

[0034] First, drones are used to collect image data of the disaster area, ensuring the acquisition of multi-view, multi-dimensional image data. At regular intervals, multiple drone image datasets are input into the mobile base station processing system.

[0035] Step S1 in the embodiment is preferably implemented using the following sub-steps: Step S1.1: Multi-view UAV image data acquisition Acquire multi-view drone imagery data, covering all directions of the disaster area.

[0036] The UAV image acquisition system of this invention is preferably configured to consist of a mobile base station and multiple UAVs, employing a distributed collaborative working mode to ensure efficient coverage and real-time processing of disaster scenarios. The mobile base station is responsible for task allocation, data reception, and fusion processing, and is equipped with a high-performance GPU computing unit to support real-time feature extraction and model inference. Each UAV in the swarm is equipped with a high-resolution optical camera and a built-in edge computing unit to support preliminary image processing and simple feature extraction operations.

[0037] See Figure 2 The UAV swarm-ground mobile base station data acquisition and feedback system shown first generates flight missions based on the disaster area, dynamically allocating the flight altitude, camera tilt angle, and coverage area for each UAV. The UAVs fly along preset paths, acquiring multi-view images to ensure coverage of all important areas in the disaster zone. The camera angle and flight altitude are automatically adjusted according to the disaster type to obtain image data at different levels.

[0038] Step S1.2: Feature extraction using adaptive attention mechanism An adaptive attention mechanism is introduced to adjust the attention weights according to changes in different regions, thereby extracting key features in the disaster environment.

[0039] The embodiment employs an adaptive attention mechanism to process image data; see [link / reference]. Figure 3 The feature extraction process is illustrated. This mechanism adjusts the attention weights for different regions based on changes in image content, as shown in the following formula:

[0040] in, It is the attention weight of the i-th block. It is a characteristic function. It is the feature of the i-th block of the input image. It is a mathematical constant. This represents the total number of blocks. Feature extraction of key regions is enhanced by adaptively adjusting the attention weights for each pixel.

[0041] To reduce the computational cost of attention feature extraction, a variable attention mechanism is introduced during the feature function extraction process. At preset keypoint locations in the image, the positions of the keypoints are dynamically adjusted using an offset learning network, and attention weights for each region are calculated based on the adjusted keypoint regions. Specifically, multiple uniformly distributed keypoints are pre-defined on the image. As the initial block center of the image, This refers to the number of keypoints. The present invention proposes that an offset learning network guides the model to learn the offset of each keypoint, with the offset being learned through a learnable function. This enables the model to adaptively and dynamically select the region of interest based on the content of the image.

[0042] in, The position on the image is point, The learning function adjusts the position of each center point by learning the features of the image, selectively focusing on regions with high information density. This allows the model to flexibly adjust the region of interest according to actual needs, improving computational efficiency while ensuring accurate identification of key regions.

[0043] Thus, the characteristic function is obtained. The expression is:

[0044] in This is the output fusion matrix of the multi-head attention, where M represents the number of multi-head attention points, with attention head indices m=1,2,…M, and K represents the number of samples in the global key-value vector of the attention, with sample indices k=1,2,…K. Indicates input feature attention. It is a linear transformation matrix of the value vector. These are the preset initial key point positions. It is the offset calculated from point i. It is the input feature at the offset keypoint location.

[0045] Step S1.3: Multi-scale feature fusion analysis By using multi-scale analysis, important information is extracted and enhanced, helping the system to identify disaster targets of different sizes and improving the accuracy of disaster site element identification.

[0046] Because disaster environments contain multi-scale targets (such as large-scale building collapses and small trapped individuals), traditional single-scale feature extraction is prone to losing information about small targets. This step proposes a multi-scale feature pyramid and a dynamic adaptive fusion mechanism to achieve multi-scale feature fusion. This includes using convolutional kernels with different receptive fields to extract multi-scale features simultaneously, integrating features of different scales through a bidirectional weighted fusion mechanism, and using learnable parameters to balance the contribution of features at each scale.

[0047] To reduce the additional computational burden of multi-scale feature extraction, a common convolutional kernel is used in the early low-level feature extraction process of the model to extract features at different scales through convolution operations. An improved depthwise separable dilated convolution is used to construct a multi-scale feature pyramid, expanding the receptive field while maintaining computational efficiency. A set of dilation rates is defined. }, corresponding to the extraction of features at different scales:

[0048] in, This is the dilated convolution feature extraction function. For multi-scale output features, These are the sampling coordinates for dilated convolution.

[0049] After extracting multi-scale features, a bidirectional feature pyramid network (BiFPN) is constructed to enhance information transmission through cross-scale feature weighted fusion.

[0050] in, , , For learnable fusion weights, satisfying + + = 1, Up / Down indicates upsampling / downsampling operation. These are the original features of the first layer of the pyramid. This refers to the enhanced features after fusion.

[0051] Multi-scale analysis can help the system identify disaster targets of different sizes (such as buildings, disaster victims, etc.), thereby improving the accuracy of target detection.

[0052] S2. Cross-modal semantic space alignment and dynamic open lexical analysis By combining an open vocabulary with a visual language model, semantic association analysis is performed on multidimensional features extracted from UAV imagery. This method can identify and extract key elements in disaster scenarios without the need for manual labeling.

[0053] Step S2 in the embodiment is preferably implemented using the following sub-steps: To address the issues of sparsity and semantic gap in disaster scenario labels, this invention proposes a cross-modal semantic space projection method based on dual-stream coding, which achieves unsupervised semantic association mining through a two-level progressive analysis.

[0054] Step S2.1: Construction of a dynamic open vocabulary driven by disaster ontology A dynamic, open disaster vocabulary driven by a disaster ontology is constructed, containing various disaster-related elements (such as buildings, personnel, equipment, roads, etc.). The vocabulary can be updated in real time based on risk mining. The construction method of the disaster open vocabulary includes establishing a knowledge graph ontology structure containing disaster entities and semantic relationships, extracting new disaster terms from the emergency database in real time through an online new word discovery function, and periodically expanding the disaster entity set in the vocabulary.

[0055] The implementation example constructs an open vocabulary database containing disaster-related elements, such as buildings, affected people, equipment, and roads. A hierarchical disaster ontology database is built using knowledge graph technology, supporting dynamic vocabulary expansion.

[0056] in Indicates semantic relationships between disaster elements (such as "road blockage") "Rescue delay", For online new word discovery functions, real-time data is collected from emergency databases. Extracting terms related to emerging disasters, For disaster knowledge graphs, For the set of disaster entities, Define association rules for disaster elements as a set of semantic relationships between entities. Let K be the i-th disaster entity or relation in the vocabulary, K be the number of basic terms, t be the time variable, and T be the number of dynamic expansion cycles.

[0057] This invention further proposes that, when constructing an open disaster vocabulary, textual semantic embeddings are generated for each disaster entity, and structured embeddings are generated based on a knowledge graph structure. The textual embeddings and structured embeddings are then concatenated to form a hybrid feature vector. In this embodiment, the vocabulary representation uses a hybrid embedding method:

[0058] in This represents the concatenation operation of text embedding and knowledge graph embedding, enhancing semantic expressiveness. For knowledge graph-based Structured embedding, For context-sensitive text embeddings generated by the pre-trained language model BERT, For the i-th disaster entity or relation in the vocabulary, It is a hybrid embedding vector representation.

[0059] Step S2.2: Semantic matching of the visual language model Visual language models are used to semantically match and associate visual features in UAV imagery with elements in an open vocabulary.

[0060] The visual language model is used to perform semantic matching between visual features in images and elements in an open vocabulary. This includes extracting visual and text features separately using a two-stream encoder, mapping heterogeneous features to a unified space using a learnable projection matrix, and optimizing cross-modal alignment based on temperature scaling contrast loss.

[0061] Suppose that the visual features extracted from the image in the embodiment are A visual-language contrastive learning framework is constructed to achieve cross-modal semantic matching. First, visual and textual features are extracted using a two-stream encoder architecture, and then aligned using a learnable projection matrix.

[0062]

[0063] Where f represents the original features obtained by the feature encoder, and h represents the aligned features obtained by the projection process W. It is the visual feature vector of the i-th input image. It is the text feature vector of the j-th word. It is a visual encoder. It is the i-th input image. yes 3D real space, It is a text encoder. yes 3D real space, It is the field of real numbers. These are the visual features after projection alignment. It is a visual feature projection matrix. yes, These are the text features after projection alignment. It is the text feature projection matrix.

[0064] After obtaining the aligned features, the similarity between each feature and each word in the vocabulary is calculated. Based on the cosine similarity, a learnable temperature parameter is introduced. The specific formula for the temperature scaling contrast loss is as follows:

[0065]

[0066] in, Here, B is the learnable temperature parameter, B is the batch size, and k is the sample index within the batch. It is a visual feature Text features cosine similarity, It is a temperature scaling contrast loss.

[0067] Using this semantic matching method of visual language model, the model can effectively match key elements in an image with disaster-related words.

[0068] S3. Personnel Risk Assessment and Dynamic Analysis Based on Large Language Models This module constructs a spatiotemporal knowledge graph of disaster scenarios, integrates multimodal perception data with the causal reasoning capabilities of large language models, and comprehensively analyzes multidimensional elements in disaster scenarios to achieve accurate risk assessment and dynamic reasoning for disaster victims, thus completing dynamic risk quantification assessment.

[0069] Step S1 of the embodiment preferably includes the following core technical aspects: Step S3.1: Multimodal entity relationship modeling This step involves multimodal entity relationship modeling, inputting the extracted disaster scenario element information and the large language model to generate risk assessment prompts.

[0070] By using various elements in a disaster scenario (such as personnel location and environmental information) as nodes and the relationships between elements as graph edges, a graph structure entity model is created. This allows the model to receive a more complete description of disaster information, thereby enabling a more accurate risk assessment.

[0071] The example first defines the disaster scenario graph structure. :

[0072] in Functions representing disaster risk factors This represents a relationship strength prediction network. This represents the sensory input at time t. It is the set of relationships between risk factors. It is a risk element entity. S2 represents the output elements, and M is the total number of risk elements. It is an element and The strength of the relationship, where i and j represent risk element index identifiers.

[0073] In practice, the edge weights of the graph are dynamically calculated through a relationship strength prediction network, and the relationship strength between nodes and edges is updated by integrating real-time sensing data. The relationship strength prediction network takes node feature pairs and real-time environmental data as input and outputs edge weight values ​​that characterize the dynamic correlation strength between disaster elements. These edge weight values ​​can be continuously iterated and updated as the disaster evolves.

[0074] Step S3.2: Semantic Reasoning and Dynamic Analysis To transform disaster element information and its topological relationships into features that can be understood by a large language model, a spatiotemporal graph attention network (ST-GAT) is first used to perform dynamic representation learning to transform the disaster scene graph structure into high-dimensional features and capture the spatiotemporal dependencies between entities in the disaster scene.

[0075] ST-GAT achieves dynamic representation of disaster elements by jointly learning the topological relationships in the spatial dimension and the evolutionary patterns in the temporal dimension. ST-GAT learns the interaction strength between different entities through an attention mechanism. For each node (entity) in the graph, through the joint effect of spatial and temporal attention, the representation update formula for node i is:

[0076] in Let these represent the feature vectors of nodes j and i in the l-th and l+1-th layers of the network, respectively. Let i be the set of neighbors of node i. Let i be the attention weight between node i and node j. For time-based attention weights, where t is the time variable. It is the learnable parameter matrix of the l-th layer.

[0077] A high-dimensional vector containing the topological relationships of disaster elements is input into a large language model for deep reasoning. The model will construct an individual risk probability model for disaster victims based on the risk feature vector. The risk rate function of disaster victims over time is:

[0078] in, The dynamic characteristics from the spatiotemporal diagram at time t represent the degree of influence of the relationships between disaster elements. For learnable parameters, Let i be the risk rate of node i at time t. Here, t is the baseline risk function, j is the feature dimension index, i is the node index, and p is the total number of dynamic feature dimensions.

[0079] Step S3.3: Rescue Plan Recommendations This step leverages the causal reasoning capabilities of the Big Prophecy model to analyze the spatiotemporal map features, assess personnel risk levels, and provide rescue recommendations.

[0080] Furthermore, the present invention proposes a rescue path generation scheme that includes constructing a disaster risk gravitational field based on individual risk distribution, integrating the terrain potential field to generate a comprehensive gradient field, balancing the influence of risk and terrain through weighted coefficients, and planning the optimal rescue path according to the negative gradient direction of the comprehensive potential field.

[0081] In practice, the disaster risk gravitational field is generated by superimposing the effects of each risk source, and the effects decrease as the distance increases; the terrain accessibility field is constructed independently of the risk gravitational field and characterizes the terrain accessibility; the weighting coefficients are dynamically adjusted according to the disaster environment.

[0082] The implementation example models the risk field of the disaster area, generates a disaster risk gravitational field, and generates disaster relief priority suggestions and the best rescue path at the time based on actual risk feedback. The specific implementation is as follows: First, the model constructs a disaster risk gravitational field based on the individual risk probability model, and generates the optimal rescue path through the potential field gradient, as shown in the following formula:

[0083]

[0084] in, Represents the gravitational field of disaster risk. Indicates the risk smoothing factor. This represents the weighting coefficient for terrain constraints, used to balance the impact of risk and terrain. These are the spatial coordinates of the rescue team or resource point. These are the location coordinates of the i-th risk source. It is the risk intensity of the i-th risk source. It is the gravitational field function of disaster risk. It is the gradient vector of the risk gravitational field. It is a potential field that facilitates terrain access. It is the gradient vector of the terrain field. It is the direction of the negative gradient of the combined potential field. This represents the total number of risk sources.

[0085] The final large language model is based on the comprehensive potential field function, and provides the optimal rescue path suggestion through the potential field gradient.

[0086] To facilitate understanding of the technical effects of this invention, an experimental example of the method in this embodiment is provided below: like Figure 4The image shows a rescue scenario following a geological disaster, with the red area representing the disaster danger zone and the blue area representing the surrounding safe zone. During the rescue operation, mobile base station S is docked in the safe zone and releases rescue drones A, B, C, and D, performing process S1: the drones collect data on the disaster area according to instructions from the mobile base station, perform preliminary data processing, and transmit the data back after a certain interval. Based on the transmitted data, the base station performs process S2: cross-modal spatial alignment and open vocabulary analysis of the disaster images. Simultaneously, the base station performs process S3: using the high-computing power equipment deployed at the base station to conduct personnel risk assessment and dynamic analysis based on a large language model. Rescue recommendations for the disaster site are obtained, and new instructions are issued to drones A, B, C, and D to search the next area.

[0087] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of this invention and computer equipment including the computer program running the corresponding computer program, should also be within the protection scope of this invention.

[0088] The following embodiments describe the electronic device provided by the present invention. The electronic device described below can be referred to in correspondence with the open vocabulary risk assessment method for UAV disaster images described above.

[0089] The electronic device may include a processor, a communications interface, memory, and a communication bus. The processor, communications interface, and memory communicate with each other via the communication bus. The processor can invoke logical instructions from the memory to execute the open-vocabulary risk assessment method for UAV disaster imagery, which mainly includes the software processing portion described above.

[0090] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0091] On the other hand, embodiments of the present invention also provide a computer program product, the computer program product including a computer program, the computer program being stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer is able to execute the software processing portion of the open vocabulary risk assessment method for UAV disaster images provided by the above methods.

[0092] In another aspect, embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the software processing portion of the open vocabulary risk assessment method for unmanned aerial vehicle disaster images provided by the above methods.

[0093] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0094] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for risk assessment of unmanned aerial vehicle (UAV) disaster imagery using an open vocabulary, characterized in that, Includes the following processes: By collecting multi-view image data of the disaster area through drone swarms, adopting an adaptive attention mechanism to dynamically focus on key areas, and extracting disaster scene elements through multi-scale feature fusion; Construct a dynamically updatable open disaster vocabulary and use a visual language model to semantically match image features with vocabulary elements; Based on spatiotemporal graph network modeling of disaster element relationships, combined with large language modeling, the risk of disaster victims is dynamically quantified and rescue route plans are generated.

2. The method for risk assessment of open vocabulary for unmanned aerial vehicle (UAV) disaster images according to claim 1, characterized in that: The adaptive attention mechanism includes dynamically adjusting the positions of key points at preset key point locations in the image through an offset learning network, and calculating the attention weights of each region based on the adjusted key point regions.

3. The method for risk assessment of open vocabulary for unmanned aerial vehicle (UAV) disaster images according to claim 1, characterized in that: The multi-scale feature fusion includes simultaneously extracting multi-scale features using convolutional kernels with different receptive fields, integrating features of different scales through a bidirectional weighted fusion mechanism, and balancing the contribution of each scale feature using learnable parameters.

4. The method for risk assessment of open vocabulary for unmanned aerial vehicle (UAV) disaster images according to claim 1, characterized in that: The construction method of the disaster open vocabulary database includes establishing a knowledge graph ontology structure containing disaster entities and semantic relationships, extracting new disaster terms from the emergency database in real time through an online new word discovery function, and periodically expanding the disaster entity set in the vocabulary database.

5. The method for risk assessment of open vocabulary for unmanned aerial vehicle (UAV) disaster images according to claim 4, characterized in that: When constructing the disaster open vocabulary, text semantic embeddings are generated for each disaster entity, structured embeddings are generated based on the knowledge graph structure, and text embeddings and structured embeddings are concatenated to form a hybrid feature vector.

6. The method for risk assessment of open vocabulary for unmanned aerial vehicle (UAV) disaster images according to claim 1, characterized in that: When modeling the relationships between disaster elements based on spatiotemporal graph networks, disaster elements are abstracted as graph nodes, and the relationships between elements are abstracted as graph edges. A spatiotemporal graph structure is constructed, and the graph edge weights are dynamically calculated through the relationship strength prediction network. Real-time sensing data is then integrated to update the relationship strength between nodes and edges.

7. The method for risk assessment of open vocabulary for unmanned aerial vehicle (UAV) disaster images according to claim 1, characterized in that: The method for generating rescue paths includes constructing a disaster risk gravitational field based on individual risk distribution and generating the optimal rescue path through the potential field gradient.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the open vocabulary risk assessment method for unmanned aerial vehicle disaster images as described in any one of claims 1 to 7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the open vocabulary risk assessment method for unmanned aerial vehicle disaster images as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by the processor, it implements the open vocabulary risk assessment method for unmanned aerial vehicle disaster images as described in any one of claims 1 to 7.

Images