A method and system for collaborative three-dimensional perception of space and ground in block emergency scenarios

Abstract

The present invention belongs to the field of smart city and Internet of Things technology, and discloses a method and system for air-ground collaborative three-dimensional perception of block emergency scenarios. The present invention takes block emergency events as the center, regards each type of observation equipment as a basic event, and constructs an observation planning model for air-ground collaboration; constructs a resource description ontology model of the block perception network, and establishes inference rules based on the resource description ontology model to realize network planning; aggregates and organizes multi-source terminal heterogeneous data obtained by several observation devices to obtain air-ground collaborative observation data; and integrates geographic information display of the air-ground collaborative observation data and its inversion data to realize three-dimensional perception of block emergency scenarios. The present invention can realize real-time reflection and visualization of block observation elements in emergency scenarios, and can support the efficient operation of emergency response systems and mechanisms, so as to respond to emergency responses in a scientific, orderly and efficient manner.

Description

Technical Field

[0001] The present invention belongs to the field of smart city and Internet of Things technology, and more specifically, relates to a method and system for collaborative three-dimensional perception of space and ground in block emergency scenarios. Background Art

[0002] Emergency events such as fires, flooding, ecological damage, and traffic accidents occur frequently. Monitoring and predicting these events to ensure efficient emergency management is a technical challenge in this field. The following describes the current state of research in China and abroad, focusing on four aspects of this technical challenge.

[0003] Current research on event observation planning, both domestically and internationally, focuses primarily on observation model construction and observation equipment scheduling. The primary approach is a centralized joint task planning approach, which solves and optimizes task requirements from a global perspective. As the variety of observation resources increases, centralized planning approaches become time-consuming, lack robustness, and have poor scalability. To address the heterogeneity of observation resources, numerous organizations and companies have conducted research and proposed numerous common standards and models. For example, the W3C has proposed the SSN description standard for sensor networks. At the network layer, some research abstracts communication resources and designs abstract interfaces that are universally scalable. However, these standards and models are often single-layered or targeted at a specific level of the neighborhood perception network. While addressing the heterogeneity of resources within a layer, significant differences exist between the standards and models at each level of the neighborhood perception network as a whole, making resource matching and planning across these layers difficult.

[0004] Current research on network planning, whether targeting network structure or device access, focuses on the networking technology and planning of dedicated networks. However, neighborhood perception networks encompass a rich variety of communication resources, each with its own unique distribution characteristics. This requires integrating and matching various network-layer communication resources based on the needs and characteristics of perception-layer observation resources to form a rational, efficient, and targeted communication network. Dedicated network planning methods are unable to adapt to network structures with multiple communication resource types. Furthermore, in the actual construction of neighborhood perception networks, manual judgment is often used to select network-layer communication resources for perception-layer observation resources. Typically, starting from the observation resources, appropriate communication resources are selected from the network interfaces supported by the observation resources to complete device networking and network coverage. This approach can lead to problems such as uneven distribution of communication resources and low utilization, resulting in fragmented neighborhood perception networks and difficulties in organization and management.

[0005] The goal of integrating 3D perception data for collaborative air-ground and street-level emergency scenarios is to aggregate and integrate observation data from various sensing devices, such as drone-mounted video and various sensors, mobile phone video and sensors, and fixed and mobile sensors. This will effectively address objective issues such as information silos and information barriers between devices, establish a set of methods to scientifically manage and govern various types of perception data, and form a data service system that is interconnected, horizontally connected, and logically integrated. However, most current emergency systems only achieve the aggregation and standardization of data from various business systems, and the value inherent in the data cannot be realized. There is a lack of effective correlation between various types of data, and structured and unstructured data cannot be horizontally expanded. Historical images and videos are also difficult to re-analyze and utilize. The entire data cycle cannot achieve the effect of closed-loop transmission and utilization, making it difficult to quickly and accurately discover and troubleshoot target information from massive amounts of dynamic data.

[0006] Geographic observation, as a means of perceiving nature and society, has become increasingly advanced with the advancement of science and technology. Depending on the application domain, multi-scale and multi-angle observation methods have been developed, including ground-based observation, aerial observation, and aviation observation. Furthermore, geographic information system (GIS) technology, which began to develop in my country in the 1970s and reached significant development by the end of the last century, provided a technical foundation and platform for the visualization of observation data. Consequently, numerous GIS-based observation data management and display platforms have emerged, targeting various applications. Visualizing observation data incorporating geographic location can more fully explore the potential information within these observation data. Currently, most GIS-based data management and display platforms only cover a limited range of observation data, such as the aforementioned ground-based and aerial observation data. These platforms often focus on displaying only a specific type of observation data. This results in incomplete information about the target space and hinders the subsequent fusion and mining of scene data. Summary of the Invention

[0007] To address the aforementioned issues with existing technologies, the present invention provides a method and system for collaborative 3D perception of street-level emergency scenarios. This method enables real-time visualization of street-level observation elements in emergency scenarios, supporting the efficient operation of emergency response systems and mechanisms, and enabling a scientific, orderly, and efficient response to emergencies.

[0008] The present invention provides a method for collaborative three-dimensional perception of space and ground in a block emergency scene, comprising:

[0009] Focusing on block emergency events, each type of observation equipment is considered as a basic event to build an air-ground collaborative observation planning model.

[0010] Constructing a resource description ontology model of the block perception network, and establishing inference rules based on the resource description ontology model to implement network planning;

[0011] Aggregate and organize heterogeneous data from multiple sources of observation equipment to obtain air-ground collaborative observation data;

[0012] The air-ground collaborative observation data and its inversion data are integrated with geographic information display to achieve three-dimensional perception of block emergency scenes.

[0013] Preferably, when constructing an observation planning model for air-ground collaboration, collaborative modeling is performed in terms of three attributes: time, space, and events;

[0014] Construct collaborative observation function H α ,as follows:

[0015]

[0016] Where, represents the time parameter of the i-th type of observation equipment in the event observation task, represents the spatial parameters of the i-th type of observation equipment in the event observation task, represents the observation element parameter of the i-th type of observation equipment in the event observation task, and n represents the total number of different types of observation equipment;

[0017] Based on the collaborative observation function, it is determined whether each type of observation equipment can join the collaborative observation; for the i-th type of observation equipment, when the type of observation equipment meets When such observation equipment is added to the collaborative observation;

[0018] In combination with the time parameter, the space parameter and the observation element parameter, weights are set for the observation devices that join the collaborative observation, and the weights are used to represent the priorities of different types of observation devices.

[0019] Preferably, when constructing the air-ground collaborative observation planning model, it further includes setting a confidence level, and adjusting the degree of effectiveness of different types of observation equipment based on the confidence level to obtain the best observation plan;

[0020] The confidence level includes a first indicator representing the timeliness of observation, a second indicator representing the coverage of observation space, and a third indicator representing the performance of observation elements;

[0021] The first indicator represents the proportion of the observation area that a type of observation equipment can cover per unit time, as follows:

[0022]

[0023]

[0024] Where η tRepresents the first indicator, S i represents the spatial range that the i-th observation device in this type of observation equipment can observe, m represents the total number of observation devices in this type of observation equipment, S represents the set of spatial ranges that the event observation task needs to observe, t α Indicates the time window value that this type of observation equipment can achieve the observation task requirements, T is T represents the start time of the observation task requirement of the i-th observation device in this type of observation equipment, ie Indicates the end time of the observation task requirement for the i-th observation device in this type of observation equipment; It represents the observation status index of the i-th observation device in this type of observation equipment within the time window. If the time window of the observation device intersects with the time window required by the observation task, Is 1, if there is no intersection is 0;

[0025] The second indicator represents the total coverage of the observation space of a type of observation equipment to the composite event target, as follows:

[0026]

[0027] Where η s represents the second indicator;

[0028] The third indicator represents the degree of matching of the collaborative observation device to the required attributes of the composite event, as follows:

[0029]

[0030] Where η n represents the third index, k represents the number of categories of observed basic events contained in the composite event; It represents the weight of the observation task requirement that the i-th type of observation equipment can achieve for the j-th type of observation basic event in the composite event; Represents the observation performance status indicator of the i-th type of observation equipment. If the observation status of this type of observation equipment meets the observation task requirements, is 1, otherwise is 0; i It represents the number of observation elements that can be observed by the observation equipment of the i-th type to meet the observation mission requirements.

[0031] Preferably, when implementing network planning, perception layer observation resources, network layer communication resources, and application layer resources are taken as three first-level categories, and internal subcategories are performed based on the hierarchical structure of each first-level category to form the resource description ontology model;

[0032] On the resource description ontology model that has been constructed, SWRL rules are constructed based on the constraint relationships among the perception layer, network layer, and application layer in network planning;

[0033] After receiving the network planning scheme request, the resource description ontology model is used to perform instance mapping, and the SWRL rules are used to perform logical reasoning to generate a network planning scheme matching resources at each layer.

[0034] Preferably, the perception layer observation resources include the following four secondary categories: basic attributes of the device, observation attributes, network attributes and data attributes; the basic attributes of the device include device ID, device name and device model; the observation attributes include mobility attributes, measurement attributes and location attributes; the mobility attributes include mobility and mobility range, the measurement attributes include sampling frequency and measurement range, and the location attributes include longitude and latitude; the data attributes include data type, data format, data packet size and data reporting frequency; the network attributes include supported interfaces and transmission requirements;

[0035] The network layer communication resources include the following two secondary categories: basic network attributes and transmission attributes; the basic network attributes include ID, name, longitude, latitude, coverage radius, node capacity and networking mode; the transmission attributes include network delay, transmission speed and transmission bandwidth.

[0036] Preferably, the matching process when using SWRL rules for logical reasoning includes:

[0037] Establish a first network planning goal and a second network planning goal; the first network planning goal is to enable the perception layer observation resource to establish a communication connection with the server of the application layer resource through the network layer communication resource, so that the server can receive data collected by the perception layer observation resource; the second network planning goal is to improve network transmission performance, reduce network latency, increase network coverage, and reduce packet loss rate;

[0038] Establishing basic networking rules around the first network planning goal based on the network data and the observed attributes of the perception layer observation resources, the basic attributes of the network of the network layer communication resources, and the bandwidth of the application layer resources;

[0039] Based on the data attributes of the perception layer observation resources and the transmission attributes of the network layer communication resources, network performance improvement rules are established around the second network planning goal.

[0040] Preferably, when aggregating and organizing the heterogeneous data of multiple terminals obtained by a number of observation devices, the heterogeneous data of multiple sources are divided into structured data and unstructured data according to the data structure, and the structured data and the unstructured data are transmitted to the data aggregation platform in parallel and in different channels;

[0041] The data aggregation platform performs storage based on data structure and semantic association; wherein each observation device is registered with a node ID as the ID of the observation device, and heterogeneous data collected by the same observation device are associated through the node ID.

[0042] Preferably, the data for displaying fused geographic information includes: first-category data in text format, second-category data in video format, and third-category data in image format; the first-category data includes ground mobile sensor observation data, the second-category data includes ground mobile phone observation data, and the third-category data includes observation inversion maps and observation mosaic maps;

[0043] The observation inversion map is obtained by performing spatiotemporal matching on the air-ground collaborative observation data to obtain a data set, and then establishing an inversion model based on the data set to calculate and generate the observation inversion map.

[0044] The first type of data, the second type of data, and the third type of data all include the data itself and the geographical location coordinates corresponding to the data; the data is associated with and integrated with the three-dimensional scene of the geographic information system platform through the geographical location coordinates;

[0045] Among them, the first type of data is displayed in the corresponding position of the three-dimensional scene in the form of a text annotation box; the second type of data displays the video in the form of a floating window, and its movement trajectory is marked in the three-dimensional scene according to the geographic location coordinates; the third type of data is displayed in the corresponding position of the three-dimensional scene in the form of a layer.

[0046] In another aspect, the present invention provides a block emergency scene air-ground collaborative three-dimensional perception system, comprising:

[0047] The observation planning unit is used to build an air-ground collaborative observation planning model centered on block emergency events and taking each type of observation equipment as a basic event;

[0048] A network planning unit, configured to construct a resource description ontology model of the block perception network and establish inference rules based on the resource description ontology model to implement network planning;

[0049] The data aggregation and organization unit is used to aggregate and organize the heterogeneous data from multiple sources of observation equipment to obtain air-ground collaborative observation data;

[0050] A fusion display unit is used to display the air-ground collaborative observation data and its inversion data in a fused geographic information manner to achieve a three-dimensional perception of the emergency scene in the block;

[0051] The block emergency scene air-ground collaborative stereoscopic perception system is used to implement the steps in the block emergency scene air-ground collaborative stereoscopic perception method as described above.

[0052] One or more technical solutions provided in the present invention have at least the following technical effects or advantages:

[0053] The present invention involves air-ground collaborative observation planning, communication networking resource allocation and planning, organization and association of air-ground collaborative block stereoscopic perception data in emergency scenarios, and air-ground collaborative joint inversion prediction. It can realize real-time reflection and visualization of block observation elements in emergency scenarios, support the efficient operation of emergency response systems and mechanisms, and respond to emergency responses in a scientific, orderly and efficient manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is an overall framework diagram of a method for collaborative three-dimensional perception of space and ground in a block emergency scenario provided by an embodiment of the present invention;

[0055] Figure 2 This is a flow chart of constructing an observation planning model for air-ground collaboration in an air-ground collaborative stereoscopic perception method for a block emergency scenario provided by an embodiment of the present invention;

[0056] Figure 3 This is an architectural diagram of network planning in a method for collaborative three-dimensional perception of space and ground in a block emergency scenario provided by an embodiment of the present invention;

[0057] Figure 4 This is a framework diagram of a resource description ontology model of a block perception network in a method for space-ground collaborative stereoscopic perception of block emergency scenarios provided by an embodiment of the present invention;

[0058] Figure 5 It is the ontology description diagram of the observation resources of the perception layer in the resource description ontology model;

[0059] Figure 6 It is the ontology description diagram of the network layer communication resources in the resource description ontology model;

[0060] Figure 7 This is a reasoning flow chart for generating a network planning scheme in a method for collaborative three-dimensional perception of space and ground in a block emergency scenario provided by an embodiment of the present invention;

[0061] Figure 8 It is a matching flow chart when using SWRL rules for logical reasoning;

[0062] Figure 9This is a framework diagram for aggregating and organizing heterogeneous multi-source terminal data obtained by multiple observation devices in a method for collaborative space-ground stereoscopic perception of a block emergency scene provided by an embodiment of the present invention;

[0063] Figure 10 It is a correlation model of heterogeneous data collected by the same observation equipment;

[0064] Figure 11 It is a flowchart of device-level aggregation and associative storage of multi-source heterogeneous data;

[0065] Figure 12 This is a flow chart of performing air-ground collaborative carbon dioxide inversion using an air-ground collaborative stereoscopic perception method for a block emergency scene provided by an embodiment of the present invention;

[0066] Figure 13 This is an architectural diagram of visualization of air-ground observation data in an air-ground collaborative stereoscopic perception method for block emergency scenarios provided by an embodiment of the present invention;

[0067] Figure 14 It is a flowchart of the geographic fusion display of air-ground observation data in an air-ground collaborative stereoscopic perception method for block emergency scenarios provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0068] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0069] Example 1:

[0070] Example 1 provides a method for space-ground collaborative stereoscopic perception of a block emergency scene, see Figure 1 , including the following steps:

[0071] S1. Centered on block emergency events, each type of observation equipment is considered as a basic event to construct an air-ground collaborative observation planning model.

[0072] Among them, when constructing the observation planning model of air-ground collaboration, collaborative modeling is carried out in terms of three attributes: time, space and events.

[0073] Construct collaborative observation function H α ,as follows:

[0074]

[0075] Where, represents the time parameter of the i-th type of observation equipment in the event observation task, represents the spatial parameters of the i-th type of observation equipment in the event observation task, It represents the observation element parameter of the i-th type of observation equipment in the event observation task, and n represents the total number of different types of observation equipment.

[0076] Based on the collaborative observation function, it is determined whether each type of observation equipment can join the collaborative observation; for the i-th type of observation equipment, when the type of observation equipment meets When , this type of observation device is added to the collaborative observation. Based on the time parameter, the space parameter, and the observation element parameter, weights are set for the observation devices added to the collaborative observation. The weights are used to represent the priorities of different types of observation devices.

[0077] The preferred solution also includes setting a confidence level, based on which the effectiveness of different types of observation equipment is adjusted to obtain the optimal observation plan. The confidence level includes a first indicator representing the timeliness of the observation, a second indicator representing the coverage of the observation space, and a third indicator representing the performance of the observation elements.

[0078] The first indicator represents the proportion of the observation area that a type of observation equipment can cover per unit time, as follows:

[0079]

[0080]

[0081] Where η t Represents the first indicator, S i represents the spatial range that the i-th observation device in this type of observation equipment can observe, m represents the total number of observation devices in this type of observation equipment, S represents the set of spatial ranges that the event observation task needs to observe, t α Indicates the time window value that this type of observation equipment can achieve the observation task requirements, T is T represents the start time of the observation task requirement of the i-th observation device in this type of observation equipment, ie Indicates the end time of the observation task requirement for the i-th observation device in this type of observation equipment; It represents the observation status index of the i-th observation device in this type of observation equipment within the time window. If the time window of the observation device intersects with the time window required by the observation task, Is 1, if there is no intersection is 0.

[0082] The second indicator represents the total coverage of the observation space of a type of observation equipment to the composite event target, as follows:

[0083]

[0084] Where η s Indicates the second indicator.

[0085] The third indicator represents the degree of matching of the collaborative observation device to the required attributes of the composite event, as follows:

[0086]

[0087] Where η n represents the third index, k represents the number of categories of observed basic events contained in the composite event; It represents the weight of the observation task requirement that the i-th type of observation equipment can achieve for the j-th type of observation basic event in the composite event; Represents the observation performance status indicator of the i-th type of observation equipment. If the observation status of this type of observation equipment meets the observation task requirements, is 1, otherwise is 0; i It represents the number of observation elements that can be observed by the observation equipment of the i-th type to meet the observation mission requirements.

[0088] S2. Construct a resource description ontology model of the block perception network, and establish inference rules based on the resource description ontology model to implement network planning.

[0089] Among them, when implementing network planning, the perception layer observation resources, network layer communication resources and application layer resources are taken as three first-level categories, and the internal categories are divided based on the hierarchical structure of each first-level category to form the resource description ontology model; on the constructed resource description ontology model, SWRL rules are constructed based on the constraint relationship between the perception layer, network layer and application layer in network planning; after receiving a network planning scheme request, instance mapping is performed through the resource description ontology model, and logical reasoning is performed using the SWRL rules to generate a network planning scheme that matches the resources at each layer.

[0090] The perception layer observation resources include the following four secondary categories: basic device attributes, observation attributes, network attributes, and data attributes. The basic device attributes include device ID, device name, and device model. The observation attributes include mobility attributes, measurement attributes, and location attributes. The mobility attributes include mobility and range of movement, the measurement attributes include sampling frequency and measurement range, and the location attributes include longitude and latitude. The data attributes include data type, data format, packet size, and data reporting frequency. The network attributes include supported interfaces and transmission requirements. The network layer communication resources include the following two secondary categories: basic network attributes and transmission attributes. The basic network attributes include ID, name, longitude, latitude, coverage radius, node capacity, and networking mode. The transmission attributes include network delay, transmission speed, and transmission bandwidth.

[0091] The matching process when using SWRL rules for logical reasoning includes: constructing a first network planning goal and a second network planning goal; the first network planning goal is to enable the perception layer observation resource to establish a communication connection with the server of the application layer resource through the network layer communication resource, and the server can receive the data collected by the perception layer observation resource; the second network planning goal is to improve network transmission performance, reduce network delay, increase network coverage, and reduce packet loss rate. Based on the network data and observation attributes of the perception layer observation resource, the basic attributes of the network of the network layer communication resource, and the bandwidth of the application layer resource, basic networking rules are established around the first network planning goal. Based on the data attributes of the perception layer observation resource and the transmission attributes of the network layer communication resource, network performance improvement rules are established around the second network planning goal.

[0092] S3. Aggregate and organize the multi-source terminal heterogeneous data obtained by several observation devices to obtain air-ground collaborative observation data.

[0093] When aggregating and organizing heterogeneous data from multiple observation devices, the data is divided into structured and unstructured data based on the data structure. The structured and unstructured data are then transmitted in parallel and via separate channels to a data aggregation platform. The data aggregation platform stores data based on structure and semantic associations. Each observation device is registered with a node ID, which serves as its ID. Heterogeneous data collected by the same device is associated using the node ID.

[0094] S4. Fusing the air-ground collaborative observation data and its inversion data into geographic information for display, to achieve three-dimensional perception of the emergency scene in the block.

[0095] Specifically, the data used for displaying fused geographic information includes: first-category data in text format, second-category data in video format, and third-category data in image format; the first-category data includes ground mobile sensor observation data, the second-category data includes ground mobile phone observation data, and the third-category data includes observation inversion maps and observation mosaic maps.

[0096] The observation inversion map is obtained by performing spatiotemporal matching on the air-ground collaborative observation data to obtain a data set, and then establishing an inversion model based on the data set to calculate and generate the observation inversion map.

[0097] The first type of data, the second type of data, and the third type of data all include the data itself and the geographical location coordinates corresponding to the data; the data is associated and integrated with the three-dimensional scene of the geographic information system platform through the geographical location coordinates.

[0098] Among them, the first type of data is displayed in the corresponding position of the three-dimensional scene in the form of a text annotation box; the second type of data displays the video in the form of a floating window, and its movement trajectory is marked in the three-dimensional scene according to the geographic location coordinates; the third type of data is displayed in the corresponding position of the three-dimensional scene in the form of a layer.

[0099] Example 2:

[0100] Example 2 provides a block emergency scene air-ground collaborative three-dimensional perception system, including:

[0101] The observation planning unit is used to build an air-ground collaborative observation planning model centered on block emergency events and taking each type of observation equipment as a basic event;

[0102] A network planning unit, configured to construct a resource description ontology model of the block perception network and establish inference rules based on the resource description ontology model to implement network planning;

[0103] The data aggregation and organization unit is used to aggregate and organize the heterogeneous data from multiple sources of observation equipment to obtain air-ground collaborative observation data;

[0104] A fusion display unit is used to display the air-ground collaborative observation data and its inversion data in a fused geographic information manner to achieve a three-dimensional perception of the emergency scene in the block;

[0105] The block emergency scene air-ground collaborative stereoscopic perception system is used to implement the steps in the block emergency scene air-ground collaborative stereoscopic perception method as described in Example 1.

[0106] The system provided in Example 2 corresponds to the method provided in Example 1, and therefore will not be described in detail.

[0107] The present invention will be further described below.

[0108] In the case of an emergency, the construction of a block 3D perception system plays a decisive role in monitoring block emergency events. In order to build a system with strong real-time performance, high accuracy and wide coverage, this paper constructs a system framework and method flow for the coordinated 3D perception of block emergency scenes. Figure 1First, in response to the monitoring needs of emergency scenarios, we collaboratively plan air-ground sensor observation tasks, and perceive the changes in the blocks to be observed by deploying air-ground sensor observation equipment; secondly, we build a communication network and resource allocation model to maximize the use of communication resources to transmit perception data; then, we build a multi-source terminal heterogeneous data aggregation and organization method, and upload the multi-source heterogeneous data perceived by the blocks to the aggregation platform in a branch way. The data received by the platform is divided into tables and stored in topic semantic associations; finally, we integrate the air-ground observation data to invert the block scene status of blocks larger than one square kilometer in real time and reliably, and realize situational awareness of emergency scenarios.

[0109] 1. Open space collaborative planning method for block emergency scenarios

[0110] By analyzing the observation needs of emergency scene events in block areas, the present invention takes each observation device as a basic event on the basis of analyzing and comparing the advantages and disadvantages of different observation devices, and constructs an observation planning model for the collaboration of multiple observation devices centered on block emergency events. By constructing a UML (Unified Modeling Language) class diagram of block emergency observation events, an overall event description of emergency event observation is performed, and the conceptual model is mathematically expressed to establish a collaborative observation planning model for multiple observation devices, construct evaluation indicators including timeliness, spatial coverage and observation element performance, and calculate corresponding performance evaluation indicators to verify the effectiveness of the constructed collaborative observation planning model to obtain the optimal observation plan.

[0111] 2. Block-aware network resource description model based on ontology model

[0112] Aiming at the block perception network, this paper studies the construction method of the block perception network resource description model and the network planning method based on the description model, aiming to achieve effective planning and management of block perception network resources and improve networking efficiency and quality.

[0113] To address the heterogeneity of block perception network resources, the present invention abstracts the reference elements and contents involved in network planning for each layer of resources and extracts common concepts based on observation and communication requirements. At the same time, it studies the intrinsic relationships and hierarchical structures among the perception layer, network layer, and application layer of the block, and forms a resource description ontology model for the block perception network.

[0114] To address the low efficiency of manual networking, this paper conducts network planning based on a neighborhood-aware network resource description ontology model. Taking into account network coverage and resource matching during data transmission, the constraints between data and network attributes of neighborhood-aware network resources are converted into inference rules. A Semantic Web Rule Language (SWRL) rule library is constructed to perform rule-based logical reasoning for planning and matching actual network resources.

[0115] 3. Method for organizing and aggregating multi-source heterogeneous data at the terminal

[0116] In view of the multi-source and heterogeneous nature of terminal data, this paper studies a method for aggregating and organizing terminal multi-source heterogeneous data, aiming to achieve effective aggregation, organization and management of terminal device-level data within a block.

[0117] In response to the problem of multi-source heterogeneous data access and aggregation at the terminal, the present invention constructs a data aggregation framework for emergency scenario air-ground collaborative block stereoscopic perception technology and system, divides the data into two categories of channels: structured and unstructured according to the data structure, and transmits the two categories of data in parallel. The data aggregation platform efficiently, channel-wise and parallelly accesses data collected by drone-mounted videos and various sensors, mobile phone videos and sensors, fixed and mobile sensors and other devices, and stores the parallel access data based on data structure and semantic association.

[0118] To address the problem of organizing heterogeneous data from multiple sources on terminals, this paper constructs a device-level heterogeneous data association model. Using the registered device node ID as a device-level identifier, each device collects heterogeneous data, tagged with the device's registered node ID, for categorized storage. Using the device's registered node ID as a key, this model enables device-level association between structured sensor data and unstructured video data.

[0119] 4. Fusion display of air-ground observation inversion data and spatiotemporal label extraction for multi-source data

[0120] Air-ground observation inversion data are divided into three categories according to the data format: ground mobile sensor observation data in text format, ground mobile phone observation data in video format, observation inversion map in image format, and observation mosaic map. The observation inversion map is based on the data set obtained by time-space matching of air-ground collaborative observation data, and is then calculated and generated by establishing an inversion model. In addition to the data itself, the three types of data also have the geographical location coordinates corresponding to the data. The present invention associates and integrates the inversion data with the three-dimensional scene of the geographic information system platform through the geographical coordinates of the inverted observation data, wherein the text-type mobile sensor observation data is displayed in the form of a text annotation box at the corresponding position in the three-dimensional scene; the video-type mobile phone observation data displays the video in the form of a floating window, and at the same time, its movement trajectory is marked in the three-dimensional scene according to the position coordinates; the image-type inversion map and mosaic map carry the geographical coordinates themselves and will be displayed in the form of a layer at the corresponding position in the three-dimensional geographical scene.

[0121] The above four aspects are explained in detail below.

[0122] like Figure 2 As shown in the figure, the entire collaborative planning modeling process is based on the needs of emergency observation events in the block, and collaborative modeling is carried out in terms of three attributes: time, space and events. Each observation device is regarded as a basic event.

[0123] Assume that A represents an emergency event and model it based on the spatiotemporal characteristics and attribute characteristics of the observation task requirements. Assume that T represents the temporal attribute of the event, S represents the spatial attribute of the event, and N represents the observation element attribute of the event. Where T is the time point or time period, representing the instantaneous time or time interval of the event; S describes the spatial information of the event, including the description of the location of the event, which can represent the set of spatial ranges that the event observation task needs to observe; N represents a series of observation conditions in the event. Based on the above description, the observation requirement event model is established, and the corresponding set formula is as follows:

[0124] A=E(T,S,N), T=[T s , T e ]

[0125] S={[(lng i ,lat i ), h], i = 1, 2, ..., n}

[0126] N={n1,n2,…,n k}

[0127] Where: A represents a set of events, T s 、T eThey are the start time and end time required for the observation task, which belong to the interval formula; S is the spatial attribute of the event, indicating the location information of the event, and the commonly used latitude and longitude data lng i and lat i , and the height information h relative to the baseline are used to determine whether the observation equipment with height attributes can meet the observation requirements; N represents the set of various requirements contained in the event, including observation equipment conditions, temporal resolution, spatial resolution, etc. Different parameter lists are established according to different event scenarios. For different T and S, the corresponding N is also different.

[0128] Collaborative modeling is performed for the same observation attribute of different observation devices. Different observation devices of the same observation event α i , α j ,have If the observation time meets And T i ≠T j , then two different observation devices α i With α j Constitute time coordination. Similarly, if α i With α j Satisfy in the observation space and Then the two observation devices meet the spatial coordination; if the observation event attributes of the two observation devices meet And N i ≠N j , then the two observation devices meet the collaborative observation requirements of the event.

[0129] For the block emergency event model A(T, S, N), T, S, and N represent the time parameter, space parameter, and observation element parameter in the event observation task, respectively. The planning function matrix in Equation 1.1 is obtained by describing each parameter:

[0130]

[0131] The parameters in formula (1.1) are used to construct collaborative planning functions in the time dimension, space dimension, and event demand dimension respectively, if and only if each column When the observation device α i Able to participate in the collaborative observation of this event. And the weight is set according to the importance of the observation equipment, that is, formula 1.2:

[0132] λ α =[λ1,…,λ n ] (1.2)

[0133] Where λ iIndicates the priority of different types of observation equipment, with a value range of (0,1). The three parameters H for the same observation event requirement T 、H s 、H N The higher the performance index, the higher the priority. Combining the overall value of the three performance parameters, we can get λ i The value of λ i The weight of the value is determined based on the opinions of authoritative persons or related phenomena. The larger the value, the higher the priority and the greater the role it plays in the observation event task.

[0134] The coordination of different types of equipment is defined as a composite event. The observation data requirements corresponding to a block emergency observation composite event in different time periods will change over time. Therefore, the roles played by basic events of different types of observation equipment in composite events vary greatly. Therefore, setting confidence levels to adjust the degree of role played by different basic events in composite events can provide a strong basis for the calculation of collaborative planning models by evaluating task requirements, determining observation elements, and establishing an indicator parameter system. Comprehensive evaluation indicators for collaborative planning can be constructed in terms of observation timeliness performance, spatial coverage, and observation element performance.

[0135] Taking urban and rural fire incidents as an example, the event model consists of basic events and compound events. Basic events consist of a single structure or function, expressing partial information about a complex phenomenon. For example, the observation behavior of a certain observation device is a basic event, as is the storage of structured data. Compound events are collections of single events, containing data of various different structures, capable of expressing the full range of information under complex conditions. For example, the overall observation of a fire incident is a compound event. Basic events describe compound events by establishing connections. For example, the observation device class aggregates the specific observation methods of multiple observation devices. Compound events are broken down into a series of basic events. For example, the storage tool class can be broken down into structured data storage basic events and unstructured storage basic events.

[0136] (1) Aging performance η t

[0137] Timeliness performance indicates the area ratio of the composite event that can be observed by a type of basic event of an observation device within a specific time. It is described as the observation coverage ratio of the observation task per unit time, that is, the proportion of the observation area that can be covered by a type of observation device per unit time. The specific expression is as follows:

[0138]

[0139] Where η t Represents the first indicator, S irepresents the spatial range that the i-th observation device in this type of observation equipment can observe, m represents the total number of observation devices in this type of observation equipment, and S represents the set of spatial ranges that the event observation task needs to observe; t α is the time window value that this type of observation equipment can realize the observation requirement, that is, formula 1.4:

[0140]

[0141] Where, T is T represents the start time of the observation task requirement of the i-th observation device in this type of observation equipment, ie Indicates the end time of the observation task requirement for the i-th observation device in this type of observation equipment; It represents the observation status index of the i-th observation device in this type of observation equipment within the time window. If the time window of the observation device intersects with the time window required by the observation task, Is 1, if there is no intersection is 0; that is is represented as follows:

[0142]

[0143] As can be seen from the formula, if the time window of a certain observation device intersects with the time window of the observation event requirement, then the observation device is guaranteed to execute the observation requirement in the time dimension within the time intersection of the observation task requirement. If the two do not intersect, it means that the observation device is unavailable under this event.

[0144] (2) Spatial coverage η s

[0145] The spatial coverage rate indicates the total coverage of the observation space of the basic events of a type of observation equipment to the composite event target, which is expressed as follows:

[0146]

[0147] Where η s Indicates the second indicator.

[0148] (3) Observation factor performance η n

[0149] The performance of observation elements is used to describe the degree of matching between the basic events of collaborative observation equipment and the required attributes of composite events, that is, Equation 1.7:

[0150]

[0151] Where η n represents the third index, k represents the number of categories of observed basic events contained in the composite event; It represents the weight of the observation task requirement that the i-th type of observation equipment can achieve for the j-th type of observation basic event in the composite event; Represents the observation performance status indicator of the i-th type of observation equipment. If the observation status of this type of observation equipment meets the observation task requirements, is 1, otherwise is 0; i It represents the number of observation elements that can be observed by the observation equipment of the i-th type to meet the observation mission requirements.

[0152] It can be understood as the confidence level of each type of observation equipment basic event that can meet the jth requirement among the k types of observation basic events for the composite event. According to its definition, The range of M is [0,1]. In order to describe the observation performance more concisely and clearly, we can further i Quantify, if it fully meets the observation requirements of event observation, then M i =1; if the higher degree of satisfaction is met, M i The value range is [0.8,1]; if it is generally satisfied, then M i The value range is [0.6, 0.8]; if it is basically satisfied, then M i The value range is [0.4, 0.6]. If a small amount of i The value range is [0.2, 0.4]; when the trace amount satisfies M i The value range is (0,0.2]; if this requirement is not met, M i =0.

[0153] N i It represents the set of observation elements for an observation device, that is, Equation 1.8:

[0154] N i ={n i1 ,n i2 ,…,n ik} (1.8)

[0155] Where n ik Represents the observation device α i An observation element, N i It is a set of observation elements, including data from multiple types of sensors and data information carried by the observation equipment itself. For a specific observation event, the observation equipment α i It has multiple observation elements. Based on this, the observation performance status indicators of the observation equipment are introduced. That is, formula 1.9:

[0156]

[0157] Where N is a set of observation conditions for the observed event. Represents the observation device α i The observation situation meets the observation task requirements. Similarly, Represents the observation device α i The observation capability does not meet the event requirements and therefore cannot support the event observation requirements.

[0158] like Figure 3 As shown in the example, in a specific embodiment, the network planning method of the present invention registers neighborhood-aware network resources in the form of a web page and queries and displays network planning solutions. The server receives the resource registration information and stores it in a relational database. Upon receiving a network planning solution request, it uses the constructed neighborhood-aware network resource description ontology model to perform instance mapping, employs SWRL rules for logical reasoning, generates a final network planning solution, and parses and stores the solution.

[0159] like Figure 4 As shown, in a specific example, the block perception network resource description model of the present invention determines a basic block perception network resource description ontology model with perception layer observation resources, network layer communication resources and application layer resources as the top-level core concepts. And the relationship between each layer of resources is mined and expanded to form a complete block perception network resource description ontology model, with three first-level classes as the top-level concepts, mainly including: perception layer observation resources (Sensor), network layer communication resources (Net) and application layer resources (Server). The first-level class is further divided into second-level classes and third-level classes, see the specific description. Figure 4 、 Figure 5 .

[0160] like Figure 5As shown in the example, the neighborhood perception network resource description model defines the relevant concepts of perception-layer observation resources and constructs an ontological description structure for these observation resources using conceptual terminology. It primarily includes four secondary categories: basic device attributes, observation attributes, network attributes, and data attributes. Basic device attributes include device ID, device name, and device model; observation attributes include movement, measurement, and position. Mobility attributes are divided into mobility and moving range; measurement attributes are divided into sampling frequency and measuring range; and position attributes are divided into latitude and longitude. Data attributes include data type, data format, data packet size, and data reporting frequency; and network attributes include supported interfaces and transmission requirements.

[0161] like Figure 6As shown in the example, the neighborhood-aware network resource description model defines the relevant concepts of network-layer communication resources and constructs an ontological description structure for these resources using conceptual terminology. It primarily includes two secondary categories: network basic attributes and transmission attributes. Basic attributes include ID, Name, longitude (Net_Longitude), latitude (Net_Latitude), coverage radius (Coverage), node capacity (Capacity), and networking mode (Construction). ID refers to the ID number automatically assigned to a network resource after registration in the network planning system. Name refers to the category name of the network resource, such as fiber or Wi-Fi. Net_Longitude and Net_Latitude refer to the longitude and latitude of the network resource's gateway / base station. Coverage refers to the network resource's maximum coverage radius. Capacity refers to the maximum number of communication nodes accessible to a single base station / gateway. Construction indicates whether the network resource transmits data via wired or wireless communication. Transmission attributes include network delay, transfer rate, and bandwidth.

[0162] like Figure 7 As shown, in a specific embodiment, based on the constructed block perception network resource description ontology model, the potential relationships and requirements of the block perception network perception layer, network layer and application layer in network planning are explored, SWRL rules are constructed, and logical reasoning for planning and matching actual resources based on the rules is completed in combination with mapping instances, and finally a network planning scheme with mutual matching of resources at each layer is derived and generated.

[0163] like Figure 8 The figure shows the matching process when the ontology model uses the network planning SWRL rule base for reasoning, which needs to focus on the two goals of network planning.

[0164] Network planning goal 1: Enable observation resources at the perception layer to establish a communication connection with application layer servers via network layer communication resources, enabling the servers to receive data collected by observation resources at the perception layer. Planning focuses on factors such as observation resource mobility, communication resource networking, and coverage.

[0165] Network planning goal two: Improve network transmission performance, reduce network latency, increase network coverage, and reduce packet loss. When planning, focus on the data volume of observation resources, the data transmission capacity of communication resources, and the data reception capacity of application layer resources.

[0166] Based on the basic attributes and observation attributes of the perception layer observation resources, the basic attributes of the network layer communication resources, and the bandwidth of the application layer resources, basic networking rules are established around the first network planning goal.

[0167] Based on the data attributes of the perception layer observation resources and the transmission attributes of the network layer communication resources, network performance improvement rules are established around the second network planning goal.

[0168] See also Figure 8 When conducting network planning, first determine whether the selected perception layer, network layer, and application layer resources can complete the formation of the network structure and whether they can be networked. If they can be networked, then use the mobility, supported network interfaces, and mobility range of the perception layer observation resources to infer and match the coverage of the network layer communication resources; through the established basic networking rules, complete the network planning of the access of the perception layer observation resources to the application layer, and achieve the first network planning goal. After meeting the conditions for the first network planning goal, the network performance improvement rules are used to match the perception layer observation resources with network layer communication resources that are more suitable for data transmission; finally, match the bandwidth and data acceptance capacity of the application layer resources to the data transmission requirements of the perception layer observation resources, achieve the second network planning goal, and complete the network planning.

[0169] The inference rule is expressed using formula 1.10, where E ij is the conditional part of the rule, H i It is the conclusion of rule-based reasoning. In rule-based reasoning, only when all the prerequisites of a rule are met can the conclusion be derived using this rule.

[0170]

[0171]

[0172] …

[0173]

[0174] For example: Sensor(?x)^has_deviceID(?x,?y)→register(?x,true). This rule means: if there is an instance x of the Sensor class, and x has device ID y, then this perception layer observation resource instance has been registered, and the register attribute of instance x can be deduced to be true. Figure 8 As shown in the figure, the network planning results are generated based on actual case reasoning. The reasoning results show the communication resources and application layer resources that can be selected by the perception layer observation resources, and provide network planning suggestions in a visual way.

[0175] like Figure 9Figure 1 shows the data aggregation framework for the air-ground collaborative block-level stereoscopic perception technology and system for emergency scenarios. The perception data from hardware devices, such as drone-mounted video and various sensors, mobile phone video and sensors, and fixed and mobile sensors, is divided into structured and unstructured data based on data structure. This data is then transmitted in parallel across two broad channels. Data from vehicle-mounted atmospheric sensors, vehicle-mounted environmental sensors, drone-mounted atmospheric and environmental sensors, and fixed atmospheric and environmental sensors is transmitted via structured data channels, while data from drone and mobile phone live video streams is transmitted via unstructured data channels. The data aggregation platform parses and repackages the received multi-source heterogeneous data, annotating all data with event semantics to differentiate and correlate events. This data is then stored in a database, providing data support for the semantic association, aggregation, and interpretation of multi-source heterogeneous data.

[0176] like Figure 10 The figure below shows a model for associating heterogeneous data from multiple types of device observations. All observation devices register a node ID to identify and distinguish different devices. All data collected by the same device is uniquely identified by a node ID. Environmental and atmospheric data collected by sensors are structured data, while images and videos collected by mobile phones and drones are unstructured data. Heterogeneous data collected by the same device is associated with each other through node IDs.

[0177] like Figure 11 The figure below illustrates the process of accessing, aggregating, storing, and organizing video and sensor data. This involves setting up an NGINX streaming server to receive video streams using various communication protocols. Based on the device registration node ID, a video file storage area is established, and the video file storage path is provided to a MySQL database. Sensor data is encapsulated in JSON format and transmitted using sockets. A relational database is then built, associating and storing data based on the device registration node IDs to which the sensors are attached. Using device registration nodes as tags, device-level association is achieved between structured sensor data and unstructured video data.

[0178] like Figure 12 Using carbon dioxide as an example, this paper illustrates the collaborative air-ground inversion process for surface environmental parameters. In this specific example, drones use multispectral cameras to collect spectral image data. After preprocessing with radiometric calibration and geometric correction, image stitching is performed to generate multispectral images of a street block larger than one square kilometer. Ground-based carbon dioxide sensor observation data is preprocessed through noise reduction and data augmentation, and then spatiotemporally matched with drone imagery to produce an air-ground matching dataset for surface carbon dioxide inversion. Based on a deep learning model, feature vectors are extracted, and a surface carbon dioxide concentration inversion model is established. The deep learning model is then adjusted to obtain the carbon dioxide inversion model and inversion map.

[0179] like Figure 13 As shown in the software architecture diagram, the online visualization service for air-ground observation of the present invention is developed based on the Vue.js and Cesium.js framework of the Web side, and consists of an application layer and a data layer. In a specific embodiment, the application layer provides a fusion display of the following four types of application data: ground mobile sensor observation data, ground mobile phone observation video, ground air-ground collaborative observation inversion map, and drone block observation mosaic map. The data layer is responsible for requesting the corresponding data from the data storage server according to the needs of the application layer, and transmitting it to the application layer for display. The interfaces for data interaction between the data layer and the application layer include two types of restful interfaces and geoserver services. The restful interface is used for the application layer to request the observation data of ground mobile sensors and the observation video data of ground mobile phones from the data layer; geoserver is a geographic data service used to request air-ground collaborative observation inversion maps and drone observation block mosaic maps.

[0180] like Figure 14 As shown in the flow chart, in response to the demand for integrated display of air-ground observation and inversion data in the geographic information system, the present invention designs a web-based display browsing page. After the user enters the page, he can browse and select different types of observation data for display. In a specific embodiment, it mainly includes three categories: observation data is ground mobile sensor atmospheric observation data and environmental observation data in text format; observation data is ground mobile phone observation data in video format; observation data is image-type surface PM2.5 concentration inversion map, surface carbon dioxide concentration inversion map, surface temperature inversion map, and drone block observation image mosaic map. In addition to the data itself, the above types of data also include the geographical location coordinates of the data. After selecting the application data category, for text observation data, data will be obtained from the data layer through the restful interface. According to the coordinates, markers will be created in the scene, and a text annotation pop-up window will be added to it. Click to view the sensor observation data at that point; for ground mobile phone video observation data, videos will be obtained from the data layer through the restful interface, and a video floating pop-up window will be added to the scene to play the video stream. At the same time, the geographic location will be marked in the three-dimensional scene to display the dynamic change trajectory of the mobile phone; for image data, due to the large geographical area covered, the data volume is large, and the data is mostly in tif format, which carries geographic location information. Therefore, the data will be read from the data layer through the access interface provided by geoserver and added to the three-dimensional map in the form of a layer to realize the geographic fusion display of the image data.

[0181] The embodiment of the present invention provides a method and system for collaborative three-dimensional perception of space and ground in a block emergency scenario, which has at least the following technical effects:

[0182] (1) This invention proposes an event-centered observation planning model for sudden emergency events. By identifying event tasks, the observation planning model is constructed in a hierarchical manner. Aiming at specific task requirements, the collaborative observation problem of observation resources is solved, and an overall event observation planning model is established. Through calculation, different types of observation equipment are enabled to collaboratively complete the task of observing a certain element, resulting in an air-ground collaborative observation planning model. On this basis, an evaluation index system is also established to evaluate the rationality of observation equipment scheduling, thereby improving the overall observation performance of events in different dimensions.

[0183] (2) Based on the observation requirements and communication requirements, the present invention starts from the observation attributes of the perception layer observation resources, the transmission attributes of the network layer communication resources, and the data reception attributes of the application layer resources, and uses the ontology model to describe the resources of each layer and associate and combine the three layers of resources to construct a resource description ontology model for the entire block perception network. In addition, the present invention proposes a network planning method based on the block perception network resource description ontology model. By mining the potential relationships and requirements of the perception layer, network layer and application layer of the block perception network in network planning, SWRL rules are constructed. Based on the rules, logical reasoning is established to plan and match the actual perception layer, network layer and application layer resources. Finally, a network planning scheme is derived to match the resources of each layer with each other, and a reasonable, efficient and targeted communication network is obtained.

[0184] (3) The present invention proposes a design scheme for a multi-source heterogeneous big data integration platform in emergency scenarios, which can solve many problems existing in the integration platform in emergency scenarios, give full play to the role of various types of perception data, and effectively aggregate and organize the terminal device-level data in the block.

[0185] (4) Based on the multi-angle collection of scene spatial data by air-ground collaborative observation, the present invention builds a geographic information system platform and performs integrated geographic information display of air-ground observation data and their inversion data. It can mine scene information from multi-scale and multi-directional analysis, and can make up for the shortcoming of insufficient display of scene observation data.

[0186] Finally, it should be noted that the above specific implementation methods are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to examples, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for collaborative three-dimensional perception of space and ground in a block emergency scene, characterized by: include: Focusing on block emergency events, each type of observation equipment is considered as a basic event to build an air-ground collaborative observation planning model. Constructing a resource description ontology model of the block perception network, and establishing inference rules based on the resource description ontology model to implement network planning; Aggregate and organize heterogeneous data from multiple sources of observation equipment to obtain air-ground collaborative observation data; The air-ground collaborative observation data and its inversion data are integrated with geographic information display to achieve three-dimensional perception of street emergency scenes; Among them, when constructing the observation planning model of air-ground collaboration, collaborative modeling is carried out in terms of three attributes: time, space and events; Constructing collaborative observation functions ,as follows: Where, represents the time parameter of the i-th type of observation equipment in the event observation task, represents the spatial parameters of the i-th type of observation equipment in the event observation task, represents the observation element parameter of the i-th type of observation equipment in the event observation task, and n represents the total number of different types of observation equipment; Based on the collaborative observation function, it is determined whether each type of observation equipment can join the collaborative observation; for the i-th type of observation equipment, when the type of observation equipment meets When such observation equipment is added to the collaborative observation; In combination with the time parameter, the space parameter, and the observation element parameter, weights are set for the observation devices participating in the collaborative observation, where the weights are used to represent the priorities of different types of observation devices; When implementing network planning, perception layer observation resources, network layer communication resources, and application layer resources are taken as three first-level categories, and internal subcategories are performed based on the hierarchical structure of each first-level category to form the resource description ontology model; On the resource description ontology model that has been constructed, SWRL rules are constructed based on the constraint relationships among the perception layer, network layer, and application layer in network planning; After receiving the network planning scheme request, the resource description ontology model is used to perform instance mapping, and the SWRL rules are used to perform logical reasoning to generate a network planning scheme matching resources at each layer.

2. The method for collaborative three-dimensional perception of space and ground in block emergency scenes according to claim 1 is characterized in that: The construction of an observation planning model for air-ground collaboration also includes setting a confidence level, and adjusting the effectiveness of different types of observation equipment based on the confidence level to obtain the best observation plan; The confidence level includes a first indicator representing the timeliness of observation, a second indicator representing the coverage of observation space, and a third indicator representing the performance of observation elements; The first indicator represents the proportion of the observation area that a type of observation equipment can cover per unit time, as follows: Where, Represents the first indicator, It represents the spatial range that the i-th observation device in this type of observation equipment can observe, m represents the total number of observation devices in this type of observation equipment, Represents the set of spatial ranges that the event observation task needs to observe, Indicates the time window value that this type of observation equipment can achieve the observation task requirements. Indicates the start time of the observation task requirement of the i-th observation device in this type of observation equipment, Indicates the end time of the observation task requirement for the i-th observation device in this type of observation equipment; It represents the observation status index of the i-th observation device in this type of observation equipment within the time window. If the time window of the observation device intersects with the time window required by the observation task, Is 1, if there is no intersection is 0; The second indicator represents the total coverage of the observation space of a type of observation equipment to the composite event target, as follows: Where, represents the second indicator; The third indicator represents the degree of matching of the collaborative observation device to the required attributes of the composite event, as follows: Where, represents the third index, Indicates the number of categories of observed basic events contained in the composite event; It represents the weight of the observation task requirement that the i-th type of observation equipment can achieve for the j-th type of observation basic event in the composite event; Represents the observation performance status indicator of the i-th type of observation equipment. If the observation status of this type of observation equipment meets the observation task requirements, is 1, otherwise is 0; It represents the number of observation elements that can be observed by the observation equipment of the i-th type to meet the observation mission requirements.

3. The method for collaborative three-dimensional perception of space and ground in block emergency scenes according to claim 1 is characterized in that: The perception layer observation resources include the following four secondary categories: basic attributes of the device, observation attributes, network attributes and data attributes; the basic attributes of the device include device ID, device name and device model; the observation attributes include mobility attributes, measurement attributes and location attributes; the mobility attributes include mobility and mobility range, the measurement attributes include sampling frequency and measurement range, and the location attributes include longitude and latitude; the data attributes include data type, data format, data packet size and data reporting frequency; the network attributes include supported interfaces and transmission requirements; The network layer communication resources include the following two secondary categories: basic network attributes and transmission attributes; the basic network attributes include ID, name, longitude, latitude, coverage radius, node capacity and networking mode; the transmission attributes include network delay, transmission speed and transmission bandwidth.

4. The method for collaborative three-dimensional perception of space and ground in a block emergency scene according to claim 3 is characterized in that: The matching process when using SWRL rules for logical reasoning includes: Establish a first network planning goal and a second network planning goal; the first network planning goal is to enable the perception layer observation resource to establish a communication connection with the server of the application layer resource through the network layer communication resource, so that the server can receive data collected by the perception layer observation resource; the second network planning goal is to improve network transmission performance, reduce network latency, increase network coverage, and reduce packet loss rate; Establishing basic networking rules around the first network planning goal based on the network data and the observed attributes of the perception layer observation resources, the basic attributes of the network of the network layer communication resources, and the bandwidth of the application layer resources; Based on the data attributes of the perception layer observation resources and the transmission attributes of the network layer communication resources, network performance improvement rules are established around the second network planning goal.

5. The method for space-ground collaborative 3D perception of block emergency scenes according to claim 1 is characterized in that: When aggregating and organizing heterogeneous data from multiple sources of terminals obtained by several observation devices, the heterogeneous data from multiple sources are divided into structured data and unstructured data according to the data structure, and the structured data and the unstructured data are transmitted to the data aggregation platform in parallel and in different channels; The data aggregation platform performs storage based on data structure and semantic association; wherein each observation device is registered with a node ID as the ID of the observation device, and heterogeneous data collected by the same observation device are associated through the node ID.

6. The method for collaborative three-dimensional perception of space and ground in a block emergency scene according to claim 1 is characterized in that: The data used for displaying fused geographic information includes: first-category data in text format, second-category data in video format, and third-category data in image format; the first-category data includes ground mobile sensor observation data, the second-category data includes ground mobile phone observation data, and the third-category data includes observation inversion maps and observation mosaic maps; The observation inversion map is obtained by performing spatiotemporal matching on the air-ground collaborative observation data to obtain a data set, and then establishing an inversion model based on the data set to calculate and generate the observation inversion map. The first type of data, the second type of data, and the third type of data all include the data itself and the geographical location coordinates corresponding to the data; the data is associated with and integrated with the three-dimensional scene of the geographic information system platform through the geographical location coordinates; Among them, the first type of data is displayed in the corresponding position of the three-dimensional scene in the form of a text annotation box; the second type of data displays the video in the form of a floating window, and its movement trajectory is marked in the three-dimensional scene according to the geographic location coordinates; the third type of data is displayed in the corresponding position of the three-dimensional scene in the form of a layer.

7. A block emergency scene space-ground collaborative three-dimensional perception system, characterized by: include: The observation planning unit is used to build an air-ground collaborative observation planning model centered on block emergency events and taking each type of observation equipment as a basic event; A network planning unit, configured to construct a resource description ontology model of the block perception network and establish inference rules based on the resource description ontology model to implement network planning; The data aggregation and organization unit is used to aggregate and organize the heterogeneous data from multiple sources of observation equipment to obtain air-ground collaborative observation data; A fusion display unit is used to display the air-ground collaborative observation data and its inversion data in a fused geographic information manner to achieve a three-dimensional perception of the emergency scene in the block; The block emergency scene air-ground collaborative stereoscopic perception system is used to implement the steps in the block emergency scene air-ground collaborative stereoscopic perception method as described in any one of claims 1-6.

Images