Oceanic and remote environmental information monitoring system and method based on semantic information extraction

Abstract

The application relates to a long-ocean long-range environment information monitoring system and method based on semantic information extraction, which comprises a sea control center and an unmanned equipment cluster, the unmanned equipment cluster is embedded with an environment perception module, a data storage module, a semantic coding module and an intention prediction module, and the sea control center is embedded with a semantic recovery module; the sea control center controls the unmanned equipment cluster to a target area, the environment perception module performs information perception and data collection to generate large-scale multi-dimensional data and caches the data into the data storage module, the semantic coding module performs semantic extraction according to the perception data in the data storage module and task demand parameters, the intention prediction module predicts the behavior and intention of a detection target based on the extracted semantic information and transmits the information to the sea control center through a sea wireless channel. Compared with the prior art, the application has the advantages of greatly reducing the overhead and time delay of data transmission.

Description

Technical Field

[0001] This invention relates to intelligent acquisition technology for marine environmental information in distant waters, and in particular to a marine environmental information monitoring system and method based on semantic information extraction. Background Technology

[0002] With the development of technology and the continuous exploitation of marine resources, security issues in the open ocean are becoming increasingly prominent. Traditional methods of ocean information processing mainly rely on a limited number of sensors and manual observation, which cannot meet the comprehensive and real-time requirements of ocean monitoring. Statistics show that current traditional ocean monitoring methods can only cover about 30% of the ocean area, and the efficiency of data collection and processing is low, wasting a significant amount of human and financial resources annually. Furthermore, due to the limited number of sensors, the accuracy and coverage of ocean monitoring data are restricted, posing challenges to a comprehensive understanding of the maritime situation and timely response, making it difficult to meet the comprehensive and multi-dimensional monitoring needs of the marine environment.

[0003] A search of Chinese Patent Publication No. CN108693849A reveals an automatic long-range maritime reconnaissance system. This system involves an automated ocean-going vessel carrying a swarm of unmanned aerial vehicles (UAVs) and underwater unmanned surface vessels (USVs) to a designated sea area. The system then automatically releases the UAVs and USVs to conduct long-range maritime reconnaissance and feeds back information from both systems to a land-based command system. The UAVs and USVs acquire real-time image information within the target sea area and transmit it to the automated ocean-going vessel system. The land-based command system receives the image information from the automated ocean-going vessel system and generates decision information based on the location of suspicious targets marked in the image information and the weather conditions at their location. The automated ocean-going vessel system receives the decision information, executes it, and sends the execution result back to the land-based command system. However, this existing patent suffers from problems such as single-information perception, large information redundancy, and high data transmission overhead and latency. Therefore, how to address the shortcomings of existing single-information perception methods for long-range maritime scenarios and reduce data transmission overhead and latency has become a technical problem that needs to be solved. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a system and method for monitoring marine environmental information based on semantic information extraction.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] According to one aspect of the present invention, a marine environmental information monitoring system based on semantic information extraction is provided. The system includes a marine control center and an unmanned equipment cluster. The unmanned equipment cluster embeds an environmental perception module, a data storage module, a semantic encoding module, and an intent prediction module. The marine control center embeds a semantic recovery module.

[0007] The maritime control center controls the unmanned equipment cluster to the target area. The environmental perception module performs information perception and data collection to generate large-scale multi-dimensional data, which is then cached in the data storage module. The semantic encoding module performs semantic extraction based on the perception data in the data storage module and the mission requirement parameters. The intent prediction module predicts the behavior and intent of the probe target based on the extracted semantic information and transmits it to the maritime control center via a maritime wireless channel. The semantic recovery module recovers the original data based on the directional probability map.

[0008] As a preferred technical solution, the unmanned equipment cluster includes sensing radar, sonar, and sensors. The environmental perception module perceives the marine environment and identifies potential targets based on the detection data of the unmanned equipment cluster, and obtains global or local three-dimensional video data, two-dimensional image data, and one-dimensional text data.

[0009] As a preferred technical solution, the data storage module is a core component equipped on the unmanned equipment cluster. It adopts a high-performance, reliable, and secure data storage device to receive, process, and store marine environmental data collected by the environmental perception module.

[0010] As a preferred technical solution, the semantic encoding module constructs a directional probability map using existing marine environmental data and known target types, and then inputs the mission requirements and information from the data storage device into the directional probability map to extract key semantic information.

[0011] As a preferred technical solution, when constructing the directional probability graph in the semantic encoding module, the information extraction system is first used to select semantic entities with task-related information, and the connection probability between entities is learned through a convolutional neural network. Then, by constructing triples of entity-probability-entity, a multi-layer directional probability graph G = (V; E) related to the task is constructed, where V identifies the entity list and E describes the adjacency matrix of entity connection probabilities.

[0012] As a preferred technical solution, the intent prediction module uses semantic information extracted by learning from a deep neural network to identify the location, movement trend, task type, and whether the potential target is dangerous.

[0013] As a preferred technical solution, the target intent prediction result of the deep neural network is one-dimensional text information.

[0014] According to another aspect of the present invention, a method for using the aforementioned semantic information extraction-based ocean-going environmental information monitoring system is provided, the method comprising the following steps:

[0015] Step S1: According to the mission requirements, the maritime control center dispatches a cluster of unmanned equipment to the target area to complete information detection work. The cluster of unmanned equipment and the environmental perception module detect the marine environment and potential targets in the area, capture multi-source data such as video, images and text, and cache the data in the data storage module after denoising and reconstruction, and then transmit it to the semantic encoding module for information processing.

[0016] In step S2, the semantic encoding module constructs a directional probability graph based on the stored information and the requirements of the task, extracts task-oriented semantic information, outputs the environmental information and potential target information required by the task, and transmits the output results to the intent prediction module.

[0017] Step S3: The intent prediction module constructs a neural network, uses the output information of the semantic encoding module as the input of the neural network, performs potential target intent processing, and outputs the possible movement direction, task intent and final purpose of the potential target.

[0018] Step S4: After the information processing is completed, the unmanned equipment cluster will return the extracted semantic information and predicted target intent information to the maritime control center through the maritime wireless channel.

[0019] In step S5, the maritime control center recovers the original data based on the constructed directional probability map and uses it for mission processing.

[0020] As a preferred technical solution, in step S4, the unmanned equipment cluster integrates the extracted semantic information and predicted target intent information based on mission requirements into a data packet, selects a data link transmission method according to communication resources and channel status, and returns the information to the maritime control center through a wireless channel.

[0021] As a preferred technical solution, in step S5, after receiving semantic information and intent information, the maritime control center recovers the original data based on the directional probability map and uses it for its own tasks.

[0022] Compared with the prior art, the present invention has the following advantages:

[0023] 1) This invention simultaneously utilizes unmanned equipment clusters and environmental perception modules to acquire multi-source ocean information, and obtains key mission-oriented information and target intent information based on semantic encoding and intent prediction modules. This effectively solves the shortcomings of existing single information perception methods in ocean scenarios, effectively removes redundant information, retains key mission-oriented information, greatly reduces data transmission overhead and latency, and further ensures the efficiency and safety of maritime mission execution.

[0024] 2) This invention introduces unmanned equipment clusters and multi-source data fusion technology, which enables comprehensive perception and information extraction of the marine environment, and can significantly improve the coverage and accuracy of monitoring;

[0025] 3) This invention employs semantic extraction technology based on deep learning, which effectively eliminates information redundancy, improves the accuracy and efficiency of data processing, and reduces the cost and latency of data transmission;

[0026] 4) This invention features rapid response and efficient task execution, effectively improving the efficiency and safety of maritime missions and providing an advanced and reliable solution for ocean-going information processing. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the system and method of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] like Figure 1 As shown, this invention proposes a marine environmental information monitoring system based on semantic information extraction. The system includes a marine control center and an unmanned equipment cluster. The unmanned equipment cluster embeds an environmental perception module, a data storage module, a semantic encoding module, and an intent prediction module. The marine control center embeds a semantic recovery module. This system mainly addresses the current problems of difficulty in deploying communication equipment at sea, limited data collection methods, insufficient information analysis capabilities, and excessive manpower consumption.

[0030] The maritime control center dispatches a cluster of unmanned equipment to the target area. The environmental perception module performs information perception and data collection, generating large-scale, multi-dimensional data, which is then cached in the data storage module. The semantic encoding module extracts semantics based on the perceived data in the data storage module and the mission requirement parameters. The intent prediction module predicts the behavior and intent of the detected target based on the extracted semantic information and transmits this prediction to the maritime control center via a maritime wireless channel for subsequent information analysis.

[0031] This invention utilizes unmanned equipment clusters and environmental perception modules to acquire multi-source ocean information, and obtains mission-oriented key information and target intent information based on semantic encoding and intent prediction modules. It effectively solves the shortcomings of existing single-information perception methods in ocean scenarios, effectively removes redundant information, retains mission-oriented key information, greatly reduces data transmission overhead and latency, and further ensures the efficiency and safety of maritime mission execution.

[0032] like Figure 1 As shown, the method corresponding to the system of the present invention includes the following steps:

[0033] Step 1: According to the mission requirements, the maritime control center dispatches a cluster of unmanned equipment to the target area. The cluster of unmanned equipment and the environmental perception module detect the marine environment and potential targets in the area, capture multi-source data such as video, images and text, and cache the data in the data storage module after denoising, reconstruction and other operations.

[0034] Step 2: Based on the stored information and the requirements of the task, the semantic encoding module constructs a directional probability graph, extracts task-oriented semantic information, outputs the environmental information and potential target information required by the task, and transmits the output results to the intent prediction module.

[0035] Step 3: The intent prediction module constructs a neural network, uses the output information of the semantic encoding module as the input of the neural network, processes the potential target intent, and outputs the possible movement direction, task intent, and final purpose of the potential target.

[0036] Step four: After information processing is completed, the unmanned equipment cluster will return the extracted semantic information and predicted target intent information to the maritime control center via the maritime wireless channel.

[0037] Step 5: The maritime control center reconstructs the original data based on the constructed directional probability map and uses it for mission processing.

[0038] In step one, the unmanned equipment cluster is equipped with various sensing devices, including radar, sonar, and sensors, enabling it to comprehensively perceive the marine environment and identify potential targets. The multi-source data acquired through these sensing devices includes global or local 3D video. Two-dimensional image and one-dimensional text Information such as these. After preprocessing and reconstruction, large-scale data is obtained. And cache it in the data storage module of the unmanned equipment cluster.

[0039] In step two, the semantic encoding module constructs a directional probability map G based on stored information and task requirements, thereby extracting task-oriented semantic information. The semantic encoding module constructs the directional probability map using existing marine environmental data and known target types. Subsequently, the task requirements and information from the data storage device are used... Key semantic information is extracted from the input to the directional probability graph G.

[0040] In the process of semantic extraction, the resulting directional probability graph G contains a set of vertices and a set of edges. Specifically, each vertex is a semantic entity used to locate task-related information, which can be identified from the dataset by an information extraction system. Let v be the i-th entity composed of data subsequences. i The edges in a directional probabilistic graph represent paths between two entity vertices, denoted by the connection probability p. Using the training dataset, two vertices... The connection probability p between i,j It can be computed using a convolutional neural network. Therefore, the directional probability map G can be represented by a triple (entity v). i -probability p i,j -Entity v j Therefore, constructing a directional probabilistic graph can be formalized as G = (V; E), where V is the list of entities and E is the adjacency matrix describing the edge connection probabilities.

[0041] In step three, the intent prediction module constructs a neural network and uses the output of the semantic encoding module. This information is used as input to a neural network. The neural network processes the intent of a potential target and outputs its possible movement direction, task intent, and final goal. This step utilizes deep learning technology to predict the target's intent by learning extracted semantic information, thus obtaining the target's movement direction, task intent, and final goal, and synthesizing one-dimensional text data. Used for subsequent transmission, it provides an important basis for decision-making in maritime missions.

[0042] In step four, the unmanned equipment cluster transmits the extracted semantic information and predicted target intent information to the maritime control center via a maritime wireless channel. This process ensures that the maritime control center receives critical information in a timely manner to support subsequent mission decisions and execution.

[0043] Finally, in step five, after receiving semantic and intent information, the maritime control center recovers the original data based on the directional probability map and uses it for the mission.

[0044] This unmanned equipment cluster semantic extraction system for ocean information processing is based on the technology of identifying and monitoring marine information and potential targets using unmanned equipment clusters and environmental perception modules. It acquires multi-source information and performs intelligent analysis of large-scale multi-source information through semantic coding and intent prediction modules to extract mission-oriented semantic information and intent information of the detected targets. By transmitting the processed semantic and intent information to the maritime control center, the system effectively improves the quality and real-time performance of information, facilitating the execution of maritime missions and enabling relevant personnel to further analyze combat plans and other related work.

[0045] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A deep-sea environmental information monitoring system based on semantic information extraction, characterized in that, The system includes a maritime control center and an unmanned equipment cluster. The unmanned equipment cluster is embedded with an environmental perception module, a data storage module, a semantic encoding module, and an intent prediction module. The maritime control center is embedded with a semantic recovery module. The maritime control center controls the unmanned equipment cluster to the target area. The environmental perception module performs information perception and data collection to generate large-scale multi-dimensional data, which is then cached in the data storage module. The semantic encoding module performs semantic extraction based on the perception data in the data storage module and the mission requirement parameters. The intent prediction module predicts the behavior and intent of the probe target based on the extracted semantic information and transmits it to the maritime control center through the maritime wireless channel. The semantic recovery module recovers the original data based on the directional probability map. The semantic encoding module constructs a directional probability map using existing marine environmental data and known target types, and then inputs the mission requirements and information from the data storage device into the directional probability map to extract key semantic information. When constructing the directional probability graph in the semantic encoding module, firstly, the information extraction system is used to select semantic entities with task-related information, and the connection probabilities between entities are learned through a convolutional neural network. Then, by constructing entity-probability-entity triples, a multi-layer directional probability graph related to the task is built. ,in Identify the entity list, An adjacency matrix that describes the probability of entity connections.

2. The ocean-going and distant-area environmental information monitoring system based on semantic information extraction according to claim 1, characterized in that, The unmanned equipment cluster includes sensing radar, sonar, and sensors. The environmental perception module perceives the marine environment and identifies potential targets based on the detection data of the unmanned equipment cluster, obtaining global or local three-dimensional video data, two-dimensional image data, and one-dimensional text data.

3. The ocean-going and distant-area environmental information monitoring system based on semantic information extraction according to claim 1, characterized in that, The data storage module is a core component equipped on the unmanned equipment cluster. It uses high-performance, reliable and secure data storage equipment to receive, process and store marine environmental data collected by the environmental perception module.

4. The ocean-going and distant-area environmental information monitoring system based on semantic information extraction according to claim 1, characterized in that, The intent prediction module uses semantic information extracted by learning from deep neural networks to identify the location, movement trend, task type, and whether the potential target is dangerous.

5. The ocean-going and distant-area environmental information monitoring system based on semantic information extraction according to claim 4, characterized in that, The target intent prediction result of the deep neural network is one-dimensional text information.

6. A method for monitoring marine environmental information based on semantic information extraction as described in claim 1, characterized in that, The method includes the following steps: Step S1: According to the mission requirements, the maritime control center dispatches a cluster of unmanned equipment to the target area to complete information detection work. The cluster of unmanned equipment and the environmental perception module detect the marine environment and potential targets in the area, capture multi-source data such as video, images and text, and cache the data in the data storage module after denoising and reconstruction, and then transmit it to the semantic encoding module for information processing. In step S2, the semantic encoding module constructs a directional probability graph based on the stored information and the requirements of the task, extracts task-oriented semantic information, outputs the environmental information and potential target information required by the task, and transmits the output results to the intent prediction module. Step S3: The intent prediction module constructs a neural network, uses the output information of the semantic encoding module as the input of the neural network, performs potential target intent processing, and outputs the possible movement direction, task intent and final purpose of the potential target. Step S4: After the information processing is completed, the unmanned equipment cluster will return the extracted semantic information and predicted target intent information to the maritime control center through the maritime wireless channel. In step S5, the maritime control center recovers the original data based on the constructed directional probability map and uses it for mission processing.

7. The method according to claim 6, characterized in that, In step S4, the unmanned equipment cluster integrates the extracted semantic information and predicted target intent information based on mission requirements into a data packet, selects a data link transmission method according to communication resources and channel status, and returns the information to the maritime control center through a wireless channel.

8. The method according to claim 6, characterized in that, In step S5, after receiving semantic and intent information, the maritime control center recovers the original data based on the directional probability map and uses it for its own mission.

Images