A target interpretation system and method based on satellite video data

Abstract

The present invention belongs to the technical field of intelligent processing and application of multimodal remote sensing images, and discloses a target interpretation system and method based on satellite video data. The system includes an interpretation template management module, a satellite video data intelligent preprocessing module, a dynamic target cloud service module, an intelligent visualization interpretation module, and an interpretation target management module. The present invention improves the accuracy and reliability of satellite video data target detection, significantly improves the performance of the target interpretation system based on satellite video data, and provides a new direction for the development of related technologies in the future.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent processing and application of multimodal remote sensing images, and in particular relates to a target interpretation system and method based on satellite video data. Background Art

[0002] With the continuous advancement of remote sensing technology, video satellites are increasingly being used for Earth observation. They not only provide high-resolution static images but also capture continuously changing dynamic processes, offering unprecedented perspectives for environmental monitoring, disaster warning, urban planning, and other fields. However, the use of satellite video data for target identification and analysis also presents challenges. While current deep learning-based object detection models can achieve high accuracy in remote sensing imagery, human intuition and experience remain indispensable in specific situations. For example, in complex and changing environments or when targets are highly similar, deep learning models may produce false positives or false negatives. In these cases, human intervention in target interpretation can not only compensate for algorithmic shortcomings but also improve the reliability of the final results.

[0003] To improve the accuracy and reliability of object detection in satellite video data, incorporating the experience and knowledge of human experts is a viable approach. By establishing a human-machine collaborative mechanism, the domain expertise of human experts can be leveraged to compensate for algorithmic deficiencies, thereby enhancing the accuracy and robustness of object recognition. Furthermore, this approach can help identify subtle differences and patterns that algorithms tend to overlook, further improving the overall performance of the system. Summary of the Invention

[0004] The purpose of the present invention is to provide a target interpretation system and method based on satellite video data. The present invention improves the accuracy and reliability of satellite video data target detection, significantly improves the performance of the target interpretation system based on satellite video data, and provides a new direction for the development of related technologies in the future.

[0005] The technical solution adopted by the present invention is a target interpretation system based on satellite video data, including an interpretation template management module, a satellite video data intelligent preprocessing module, a dynamic target cloud service module, an intelligent visualization interpretation module, and an interpretation target management module, wherein:

[0006] The interpretation template management module provides full life cycle management for interpretation templates of satellite video data, and supports creation, modification, query, and deletion of interpretation templates.

[0007] The satellite video data intelligent preprocessing module is used to realize intelligent analysis of satellite video metadata, dynamic frame sequence optimization, multi-dimensional space geometry correction, Web map service cloud publishing, and distributed storage and retrieval;

[0008] The dynamic target cloud service module is used to implement dynamic target detection result analysis, dynamic target association analysis, dynamic target storage, and dynamic target vector tile cloud publishing;

[0009] The intelligent visual interpretation module is used to realize multi-source data fusion display and intelligent interactive interpretation;

[0010] The interpretation target management module is used to realize interpretation target storage, interpretation target modification, interpretation target vector tile cloud publishing, data export and integration.

[0011] Furthermore, the interpretation template management module includes interpretation template creation, interpretation template modification, interpretation template query, and interpretation template deletion. Interpretation template creation supports customized interpretation templates. New interpretation templates are created through the browser interface, and the interpretation template description, attributes, attribute values, attribute value types, and attribute value verification rules are defined through forms. Interpretation template creation is also completed by importing JSON files or XML files, and then the defined interpretation templates are saved to the database; interpretation template modification supports modification and saving of defined interpretation template descriptions, attributes, attribute values, attribute value types, and attribute value verification rules, and supports adding and deleting attribute values; interpretation template deletion supports deleting no longer needed interpretation templates to release system resources; interpretation template query supports querying interpretation templates through multiple conditions.

[0012] Furthermore, the satellite video data intelligent preprocessing module includes satellite video metadata intelligent analysis, dynamic frame sequence optimization, multi-dimensional space geometry correction, Web map service cloud publishing, and distributed storage retrieval. The satellite video metadata intelligent analysis uses a custom auxiliary information field parsing template to parse the auxiliary information of different satellite video data. At the same time, it uses the word embedding Word2Vec and cosine similarity technology in natural language processing to parse fields that have not appeared in the template and dynamically update the field mapping; the dynamic frame sequence optimization processes each frame in the satellite video data, including noise suppression, contrast enhancement, and super-resolution reconstruction, outputs satellite video frame data, and re-encodes and encapsulates the satellite video frame data into satellite video browsing data; the multi-dimensional space geometry correction optimizes the satellite video output by the dynamic frame sequence. The video frame data is geometrically corrected in combination with metadata information, and the corrected data is reprojected to the WGS84 coordinate system; the Web map service cloud publishing quickly converts the satellite video frame data that has undergone dynamic frame sequence optimization and multi-dimensional spatial geometric correction into satellite video frame data in cloud-optimized TIFF file format, uploads it to the distributed object storage platform, and seamlessly publishes it as a WMS layer to support real-time query and visual browsing; distributed storage retrieval uploads satellite video data, satellite video frame data, and satellite video display data to the distributed object storage platform, associates and stores the satellite video data information, satellite video frame data information, satellite video display data information, and fields after intelligent analysis of satellite video metadata in a structured database, and establishes a Geohash retrieval index.

[0013] Furthermore, the operation steps of the distributed storage and retrieval process use distributed transaction management technology to ensure data consistency and integrity.

[0014] Furthermore, the dynamic target cloud service module includes dynamic target detection result parsing, dynamic target association analysis, dynamic target storage, and dynamic target vector tile cloud publishing. The dynamic target detection result parsing will parse the dynamic target detection result based on the satellite video data to obtain the dynamic target information in the dynamic target detection result, among which the pixel coordinates, latitude and longitude coordinates, the video frame number, ID, heading, speed, and identity information of the dynamic target in the dynamic target detection result are mainly parsed; the dynamic target association analysis will associate the dynamic target information with the corresponding satellite video frame data and dynamic target historical data to obtain the dynamic target association analysis results; the dynamic target storage will store the dynamic target information in a structured database and the dynamic target association analysis results in an unstructured database to generate a geographic spatial retrieval index; the dynamic target vector tile cloud publishing will seamlessly integrate the dynamic target information and the dynamic target association analysis results and publish them as a dynamic target vector tile cloud service.

[0015] Furthermore, the dynamic target association analysis associates the dynamic target information with the corresponding satellite video frame data and dynamic target historical data. The specific steps for obtaining the dynamic target association analysis results are: based on the dynamic target ID and the video frame number of the dynamic target, the dynamic target trajectory information, dynamic target distribution area and dynamic target activity heat are generated. At the same time, based on the upper and lower frame information, the dynamic target heading and dynamic target speed in the dynamic target detection result are re-verified and calculated. If the dynamic target detection result contains the identity information of the dynamic target, the dynamic target historical data is further associated with the identity information to generate the dynamic target trajectory information, dynamic target distribution area and dynamic target activity heat based on the dynamic target historical data.

[0016] Furthermore, the intelligent visualization interpretation module includes multi-source data fusion display and intelligent interactive interpretation. The multi-source data fusion display supports the associated fusion display of satellite video data, satellite video frame data, dynamic target information, dynamic target historical trajectory, and interpretation target; the intelligent interactive interpretation supports clicking on dynamic target attribute information, and automatically filling the dynamic target attribute information into the similar attributes in the interpretation template through the word embedding Word2Vec and cosine similarity technology in natural language processing, and adding or modifying the attribute values ​​in the interpretation template to complete the target interpretation; it supports drawing the target shape outline on the WMS layer corresponding to the satellite video frame data. After the drawing is completed, the interpretation template pops up and automatically fills in the attribute values ​​therein, and adds or modifies the attribute values ​​in the interpretation template to complete the target interpretation.

[0017] Furthermore, the multi-source data fusion display supports the associated fusion display of satellite video data, satellite video frame data, dynamic target information, dynamic target historical trajectory, and interpreted targets, among which:

[0018] Satellite video data display: supports loading and displaying satellite video data in the corresponding detection area, playing satellite video data streams through WebRTC technology, providing high-definition and smooth video display effects, and providing remote access and sharing to the outside world;

[0019] Satellite video frame data display: supports displaying a list of each frame of satellite video data on the browser interface. Click to view detailed WMS layers, which is convenient for frame-by-frame analysis, comparison and interpretation;

[0020] Dynamic target information display: supports loading and displaying dynamic target lists and dynamic target vector tiles in the browser interface, supports querying the dynamic target information closest to the mouse click position, and queries the dynamic target information in the dynamic target list selected by the mouse, clearly displaying the dynamic target position and attributes;

[0021] Interpretation target information display: supports loading and displaying the interpretation target list and interpretation target vector tiles in the browser interface, supports querying the interpretation target information closest to the mouse click position, and queries the interpretation target information in the mouse selected interpretation target list, clearly displaying the interpretation target location and attributes;

[0022] Satellite video frame data-dynamic target information associated fusion display: supports the simultaneous display of satellite video frame data WMS layer and the dynamic target information corresponding to the frame data. By clicking the corresponding dynamic target detection result, the video frame data WMS layer corresponding to the dynamic target detection result and the dynamic target detection result attribute information are displayed;

[0023] Dynamic target historical trajectory display: supports simultaneous query of multiple dynamic target trajectory information and draws the historical movement trajectories of multiple dynamic targets on the browser interface; supports selecting a specific time period to view the target's movement path over the past period of time, assisting in dynamic target interpretation.

[0024] Furthermore, the interpretation target management module includes interpretation target storage, interpretation target modification, interpretation target vector tile cloud publishing, data export and integration. Interpretation target storage is to perform data verification on the interpretation target to be saved according to the attribute value verification rules defined in the interpretation template. After the data verification passes, interpretation conflict detection is performed. If passed, the interpretation target is stored in the database. If not, the failed attribute value is prompted through the browser interface; interpretation target modification is to modify one or more attribute values ​​in the saved interpretation target. According to the attribute value verification rules defined in the interpretation template, the interpretation target to be saved after modification is verified. If passed, the interpretation target is stored in the database. If not, the failed attribute value is prompted through the browser interface; interpretation target vector tile cloud publishing is to seamlessly integrate and publish the parsed interpretation target as a interpretation target vector tile cloud service; data export and integration support, according to user needs, downloading the saved interpretation targets in JSON format to the user's local folder / according to the external system interface, constructing the interpretation target list into interface content and sending it to the external system.

[0025] Furthermore, the conflict detection rule is that the same target or similar target cannot appear in the same video frame. The same target means that the target ID / target serial number is the same, and the similar target means that the target ID / target serial number is different, but among all the defined attributes, 80% of the attribute values ​​match the target object of another target.

[0026] A target interpretation method based on satellite video data is implemented by the above-mentioned target interpretation system based on satellite video data, comprising the following steps:

[0027] S1: Create a new interpretation template in the interpretation template management module based on user needs / external system interface, fill the required interpretation attributes and attribute value verification rules into the interpretation template, enter the interpretation template name and save;

[0028] S2, loading satellite video data, and the satellite video data intelligent preprocessing module preprocesses the satellite video data;

[0029] S3, the dynamic target cloud service module parses the dynamic target detection results imported from the system, performs correlation analysis with satellite video frame data and dynamic target historical data, stores the parsing results and correlation analysis results in the database, and seamlessly integrates and publishes them into a dynamic target vector tile cloud service;

[0030] S4, the intelligent visual interpretation module correlates and fuses satellite video data, satellite video frame data, dynamic target information, dynamic target historical trajectory, and interpreted targets for display. It automatically fills dynamic target attribute information into similar attributes in the interpretation template using word embedding Word2Vec and cosine similarity technology in natural language processing to complete target interpretation. It also automatically fills in the attribute values ​​of the target shape outline drawn on the WMS layer of the satellite video frame data into the interpretation template, and adds or modifies attribute values ​​in the interpretation template to complete target interpretation.

[0031] S5, the target interpretation management module performs data verification and conflict detection on the target interpretation information generated by the intelligent visualization interpretation module. If it passes, the target interpretation is saved in the database and seamlessly integrated and published as a vector tile cloud service. If it fails, the reason for failure is prompted.

[0032] S6. Based on user needs, the interpretation target management module downloads the saved interpretation targets in JSON format to the user's local folder. Based on the external system interface, the interpretation target management module constructs the interpretation target list into interface content and sends it to the external system.

[0033] Furthermore, in S2, the specific steps of preprocessing the satellite video data by the satellite video data intelligent preprocessing module include:

[0034] S21, performs intelligent analysis of satellite video metadata;

[0035] S22, performing dynamic frame sequence optimization;

[0036] S23, performing multi-dimensional space geometric correction;

[0037] S24, Web map service cloud release;

[0038] S25, using distributed transaction management technology to store and retrieve satellite video data and satellite video data preprocessing results.

[0039] Furthermore, in S22, the specific steps of performing dynamic frame sequence optimization include:

[0040] S221, using a wavelet transform method to perform noise suppression on each frame of satellite video data;

[0041] S222, using a histogram equalization method to perform contrast enhancement on the noise-suppressed satellite video data;

[0042] S223, uses the deep learning ESRGAN model to perform super-resolution reconstruction on the contrast-enhanced satellite video data and outputs satellite video frame data;

[0043] S224: re-encode and encapsulate the satellite video frame data into satellite video browsing data.

[0044] The beneficial effects of the present invention are:

[0045] (1) Enhanced recognition accuracy: The intuition and experience of human experts in target recognition are introduced, enabling the system to identify targets more accurately, especially when dealing with blurred or complex background images.

[0046] (2) Improved robustness: Through the human-machine collaboration mechanism, the present invention can effectively handle special situations that are difficult for algorithms to cope with, such as target detection in severe weather conditions and recognition of atypical target morphologies.

[0047] (3) Reduce false alarm rates: Human experts can quickly identify and filter out false alarms, reducing the proportion of false positives in the system output and improving the quality of the decision-making process.

[0048] (4) Assisted learning and improvement: The data collected during the human-computer interaction process can be used to train and optimize the algorithm model, forming a closed loop of continuous improvement and further improving the intelligence level of the system.

[0049] (5) Strong adaptability: The system is not only applicable to known environments, but can also flexibly adapt to new scenarios and challenges through human-machine collaboration, thus expanding its scope of application.

[0050] (6) Education and training value: This system can also serve as a training platform to help new operators learn and master target interpretation skills, and promote knowledge inheritance and development.

[0051] In summary, this invention improves the accuracy and reliability of target detection in satellite video data by leveraging the knowledge of human experts, significantly improves the performance of target interpretation systems based on satellite video data, and provides new directions for the development of related technologies in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a composition diagram of the system of the present invention.

[0053] Figure 2 Flowchart of the method of the present invention. DETAILED DESCRIPTION

[0054] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings.

[0055] This paper addresses the low accuracy and reliability of target detection in satellite video data and proposes a target interpretation system based on satellite video data. This system, based on a high-performance computing platform and utilizing WebGIS and video streaming technologies, utilizes a browser / server (B / S) architecture to achieve target interpretation capabilities for large-scale satellite video data. This ensures stable operation and efficient processing, optimizes the human-computer collaboration process, and improves target interpretation quality and efficiency.

[0056] A target interpretation system based on satellite video data, such as Figure 1 As shown, it includes an interpretation template management module, a satellite video data intelligent preprocessing module, a dynamic target cloud service module, an intelligent visual interpretation module, and an interpretation target management module.

[0057] Interpretation template management module

[0058] The interpretation template management module is a highly integrated interpretation tool that provides full life cycle management for interpretation templates of satellite video data and supports the creation, modification, query, and deletion of interpretation templates.

[0059] The interpretation template management module includes creation of interpretation template, modification of interpretation template, query of interpretation template and deletion of interpretation template.

[0060] (1) Create a new interpretation template

[0061] It supports custom interpretation templates. New interpretation templates can be created through the browser interface. The interpretation template description, attributes, attribute values, attribute value types, and attribute value verification rules can be defined through the form. Interpretation templates can also be created by importing JSON files or XML files, and then the defined interpretation templates can be saved in the database.

[0062] (2) Modification of interpretation template

[0063] Supports modification and saving of defined interpretation template descriptions, attributes, attribute values, attribute value types, and attribute value verification rules, and supports adding and deleting attribute values.

[0064] (3) Deletion of interpretation template

[0065] Supports deleting interpretation templates that are no longer needed to release system resources.

[0066] (4) Interpretation template query

[0067] Supports querying interpretation templates based on multiple conditions (such as interpretation template name, creation time, modification time, and attributes).

[0068] Satellite video data intelligent preprocessing module

[0069] The satellite video data intelligent preprocessing module is used to realize intelligent analysis of satellite video metadata, dynamic frame sequence optimization, multi-dimensional space geometry correction, Web map service cloud publishing, and distributed storage and retrieval.

[0070] The satellite video data intelligent preprocessing module includes intelligent analysis of satellite video metadata, dynamic frame sequence optimization, multi-dimensional space geometry correction, Web map service cloud publishing, and distributed storage retrieval.

[0071] (1) Intelligent analysis of satellite video metadata

[0072] Auxiliary information from different satellite video data is parsed using custom auxiliary information field parsing templates. Word2Vec and cosine similarity techniques from natural language processing are used to parse fields not present in the templates and dynamically update field mappings. This process is primarily used to parse key fields from satellite video data, such as satellite ID, sensor type, resolution, reconnaissance start time, reconnaissance coverage (latitude and longitude), and the projected coordinate system used.

[0073] (2) Dynamic frame sequence optimization

[0074] Each frame in the satellite video data is processed, including noise suppression, contrast enhancement, and super-resolution reconstruction, and the satellite video frame data is output. The satellite video frame data is then re-encoded and encapsulated into satellite video browsing data.

[0075] (3) Multidimensional space geometric correction

[0076] The satellite video frame data output by dynamic frame sequence optimization is combined with metadata information for geometric correction (RPC / GCPs correction), and the corrected data is reprojected to the WGS84 coordinate system.

[0077] (4) Web map service cloud publishing

[0078] The satellite video frame data that has undergone dynamic frame sequence optimization and multi-dimensional spatial geometric correction is quickly converted into cloud-optimized TIFF file format, uploaded to the distributed object storage platform, and published as a WMS layer to support real-time query and visual browsing.

[0079] (5) Distributed storage and retrieval

[0080] Satellite video data, satellite video frame data, and satellite video display data are uploaded to a distributed object storage platform. Satellite video data information, satellite video frame data information, satellite video display data information (data path, data size, total number of satellite video frames), and fields after intelligent analysis of satellite video metadata are associated and stored in a structured database (Postgres). A Geohash retrieval index is established. The operation steps of this process use distributed transaction management technology to ensure data consistency and integrity.

[0081] Dynamic target cloud service module

[0082] The dynamic target cloud service module is used to realize dynamic target detection result analysis, dynamic target association analysis, dynamic target storage, and dynamic target vector tile cloud publishing.

[0083] The dynamic target cloud service module includes dynamic target detection result analysis, dynamic target association analysis, dynamic target storage, and dynamic target vector tile cloud publishing.

[0084] (1) Analysis of dynamic target detection results

[0085] The dynamic target detection results based on satellite video data (external source) are parsed to obtain the dynamic target information in the dynamic target detection results. Among them, the pixel coordinates, longitude and latitude coordinates, video frame number, ID, heading, speed, identity information, etc. of the dynamic target in the dynamic target detection results are mainly parsed.

[0086] (2) Dynamic target association analysis

[0087] The dynamic target information is correlated and analyzed with the corresponding satellite video frame data and dynamic target historical data to provide comprehensive dynamic target information, that is, to obtain dynamic target correlation analysis results.

[0088] Dynamic target association analysis associates dynamic target information with corresponding satellite video frame data and dynamic target historical data. The specific steps to obtain the dynamic target association analysis results are as follows: based on the dynamic target ID and the video frame number of the dynamic target, the dynamic target trajectory information, dynamic target distribution area and dynamic target activity heat are generated. At the same time, based on the upper and lower frame information, the dynamic target heading and dynamic target speed in the dynamic target detection results are re-verified and calculated. If the dynamic target detection result contains the identity information of the dynamic target (string number), the dynamic target historical data is further associated with the identity information to generate the dynamic target trajectory information, dynamic target distribution area and dynamic target activity heat based on the dynamic target historical data.

[0089] Dynamic target historical data includes externally imported dynamic target information (such as news about the target, general description of the target, etc.), related dynamic target databases of other systems, etc.

[0090] (3) Dynamic target storage

[0091] The dynamic target information is stored in a structured database, and the dynamic target association analysis results are stored in an unstructured database (Redis) to generate a geospatial retrieval index.

[0092] (4) Dynamic target vector tile cloud release

[0093] The dynamic target information is seamlessly integrated and published as a dynamic target vector tile cloud service (mainly the Point layer), and the dynamic target association analysis results are seamlessly integrated and published as a dynamic target vector tile cloud service (mainly the Line layer).

[0094] Intelligent visual interpretation module

[0095] The intelligent visual interpretation module is a highly integrated data display and interpretation tool used to achieve multi-source data fusion display and intelligent interactive interpretation.

[0096] The intelligent visual interpretation module includes multi-source data fusion display and intelligent interactive interpretation.

[0097] (1) Multi-source data fusion display

[0098] Supports associated fusion display of satellite video data, satellite video frame data, dynamic target information, dynamic target historical trajectory, and interpreted targets, including:

[0099] Satellite video data display: supports loading and displaying satellite video data in the corresponding detection area, playing satellite video data streams through WebRTC technology, providing high-definition and smooth video display effects, and providing remote access and sharing to the outside world;

[0100] Satellite video frame data display: supports displaying a list of each frame of satellite video data on the browser interface. Click to view detailed WMS layers, which is convenient for frame-by-frame analysis, comparison and interpretation;

[0101] Dynamic target information display: supports loading and displaying dynamic target lists and dynamic target vector tiles in the browser interface, supports querying the dynamic target information closest to the mouse click position, and queries the dynamic target information in the dynamic target list selected by the mouse, clearly displaying the dynamic target position and attributes;

[0102] Interpretation target information display: supports loading and displaying the interpretation target list and interpretation target vector tiles in the browser interface, supports querying the interpretation target information closest to the mouse click position, and queries the interpretation target information in the mouse selected interpretation target list, clearly displaying the interpretation target location and attributes;

[0103] Satellite video frame data-dynamic target information associated fusion display: supports the simultaneous display of satellite video frame data WMS layer and the dynamic target information corresponding to the frame data. By clicking the corresponding dynamic target detection result, the video frame data WMS layer corresponding to the dynamic target detection result and the dynamic target detection result attribute information are displayed;

[0104] Dynamic target historical trajectory display: supports simultaneous query of multiple dynamic target trajectory information, and draws the historical movement trajectories of multiple dynamic targets on the browser interface. It supports selecting a specific time period to view the target's movement path over the past period of time, assisting in dynamic target interpretation.

[0105] (2) Intelligent interactive interpretation

[0106] It supports clicking on dynamic target attribute information and automatically filling the dynamic target attribute information into similar attributes in the interpretation template through word embedding Word2Vec and cosine similarity technology in natural language processing. The attribute values ​​in the interpretation template can be added or modified to complete target interpretation; it supports drawing target shape outlines such as rectangles, circles, and polygons on the WMS layer corresponding to the satellite video frame data. After drawing, the interpretation template pops up and automatically fills in the attribute values ​​such as latitude and longitude, area, length, and azimuth. The attribute values ​​in the interpretation template can be added or modified to complete target interpretation.

[0107] Interpretation target management module

[0108] The interpretation target management module is used to realize interpretation target storage, interpretation target modification, interpretation target vector tile cloud publishing, data export and integration.

[0109] The target interpretation management module includes target interpretation storage, target interpretation modification, target interpretation vector tile cloud publishing, data export and integration.

[0110] (1) Determine target storage

[0111] According to the attribute value verification rules defined in the interpretation template, data verification is performed on the interpretation target to be saved. After the data verification passes, interpretation conflict detection is performed. If it passes, the interpretation target is stored in the database. If it fails, the failed attribute value is prompted through the browser interface.

[0112] The main rule for conflict detection is that identical or similar targets cannot appear in the same video frame. Identical targets have the same target ID / target serial number, while similar targets have different target IDs / target serial numbers. However, at least 80 percent of the attribute values ​​of all defined attributes must match those of another target (this ratio, as a threshold, can be adjusted according to specific needs).

[0113] (2) Modification of target interpretation

[0114] Modify one or more attribute values ​​in the saved interpretation target, and perform data verification on the interpretation target that needs to be saved after the modification according to the attribute value verification rules defined in the interpretation template. If it passes, the interpretation target will be stored in the database. If it fails, the browser interface will prompt the failed attribute value.

[0115] (3) Interpretation of target vector tile cloud release

[0116] The parsed interpretation targets are seamlessly integrated and published as interpretation target vector tile cloud services (mainly Point layers).

[0117] (4) Data export and integration

[0118] Supports downloading saved interpretation targets in JSON format to the user's local folder according to user needs / constructing the interpretation target list into interface content according to the external system interface and sending it to the external system.

[0119] The target interpretation method based on satellite video data of the present invention is realized by the target interpretation system based on satellite video data of the present invention. Taking the satellite video data of Jilin-1 Gaofen 03C as an example, Figure 2 As shown, the following steps are included:

[0120] S1. Create a new interpretation template in the interpretation template management module according to user requirements / external system interface, fill the required interpretation attributes and attribute value verification rules into the interpretation template, enter the interpretation template name and save it.

[0121] Create a new interpretation template in the interpretation template management module, enter the interpretation attribute fields (target ID, target serial number, target model, longitude, latitude, confidence, length, width, target type, etc.), enter the constraint information of the corresponding fields (such as number size range, latitude and longitude accuracy range, target type selection range, string length range), enter the interpretation template name and save it.

[0122] S2, loading satellite video data, the satellite video data intelligent preprocessing module preprocesses the satellite video data, the specific steps include:

[0123] S21, performs intelligent analysis of satellite video metadata;

[0124] The XML file format of the satellite video metadata of Jilin-1 Gaofen 03C satellite is intelligently parsed to obtain key field information in the XML file, such as satellite code, sensor type, resolution, detection start time, frame coverage (latitude and longitude), projection coordinate system, and frame information, mainly including the total number of frames in the video and the start and end time of each frame.

[0125] S22, performing dynamic frame sequence optimization;

[0126] The MP4 format of Jilin-1 Gaofen 03C satellite video data is split into each frame for processing to improve the image details and clarity of the satellite video data. The specific steps include:

[0127] S221, using a wavelet transform method to perform noise suppression on each frame of satellite video data;

[0128] The wavelet transform method is used to suppress noise in each frame of PNG image.

[0129] S222, using a histogram equalization method to perform contrast enhancement on the noise-suppressed satellite video data;

[0130] The histogram equalization method is used to enhance the contrast of the PNG image after noise suppression.

[0131] S223, uses the deep learning ESRGAN model to perform super-resolution reconstruction on the contrast-enhanced satellite video data and outputs satellite video frame data;

[0132] The deep learning ESRGAN model is used to perform super-resolution reconstruction on the contrast-enhanced PNG image and output the browsed PNG image.

[0133] S224, re-encoding and packaging the satellite video frame data into satellite video browsing data;

[0134] Re-encode and encapsulate the browsing PNG image into satellite video browsing data.

[0135] S23, performing multi-dimensional space geometric correction;

[0136] The browsed PNG image is combined with metadata information (latitude and longitude coordinates of the four corner points of upper left, lower left, upper right and lower right, projection coordinate system, pixel coordinates of the four corner points of upper left, lower left, upper right and lower right) for geometric correction (RPC / GCPs correction), and the corrected data is reprojected to the WGS84 coordinate system.

[0137] S24, Web map service cloud release;

[0138] The browsed PNG images that have undergone dynamic frame sequence optimization and multi-dimensional spatial geometric correction are quickly converted into cloud-optimized TIFF file format, uploaded to the distributed object storage platform, and seamlessly integrated and published as WMS layers to support real-time query and visual browsing.

[0139] S25, using distributed transaction management technology to store and retrieve satellite video data and satellite video data preprocessing results;

[0140] Satellite video data (PNG images), satellite video frame data (browsing PNG images), and satellite video browsing data are uploaded to a distributed object storage platform. Satellite video data information, satellite video frame data information, satellite video browsing data information (data path, data size, total number of satellite video frames), and fields after intelligent metadata analysis are associated and stored in a structured database (Postgres). A Geohash retrieval index is established. The operation steps of this process use distributed transaction management technology to ensure data consistency and integrity.

[0141] S3, the dynamic target cloud service module parses the dynamic target detection results imported from outside the system, performs correlation analysis with satellite video frame data and dynamic target historical data, stores the parsing results and correlation analysis results in the database, and seamlessly integrates and publishes them into a dynamic target vector tile cloud service.

[0142] Specifically, the dynamic target cloud service module loads the dynamic target detection results (external source) and parses them to obtain the dynamic target information in the dynamic target detection results; associates and analyzes the dynamic target information with the corresponding satellite video frame data and dynamic target historical data to obtain the dynamic target association analysis results; stores the dynamic target information in a structured database (Postgres), and stores the dynamic target association analysis results in an unstructured database (Redis) to generate a geographic spatial retrieval index; seamlessly integrates and publishes the dynamic target information as a dynamic target vector tile cloud service (mainly the Point layer), and seamlessly integrates and publishes the dynamic target association analysis results as a dynamic target vector tile cloud service (mainly the Line layer).

[0143] S4, the intelligent visual interpretation module associates and fuses satellite video data, satellite video frame data, dynamic target information, dynamic target historical trajectory, and interpretation target for display. It automatically fills dynamic target attribute information into similar attributes in the interpretation template through word embedding Word2Vec and cosine similarity technology in natural language processing to complete target interpretation. It also draws the target shape outline on the WMS layer of satellite video frame data, automatically fills the attribute values ​​into the interpretation template, and adds or modifies attribute values ​​in the interpretation template to complete target interpretation.

[0144] Specifically, the intelligent visualization interpretation module loads and displays the satellite video data in the corresponding reconnaissance area, and plays the satellite video data stream through WebRTC technology; displays the list of each frame of satellite video data on the browser interface, and clicks to view the detailed WMS layer, which is convenient for frame-by-frame analysis, comparison and interpretation; loads and displays the dynamic target list and dynamic target vector tiles on the browser interface, queries the dynamic target information closest to the mouse click position, and queries the dynamic target information in the dynamic target list selected by the mouse, and clearly displays the dynamic target position and attributes; loads and displays the interpretation target list and interpretation target vector tiles on the browser interface, queries the interpretation target information closest to the mouse click position, and queries the interpretation target information in the interpretation target list selected by the mouse, and clearly displays the interpretation target position and attributes; displays the satellite video frame data WMS layer and the dynamic target information corresponding to the frame data at the same time, and clicks on the corresponding dynamic target to detect The detection results are displayed, and the video frame data WMS layer corresponding to the dynamic target detection result and the dynamic target detection result attribute information are displayed; the historical movement trajectories of multiple dynamic targets are drawn on the browser interface, and a specific time period is selected to view the movement path of the dynamic target in the past period of time, which assists in the interpretation of dynamic targets; by clicking on the dynamic target attribute information, the dynamic target attribute information is automatically filled into the similar attributes in the interpretation template through the word embedding Word2Vec and cosine similarity technology in natural language processing, and the attribute values ​​in the interpretation template can be added or modified to complete the target interpretation; if the interpretation target does not appear in the dynamic target list, the outline of the target shape such as rectangle, circle, polygon, etc. can be drawn on the WMS layer corresponding to the video frame data. After the drawing is completed, the interpretation template pops up and automatically fills in the attribute values ​​such as latitude and longitude, area, length, azimuth, etc., and the attribute values ​​in the interpretation template can be added or modified to complete the target interpretation.

[0145] S5, the target interpretation management module performs data verification and conflict detection on the target interpretation information generated by the intelligent visualization interpretation module. If it passes, the target interpretation will be saved in the database and seamlessly integrated and published as a vector tile cloud service. If it fails, the reason for failure will be prompted.

[0146] S6. Based on user needs, the interpretation target management module downloads the saved interpretation targets in JSON format to the user's local folder. Based on the external system interface, the interpretation target management module constructs the interpretation target list into interface content and sends it to the external system.

[0147] The contents not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. A target interpretation system based on satellite video data, characterized in that: It includes a judgment template management module, a satellite video data intelligent preprocessing module, a dynamic target cloud service module, an intelligent visualization judgment module and a judgment target management module, among which: The interpretation template management module provides full life cycle management for interpretation templates of satellite video data, and supports creation, modification, query and deletion of interpretation templates; The satellite video data intelligent preprocessing module is used to realize intelligent analysis of satellite video metadata, dynamic frame sequence optimization, multi-dimensional space geometry correction, Web map service cloud publishing and distributed storage and retrieval; The satellite video metadata intelligent parsing parses the auxiliary information of different satellite video data through a custom auxiliary information field parsing template, and simultaneously uses word embedding Word2Vec and cosine similarity to parse fields that have not appeared in the template, and dynamically updates the field mapping; The dynamic frame sequence optimization comprises: Wavelet transform method is used to suppress noise in each frame of satellite video data; The histogram equalization method is used to enhance the contrast of the satellite video data after noise suppression. The deep learning ESRGAN model is used to perform super-resolution reconstruction on the contrast-enhanced satellite video data and output satellite video frame data; Re-encode and encapsulate satellite video frame data into satellite video browsing data; The dynamic target cloud service module is used to implement dynamic target detection result analysis, dynamic target association analysis, dynamic target storage and dynamic target vector tile cloud publishing; The intelligent visualization interpretation module is used to realize multi-source data fusion display and intelligent interactive interpretation; The intelligent interactive interpretation supports completing target interpretation by clicking on dynamic target attribute information, automatically filling the dynamic target attribute information into similar attributes in the interpretation template through word embedding Word2Vec and cosine similarity, and adding or modifying the attribute value in the interpretation template; the intelligent interactive interpretation also supports drawing the target shape outline on the WMS layer corresponding to the satellite video frame data, popping up the interpretation template after drawing, and automatically filling in the attribute value therein, adding or modifying the attribute value in the interpretation template, to complete target interpretation; The target interpretation management module is used to realize target interpretation storage, target interpretation modification, target interpretation vector tile cloud publishing and data export and integration.

2. The target interpretation system based on satellite video data according to claim 1, characterized in that: The new interpretation template supports custom interpretation templates. New interpretation templates are created through the browser interface. The interpretation template description, attributes, attribute values, attribute value types and attribute value verification rules are defined through the form. After the interpretation template is created by importing a JSON file or an XML file, the defined interpretation template is saved in the database. The interpretation template modification supports modification and saving of the already defined interpretation template description, attributes, attribute values, attribute value types and attribute value verification rules, and supports adding and deleting attribute values. The interpretation template deletion supports deleting interpretation templates that are no longer needed to release system resources. The interpretation template query supports querying the interpretation template through multiple conditions.

3. The target interpretation system based on satellite video data according to claim 1, characterized in that: The multi-dimensional space geometric correction combines the satellite video frame data output by the dynamic frame sequence optimization with the metadata information for geometric correction, and reprojects the corrected data into the WGS84 coordinate system; The Web map service cloud publishing quickly converts the satellite video frame data after dynamic frame sequence optimization and multi-dimensional space geometry correction into satellite video frame data in cloud optimized TIFF file format, uploads it to the distributed object storage platform, and publishes it as a WMS layer to support real-time query and visual browsing; Distributed storage retrieval uploads satellite video data, satellite video frame data, and satellite video display data to a distributed object storage platform, associates and stores satellite video data information, satellite video frame data information, satellite video display data information, and fields after intelligent analysis of satellite video metadata in a structured database, and establishes a Geohash retrieval index.

4. The target interpretation system based on satellite video data according to claim 3, characterized in that: The distributed storage and retrieval process uses distributed transaction management to ensure data consistency and integrity.

5. The target interpretation system based on satellite video data according to claim 1, characterized in that: The dynamic target detection result analysis analyzes the dynamic target detection result based on the satellite video data to obtain the dynamic target information in the dynamic target detection result, wherein the dynamic target information includes the pixel coordinates, longitude and latitude coordinates, the video frame number, ID, heading, speed and identity information of the dynamic target; the dynamic target association analysis associates the dynamic target information with the corresponding satellite video frame data and the dynamic target historical data to obtain the dynamic target association analysis result; the dynamic target storage stores the dynamic target information in a structured database and the dynamic target association analysis result in an unstructured database to generate a geographic spatial retrieval index; the dynamic target vector tile cloud publishing integrates the dynamic target information and the dynamic target association analysis result and publishes it as a dynamic target vector tile cloud service.

6. The target interpretation system based on satellite video data according to claim 5, characterized in that: The dynamic target association analysis associates the dynamic target information with the corresponding satellite video frame data and dynamic target historical data for analysis. The specific steps for obtaining the dynamic target association analysis results are: based on the dynamic target ID and the video frame number to which the dynamic target belongs, the dynamic target trajectory information, dynamic target distribution area and dynamic target activity heat are generated; at the same time, based on the upper and lower frame information, the dynamic target heading and dynamic target speed in the dynamic target detection result are re-verified and calculated; if the dynamic target detection result contains the identity information of the dynamic target, the dynamic target historical data is further associated with the identity information to generate the dynamic target trajectory information, dynamic target distribution area and dynamic target activity heat based on the dynamic target historical data.

7. The target interpretation system based on satellite video data according to claim 1, characterized in that: The multi-source data fusion display supports the associated fusion display of satellite video data, satellite video frame data, dynamic target information, dynamic target historical trajectory and judged target.

8. The target interpretation system based on satellite video data according to claim 7, characterized in that: The multi-source data fusion display supports the associated fusion display of satellite video data, satellite video frame data, dynamic target information, dynamic target historical trajectory and interpreted target, specifically including: Satellite video data display: supports loading and displaying satellite video data in the corresponding surveillance area, plays satellite video data streams through WebRTC technology, and provides remote access and sharing to the outside world; Satellite video frame data display: Supports displaying a list of each frame of satellite video data on the browser interface. Click to view detailed WMS layers, which is convenient for frame-by-frame analysis, comparison and interpretation; Dynamic target information display: supports loading and displaying dynamic target lists and dynamic target vector tiles in the browser interface, supports querying the dynamic target information closest to the mouse click position, and querying the dynamic target information in the dynamic target list selected by the mouse, clearly displaying the dynamic target position and attributes; Display of target information: supports loading and displaying the target list and target vector tiles in the browser interface, supports querying the target information closest to the mouse click position, and querying the target information in the target list selected by the mouse, clearly displaying the target position and attributes; Satellite video frame data-dynamic target information associated fusion display: supports the simultaneous display of satellite video frame data WMS layer and the dynamic target information corresponding to the frame data. By clicking the corresponding dynamic target detection result, the video frame data WMS layer corresponding to the dynamic target detection result and the dynamic target detection result attribute information are displayed; Dynamic target historical trajectory display: supports querying the trajectory information of multiple dynamic targets at the same time, and draws the historical movement trajectories of multiple dynamic targets on the browser interface; supports selecting a specific time period to view the movement path of the target in the past period of time, and assists in dynamic target interpretation.

9. The target interpretation system based on satellite video data according to claim 8, characterized in that: The judgment target storage performs data verification on the judgment target to be saved according to the attribute value verification rules defined in the judgment template. After the data verification passes, judgment conflict detection is performed. If it passes, the judgment target is stored in the database. If it fails, the failed attribute value is prompted through the browser interface; the judgment target modification is to modify one or more attribute values ​​in the saved judgment target. According to the attribute value verification rules defined in the judgment template, the judgment target to be saved after the modification is performed. If it passes, the judgment target is stored in the database. If it fails, the failed attribute value is prompted through the browser interface; the judgment target vector tile cloud publishing is to integrate and publish the parsed judgment target as a judgment target vector tile cloud service; the data export and integration support downloads the saved judgment target in JSON format to the user's local folder according to user needs; or constructs the judgment target list into interface content according to the external system interface and sends it to the external system.

10. The target interpretation system based on satellite video data according to claim 9, characterized in that: The conflict detection rule is that the same target or similar target cannot appear in the same video frame. The same target means that the target ID or target serial number is the same, and similar targets mean that the target ID or target serial number is different, but among all the defined attributes, 80% of the attribute values ​​match those of another target.

11. A target interpretation method based on satellite video data, implemented according to a target interpretation system based on satellite video data as claimed in any one of claims 1 to 10, characterized in that: The following steps are involved: S1, according to user needs or external system interface, create a new interpretation template in the interpretation template management module, fill the required interpretation attributes and attribute value verification rules into the interpretation template, enter the interpretation template name and save; S2, loading satellite video data, and the satellite video data intelligent preprocessing module preprocesses the satellite video data; S3, the dynamic target cloud service module analyzes the dynamic target detection results imported from the external system, performs correlation analysis with the satellite video frame data and dynamic target historical data, stores the parsing results and correlation analysis results in the database, and integrates and publishes them into a dynamic target vector tile cloud service; S4, the intelligent visualization interpretation module associates and fuses satellite video data, satellite video frame data, dynamic target information, dynamic target historical trajectory and interpretation target, and automatically fills the dynamic target attribute information into the similar attributes in the interpretation template through word embedding Word2Vec and cosine similarity to complete target interpretation; it also draws the target shape outline on the WMS layer of the satellite video frame data, automatically fills the attribute value into the interpretation template, and adds or modifies the attribute value in the interpretation template to complete target interpretation; S5, the target management module performs data verification and conflict detection on the target information generated by the intelligent visualization judgment module. If it passes, the target is saved in the database and integrated and published as a vector tile cloud service. If it fails, the reason for failure is prompted. S6, according to user needs, the interpretation target management module downloads the saved interpretation targets in JSON format to the user's local folder; or according to the external system interface, the interpretation target management module constructs the interpretation target list into interface content and sends it to the external system.

Images