Target interpretation system and method based on satellite video data

Abstract

The invention belongs to the technical field of intelligent processing and application of multi-modal remote sensing images, and discloses a target interpretation system and method based on satellite video data, and the system comprises an interpretation template management module, a satellite video data intelligent preprocessing module, a dynamic target cloud service module, an intelligent visual interpretation module, and an interpretation target management module. According to the method, the accuracy and reliability of satellite video data target detection are improved, the performance of a target interpretation system based on satellite video data is remarkably improved, and a new direction is provided for the development of related technologies in the future.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent processing and application of multimodal remote sensing images, and in particular relates to a target interpretation system and method based on satellite video data. Background Art

[0002] With the continuous advancement of remote sensing technology, video satellites are increasingly used in earth observation. They can not only provide high-resolution static images, but also capture continuous dynamic changes, providing unprecedented perspectives for environmental monitoring, disaster warning, urban planning and other fields. However, in the process of using satellite video data for target recognition and analysis, there are also some challenges. Although the current target detection model based on deep learning can achieve a high accuracy in remote sensing images, human intuition and experience are still indispensable in special cases. For example, in complex and changing environments or when the targets have high similarity, the deep learning model may produce false positives or false negatives. At this time, manual participation in target interpretation can not only make up for the shortcomings of the algorithm, but also improve the reliability of the final result.

[0003] In order to improve the accuracy and reliability of satellite video data target detection, combining the experience and knowledge of professionals has become a feasible method. By establishing a human-machine collaboration mechanism, the expertise of human experts in specific fields can be used to make up for the shortcomings of the algorithm, thereby enhancing the accuracy and robustness of target recognition. In addition, this approach can also help identify subtle differences and patterns that are easily overlooked by the algorithm, further improving the overall performance of the system. Summary of the invention

[0004] The purpose of the present invention is to provide a target interpretation system and method based on satellite video data. The present invention improves the accuracy and reliability of satellite video data target detection, significantly improves the performance of the target interpretation system based on satellite video data, and provides a new direction for the development of related technologies in the future.

[0005] The technical solution adopted by the present invention is a target interpretation system based on satellite video data, including an interpretation template management module, a satellite video data intelligent preprocessing module, a dynamic target cloud service module, an intelligent visualization interpretation module, and an interpretation target management module, wherein: The interpretation template management module provides full life cycle management for interpretation templates of satellite video data, and supports creation, modification, query and deletion of interpretation templates; The satellite video data intelligent preprocessing module is used to realize intelligent analysis of satellite video metadata, dynamic frame sequence optimization, multi-dimensional space geometry correction, Web map service cloud publishing, and distributed storage and retrieval; The dynamic target cloud service module is used to implement dynamic target detection result analysis, dynamic target association analysis, dynamic target storage, and dynamic target vector tile cloud publishing; The intelligent visualization interpretation module is used to realize multi-source data fusion display and intelligent interactive interpretation; The target interpretation management module is used to realize target interpretation storage, target interpretation modification, target interpretation vector tile cloud publishing, data export and integration.

[0006] Furthermore, the interpretation template management module includes the creation of interpretation templates, modification of interpretation templates, query of interpretation templates, and deletion of interpretation templates. The creation of interpretation templates supports custom interpretation templates. New interpretation templates are created through the browser interface. Definitions of interpretation template descriptions, attributes, attribute values, attribute value types, and attribute value verification rules are completed through forms. Interpretation template creation is also completed by importing Json files or Xml files, and then the defined interpretation templates are saved in the database; interpretation template modification supports modification and saving of already defined interpretation template descriptions, attributes, attribute values, attribute value types, and attribute value verification rules, and supports adding and deleting attribute values; interpretation template deletion supports deleting interpretation templates that are no longer needed to release system resources; interpretation template query supports querying interpretation templates through multiple conditions.

[0007] Furthermore, the satellite video data intelligent preprocessing module includes satellite video metadata intelligent analysis, dynamic frame sequence optimization, multi-dimensional space geometric correction, Web map service cloud publishing, and distributed storage retrieval. The satellite video metadata intelligent analysis completes the auxiliary information analysis of different satellite video data by defining auxiliary information field analysis templates, and uses the word embedding Word2Vec and cosine similarity technology in natural language processing to analyze fields that have not appeared in the template, and dynamically updates the field mapping; the dynamic frame sequence optimization processes each frame in the satellite video data, including noise suppression, contrast enhancement, and super-resolution reconstruction, outputs satellite video frame data, and re-encodes and encapsulates the satellite video frame data into satellite video browsing data; the multi-dimensional space geometric correction converts the dynamic frame sequence into The satellite video frame data output by column optimization is combined with metadata information for geometric correction, and the corrected data is reprojected to the WGS84 coordinate system; the Web map service cloud publishing quickly converts the satellite video frame data after dynamic frame sequence optimization and multi-dimensional space geometric correction into a cloud-optimized TIFF file format, uploads it to distributed storage, and seamlessly publishes it as a WMS layer to support real-time query and visual browsing; distributed storage retrieval uploads satellite video data, satellite video frame data, and satellite video display data to distributed object storage, and associates and stores satellite video data information, satellite video frame data information, satellite video display data information, and fields after intelligent analysis of satellite video metadata in a structured database to establish a Geohash retrieval index.

[0008] Furthermore, the operation steps of the distributed storage and retrieval process use distributed transactions to ensure the consistency and integrity of the data.

[0009] Furthermore, the dynamic target cloud service module includes dynamic target detection result analysis, dynamic target association analysis, dynamic target storage, and dynamic target vector tile cloud publishing. The dynamic target detection result analysis will parse the dynamic target detection result based on the satellite video data to obtain the dynamic target information in the dynamic target detection result, wherein the pixel coordinates, longitude and latitude coordinates, the video frame number, ID, heading, speed, and identity information of the dynamic target in the dynamic target detection result are mainly parsed; the dynamic target association analysis associates the dynamic target information with the corresponding satellite video frame data and the dynamic target historical data to obtain the dynamic target association analysis result; the dynamic target storage stores the dynamic target information in a structured database and the dynamic target association analysis result in an unstructured database to generate a geographic spatial retrieval index; the dynamic target vector tile cloud publishing seamlessly integrates the dynamic target information and the dynamic target association analysis result and publishes them as a dynamic target vector tile cloud service.

[0010] Furthermore, the dynamic target association analysis associates the dynamic target information with the corresponding satellite video frame data and dynamic target historical data. The specific steps for obtaining the dynamic target association analysis results are as follows: based on the dynamic target ID and the video frame number to which the dynamic target belongs, the dynamic target trajectory information, dynamic target distribution area and dynamic target activity heat are generated. At the same time, based on the upper and lower frame information, the dynamic target heading and dynamic target speed in the dynamic target detection result are re-verified and calculated. If the dynamic target detection result contains the identity information of the dynamic target, the dynamic target historical data is further associated with the identity information to generate the dynamic target trajectory information, dynamic target distribution area and dynamic target activity heat based on the dynamic target historical data.

[0011] Furthermore, the intelligent visualization interpretation module includes multi-source data fusion display and intelligent interactive interpretation. The multi-source data fusion display supports the associated fusion display of satellite video data, satellite video frame data, dynamic target information, dynamic target historical trajectory, and interpretation target; the intelligent interactive interpretation supports clicking on dynamic target attribute information, and automatically filling the dynamic target attribute information into similar attributes in the interpretation template through word embedding Word2Vec and cosine similarity technology in natural language processing, and adding or modifying the attribute values ​​in the interpretation template to complete the target interpretation; it supports drawing the target shape outline on the WMS layer corresponding to the satellite video frame data. After the drawing is completed, the interpretation template pops up and automatically fills in the attribute values ​​therein, and adds or modifies the attribute values ​​in the interpretation template to complete the target interpretation.

[0012] Furthermore, the multi-source data fusion display supports the associated fusion display of satellite video data, satellite video frame data, dynamic target information, dynamic target historical trajectory, and interpreted targets, among which: Satellite video data display: supports loading and displaying satellite video data in the corresponding surveillance area, plays satellite video data streams through WebRTC technology, provides high-definition and smooth video display effects, and provides remote access and sharing to the outside world; Satellite video frame data display: Supports displaying a list of each frame of satellite video data on the browser interface. Click to view detailed WMS layers, which is convenient for frame-by-frame analysis, comparison and interpretation; Dynamic target information display: supports loading and displaying dynamic target lists and dynamic target vector tiles in the browser interface, supports querying the dynamic target information closest to the mouse click position, and querying the dynamic target information in the dynamic target list selected by the mouse, clearly displaying the dynamic target position and attributes; Display of target information: supports loading and displaying the target list and target vector tiles in the browser interface, supports querying the target information closest to the mouse click position, and querying the target information in the target list selected by the mouse, clearly displaying the target position and attributes; Satellite video frame data-dynamic target information associated fusion display: supports the simultaneous display of satellite video frame data WMS layer and the dynamic target information corresponding to the frame data. By clicking the corresponding dynamic target detection result, the video frame data WMS layer corresponding to the dynamic target detection result and the dynamic target detection result attribute information are displayed; Dynamic target historical trajectory display: supports querying the trajectory information of multiple dynamic targets at the same time, and draws the historical movement trajectories of multiple dynamic targets on the browser interface; supports selecting a specific time period to view the movement path of the target in the past period of time, and assists in dynamic target interpretation.

[0013] Furthermore, the judgment target management module includes judgment target storage, judgment target modification, judgment target vector tile cloud publishing, data export and integration. The judgment target storage is to perform data verification on the judgment target to be saved according to the attribute value verification rules defined in the judgment template. After the data verification passes, judgment conflict detection is performed. If passed, the judgment target is stored in the database. If not, the attribute value that failed is prompted through the browser interface; the judgment target modification is to modify one or more attribute values ​​in the saved judgment target. According to the attribute value verification rules defined in the judgment template, the judgment target to be saved after the modification is performed. If passed, the judgment target is stored in the database. If not, the attribute value that failed is prompted through the browser interface; the judgment target vector tile cloud publishing is to seamlessly integrate and publish the parsed judgment target as a judgment target vector tile cloud service; data export and integration support, according to user needs, downloading the saved judgment target in Json format to the user's local folder / according to the external system interface, constructing the judgment target list into interface content and sending it to the external system.

[0014] Furthermore, the conflict detection rule is that the same target or similar target cannot appear in the same video frame. The same target means that the target ID / target serial number is consistent, and the similar target means that the target ID / target serial number is different, but the attribute values ​​are more than 80% the same.

[0015] A target interpretation method based on satellite video data is implemented by the above-mentioned target interpretation system based on satellite video data, comprising the following steps: S1, according to user needs / external system interface, create a new interpretation template in the interpretation template management module, fill the required interpretation attributes and attribute value verification rules into the interpretation template, enter the interpretation template name and save; S2, loading satellite video data, and the satellite video data intelligent preprocessing module preprocesses the satellite video data; S3, the dynamic target cloud service module analyzes the dynamic target detection results imported from outside the system, associates and analyzes them with satellite video frame data and dynamic target historical data, stores them in the database, and seamlessly integrates and publishes them as dynamic target vector tile cloud services; S4, the intelligent visualization interpretation module associates and fuses satellite video data, satellite video frame data, dynamic target information, dynamic target historical trajectory and interpretation target, and automatically fills the dynamic target attribute information into the similar attributes in the interpretation template through the word embedding Word2Vec and cosine similarity technology in natural language processing to complete the target interpretation; it also draws the target shape outline on the WMS layer of the satellite video frame data, automatically fills the attribute value into the interpretation template, and adds or modifies the attribute value in the interpretation template to complete the target interpretation; S5, the target management module performs data verification and conflict detection on the target information generated by the intelligent visualization judgment module. If it passes, the target is saved in the database and seamlessly integrated and published as a vector tile cloud service. If it fails, the reason for failure is prompted. S6, according to user needs, the judgment target management module downloads the saved judgment targets in Json format to the user's local folder / according to the external system interface, the judgment target management module constructs the judgment target list into interface content and sends it to the external system.

[0016] Furthermore, in S2, the specific steps of preprocessing the satellite video data by the satellite video data intelligent preprocessing module include: S21, performs intelligent analysis of satellite video metadata; S22, performing dynamic frame sequence optimization; S23, performing multi-dimensional space geometric correction; S24, Web map service cloud publishing; S25, perform distributed storage retrieval.

[0017] Furthermore, in S22, the specific steps of performing dynamic frame sequence optimization include: S221, using a wavelet transform method to suppress noise on each frame of satellite video data; S222, using a histogram equalization method to perform contrast enhancement on the satellite video data after noise suppression; S223, using the deep learning ESRGAN model to perform super-resolution reconstruction on the contrast-enhanced satellite video data, and output satellite video frame data; S224, re-encode and encapsulate the satellite video frame data into satellite video browsing data.

[0018] The beneficial effects of the present invention are: (1) Enhanced recognition accuracy: The intuition and experience of human experts in target recognition are introduced, enabling the system to identify targets more accurately, especially when dealing with blurred images or images with complex backgrounds.

[0019] (2) Improved robustness: Through the human-machine collaboration mechanism, the present invention can effectively handle special situations that are difficult for the algorithm to cope with, such as target detection in severe weather conditions and recognition of atypical target morphologies.

[0020] (3) Reduce false alarm rates: Human experts can quickly identify and filter out false alarms, reduce the proportion of false positives in system output, and improve the quality of the decision-making process.

[0021] (4) Assisted learning and improvement: The data collected during the human-computer interaction process can be used to train and optimize the algorithm model, forming a closed loop of continuous improvement and further improving the intelligence level of the system.

[0022] (5) Strong adaptability: The system is not only applicable to known environments, but can also flexibly adapt to new scenarios and challenges through human-machine collaboration, thus expanding its scope of application.

[0023] (6) Education and training value: This system can also serve as a training platform to help new operators learn and master target interpretation skills, thereby promoting knowledge inheritance and development.

[0024] In summary, the present invention improves the accuracy and reliability of target detection in satellite video data by relying on the knowledge of human experts, significantly improves the performance of the target interpretation system based on satellite video data, and provides a new direction for the development of related technologies in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a composition diagram of the system of the present invention.

[0026] Figure 2 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer and more understandable, the technical solution of the present invention will be further described in detail below with reference to the accompanying drawings.

[0028] Aiming at the problem of low accuracy and reliability of target detection in satellite video data, the present invention proposes a target interpretation system based on satellite video data. The system relies on a high-performance computing platform, adopts WebGIS and video streaming technology, and is based on the B / S (browser / server) architecture. It realizes the target interpretation capability of large-scale satellite video data, ensures the stable operation and efficient processing capability of the system, optimizes the human-computer collaboration process, and improves the quality and work efficiency of target interpretation.

[0029] A target interpretation system based on satellite video data, such as Figure 1 As shown, it includes a judgment template management module, a satellite video data intelligent preprocessing module, a dynamic target cloud service module, an intelligent visualization judgment module, and a judgment target management module.

[0030] Interpretation template management module The interpretation template management module is a highly integrated interpretation tool that provides full life cycle management for interpretation templates of satellite video data and supports the creation, modification, query and deletion of interpretation templates.

[0031] The interpretation template management module includes creation of interpretation template, modification of interpretation template, query of interpretation template and deletion of interpretation template.

[0032] (1) Create a new interpretation template It supports custom interpretation templates. You can create new interpretation templates through the browser interface, and use the form to define the interpretation template description, attributes, attribute values, attribute value types, and attribute value verification rules. You can also create a new interpretation template by importing Json files or Xml files, and then save the defined interpretation template to the database.

[0033] (2) Modification of interpretation template Supports modification and saving of defined interpretation template descriptions, attributes, attribute values, attribute value types, and attribute value verification rules, and supports adding and deleting attribute values.

[0034] (3) Deletion of interpretation template Supports deleting interpretation templates that are no longer needed to release system resources.

[0035] (4) Interpretation template query Supports querying the interpretation template by multiple conditions (such as interpretation template name, creation time, modification time, and attributes).

[0036] Satellite video data intelligent preprocessing module The satellite video data intelligent preprocessing module is used to realize intelligent analysis of satellite video metadata, dynamic frame sequence optimization, multi-dimensional space geometry correction, Web map service cloud publishing, and distributed storage and retrieval.

[0037] The satellite video data intelligent preprocessing module includes intelligent analysis of satellite video metadata, dynamic frame sequence optimization, multi-dimensional space geometry correction, Web map service cloud publishing, and distributed storage retrieval.

[0038] (1) Intelligent analysis of satellite video metadata By defining auxiliary information field parsing templates, auxiliary information parsing of different satellite video data is completed. At the same time, word embedding Word2Vec and cosine similarity technology in natural language processing are used to parse fields that have not appeared in the template, and the field mapping is dynamically updated. This process is mainly used to parse key field information of satellite video data, such as satellite code, sensor type, resolution, detection start time, detection coverage (latitude and longitude), and the projection coordinate system used.

[0039] (2) Dynamic frame sequence optimization Each frame in the satellite video data is processed, including noise suppression, contrast enhancement, and super-resolution reconstruction, and the satellite video frame data is output, and the satellite video frame data is re-encoded and encapsulated into satellite video browsing data.

[0040] (3) Multidimensional space geometry correction The satellite video frame data output by dynamic frame sequence optimization is combined with metadata information for geometric correction (RPC / GCPs correction), and the corrected data is reprojected to the WGS84 coordinate system.

[0041] (4) Web map service cloud publishing The satellite video frame data after dynamic frame sequence optimization and multi-dimensional space geometry correction is quickly converted into cloud-optimized TIFF file format, uploaded to distributed storage, and seamlessly published as WMS layers to support real-time query and visual browsing.

[0042] (5) Distributed storage and retrieval Upload satellite video data, satellite video frame data, and satellite video display data to distributed object storage, associate and store satellite video data information, satellite video frame data information, satellite video display data information (data path, data size, total number of satellite video frames), and fields after intelligent analysis of satellite video metadata in a structured database (Postgres), and establish a Geohash retrieval index. The operation steps of this process use distributed transactions to ensure data consistency and integrity.

[0043] Dynamic Target Cloud Service Module The dynamic target cloud service module is used to realize dynamic target detection result analysis, dynamic target association analysis, dynamic target storage, and dynamic target vector tile cloud publishing.

[0044] The dynamic target cloud service module includes dynamic target detection result analysis, dynamic target association analysis, dynamic target storage, and dynamic target vector tile cloud publishing.

[0045] (1) Analysis of dynamic target detection results The dynamic target detection results (external source) based on satellite video data are parsed to obtain the dynamic target information in the dynamic target detection results, among which the pixel coordinates, longitude and latitude coordinates, video frame number, ID, heading, speed, identity information, etc. of the dynamic targets in the dynamic target detection results are mainly parsed.

[0046] (2) Dynamic target association analysis The dynamic target information is correlated and analyzed with the corresponding satellite video frame data and dynamic target historical data to provide comprehensive dynamic target information, that is, to obtain dynamic target correlation analysis results.

[0047] The dynamic target association analysis associates the dynamic target information with the corresponding satellite video frame data and dynamic target historical data. The specific steps to obtain the dynamic target association analysis results are as follows: based on the dynamic target ID and the video frame number of the dynamic target, the dynamic target trajectory information, dynamic target distribution area and dynamic target activity heat are generated. At the same time, based on the upper and lower frame information, the dynamic target heading and dynamic target speed in the dynamic target detection result are re-verified and calculated. If the dynamic target detection result contains the identity information of the dynamic target (string number), the dynamic target historical data is further associated with the identity information to generate the dynamic target trajectory information, dynamic target distribution area and dynamic target activity heat based on the dynamic target historical data.

[0048] Dynamic target historical data includes externally imported dynamic target information (such as news about the target, general description of the target, etc.), relevant dynamic target databases of other systems, etc.

[0049] (3) Dynamic target storage The dynamic target information is stored in a structured database, and the dynamic target association analysis results are stored in an unstructured database (Redis) to generate a geospatial retrieval index.

[0050] (4) Dynamic target vector tile cloud publishing The dynamic target information is seamlessly integrated and published as a dynamic target vector tile cloud service (mainly Point layer), and the dynamic target association analysis results are seamlessly integrated and published as a dynamic target vector tile cloud service (mainly Line layer).

[0051] Intelligent visual interpretation module The intelligent visual interpretation module is a highly integrated data display and interpretation tool used to achieve multi-source data fusion display and intelligent interactive interpretation.

[0052] The intelligent visual interpretation module includes multi-source data fusion display and intelligent interactive interpretation.

[0053] (1) Multi-source data fusion display Supports the associated fusion display of satellite video data, satellite video frame data, dynamic target information, dynamic target historical trajectory, and interpreted targets, including: Satellite video data display: supports loading and displaying satellite video data in the corresponding surveillance area, plays satellite video data streams through WebRTC technology, provides high-definition and smooth video display effects, and provides remote access and sharing to the outside world; Satellite video frame data display: Supports displaying a list of each frame of satellite video data on the browser interface. Click to view detailed WMS layers, which is convenient for frame-by-frame analysis, comparison and interpretation; Dynamic target information display: supports loading and displaying dynamic target lists and dynamic target vector tiles in the browser interface, supports querying the dynamic target information closest to the mouse click position, and querying the dynamic target information in the dynamic target list selected by the mouse, clearly displaying the dynamic target position and attributes; Display of target information: supports loading and displaying the target list and target vector tiles in the browser interface, supports querying the target information closest to the mouse click position, and querying the target information in the target list selected by the mouse, clearly displaying the target position and attributes; Satellite video frame data-dynamic target information associated fusion display: supports the simultaneous display of satellite video frame data WMS layer and the dynamic target information corresponding to the frame data. By clicking the corresponding dynamic target detection result, the video frame data WMS layer corresponding to the dynamic target detection result and the dynamic target detection result attribute information are displayed; Dynamic target historical trajectory display: supports querying the trajectory information of multiple dynamic targets at the same time, and draws the historical movement trajectories of multiple dynamic targets on the browser interface. It supports selecting a specific time period to view the movement path of the target in the past period of time, and assists in dynamic target interpretation.

[0054] (2) Intelligent interactive interpretation It supports clicking on dynamic target attribute information, and automatically filling the dynamic target attribute information into similar attributes in the interpretation template through word embedding Word2Vec and cosine similarity technology in natural language processing. The attribute values ​​in the interpretation template can be added or modified to complete target interpretation; it supports drawing the outline of target shapes such as rectangles, circles, and polygons on the WMS layer corresponding to the satellite video frame data. After drawing, the interpretation template pops up and automatically fills in the attribute values ​​such as latitude and longitude, area, length, and azimuth. The attribute values ​​in the interpretation template can be added or modified to complete target interpretation.

[0055] Interpretation target management module The target interpretation management module is used to realize target interpretation storage, target interpretation modification, target interpretation vector tile cloud publishing, data export and integration.

[0056] The target interpretation management module includes target interpretation storage, target interpretation modification, target interpretation vector tile cloud publishing, data export and integration.

[0057] (1) Determine target storage According to the attribute value verification rules defined in the interpretation template, data verification is performed on the interpretation target to be saved. After the data verification passes, interpretation conflict detection is performed. If it passes, the interpretation target is stored in the database. If it fails, the failed attribute value is prompted through the browser interface.

[0058] The main rule for conflict detection is that the same target or similar targets cannot appear in the same video frame. The same target means the target ID / target serial number is the same, and similar targets mean the target ID / target serial number is different but the attribute values ​​are more than 80% the same (threshold, which can be set manually).

[0059] (2) Modification of target interpretation Modify one or more attribute values ​​in the saved interpretation target, and perform data verification on the interpretation target to be saved after the modification according to the attribute value verification rules defined in the interpretation template. If it passes, the interpretation target will be stored in the database. If it fails, the failed attribute value will be prompted through the browser interface.

[0060] (3) Target vector tile cloud publishing The parsed interpretation targets are seamlessly integrated and published as interpretation target vector tile cloud services (mainly Point layers).

[0061] (4) Data export and integration Supports downloading the saved interpretation targets in Json format to the user's local folder according to user needs / constructing the interpretation target list into interface content according to the external system interface and sending it to the external system.

[0062] The target interpretation method based on satellite video data of the present invention is realized by the target interpretation system based on satellite video data of the present invention. Taking the satellite video data of Jilin-1 Gaofen 03C as an example, Figure 2 As shown, the following steps are included: S1. Create a new interpretation template in the interpretation template management module according to user requirements / external system interface, fill the required interpretation attributes and attribute value verification rules into the interpretation template, enter the interpretation template name and save it.

[0063] Create a new interpretation template in the interpretation template management module, enter the interpretation attribute fields (target ID, target serial number, target model, longitude, latitude, confidence, length, width, target type, etc.), enter the constraint information of the corresponding fields (such as number size range, longitude and latitude accuracy range, target type selection range, string length range), enter the interpretation template name and save it.

[0064] S2, loading satellite video data, the satellite video data intelligent preprocessing module preprocesses the satellite video data, and the specific steps include: S21, performs intelligent analysis of satellite video metadata; The satellite video metadata Xml file format of Jilin-1 Gaofen 03C satellite is intelligently parsed to obtain key field information such as satellite code, sensor type, resolution, detection start time, frame coverage (latitude and longitude), projection coordinate system, and frame information including the total number of video frames and the start and end time of each frame.

[0065] S22, performing dynamic frame sequence optimization; The satellite video data MP4 format of Jilin-1 Gaofen 03C satellite is split into each frame for processing to improve the image details and clarity of the satellite video data. The specific steps include: S221, using a wavelet transform method to suppress noise on each frame of satellite video data; The wavelet transform method is used to suppress noise in each frame of PNG image.

[0066] S222, using a histogram equalization method to perform contrast enhancement on the satellite video data after noise suppression; The histogram equalization method is used to enhance the contrast of the PNG image after noise suppression.

[0067] S223, using the deep learning ESRGAN model to perform super-resolution reconstruction on the contrast-enhanced satellite video data, and output satellite video frame data; The deep learning ESRGAN model is used to perform super-resolution reconstruction on the contrast-enhanced PNG image and output the browsed PNG image.

[0068] S224, re-encoding and packaging the satellite video frame data into satellite video browsing data; Re-encode and package the browsed png images into satellite video browse data.

[0069] S23, performing multi-dimensional space geometric correction; The browsed png image is combined with metadata information (latitude and longitude coordinates of the four corner points of upper left, lower left, upper right and lower right, projection coordinate system, pixel coordinates of the four corner points of upper left, lower left, upper right and lower right) for geometric correction (RPC / GCPs correction), and the corrected data is reprojected to the WGS84 coordinate system.

[0070] S24, Web map service cloud publishing; The browsed PNG images that have undergone dynamic frame sequence optimization and multi-dimensional spatial geometry correction are quickly converted into cloud-optimized TIFF file formats, uploaded to distributed storage, and seamlessly published as WMS layers to support real-time query and visual browsing.

[0071] S25, performing distributed storage retrieval; Upload satellite video data (png images), satellite video frame data (browsing png images), and satellite video browsing data to distributed object storage, and store satellite video data information, satellite video frame data information, satellite video browsing data information (data path, data size, total number of satellite video frames), and fields after metadata intelligent parsing in a structured database (Postgres) to establish a Geohash retrieval index. The operation steps of this process use distributed transactions to ensure data consistency and integrity.

[0072] S3, the dynamic target cloud service module parses the dynamic target detection results imported from outside the system, associates and analyzes them with satellite video frame data and dynamic target historical data, stores them in the database, and seamlessly integrates and publishes them as dynamic target vector tile cloud services.

[0073] Specifically, the dynamic target cloud service module loads the dynamic target detection results (external source) and parses them to obtain the dynamic target information in the dynamic target detection results; associates and analyzes the dynamic target information with the corresponding satellite video frame data and dynamic target historical data to obtain the dynamic target association analysis results; stores the dynamic target information in a structured database (Postgres), stores the dynamic target association analysis results in an unstructured database (Redis), and generates a geographic spatial retrieval index; seamlessly integrates and publishes the dynamic target information as a dynamic target vector tile cloud service (mainly Point layer), and seamlessly integrates and publishes the dynamic target association analysis results as a dynamic target vector tile cloud service (mainly Line layer).

[0074] S4, the intelligent visualization interpretation module associates and fuses satellite video data, satellite video frame data, dynamic target information, dynamic target historical trajectory and interpretation target for display, and automatically fills dynamic target attribute information into similar attributes in the interpretation template through word embedding Word2Vec and cosine similarity technology in natural language processing to complete target interpretation; it also draws the target shape outline on the WMS layer of satellite video frame data, automatically fills the attribute value into the interpretation template, and adds or modifies the attribute value in the interpretation template to complete target interpretation.

[0075] Specifically, the intelligent visualization interpretation module loads and displays the satellite video data in the corresponding reconnaissance area, and plays the satellite video data stream through WebRTC technology; displays the satellite video data list of each frame on the browser interface, and clicks to view the detailed WMS layer, which is convenient for frame-by-frame analysis, comparison and interpretation; loads and displays the dynamic target list and dynamic target vector tiles on the browser interface, queries the dynamic target information closest to the mouse click position, and queries the dynamic target information in the dynamic target list selected by the mouse, and clearly displays the dynamic target position and attributes; loads and displays the judgment target list and judgment target vector tiles on the browser interface, queries the judgment target information closest to the mouse click position, and queries the judgment target information in the judgment target list selected by the mouse, and clearly displays the judgment target position and attributes; displays the satellite video frame data WMS layer and the dynamic target information corresponding to the frame data at the same time, and clicks on the corresponding dynamic target to detect The detection result is displayed, and the WMS layer of the video frame data corresponding to the dynamic target detection result and the attribute information of the dynamic target detection result are displayed; the historical movement trajectories of multiple dynamic targets are drawn on the browser interface, and a specific time period is selected to view the movement path of the dynamic target in the past period of time, so as to assist in the interpretation of dynamic targets; by clicking on the dynamic target attribute information, the dynamic target attribute information is automatically filled into the similar attributes in the interpretation template through the word embedding Word2Vec and cosine similarity technology in natural language processing, and the attribute values ​​in the interpretation template can be added or modified to complete the target interpretation; if the interpreted target does not appear in the dynamic target list, the outline of the target shape such as rectangle, circle, polygon, etc. can be drawn on the WMS layer corresponding to the video frame data. After the drawing is completed, the interpretation template pops up and automatically fills in the attribute values ​​such as latitude and longitude, area, length, azimuth, etc., and the attribute values ​​in the interpretation template can be added or modified to complete the target interpretation.

[0076] S5, the target management module performs data verification and conflict detection on the target information generated by the intelligent visualization judgment module. If it passes, the target is saved in the database and seamlessly integrated and published as a vector tile cloud service. If it fails, the reason for failure is prompted.

[0077] S6, according to user needs, the judgment target management module downloads the saved judgment targets in Json format to the user's local folder / according to the external system interface, the judgment target management module constructs the judgment target list into interface content and sends it to the external system.

[0078] The contents not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. A target interpretation system based on satellite video data, characterized in that: It includes interpretation template management module, satellite video data intelligent preprocessing module, dynamic target cloud service module, intelligent visualization interpretation module and interpretation target management module, among which: The interpretation template management module provides full life cycle management for interpretation templates of satellite video data, and supports creation, modification, query and deletion of interpretation templates; The satellite video data intelligent preprocessing module is used to realize intelligent analysis of satellite video metadata, dynamic frame sequence optimization, multi-dimensional space geometry correction, Web map service cloud publishing, and distributed storage and retrieval; The dynamic target cloud service module is used to implement dynamic target detection result analysis, dynamic target association analysis, dynamic target storage, and dynamic target vector tile cloud publishing; The intelligent visualization interpretation module is used to realize multi-source data fusion display and intelligent interactive interpretation; The target interpretation management module is used to realize target interpretation storage, target interpretation modification, target interpretation vector tile cloud publishing, data export and integration.

2. The target interpretation system based on satellite video data according to claim 1, characterized in that: The interpretation template management module includes creation of interpretation template, modification of interpretation template, query of interpretation template and deletion of interpretation template. Creation of interpretation template supports custom interpretation template. New interpretation template is created through the browser interface. Definition of interpretation template description, attributes, attribute values, attribute value types and attribute value verification rules are completed through the form. Creation of interpretation template is also completed by importing Json file or Xml file. Then the defined interpretation template is saved in the database. Interpretation template modification supports modification and saving of defined interpretation template description, attributes, attribute values, attribute value types and attribute value verification rules. It also supports adding and deleting attribute values. Deletion of interpretation templates supports deletion of interpretation templates that are no longer needed to release system resources; Interpretation template query supports querying interpretation templates through multiple conditions.

3. The target interpretation system based on satellite video data according to claim 1, characterized in that: The satellite video data intelligent preprocessing module includes satellite video metadata intelligent analysis, dynamic frame sequence optimization, multi-dimensional space geometry correction, Web map service cloud publishing, and distributed storage retrieval. The satellite video metadata intelligent analysis completes the auxiliary information analysis of different satellite video data by defining auxiliary information field analysis templates. At the same time, it uses the word embedding Word2Vec and cosine similarity technology in natural language processing to analyze fields that have not appeared in the template and dynamically update the field mapping; Dynamic frame sequence optimization processes each frame in the satellite video data, including noise suppression, contrast enhancement, and super-resolution reconstruction, outputs satellite video frame data, and re-encodes and encapsulates the satellite video frame data into satellite video browsing data; Multi-dimensional spatial geometric correction combines the satellite video frame data output by dynamic frame sequence optimization with metadata information for geometric correction, and reprojects the corrected data into the WGS84 coordinate system; Web map service cloud publishing quickly converts satellite video frame data after dynamic frame sequence optimization and multi-dimensional spatial geometry correction into cloud-optimized TIFF file format, uploads it to distributed storage, and seamlessly publishes it as a WMS layer to support real-time query and visual browsing; Distributed storage retrieval uploads satellite video data, satellite video frame data, and satellite video display data to distributed object storage, associates and stores satellite video data information, satellite video frame data information, satellite video display data information, and fields after intelligent analysis of satellite video metadata in a structured database, and establishes a Geohash retrieval index.

4. The target interpretation system based on satellite video data according to claim 3, characterized in that: The operation steps of the distributed storage and retrieval process use distributed transactions to ensure data consistency and integrity.

5. The target interpretation system based on satellite video data according to claim 1, characterized in that: The dynamic target cloud service module includes dynamic target detection result analysis, dynamic target association analysis, dynamic target storage, and dynamic target vector tile cloud publishing. The dynamic target detection result analysis will parse the dynamic target detection result based on the satellite video data to obtain the dynamic target information in the dynamic target detection result, among which the pixel coordinates, longitude and latitude coordinates, the video frame number, ID, heading, speed, and identity information of the dynamic target in the dynamic target detection result are mainly parsed; the dynamic target association analysis associates the dynamic target information with the corresponding satellite video frame data and the dynamic target historical data to obtain the dynamic target association analysis results; the dynamic target storage stores the dynamic target information in a structured database and the dynamic target association analysis results in an unstructured database to generate a geographic spatial retrieval index; the dynamic target vector tile cloud publishing seamlessly integrates the dynamic target information and the dynamic target association analysis results and publishes them as a dynamic target vector tile cloud service.

6. The target interpretation system based on satellite video data according to claim 5, characterized in that: The dynamic target association analysis associates the dynamic target information with the corresponding satellite video frame data and dynamic target historical data. The specific steps to obtain the dynamic target association analysis results are as follows: based on the dynamic target ID and the video frame number of the dynamic target, the dynamic target trajectory information, dynamic target distribution area and dynamic target activity heat are generated. At the same time, based on the upper and lower frame information, the dynamic target heading and dynamic target speed in the dynamic target detection result are re-verified and calculated. If the dynamic target detection result contains the identity information of the dynamic target, the dynamic target historical data is further associated with the identity information to generate the dynamic target trajectory information, dynamic target distribution area and dynamic target activity heat based on the dynamic target historical data.

7. The target interpretation system based on satellite video data according to claim 1, characterized in that: The intelligent visualization interpretation module includes multi-source data fusion display and intelligent interactive interpretation. The multi-source data fusion display supports the associated fusion display of satellite video data, satellite video frame data, dynamic target information, dynamic target historical trajectory, and interpretation target. Intelligent interactive interpretation supports clicking on dynamic target attribute information, automatically filling the dynamic target attribute information into similar attributes in the interpretation template through word embedding Word2Vec and cosine similarity technology in natural language processing, and adding or modifying the attribute values ​​in the interpretation template to complete target interpretation; it supports drawing the target shape outline on the WMS layer corresponding to the satellite video frame data. After drawing, the interpretation template pops up and automatically fills in the attribute values, and adds or modifies the attribute values ​​in the interpretation template to complete target interpretation.

8. The target interpretation system based on satellite video data according to claim 7, characterized in that: Multi-source data fusion display supports satellite video data, satellite video frame data, dynamic target information, dynamic target historical trajectory, and interpreted target for associated fusion display, including: Satellite video data display: supports loading and displaying satellite video data in the corresponding surveillance area, plays satellite video data streams through WebRTC technology, provides high-definition and smooth video display effects, and provides remote access and sharing to the outside world; Satellite video frame data display: Supports displaying a list of each frame of satellite video data on the browser interface. Click to view detailed WMS layers, which is convenient for frame-by-frame analysis, comparison and interpretation; Dynamic target information display: supports loading and displaying dynamic target lists and dynamic target vector tiles in the browser interface, supports querying the dynamic target information closest to the mouse click position, and querying the dynamic target information in the dynamic target list selected by the mouse, clearly displaying the dynamic target position and attributes; Display of target information: supports loading and displaying the target list and target vector tiles in the browser interface, supports querying the target information closest to the mouse click position, and querying the target information in the target list selected by the mouse, clearly displaying the target position and attributes; Satellite video frame data-dynamic target information associated fusion display: supports the simultaneous display of satellite video frame data WMS layer and the dynamic target information corresponding to the frame data. By clicking the corresponding dynamic target detection result, the video frame data WMS layer corresponding to the dynamic target detection result and the dynamic target detection result attribute information are displayed; Dynamic target historical trajectory display: supports querying the trajectory information of multiple dynamic targets at the same time, and draws the historical movement trajectories of multiple dynamic targets on the browser interface; supports selecting a specific time period to view the movement path of the target in the past period of time, and assists in dynamic target interpretation.

9. The target interpretation system based on satellite video data according to claim 8, characterized in that: The judgment target management module includes judgment target storage, judgment target modification, judgment target vector tile cloud publishing, data export and integration. The judgment target storage is to perform data verification on the judgment target to be saved according to the attribute value verification rules defined in the judgment template. After the data verification passes, the judgment conflict detection is performed. If it passes, the judgment target is stored in the database. If it fails, the failed attribute value is prompted through the browser interface; the judgment target modification is to modify one or more attribute values ​​in the saved judgment target. According to the attribute value verification rules defined in the judgment template, the judgment target to be saved after the modification is verified. If it passes, the judgment target is stored in the database. If it fails, the failed attribute value is prompted through the browser interface; The target vector tile cloud publishing is to seamlessly integrate and publish the analyzed target into the target vector tile cloud service; Data export and integration support: According to user needs, the saved interpretation targets are downloaded in Json format to the user's local folder / according to the external system interface, the interpretation target list is constructed into interface content and sent to the external system.

10. The target interpretation system based on satellite video data according to claim 9, characterized in that: The conflict detection rule is that the same target or similar targets cannot appear in the same video frame. The same target means that the target ID / target serial number is the same, and similar targets mean that the target ID / target serial number is different, but the attribute values ​​are more than 80% the same.

11. A method for target interpretation based on satellite video data, implemented according to any one of claims 1 to 10, characterized in that: The following steps are involved: S1, according to user needs / external system interface, create a new interpretation template in the interpretation template management module, fill the required interpretation attributes and attribute value verification rules into the interpretation template, enter the interpretation template name and save; S2, loading satellite video data, and the satellite video data intelligent preprocessing module preprocesses the satellite video data; S3, the dynamic target cloud service module analyzes the dynamic target detection results imported from outside the system, associates and analyzes them with satellite video frame data and dynamic target historical data, stores them in the database, and seamlessly integrates and publishes them as dynamic target vector tile cloud services; S4, the intelligent visualization interpretation module associates and fuses satellite video data, satellite video frame data, dynamic target information, dynamic target historical trajectory and interpretation target, and automatically fills the dynamic target attribute information into the similar attributes in the interpretation template through the word embedding Word2Vec and cosine similarity technology in natural language processing to complete the target interpretation; it also draws the target shape outline on the WMS layer of the satellite video frame data, automatically fills the attribute value into the interpretation template, and adds or modifies the attribute value in the interpretation template to complete the target interpretation; S5, the target management module performs data verification and conflict detection on the target information generated by the intelligent visualization judgment module. If it passes, the target is saved in the database and seamlessly integrated and published as a vector tile cloud service. If it fails, the reason for failure is prompted. S6, according to user needs, the judgment target management module downloads the saved judgment targets in Json format to the user's local folder / according to the external system interface, the judgment target management module constructs the judgment target list into interface content and sends it to the external system.

12. The target interpretation method based on satellite video data according to claim 11, characterized in that: In S2, the specific steps of preprocessing the satellite video data by the satellite video data intelligent preprocessing module include: S21, performs intelligent analysis of satellite video metadata; S22, performing dynamic frame sequence optimization; S23, performing multi-dimensional space geometric correction; S24, Web map service cloud publishing; S25, perform distributed storage retrieval.

13. The target interpretation method based on satellite video data according to claim 12, characterized in that: In S22, the specific steps of performing dynamic frame sequence optimization include: S221, using a wavelet transform method to suppress noise on each frame of satellite video data; S222, using a histogram equalization method to perform contrast enhancement on the satellite video data after noise suppression; S223, using the deep learning ESRGAN model to perform super-resolution reconstruction on the contrast-enhanced satellite video data, and output satellite video frame data; S224, re-encode and encapsulate the satellite video frame data into satellite video browsing data.

Images