High-availability data management system and method for satellite constellation

Abstract

The invention provides a high-availability data management system and method for a satellite constellation. The high-availability data management system comprises a data management portal module used for performing integrated display on archived data; the efficient data query and retrieval module is used for performing query and retrieval on the archived data by adopting a distributed retrieval service and taking a to-be-processed retrieval condition configured by a product model as supplement; the data archiving access and extraction module is used for performing data archiving on the data according to the data type, and performing near-line storage extraction or off-line extraction on the archived data after query and retrieval; the data management statistical analysis module is used for performing data statistics on the archived data according to the data type and displaying a data statistics result in a specified form; and the data management monitoring module is used for monitoring the running state of the system and starting an early warning mechanism when an abnormality is monitored. According to the method, the management effect on the high-availability data of the satellite constellation can be remarkably improved, and meanwhile, the user requirements can be better met.

Description

Technical Field

[0001] The present invention relates to the technical field of data management, and more particularly to a high-availability data management system and method for a satellite constellation. Background Art

[0002] Data management for a wheel-shaped satellite constellation is a complex and critical task that requires highly reliable technologies and systems to ensure the smooth operation of the constellation and meet the needs of various users and applications. These management systems usually evolve continuously to adapt to changing technologies and market requirements. However, existing data management systems are difficult to apply to scenarios of managing high-availability data for large-scale satellite constellations. Summary of the Invention

[0003] In view of this, an object of the present invention is to provide a high-availability data management system and method for a satellite constellation, which can significantly improve the management effect of high-availability data for a satellite constellation and better meet user needs at the same time.

[0004] In a first aspect, an embodiment of the present invention provides a high-availability data management system for a satellite constellation, including: a data management portal module, a data efficient query and retrieval module, a data archiving access and extraction module, a data management statistical analysis module, and a data management monitoring module;

[0005] The data management portal module is used for: integrally displaying the archived high-availability data of the satellite constellation;

[0006] The data efficient query and retrieval module is used for: using a distributed retrieval service and supplementing with the to-be-processed retrieval conditions configured by a product model to query and retrieve the archived high-availability data of the satellite constellation;

[0007] The data archiving access and extraction module is used for: archiving the high-availability data of the satellite constellation according to data types, and for performing near-line storage extraction or off-line extraction on the archived high-availability data of the satellite constellation after query and retrieval;

[0008] The data management statistical analysis module is used for: performing data statistics on the archived high-availability data of the satellite constellation according to data types, and displaying the results of the data statistics in a specified form;

[0009] The data management monitoring module is used for: monitoring the system operation status and starting an early warning mechanism when an abnormality is detected.

[0010] In one embodiment, the data management portal module is specifically configured to perform one or more of the following operations on the archived high-availability data of the satellite constellation: data processing statistics display, data summary display, data archiving and / or extraction chart display, imaging mode and / or polarization mode status display, and data center area distribution information data display.

[0011] In one embodiment, the data efficient query and retrieval module is specifically configured to: perform a preliminary query and retrieval on the archived high-availability data of the satellite constellation to obtain preselected high-availability data of the satellite constellation, and perform a query and retrieval on the preselected high-availability data of the satellite constellation based on the retrieval conditions to be processed configured by the product model.

[0012] In one embodiment, the data efficient query and retrieval module is further configured to: export the files of the extracted target high-availability data of the satellite constellation, and determine the data coverage rate based on the extracted target high-availability data of the satellite constellation to generate a data coverage rate list corresponding to the target high-availability data of the satellite constellation.

[0013] In one embodiment, the data archiving access and extraction module is further configured to: after archiving the high-availability data of the satellite constellation, if the near-line storage duration is not reached, continue to archive the new high-availability data of the satellite constellation until the near-line storage duration is reached, and perform blue-ray offline storage on all the archiving results.

[0014] In one embodiment, the data archiving access and extraction module is specifically configured to: after the query and retrieval, verify whether the target high-availability data of the satellite constellation corresponding to the query and retrieval is data prohibited from extraction; in the case where the target high-availability data of the satellite constellation corresponding to the query and retrieval is not data prohibited from extraction, perform near-line storage extraction or offline extraction on the target high-availability data of the satellite constellation.

[0015] In one embodiment, the data archiving access and extraction module is further configured to: share the extracted target high-availability data of the satellite constellation.

[0016] In one embodiment, the data management statistics and analysis module is specifically configured to perform one or more of the following operations on the archived high-availability data of the satellite constellation: archiving statistics, extraction statistics, backup statistics, and coverage rate statistics.

[0017] In one embodiment, the data management monitoring module is specifically configured to: perform real-time monitoring on the operating conditions of the data management portal module, the data efficient query and retrieval module, the data archiving access and extraction module, and the data management statistics and analysis module.

[0018] In a second aspect, the present invention further provides a method for managing high-availability data of a satellite constellation. The method is applied to the high-availability data management system of the satellite constellation provided in the first aspect, and the method includes:

[0019] Collect highly available data of the satellite constellation;

[0020] Archive the highly available data of the satellite constellation according to the data type;

[0021] Adopt a distributed retrieval service, supplemented by the to-be-processed retrieval conditions configured by the product model, to query and retrieve the archived highly available data of the satellite constellation;

[0022] After querying and retrieving, perform near-line storage extraction or off-line extraction on the archived highly available data of the satellite constellation to extract the target highly available data of the satellite constellation to the agreed directory.

[0023] A highly available data management system and method for a satellite constellation provided by the present invention includes: a data management portal module, a data efficient query and retrieval module, a data archiving access and extraction module, a data management statistical analysis module, and a data management monitoring module; wherein, the data management portal module is used for: integrally displaying the archived highly available data of the satellite constellation; the data efficient query and retrieval module is used for: adopting a distributed retrieval service, supplemented by the to-be-processed retrieval conditions configured by the product model, to query and retrieve the archived highly available data of the satellite constellation; the data archiving access and extraction module is used for: archiving the highly available data of the satellite constellation according to the data type, and for performing near-line storage extraction or off-line extraction on the archived highly available data of the satellite constellation after querying and retrieving; the data management statistical analysis module is used for statistically analyzing the archived highly available data of the satellite constellation according to the data type, and displaying the results of the data statistics in a specified form; the data management monitoring module is used for monitoring the system operation status and starting an early warning mechanism when an anomaly is monitored. The above system utilizes the data management portal module, the data efficient query and retrieval module, the data archiving access and extraction module, the data management statistical analysis module, and the data management monitoring module, and can better implement data archiving, data fast retrieval query, data extraction, etc. when facing large-scale highly available data of the satellite constellation, and can also implement functions such as data integrated display, data statistical display, and operation status monitoring, can significantly improve the management effect of the highly available data of the satellite constellation, and can better meet user needs at the same time.

[0024] Other features and advantages of the present invention will be described in the subsequent specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention are realized and obtained by the structures specifically pointed out in the specification, the claims, and the drawings.

[0025] To make the above objectives, features, and advantages of the present invention more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, makes the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0027] Figure 1 It is a schematic structural diagram of a high-availability data management system for a satellite constellation provided by an embodiment of the present invention;

[0028] Figure 2 It is a schematic structural diagram of a data management portal module provided by an embodiment of the present invention;

[0029] Figure 3 It is a logical flowchart of a data management portal module provided by an embodiment of the present invention;

[0030] Figure 4 It is a schematic structural diagram of a data efficient query and retrieval module provided by an embodiment of the present invention;

[0031] Figure 5 It is a query and retrieval flowchart of a data efficient query and retrieval module provided by an embodiment of the present invention;

[0032] Figure 6 It is a query function flowchart of a data efficient query and retrieval module provided by an embodiment of the present invention;

[0033] Figure 7 It is an export function flowchart of a data efficient query and retrieval module provided by an embodiment of the present invention;

[0034] Figure 8 It is a coverage function flowchart of a data efficient query and retrieval module provided by an embodiment of the present invention;

[0035] Figure 9 It is a schematic structural diagram of a data archiving access and extraction module provided by an embodiment of the present invention;

[0036] Figure 10 It is a data archiving flowchart of a data archiving access and extraction module provided by an embodiment of the present invention;

[0037] Figure 11 It is a data extraction flowchart of a data archiving access and extraction module provided by an embodiment of the present invention;

[0038] Figure 12 It is a data sharing flowchart of a data archiving access and extraction module provided by an embodiment of the present invention;

[0039] Figure 13 It is a system configuration flowchart of a data archiving access and extraction module provided by an embodiment of the present invention;

[0040] Figure 14 It is an archiving task flowchart of a data archiving access and extraction module provided by an embodiment of the present invention;

[0041] Figure 15 It is an extraction task flowchart of a data archiving access and extraction module provided by an embodiment of the present invention;

[0042] Figure 16 It is an extraction subtask flowchart of a data archiving access and extraction module provided by an embodiment of the present invention;

[0043] Figure 17 It is an automatic data sharing sending / receiving flowchart of a data archiving access and extraction module provided by an embodiment of the present invention;

[0044] Figure 18 It is a manual data sharing sending / receiving flowchart of a data archiving access and extraction module provided by an embodiment of the present invention;

[0045] Figure 19 It is an extraction management flowchart of a data archiving access and extraction module provided by an embodiment of the present invention;

[0046] Figure 20 It is an extraction audit function flowchart of a data archiving access and extraction module provided by an embodiment of the present invention;

[0047] Figure 21 It is a structural schematic diagram of a data management statistical analysis module provided by an embodiment of the present invention;

[0048] Figure 22 It is a data statistics flowchart of a data management statistical analysis module provided by an embodiment of the present invention;

[0049] Figure 23 It is an archiving statistics function flowchart of a data management statistical analysis module provided by an embodiment of the present invention;

[0050] Figure 24 It is an extraction statistics function flowchart of a data management statistical analysis module provided by an embodiment of the present invention;

[0051] Figure 25 It is a backup statistics function flowchart of a data management statistical analysis module provided by an embodiment of the present invention;

[0052] Figure 26 It is a coverage rate statistics function flowchart of a data management statistical analysis module provided by an embodiment of the present invention;

[0053] Figure 27 A schematic structural diagram of a data management and monitoring module provided by an embodiment of the present invention;

[0054] Figure 28 A data management and monitoring function flowchart of a data management and monitoring module provided by an embodiment of the present invention;

[0055] Figure 29 A data push function flowchart of a data management and monitoring module provided by an embodiment of the present invention;

[0056] Figure 30 A data reception status function flowchart of a data management and monitoring module provided by an embodiment of the present invention;

[0057] Figure 31 A data management and monitoring function flowchart of a data management and monitoring module provided by an embodiment of the present invention. Specific implementation manners

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0059] The deficiencies of satellite data management technology are mainly concentrated in aspects such as data transmission, storage, quality, security, and analysis. Continuous technological innovation and improvement are needed to solve these problems in order to better utilize satellite data for scientific research, commercial applications, and social services. Currently, there are mainly 4 challenges in the data management system: 1. When processing massive data, large-scale data requires huge storage capacity. How to effectively expand storage resources is a challenge. 2. How to quickly search and retrieve large-scale data to meet user needs. 3. Securely sharing data with internal and external partners. When managing a large amount of sensitive data, there is a risk of data leakage, and data security measures need to be strengthened. 4. The data is not easy for users to use. Users cannot easily find the data they need and are unable to perform advanced data analysis and visualization and manage the data.

[0060] Based on this, the embodiments of the present invention provide a highly available data management system and method for a satellite constellation, which can provide archived storage of product data at all levels, on-demand extraction of data, unified shared management of data, efficient query and retrieval of data, and multi-dimensional effective display of data, which can significantly improve the management effect of highly available data of the satellite constellation and better meet user needs at the same time.

[0061] To facilitate the understanding of this embodiment, first, a highly available data management system for a satellite constellation disclosed in the embodiments of the present invention will be introduced in detail. The satellite constellation can be a "wheel-type" formation constellation. Refer to Figure 1 the schematic structural diagram of a highly available data management system for a satellite constellation shown in the figure. The system includes: a data management portal module 1, a data efficient query and retrieval module 2, a data archiving access and extraction module 3, a data management statistical analysis module 4, and a data management monitoring module 5.

[0062] The data management portal module 1 is used for: integrally displaying the highly available data of the satellite constellation after archiving. Specifically, the data management portal module 1 is mainly responsible for the integration and display of data management information of the portal system, including data archiving, data extraction status, news dynamics, data resource statistics, satellite product introduction, etc.

[0063] The data efficient query and retrieval module 2 is used for: adopting a distributed retrieval service, and supplementing with the to-be-processed retrieval conditions configured by the product model, querying and retrieving the highly available data of the satellite constellation after archiving. Specifically, the data efficient query and retrieval module 2 mainly realizes the efficient query and retrieval of data. By prefabricating common data retrieval query modules and supplementing with the data retrieval conditions configured by the product model, a full-scale data query set is formed. Based on the Elasticsearch distributed retrieval service in the background, the data retrieval ability of the system is rapidly improved.

[0064] The data archiving access and extraction module 3 is used for: archiving the highly available data of the satellite constellation according to the data type, and for performing near-line storage extraction or off-line extraction on the highly available data of the satellite constellation after query and retrieval. Specifically, the data archiving access and extraction module 3 mainly realizes functions such as data archiving and extraction. Based on the archived product model, it conducts archiving access of multi-source satellite data products; receives a data extraction instruction and performs near-line storage extraction or off-line service off-line extraction; pushes the extracted highly available data of the target satellite constellation to the shared exchange area to better manage the data.

[0065] The data management statistical analysis module 4 is used for: performing data statistics on the highly available data of the satellite constellation after archiving according to the data type, and displaying the results of the data statistics in a specified form. Specifically, the data management statistical analysis module 4 mainly realizes the statistical analysis functions of various levels and types of data, including multi-dimensional statistics and intelligent analysis of data resource status, data coverage, and multi-level product result data. The analysis results are effectively displayed through various visual charts such as bar charts and line charts, realizing an integrated and one-stop service for image data, result data, and corresponding computing resources.

[0066] The data management monitoring module 5 is used to: monitor the system operation status and start the warning mechanism when an anomaly is detected. Specifically, the data management monitoring module 5 mainly realizes the monitoring of the data management process, including the real-time monitoring of application services such as archiving tasks and extraction tasks, and the operation status of resources such as near-line storage spaces and data calculation nodes, and starting the corresponding warning mechanism for abnormal data, etc.

[0067] The system provided by the embodiments of the present invention, by using the data management portal module, the data efficient query and retrieval module, the data archiving access and extraction module, the data management statistical analysis module, and the data management monitoring module, can better realize data archiving, data fast retrieval and query, data extraction, etc. when facing large-scale satellite constellation highly available data, and can also realize functions such as data integration display, data statistical display, and operation status monitoring, can significantly improve the management effect of the highly available data of the satellite constellation, and can better meet user requirements at the same time.

[0068] For ease of understanding, the embodiments of the present invention provide a specific implementation manner of a highly available data management system for a satellite constellation. The main processes of the system are as follows: 1. Configure the system model according to the data types. The data types include satellite name, imaging mode, satellite type, product level, number of files, and file size. 2. Perform task archiving according to the data types. After dynamically collecting data information according to the product level, satellite type, and imaging mode, archive and backup the data. The prerequisite for data backup is that multiple data centers are required to back up each other. For example, the data of data center A is backed up to data center B. The archiving message list is sent by the production system, and the near-line storage duration is configured by the data management personnel. If the longest duration is not reached, no operation is performed. 3. Share the data according to the data tasks. The prerequisite for data sharing is that multiple data centers are required. For example, the data of data center A is shared to data center B; 4. Query the data according to conditions such as satellite type, product level, and data center. The queried data can be directly added to the preselected order (i.e., the preselected highly available data of the satellite constellation), or the queried data files can be exported to calculate the coverage rate; 5. Extract relevant data according to the exported file name. First, screen the data that meets the requirements, preselect the data, delete the data that does not meet the requirements, and review the data that meets the preselected requirements; the data that passes the review is verified to check whether the data can be extracted. If data extraction is prohibited, the extraction fails, and the result of the extraction failure will be fed back to the application system that initiated the data extraction. If the data extraction is successful, then judge whether it is for near-line storage. When the judgment result is yes, perform near-line storage, and the data will be pushed to each application system and recorded. Otherwise, perform offline storage extraction, extract the data in the blue-ray offline library and extract the offline data to the agreed directory.

[0069] On this basis, the embodiments of the present invention will give a more specific explanation of the data management portal module 1, the data efficient query and retrieval module 2, the data archiving access and extraction module 3, the data management statistical analysis module 4, and the data management monitoring module 5.

[0070] Refer to Figure 2 As shown in the structural schematic diagram of a data management portal module, the data management portal module 1 is mainly responsible for the integration and display of data management information of the portal system, including: one or more of data processing statistics display, data summary display, data archiving and / or extraction chart display, imaging mode and / or polarization mode status display, and data center area distribution information data display. The data management portal module 1 is mainly responsible for the integration and display of data management information, including data archiving, data extraction status, news dynamics, data resource statistics, and satellite product introduction. Further, refer to Figure 3 As shown in the logic flow chart of a data management portal module, it includes processes such as selecting data types, displaying data information and lists, etc.

[0071] The data management portal module 1 provided by the embodiments of the present invention is easy to use, and users can easily find the data they need and can perform advanced data analysis and visualization.

[0072] Refer to Figure 4 As shown in the structural schematic diagram of a data efficient query and retrieval module, it includes the following functions: administrative region query, longitude and latitude query, file query, file export, quick view export, file name export, calculation of coverage rate, addition of extraction tasks, text import, etc.

[0073] In one example, preliminary query and retrieval can be performed on the archived satellite constellation highly available data to obtain preselected satellite constellation highly available data, and the preselected satellite constellation highly available data can be audited. In the case where the preselected satellite constellation highly available data passes the audit, query and retrieval are performed on the preselected satellite constellation highly available data based on the to-be-processed retrieval conditions configured by the product model to obtain the target satellite constellation highly available data. In another example, file export can also be performed on the extracted target satellite constellation highly available data, and the data coverage rate is determined based on the extracted target satellite constellation highly available data to generate a data coverage rate list corresponding to the target satellite constellation highly available data.

[0074] In a specific implementation manner, refer to Figure 5 As shown in the query and retrieval flow chart of a data efficient query and retrieval module, it includes: using the administrative region, longitude and latitude, shp file, and text file as conditions, and when there is corresponding data, performing data selection and adding it to the preselected order, and at the same time performing operations such as file export and calculation of coverage rate, and finally performing operations such as file export and display of the coverage rate data list.

[0075] In a specific embodiment, refer to Figure 6 the query function flowchart of a data-efficient query and retrieval module shown in the figure, including: using administrative regions, longitude and latitude, shp files, and text files as conditions to filter data, so as to determine the query results, and finally update the data information and display it.

[0076] In a specific embodiment, refer to Figure 7 the export function flowchart of a data-efficient query and retrieval module shown in the figure, including: selecting data from the query results and adding them to a preselected order, and at the same time performing operations such as file export and coverage calculation, and finally performing operations such as file export and display of the coverage data list.

[0077] In a specific embodiment, refer to Figure 8 the coverage function flowchart of a data-efficient query and retrieval module shown in the figure, including: checking the query result data and clicking on coverage calculation, display of list data and map data, etc.

[0078] In summary, the data-efficient query and retrieval module 2 is based on satellite imagery data, queries relevant imagery data information according to administrative regions, longitude and latitude, shp files, and text, and views the data details; calculates the coverage rate of the imagery data, adds it to the preselected order, exports the quick view, exports the file name, exports the shp file, etc. For example, when querying the imagery data taken by satellites in the Chinese region, through the query and retrieval module of the data, select the query and retrieval range of Asia / China administrative region, the product level is single-look slant-range complex imagery data (L1A), the satellite number is HT1-A, the satellite imaging mode is multi-satellite strip interference mode, the shooting time is from May to June 2023, etc. as conditions for the data, and query the data results; calculate the data coverage rate of the imagery data taken by the HT1-A satellite in the Chinese region, and select the covered or uncovered data files for export. The data-efficient query and retrieval module provided by the embodiments of the present invention can achieve rapid search and retrieval of large-scale data to meet the needs of users.

[0079] Determining which data needs to be archived and for how long is an important decision. Different types of data may require different archiving strategies, which need to match business requirements and regulatory requirements. Once the data is archived, how to retrieve it effectively is a challenge. The data archiving module needs to provide powerful retrieval functions so that users can easily find and extract the required data. Archived data is usually stored in a specific format, but users may need different data formats for analysis or reporting. The data extraction module needs to provide data format conversion functions to meet the needs of users. Archived data needs to be managed and maintained regularly, including data migration, backup, and retention period management. This requires an effective data management process. As the amount of archived data increases, the performance of the data extraction module may become a problem. The data extraction process needs to be optimized to ensure that it can be completed within a reasonable time. Archived data usually requires detailed metadata and documentation to help users understand the meaning and use of the data. Managing this data is a challenge. Based on this, refer to Figure 9 the schematic structural diagram of a data archiving access and extraction module shown in

[0080] In one example, after archiving the high-availability data of the satellite constellation, if the near-line storage duration has not been reached, continue to archive the new high-availability data of the satellite constellation until the near-line storage duration is reached, and perform blue-ray offline storage on all the archiving results. In another example, after querying and retrieving, verify whether the target high-availability data of the satellite constellation corresponding to the query and retrieval is data prohibited from extraction; in the case where the target high-availability data of the satellite constellation corresponding to the query and retrieval is not data prohibited from extraction, perform near-line storage extraction or offline extraction on the target high-availability data of the satellite constellation. In another example, the data archiving access and extraction module can also be used to: share the extracted target high-availability data of the satellite constellation.

[0081] In a specific implementation manner, refer to Figure 10The data archiving flow chart of a data archiving access and extraction module shown includes: receiving an "archiving message list"; message parsing and storage; archiving tasks and products can only be viewed in the task management-archiving task list; verification and metadata collection are performed according to the rules of the model management dynamic modeling configuration, wherein the verification content includes the number of files, file size, and whether the metadata is complete; if the rule verification fails, the archiving failure information is fed back to each application system; if the rule verification passes, metadata collection and file collection are performed, the data cataloging information is synchronized to the special service production system, the browsing map is extracted, the data is compressed and stored according to the near-line rules; the archiving results are fed back to each application system; it is determined whether the archived product is original data, if it is original data, it is backed up to other data centers, if it is image data, it is determined whether the maximum near-line storage time has been reached, and Blu-ray offline storage is performed when the maximum near-line storage time has been reached.

[0082] In a specific embodiment, see Figure 11 The data extraction flow chart of a data archiving access and extraction module shown includes: receiving a "data extraction request" and performing message parsing and storage, and selecting data that meets the requirements through a data efficient query and retrieval module 2 and retaining pre-selected data that meets the requirements, and conducting leadership review of the pre-selected data; the extraction task machine product can be viewed in the extraction task management list; verifying whether the data is prohibited from extraction, if so, the extraction failure result is fed back to the application system that initiated the extraction request, if not, checking whether the data is near-line stored, and the data carries a mark of near-line storage or offline storage; if it carries a mark of offline storage, extracting the data from the Blu-ray offline library, and extracting the offline data to the agreed directory; if it carries a mark of near-line storage, pushing the data to each application system.

[0083] In a specific embodiment, see Figure 12 The data sharing flowchart of a data archiving access and extraction module shown includes: receiving a "data sharing request"; message parsing and storage; shared tasks and products can be viewed in the task management-shared task list; verification and metadata collection are performed according to the rules of the model management dynamic modeling configuration; if the rule verification fails, the sharing failure information is fed back to each application system; if the rule verification succeeds, metadata collection is performed; if it is data sharing, the shared data of this type is published in the form of a service through the service structure, and if it is file sharing, the application system that subscribes to this type of product is notified.

[0084] In a specific embodiment, see Figure 13 The system configuration flow chart of a data archiving access and extraction module shown includes: selecting a data type; adding, modifying, and querying data; and displaying data list information.

[0085] In a specific embodiment, refer to Figure 14 The archiving task flow chart of a data archiving access and extraction module shown in the figure, including: archiving task data types; automatically and manually sending data; displaying data list information.

[0086] In a specific embodiment, refer to Figure 15 The extraction task flow chart of a data archiving access and extraction module shown in the figure, including: extraction task data types; displaying data information list.

[0087] In a specific embodiment, refer to Figure 16 The extraction subtask flow chart of a data archiving access and extraction module shown in the figure, including: extraction subtask data types; displaying extraction subtask data.

[0088] In a specific embodiment, refer to Figure 17 The automatic data sharing sending / receiving flow chart of a data archiving access and extraction module shown in the figure, including: selecting data for automatic sending / receiving; successful sending / receiving.

[0089] In a specific embodiment, refer to Figure 18 The manual data sharing sending / receiving flow chart of a data archiving access and extraction module shown in the figure, including: selecting data for manual sending / receiving; successful sending / receiving.

[0090] In a specific embodiment, refer to Figure 19 The extraction management flow chart of a data archiving access and extraction module shown in the figure, including: pre-extracting data; displaying data list information.

[0091] In a specific embodiment, refer to Figure 20 The extraction audit function flow chart of a data archiving access and extraction module shown in the figure, including: auditing relevant data; displaying data list information.

[0092] In summary, the data archiving access and extraction module 3 mainly includes: when a data task occurs, data archiving is performed according to the data type; data extraction is performed according to the file name. If the data is not in this center, subtask data extraction needs to be performed on the data of the sub-center. Data archiving automatically archives or manually archives the image products of the upstream production system. Automatic archiving does not require any operation. For manual archiving, according to the archiving task name (such as L1A, L3B, etc.), archiving model (such as GD_L6A), priority level three, and data center, the archiving product path is finally selected to archive the data. The original data is backed up to other data centers, and when the maximum offline storage duration is reached, it is stored offline on Blu-ray. For data extraction, for example, when querying HT1-A satellite image data, first query the data, then add the data to the preselected order, select to delete or generate an order in the preselected data management. The generated order is reviewed by the leader, and the data that passes the review is extracted. The background performs Blu-ray offline storage when the maximum offline storage duration is reached.

[0093] To make the analysis results understandable and reproducible, it is necessary to effectively manage data documents and metadata. This can help other users understand the background and methods of the analysis. Users may need training and support to correctly use the analysis module. It is necessary to provide training materials and support channels. The analysis results usually need to be presented to business decision-makers in the form of visual charts or reports. Creating impactful visualizations and reports requires professional skills. See Figure 21 As shown in the schematic structural diagram of a data management statistical analysis module, it includes one or more of the following functions: archiving statistics, extraction statistics, backup statistics, coverage rate statistics, and records data changes. Specific data is found through conditional filtering, and data details information is viewed, etc. The data management statistical analysis module mainly realizes the statistical analysis functions of various levels and types of data, including multi-dimensional statistics and intelligent analysis of data resource conditions, data coverage conditions, and multi-level product result data. The analysis results are effectively displayed through various visual charts such as bar charts and line charts, realizing an integrated and one-stop service for image data, result data, and corresponding computing resources.

[0094] In a specific implementation manner, see Figure 22 As shown in the data statistical flowchart of a data management statistical analysis module, it includes: archiving statistics / extraction statistics / backup statistics / coverage rate statistics; displaying data statistical lists / charts.

[0095] In a specific implementation manner, see Figure 23 As shown in the archiving statistics function flowchart of a data management statistical analysis module, it includes: archiving statistical data; displaying data lists.

[0096] In a specific implementation manner, see Figure 24The extraction and statistics function flowchart of a data management statistics analysis module shown in the figure includes: extracting and statistics data; displaying the data list.

[0097] In a specific implementation manner, refer to Figure 25 The backup and statistics function flowchart of a data management statistics analysis module shown in the figure includes: backing up and statistics data; displaying the data list.

[0098] In a specific implementation manner, refer to Figure 26 The coverage rate statistics function flowchart of a data management statistics analysis module shown in the figure includes: coverage rate data; displaying the data list.

[0099] In summary, the data management statistics analysis module 4 mainly conducts data statistical management on the functions of archived data, extracted data, backup data, and coverage rate data. Different types of data are displayed in lists and charts, and the data change trends can be directly seen, which is conducive to better statistical management of different types of data.

[0100] If the data of an organization comes from multiple different data sources, these sources may have different data formats, structures, and integration challenges. The monitoring module needs to be able to effectively handle this diversity. For large-scale data management systems, real-time monitoring may become complex. Processing real-time events and metrics for large amounts of data may require a highly optimized system. The data monitoring module itself needs to rely on accurate and reliable data for monitoring. If there are quality problems with the source data, the monitoring module may not be able to provide accurate information. To ensure effective monitoring, the monitoring module needs to be correctly configured. Complex configuration and management may lead to configuration errors or omissions, thus affecting the effectiveness of monitoring. The monitoring module itself will generate a certain performance overhead, which may lead to resource contention or performance degradation. It is necessary to weigh the monitoring overhead and the value of the information obtained. Setting appropriate alarm and notification thresholds is crucial. Setting them too high may lead to missed alarms, and setting them too low may lead to false alarms. These parameters need to be carefully adjusted and managed. Storing and processing monitoring data may involve security and privacy issues. It is necessary to ensure that the monitoring data itself is not accessed or leaked by unauthorized persons. Based on this, refer to Figure 27 The structural schematic diagram of a data management monitoring module shown in the figure includes the following functions: real-time monitoring of application services such as archived summary, backup summary, data backup, and data sharing, and the operating status of resources such as near-line storage space and data calculation nodes, starting corresponding warning mechanisms for abnormal data, and monitoring the resource usage of hardware devices, such as monitoring the usage status of hosts, including host disk monitoring, CPU monitoring, memory monitoring, etc. This module is used to conduct real-time monitoring of the operating status of the data management portal module, data efficient query and retrieval module, data archiving access and extraction module, and data management statistics analysis module.

[0101] In a specific embodiment, refer to Figure 28 The data management and monitoring function flowchart of a data management and monitoring module shown in the figure, including: selecting product data; monitoring data push status; monitoring data reception status; recording data logs.

[0102] In a specific embodiment, refer to Figure 29 The data push function flowchart of a data management and monitoring module shown in the figure, including: selecting data; monitoring data push.

[0103] In a specific embodiment, refer to Figure 30 The data reception status function flowchart of a data management and monitoring module shown in the figure, including: monitoring data reception status; displaying the archived summary / data backup / backup summary / data sharing data list.

[0104] In a specific embodiment, refer to Figure 31 The data management and monitoring function flowchart of a data management and monitoring module shown in the figure, including: the monitoring center displays the chart data of base, search, interface, and taskdeal services.

[0105] In summary, the data management and monitoring module 5 mainly includes reading different product level data pushed through the data push interface, archiving summary, backup summary, data backup, data sharing, etc., to view different product level data in different types; re-pushing, stopping, and adjusting the priority function to operate the data. The data push rule list displays the data ID, product name, task number, priority, push status, push rule number, affiliated data center, target data center, application system code, whether to push catalog metadata, whether to push entity products, whether the target product is decompressed, submission time, submission start time, submission completion time, and submission time spent. The main achievements are: data push status data management, viewing information such as the push data status. Through the monitoring center, the hardware usage can be monitored, and the status display of the normal operation, abnormal operation, or unknown operation of services such as the data management analysis front-end service, base service, search service, interface service, and taskdeal service can be monitored; the total amount, normal, and abnormal data status of the host monitoring; the alarm status display, and the information chart display of the cpu, memory, and disk usage rate distribution of the host monitoring, etc.

[0106] In addition, transmitting a large amount of data may put pressure on network bandwidth and speed. Embodiments of the present invention also establish and configure 10 Gigabit Ethernet; establish network partitions such as service networks and data storage networks according to the service usage network. Configure the computer network to ensure network availability; configure the connection between the network and the server. Each server has 4 10 Gigabit interfaces, and two of the 10 Gigabit interfaces are configured in bond4 mode and connected to two front-end service access switches respectively. The switch ports are configured in dynamic link aggregation mode. In this way, the network bandwidth is increased to 20 Gbps, and network interruption caused by a single-link failure can be effectively prevented. Similarly, the other two 10 Gigabit interfaces are configured in bond4 mode and connected to the storage switches respectively.

[0107] In addition, how to securely share data with internal and external partners. The need to transmit data to remote locations. Ethical and moral issues need to be considered when dealing with large-scale data, such as data abuse and privacy issues. The system provided by the embodiments of the present invention distinguishes system roles from a business perspective and can be divided into: system administrators, data administrators, and ordinary users. System administrators are responsible for system configuration, system maintenance, and system management; data administrators are responsible for monitoring data reception, archiving, and processing, and analyzing the causes and manually processing abnormal situations; ordinary users can browse and view query retrieval, data sharing, task management, etc.

[0108] The data product storage management and application service center of the ground system of the "wheel-type" formation satellite constellation in the embodiments of the present invention provides data archiving management and services for the satellites of the "wheel-type" formation constellation. Its main tasks are to provide archived storage of product data at all levels, on-demand extraction of data, unified sharing management of data, efficient query retrieval of data, and multi-dimensional effective display of data. At the same time, by using various types of database file systems and distributed cluster services, sub-library storage and efficient access of image-related metadata information entities are realized to meet the requirements of massive image storage management and applications. Further, it mainly performs integrated archived management of metadata captured by satellites, single-look slant-range complex images, single-look slant-range complex image interferometric pairs, system geometric correction images, geocoded terrain correction images, geometric fine correction, orthorectified images, digital surface models, digital elevation models, surface deformation monitoring products, land cover change monitoring products, soil water products, water body extraction products, ship monitoring product data, etc.

[0109] The highly available data management system of the satellite constellation provided by the embodiments of the present invention has at least the following characteristics:

[0110] 1) In terms of storage, the data archiving access and data extraction module mainly realizes functions such as data archiving and extraction. Based on the archived product model, it conducts the archiving access of sar satellite data products; upon receiving a data extraction instruction, it performs near-line storage extraction or offline extraction of data for offline services; it pushes the target data to the shared exchange area to better manage the data and expand the storage resources.

[0111] 2) In terms of query, the data efficient query and retrieval module mainly realizes the efficient query and retrieval of data. By prefabricating common data retrieval query modules and supplementing them with the data retrieval conditions configured by the product model, a full-volume data query set is formed. The background is based on the Elasticsearch distributed retrieval service to quickly improve the system's data retrieval ability.

[0112] 3) In terms of transmission security, according to the "wheel formation" satellite data transmission rule, data is transmitted from the data source to the data terminal through one or more data links. Its main function is to realize the information transmission and exchange between points. This data transmission method is mainly applied to the data transmission between big data, multiple files, and cross-services, which can improve the real-time performance and reliability of data transmission.

[0113] 4) In terms of user experience, users can easily find the data they need, conduct data analysis and visualization, monitor the usage resources of hardware devices and business data, such as the usage status of the monitoring host for the hardware monitoring host disk, CPU, memory, etc., and the "wheel formation" series satellite data push and received data status for business data monitoring. If the data expires and changes from online to offline status, the system will automatically delete the data to realize the management of the data.

[0114] Based on the foregoing embodiments, the embodiment of the present invention provides a high-availability data management method for a satellite constellation. This method is applied to the high-availability data management system of the foregoing satellite constellation, and this method includes:

[0115] Collect high-availability data of the satellite constellation;

[0116] Archive the high-availability data of the satellite constellation according to the data type;

[0117] Adopt a distributed retrieval service and supplement it with the to-be-processed retrieval conditions configured by the product model to query and retrieve the archived high-availability data of the satellite constellation;

[0118] After querying and retrieving, perform near-line storage extraction or offline extraction on the archived high-availability data of the satellite constellation to extract the target high-availability data of the satellite constellation to the agreed directory.

[0119] The method provided by the embodiment of the present invention can better achieve data archiving, rapid data retrieval and query, data extraction, etc. when facing a large amount of highly available data of a satellite constellation by using a data management portal module, a data efficient query and retrieval module, a data archiving access and extraction module, a data management statistical analysis module, and a data management monitoring module. It can also implement functions such as data integrated display, data statistical display, and operation status monitoring, which can significantly improve the management effect of the highly available data of the satellite constellation and better meet user requirements at the same time.

[0120] The implementation principle and the technical effects generated by the method provided by the embodiment of the present invention are the same as those of the foregoing apparatus embodiment. For the sake of brief description, for the parts not mentioned in the method embodiment, reference may be made to the corresponding content in the foregoing apparatus embodiment.

[0121] In the description of the embodiment of the present invention, unless otherwise clearly specified and limited, the terms "installation", "connection", and "connection" shall be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be a direct connection or an indirect connection through an intermediate medium, and it may be the communication inside two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0122] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0123] Finally, it should be noted that the above-mentioned embodiments are only specific embodiments of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or equivalently replace some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A highly available data management system for a satellite constellation, characterized in that, Including: A data management portal module, a data efficient query and retrieval module, a data archiving access and extraction module, a data management statistical analysis module, and a data management monitoring module; among which, The data management portal module is used for: integrally displaying the archived satellite constellation highly available data; The data efficient query and retrieval module is used for: using a distributed retrieval service, and supplementing with the to-be-processed retrieval conditions configured by a product model, querying and retrieving the archived satellite constellation highly available data; The data archiving access and extraction module is used for: archiving the satellite constellation highly available data according to data types, and for near-line storage extraction or off-line extraction of the archived satellite constellation highly available data after querying and retrieving; The data management statistical analysis module is used for: performing data statistics on the archived satellite constellation highly available data according to the data types, and displaying the results of the data statistics in a specified form; The data management monitoring module is used for: monitoring the system operation status, and starting an early warning mechanism when an anomaly is monitored.

2. The highly available data management system of the satellite constellation according to claim 1, characterized in that The data management portal module is specifically used for performing one or more of the following on the archived satellite constellation highly available data: data processing statistical display, data summary situation display, data archiving and / or extraction chart display, imaging mode and / or polarization mode status display, data center area distribution information data display.

3. The highly available data management system of the satellite constellation according to claim 1, characterized in that, The data efficient query and retrieval module is specifically used for: performing preliminary query and retrieval on the archived satellite constellation highly available data to obtain preselected satellite constellation highly available data, and querying and retrieving the preselected satellite constellation highly available data based on the to-be-processed retrieval conditions configured by the product model.

4. The highly available data management system for a satellite constellation according to claim 1, wherein The data efficient query and retrieval module is further used for: exporting files of the extracted target satellite constellation highly available data, and determining data coverage rate based on the extracted target satellite constellation highly available data to generate a data coverage rate list corresponding to the target satellite constellation highly available data.

5. The highly available data management system of the satellite constellation according to claim 1, characterized in that, The data archiving access and extraction module is further used for: after archiving the satellite constellation highly available data, if the near-line storage duration is not reached, continuing to archive new satellite constellation highly available data until the near-line storage duration is reached, and performing blue-ray off-line storage on all the archiving results.

6. The highly available data management system for a satellite constellation according to claim 1, wherein, The data archiving access and extraction module is specifically used for: after querying and retrieving, verifying whether the target satellite constellation highly available data corresponding to the querying and retrieving is data prohibited from extraction; in the case that the target satellite constellation highly available data corresponding to the querying and retrieving is not the data prohibited from extraction, performing near-line storage extraction or off-line extraction on the target satellite constellation highly available data.

7. The highly available data management system of the satellite constellation according to claim 1, characterized in that, The data archiving access and extraction module is further used for: sharing the extracted target satellite constellation highly available data.

8. The highly available data management system for a satellite constellation according to claim 1, characterized in that, The data management statistical analysis module is specifically used for performing one or more of the following on the archived satellite constellation highly available data: archiving statistics, extraction statistics, backup statistics, coverage rate statistics.

9. The highly available data management system for a satellite constellation according to claim 1, wherein The data management monitoring module is specifically used for: real-time monitoring of the operating conditions of the data management portal module, the data efficient query and retrieval module, the data archiving access and extraction module, and the data management statistical analysis module.

10. A high-availability data management method for a satellite constellation, characterized in that, The method is applied to the high-availability data management system of the satellite constellation described in claim 1, and the method includes: Collecting high-availability data of the satellite constellation; Archiving the high-availability data of the satellite constellation according to the data type; Using a distributed retrieval service and supplementing with the to-be-processed retrieval conditions configured by the product model to query and retrieve the archived high-availability data of the satellite constellation; Performing near-line storage extraction or off-line extraction on the archived high-availability data of the satellite constellation after query and retrieval, so as to extract the target high-availability data of the satellite constellation to the agreed directory.

Images