Scientific satellite data distribution and collection platform, unified service system and method

Abstract

The invention belongs to the technical field of scientific satellite technology ground systems, and discloses a scientific satellite data distribution and collection platform, unified service system and method, and the platform comprises a scientific satellite data distribution and collection unified service system and a scientific satellite data acquisition and collection client. The service system is used for monitoring and receiving a task distribution request sent by an external system; interpreting a distribution condition according to the data product distribution strategy, and generating a data distribution job when the distribution condition is met; carrying out distribution operation task scheduling, and storing the satellite data products into a satellite data product distribution storage directory; generating and sending a data product distribution list according to the user distribution strategy, and providing a data acquisition interface service for the client; and the client is used for monitoring and receiving the data product distribution list, downloading the data product to the user storage directory, and storing the data product synchronous list to the user storage directory. Through the platform, unified and effective distribution of multi-satellite data can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of scientific satellite technology ground system, and in particular to a scientific satellite data distribution and collection platform, a unified service system and a method. Background Art

[0002] The ground system is an important part of the scientific satellite engineering mission. After receiving the raw data downlinked by the satellite, the ground system will process and produce data products of different levels and types according to the data characteristics and processing requirements of the satellite and the payload. After completing the production of standard primary scientific data products, the ground system will send the scientific data products to the scientific application system for the generation and analysis of higher-level data products; finally, the scientific application system will collect the generated advanced data products to the ground system for long-term archiving and preservation. However, with the increasing number of scientific satellites in my country and the rapid development of space technology, scientific satellites can timely downlink scientific data to the ground system through multiple channels such as digital transmission X-band channels, telemetry S-band channels, Beidou short messages, and VHF (Very High Frequency). The scientific application system has higher and higher requirements for the timeliness of data distribution, and the requirements for data distribution conditions are becoming more and more stringent, especially for space astronomical science satellites, which require the timeliness of data product distribution in seconds, and the distribution scene needs to be interpreted before distribution.

[0003] In order to meet the data product distribution needs of different scientific satellite engineering missions, there is an urgent need for a platform and service system that can support the distribution of multi-satellite data products, so as to be able to summarize the common and individual needs of various scientific satellite missions, save research and development costs and improve research and development efficiency. Summary of the invention

[0004] The purpose of the present invention is to provide a scientific satellite data distribution and collection platform, a unified service system and a method, which are conducive to the data distribution and collection needs of different scientific satellite engineering tasks, especially the unified data distribution and collection needs of multiple satellites, establish a unified distribution and collection service and heterogeneous data acquisition and collection clients, and at the same time, design a data product distribution scenario interpretation method for the data distribution needs of different scientific application systems to achieve effective distribution of scientific data products.

[0005] In the first aspect, a scientific satellite data distribution and collection platform is provided, including: a scientific satellite data distribution and collection unified service system and a scientific satellite data acquisition and collection client;

[0006] The scientific satellite data distribution and collection unified service system is used to monitor and receive data product distribution task requests carrying satellite data products and their information sent by external systems; to judge distribution conditions according to data product distribution strategies, and when distribution conditions are met, to generate data distribution jobs according to data product distribution strategies; to schedule distribution job tasks, store satellite data product information and distribution status in a satellite data product distribution library, and store satellite data products in a satellite data product distribution storage directory; to generate a user's data product distribution list according to a user distribution strategy; and after completing data distribution, to provide a data acquisition interface service for a scientific satellite data acquisition and collection client and send a data product distribution list;

[0007] The scientific satellite data acquisition and collection client is used to monitor and receive the data product distribution list of the scientific satellite data distribution and collection unified service system, download the data products from the satellite data product distribution storage directory to the local user storage directory according to the data product distribution list and the provided data acquisition interface, and generate the user's data product synchronization list after completion, and store the data product synchronization list in the local user storage directory.

[0008] Preferably, the data product distribution task request also carries a data list;

[0009] The scientific satellite data distribution and collection unified service system is specifically used to obtain and match the observation number ID of the scientific data product and the observation number ID of the engineering data product respectively according to the data list carried in the distribution task request; when matched, if it is judged that the satellite has completed the observation task and the data product is complete, or the satellite has not completed the observation task and the data product is incomplete, it is determined that the received data product meets the distribution conditions.

[0010] Preferably, the scientific satellite data distribution and collection unified service system is further used to calculate MD5 for data products in the data product distribution list that meet the distribution conditions, obtain MD5 values, record the MD5 values ​​in the data product distribution list and send them to the scientific satellite data acquisition and collection client;

[0011] The scientific satellite data acquisition and collection client is also used to calculate the MD5 of the data product according to the data product distribution list to obtain the MD5 value when downloading the data product to the local user storage directory, and compare it with the MD5 value recorded in the data product distribution list. When they are consistent and the data product download is completed, the successful distribution status of the data product is fed back to the scientific satellite data distribution and collection unified service system.

[0012] Preferably, the scientific satellite data acquisition and collection client is further used to receive a collection order created by a user, which includes a collection data product and a collection data list, and perform MD5 data verification on the collection data product in the collection order; if the verification is successful, the MD5 value is calculated and recorded in the collection data list, and the collection order is carried in the data product collection request and sent to the scientific satellite data distribution and collection unified service system;

[0013] The scientific satellite data distribution and collection unified service system is also used to monitor and receive data product collection requests sent by scientific satellite data acquisition and collection clients, generate data product collection tasks according to the requested task type and quantity, perform collection task scheduling, establish a data transmission link with the scientific satellite data acquisition and collection clients, receive the collected data products carried in the data product collection request, calculate MD5 for the collected data products according to the collected data list to obtain the MD5 value, and compare it with the MD5 value recorded in the collected data list. When they are consistent, the collected data list is saved in the satellite data product information collection library; the collected data products are saved in the data product collection storage directory, and when the collected data products are received, the collected product metadata information and product file storage information are saved in the satellite data product information collection library.

[0014] In the second aspect, a scientific satellite data distribution and collection unified service system is provided, including: a data distribution object and distribution strategy management module, a satellite data distribution and collection operation scheduling module and a satellite data product distribution service module;

[0015] The data distribution object and distribution strategy management module are used to configure the data product distribution strategy, uniformly manage the data product distribution users, and configure the distribution strategy for each user;

[0016] The satellite data distribution and collection operation scheduling module is used to monitor and receive data product distribution task requests carrying satellite data products and their information sent by an external system; perform distribution operation task scheduling, store satellite data product information and distribution status in a satellite data product distribution library, and store satellite data products in a satellite data product distribution storage directory;

[0017] The satellite data product distribution service module is used to judge the distribution conditions according to the data product distribution strategy, generate a data distribution job according to the data product distribution strategy when the distribution conditions are met; generate a user's data product distribution list according to the user distribution strategy; after completing the data distribution, provide a data acquisition interface service for the scientific satellite data acquisition and collection client and send a data product distribution list.

[0018] Preferably, the data product distribution task request also carries a data list;

[0019] The satellite data product distribution service module is specifically used to obtain and match the observation number ID of the scientific data product and the observation number ID of the engineering data product according to the data list carried in the distribution task request; when matching, if it is judged that the satellite has completed the observation task and the data product is complete, or the satellite has not completed the observation task and the data product is incomplete, it is determined that the received data product meets the distribution conditions.

[0020] Preferably, the satellite data product distribution service module is further used to calculate MD5 for data products in the data product distribution list that meet the distribution conditions, obtain MD5 values, save the MD5 values ​​in the data product distribution list and send them to the scientific satellite data acquisition and aggregation client.

[0021] Preferably, the satellite data distribution and collection job scheduling module is further used to monitor and receive data product collection requests sent by scientific satellite data acquisition and collection clients, generate data product collection tasks according to the requested task type and quantity, and perform collection task scheduling, wherein the data product collection request carries a collection order, and the collection order includes a collection data product and a collection data list;

[0022] The scientific satellite data distribution and collection unified service system also includes: a satellite data product collection service module;

[0023] The satellite data product aggregation service module is used to establish a data transmission link with a scientific satellite data acquisition and aggregation client, receive the aggregated data product carried in the data product aggregation request, calculate the MD5 of the aggregated data product according to the aggregated data list to obtain the MD5 value, and compare it with the MD5 value recorded in the aggregated data list. When they are consistent, the aggregated data list is saved in the satellite data product information aggregation library; call the standardized audit tool to verify the data product, and save the aggregated data product that passes the verification to the data product aggregation storage directory. When the aggregated data product is received, the aggregated product metadata information and product file storage information are saved in the satellite data product information aggregation library.

[0024] Preferably, the scientific satellite data distribution and collection unified service system further comprises: a satellite data distribution and collection supervision module;

[0025] The satellite data distribution and collection supervision module is used to supervise the status of the distribution tasks and collection tasks of each scientific satellite product, including the scheduling execution status of the distribution tasks, the scheduling execution status of the collection tasks, and the collection task status feedback sent by the scientific satellite data acquisition and collection client.

[0026] In a third aspect, a method for distributing and collecting scientific satellite data is provided, including:

[0027] Monitor and receive data product distribution task requests sent by external systems that carry satellite data products and their information;

[0028] Determine the distribution conditions according to the data product distribution strategy, and generate data distribution jobs according to the data product distribution strategy when the distribution conditions are met;

[0029] Perform distribution task scheduling, store satellite data product information and distribution status into the satellite data product distribution library, and store satellite data products into the satellite data product distribution storage directory;

[0030] Generate a data product distribution list for the user based on the user distribution strategy;

[0031] After completing the data distribution, provide data acquisition interface services for scientific satellite data acquisition and aggregation clients and send data product distribution lists.

[0032] Preferably, the data product distribution task request carries a data list;

[0033] The distribution conditions are judged according to the data product distribution strategy. When the distribution conditions are met, a data distribution job is generated according to the data product distribution strategy, including:

[0034] According to the data list carried in the distribution task request, the observation number ID of the scientific data product and the observation number ID of the engineering data product are obtained and matched respectively; when matched, if it is judged that the satellite has completed this observation task and the data product is complete, or the satellite has not completed this observation task and the data product is incomplete, it is determined that the received data product meets the distribution conditions.

[0035] Preferably, the scientific satellite data distribution and collection method further comprises:

[0036] Calculate MD5 for data products in the data product distribution list that meet the distribution conditions to obtain MD5 values, save the MD5 values ​​in the data product distribution list and send them to the scientific satellite data acquisition and aggregation client;

[0037] and / or

[0038] Listen to and receive the data product collection request sent by the scientific satellite data acquisition and collection client, generate data product collection tasks according to the requested task type and quantity, schedule the collection tasks, establish a data transmission link with the scientific satellite data acquisition and collection client, receive the collected data products carried in the data product collection request, calculate the MD5 of the collected data products according to the collection data list to obtain the MD5 value, and compare it with the MD5 value recorded in the collection data list. When they are consistent, save the collection data list to the satellite data product information collection library; save the collected data products to the data product collection storage directory. When the collection data products are received, save the collected product metadata information and product file storage information to the satellite data product information collection library.

[0039] Compared with the prior art, the advantages of the present invention are:

[0040] 1. The present invention designs a scientific satellite data product distribution scenario judgment algorithm, which can accurately determine the data products that meet the distribution conditions from multiple batches of data products;

[0041] 2. The present invention adopts the C / S (server / client) model to design a unified scientific satellite data distribution and collection service platform, which can simultaneously support the data distribution and collection services of multiple scientific satellite missions;

[0042] 3. The present invention designs a unified data distribution and aggregation task scheduling function. Since the data distribution task and data aggregation task execution of each scientific satellite usually do not overlap, and the amount of data transmitted by the task distribution task and data aggregation task of each scientific satellite is quite different, the design of a unified data distribution and aggregation task scheduling can efficiently utilize computing resources and respond to satellite data distribution and aggregation task requirements in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a composition diagram of a scientific satellite data distribution and collection platform according to an embodiment of the present invention;

[0044] Figure 2 This is a data product distribution flow chart of the scientific satellite data distribution and collection unified service system according to an embodiment of the present invention;

[0045] Figure 3 is a flow chart of a scene condition interpretation algorithm for distributing scientific satellite data products according to an embodiment of the present invention;

[0046] Figure 4 is a flow chart of scientific satellite data acquisition and client data acquisition in an embodiment of the present invention;

[0047] Figure 5 This is a flow chart of scientific satellite data acquisition and data uploading by a collection client according to an embodiment of the present invention;

[0048] Figure 6 It is a collection service flow chart of the unified service system for distributing and collecting scientific satellite data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The present invention provides a scientific satellite data distribution and collection platform, including a scientific satellite data distribution and collection unified service system and a scientific satellite data acquisition and collection client. The scientific satellite data distribution and collection unified service system supports the operation of one or more scientific satellite data product acquisition and collection clients in a unified service manner, and has the functions of data product distribution and collection supervision, data product distribution user and distribution strategy control, data product distribution condition interpretation, data product distribution and collection task unified scheduling; the scientific satellite data acquisition and collection client supports multi-site heterogeneous environment deployment, can obtain data product distribution status in time to obtain data products, and can also support uploading local collection data to the scientific satellite data distribution and collection unified service system.

[0050] The scientific satellite data distribution and collection unified service system is used to monitor and receive data product distribution task requests carrying satellite data products and their information sent by external systems; to judge distribution conditions according to data product distribution strategies, and when distribution conditions are met, to generate data distribution jobs according to data product distribution strategies; to schedule distribution job tasks, store satellite data product information and distribution status in a satellite data product distribution library, and store satellite data products in a satellite data product distribution storage directory; to generate a user's data product distribution list according to a user distribution strategy; and after completing data distribution, to provide a data acquisition interface service for a scientific satellite data acquisition and collection client and send a data product distribution list;

[0051] The scientific satellite data acquisition and collection client is used to monitor and receive the data product distribution list of the scientific satellite data distribution and collection unified service system, download the data products from the satellite data product distribution storage directory to the local user storage directory according to the data product distribution list and the provided data acquisition interface, and generate the user's data product synchronization list after completion, and store the data product synchronization list in the local user storage directory.

[0052] Preferably, the data product distribution task request also carries a data list;

[0053] The scientific satellite data distribution and collection unified service system is specifically used to obtain and match the observation number ID of the scientific data product and the observation number ID of the engineering data product respectively according to the data list carried in the distribution task request; when matched, if it is judged that the satellite has completed the observation task and the data product is complete, or the satellite has not completed the observation task and the data product is incomplete, it is determined that the received data product meets the distribution conditions.

[0054] Preferably, the scientific satellite data distribution and collection unified service system is further used to calculate MD5 for data products in the data product distribution list that meet the distribution conditions, obtain MD5 values, record the MD5 values ​​in the data product distribution list and send them to the scientific satellite data acquisition and collection client;

[0055] The scientific satellite data acquisition and collection client is also used to calculate the MD5 of the data product according to the data product distribution list to obtain the MD5 value when downloading the data product to the local user storage directory, and compare it with the MD5 value recorded in the data product distribution list. When they are consistent and the data product download is completed, the successful distribution status of the data product is fed back to the scientific satellite data distribution and collection unified service system.

[0056] Preferably, the scientific satellite data acquisition and collection client is further used to receive a collection order created by a user, which includes a collection data product and a collection data list, and perform MD5 data verification on the collection data product in the collection order; if the verification is successful, the MD5 value is calculated and recorded in the collection data list, and the collection order is carried in the data product collection request and sent to the scientific satellite data distribution and collection unified service system;

[0057] The scientific satellite data distribution and collection unified service system is also used to monitor and receive data product collection requests sent by scientific satellite data acquisition and collection clients, generate data product collection tasks according to the requested task type and quantity, perform collection task scheduling, establish a data transmission link with the scientific satellite data acquisition and collection clients, receive the collected data products carried in the data product collection request, calculate MD5 for the collected data products according to the collected data list to obtain the MD5 value, and compare it with the MD5 value recorded in the collected data list. When they are consistent, the collected data list is saved in the satellite data product information collection library; the collected data products are saved in the data product collection storage directory, and when the collected data products are received, the collected product metadata information and product file storage information are saved in the satellite data product information collection library.

[0058] The present invention also provides a scientific satellite data distribution and collection unified service system, including: a data distribution object and distribution strategy management module, a satellite data distribution and collection operation scheduling module and a satellite data product distribution service module;

[0059] The data distribution object and distribution strategy management module are used to configure the data product distribution strategy, uniformly manage the data product distribution users, and configure the distribution strategy for each user;

[0060] The satellite data distribution and collection operation scheduling module is used to monitor and receive data product distribution task requests carrying satellite data products and their information sent by an external system; perform distribution operation task scheduling, store satellite data product information and distribution status in a satellite data product distribution library, and store satellite data products in a satellite data product distribution storage directory;

[0061] The satellite data product distribution service module is used to judge the distribution conditions according to the data product distribution strategy, generate a data distribution job according to the data product distribution strategy when the distribution conditions are met; generate a user's data product distribution list according to the user distribution strategy; after completing the data distribution, provide a data acquisition interface service for the scientific satellite data acquisition and collection client and send a data product distribution list.

[0062] Preferably, the data product distribution task request also carries a data list;

[0063] The satellite data product distribution service module is specifically used to obtain and match the observation number ID of the scientific data product and the observation number ID of the engineering data product according to the data list carried in the distribution task request; when matching, if it is judged that the satellite has completed the observation task and the data product is complete, or the satellite has not completed the observation task and the data product is incomplete, it is determined that the received data product meets the distribution conditions.

[0064] Preferably, the satellite data product distribution service module is further used to calculate MD5 for data products in the data product distribution list that meet the distribution conditions, obtain MD5 values, save the MD5 values ​​in the data product distribution list and send them to the scientific satellite data acquisition and aggregation client.

[0065] Preferably, the satellite data distribution and collection job scheduling module is further used to monitor and receive data product collection requests sent by scientific satellite data acquisition and collection clients, generate data product collection tasks according to the requested task type and quantity, and perform collection task scheduling, wherein the data product collection request carries a collection order, and the collection order includes a collection data product and a collection data list;

[0066] The scientific satellite data distribution and collection unified service system also includes: a satellite data product collection service module;

[0067] The satellite data product aggregation service module is used to establish a data transmission link with a scientific satellite data acquisition and aggregation client, receive the aggregated data product carried in the data product aggregation request, calculate the MD5 of the aggregated data product according to the aggregated data list to obtain the MD5 value, and compare it with the MD5 value recorded in the aggregated data list. When they are consistent, the aggregated data list is saved in the satellite data product information aggregation library; call the standardized audit tool to verify the data product, and save the aggregated data product that passes the verification to the data product aggregation storage directory. When the aggregated data product is received, the aggregated product metadata information and product file storage information are saved in the satellite data product information aggregation library.

[0068] Preferably, the scientific satellite data distribution and collection unified service system further comprises: a satellite data distribution and collection supervision module;

[0069] The satellite data distribution and collection supervision module is used to supervise the status of the distribution tasks and collection tasks of each scientific satellite product, including the scheduling execution status of the distribution tasks, the scheduling execution status of the collection tasks, and the collection task status feedback sent by the scientific satellite data acquisition and collection client.

[0070] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0071] The system disclosed in the present invention comprises: Figure 1 As shown in the figure, it includes two parts: the unified service system for distributing and collecting scientific satellite data and the client for acquiring and collecting scientific satellite data. Each client can correspond to different scientific satellites. Figure 1 The client corresponding to star A, the client corresponding to star B and the client corresponding to star C are shown in FIG. The module composition of each part is as follows: Figure 1 As shown, the scientific satellite data acquisition and collection client supports deployment in a heterogeneous environment, supports one service system corresponding to multiple clients, and the client can be deployed in the scientific application system of each scientific satellite engineering task according to the task requirements, and supports multiple sets of clients to run in parallel at the same time. The scientific satellite data distribution and collection unified service system needs to receive the distribution data pushed by the external system. The unified service system can be deployed separately to provide distribution and collection service support for each scientific satellite engineering task; the interface definition with the external system is not within the definition scope of the present invention.

[0072] 1. The scientific satellite data distribution and collection unified service system consists of five functional modules, including satellite data distribution and collection supervision module, data distribution object and distribution strategy management module, satellite data distribution and collection operation scheduling module, satellite data product distribution service module and satellite data product collection service module.

[0073] 1) Satellite data distribution and collection supervision module: This module is mainly responsible for the status supervision of the distribution and collection tasks of various scientific satellite missions, including the scheduling and execution status of the distribution and collection tasks, as well as the task status feedback sent by the scientific satellite data acquisition and collection client.

[0074] 2) Data distribution object and distribution strategy management module: This module is mainly responsible for unified data distribution object management, configuring each user's data distribution strategy (part or all of the information in the payload type, payload level, and payload data type, etc.), and determining the distribution data range of each user. The user here is the data distribution object.

[0075] 3) Satellite data distribution and collection job scheduling module: This module is responsible for receiving data product distribution requests sent by the upstream system and data product collection requests sent by the scientific satellite data acquisition and collection client; for data product distribution requests, this module first obtains the user distribution strategy, identifies the user of the data product request, and then generates a distribution task. The same batch of data product distribution requests will generate multiple data distribution tasks for different users. For data product collection requests, this module is responsible for receiving and parsing the content of the request, and generating data product collection tasks based on the requested task type and quantity.

[0076] There are two main types of data product collection request tasks: one is to collect new archived data, and the other is to update the already collected data; these two types of collection tasks are distinguished. Specifically, based on the above task types, "update" tasks and "additional" tasks can be generated. In addition, considering the data volume constraint, if the file volume and data volume exceed the constraint of 10GB (configurable), the information in the request will be split and multiple collection tasks will be generated.

[0077] 4) Satellite data product distribution service module: This module is mainly responsible for judging whether the distribution conditions are met according to the data product distribution strategy, calculating the MD5 value of the data product distribution list that meets the distribution conditions, and sending the data product distribution list to the scientific satellite data acquisition and aggregation client after the calculation is completed, notifying the client that the distribution product has met the acquisition conditions, and providing the client with data acquisition interface services.

[0078] The data product distribution strategy includes part or all of the matching of scientific products and engineering data products, completion of observation tasks, data integrity and timeliness of data requirements.

[0079] 5) Satellite data product aggregation service module: This module is mainly responsible for calculating the MD5 of the aggregated data product based on the data product aggregation data list, and comparing the calculated MD5 value with the MD5 value recorded in the aggregation data list. If the contents of the two are exactly the same, the standardized audit tool is called to verify the data product, and the aggregated data product that passes the verification is stored in the warehouse.

[0080] 2. The scientific satellite data acquisition and collection client consists of two modules related to the data product collection business: the data collection order management module and the collection data upload module; and three modules related to the data product distribution business: the data distribution list receiving module, the data subscription application module and the distribution data product acquisition module.

[0081] 1) Data collection order management module: The main function of this module is to scan the collection data products and collection data lists generated by local users. The collection order is initiated by the user, who actively fills in the data product information, selects the included files, and generates the collection data list. After the module scans the collection data list, it verifies the list content, including the review of the MD5 value. Finally, the module sends the collection data list that passes the MD5 verification to the collection server.

[0082] 2) Aggregate data upload module: This module is mainly responsible for uploading local aggregate data products to the server based on the contents in the aggregate data list.

[0083] 3) Data distribution list receiving module: This module is responsible for receiving the data product distribution list sent by the scientific satellite data distribution and collection unified service system, parsing the content of the list, and notifying users of the availability of new data products through message notifications.

[0084] 4) Data subscription application module: This module is mainly responsible for providing users with a list of subscribable data product types, supporting users to submit subscription applications for several data products based on data product types, and providing the approval status of existing subscription applications (accepted / unprocessed / rejected).

[0085] 5) Distribution data product acquisition module: This module is mainly responsible for acquiring data products from the server, downloading the data products to the local storage space, and verifying the MD5 value. When the verification passes and the data product acquisition is completed, the successful distribution status of the data product is fed back to the satellite data distribution and aggregation supervision module.

[0086] The above-mentioned server is the unified service system for distributing and collecting scientific satellite data.

[0087] like Figure 2 As shown, it is a data distribution process designed by the scientific satellite data distribution and collection unified service system according to an embodiment of the present invention, including the following steps:

[0088] Step 1: The satellite data distribution and collection job scheduling module listens to the data product distribution task requests sent by the external system through the message queue. When receiving the data product distribution task request, it notifies the satellite data product distribution service module to parse the distribution task request and obtain the data product distribution strategy information. After that, the satellite data product distribution service module interprets the data product distribution conditions. Since the data of a scientific satellite observation mission will be transmitted downlink multiple times, the data of the same group of observation IDs will be continuously updated and iterated. To avoid wasting storage and transmission resources, in general, only data products that have completed the observation mission and the first downlink data will be distributed as data products. The specific distribution condition interpretation algorithm is as follows: Figure 3 As shown, the algorithm flow for judging the data product distribution conditions is as follows:

[0089] 1) According to the data list in the distribution task, obtain the observation number ID information of scientific data products and engineering data products respectively.

[0090] 2) Match the observation ID information of the scientific data products and the engineering data products to be distributed. If the scientific data products and engineering data products with the same set of observation IDs cannot be matched, it means that the observation data is not complete and does not meet the distribution conditions.

[0091] 3) When matching the scientific data products and engineering data products with the same observation ID, it is determined whether the satellite has completed the observation.

[0092] 4) If the satellite is judged to have completed the observation mission, the data integrity is judged; if the data product is complete, the received batch data product is judged to be eligible for distribution. If the data product is incomplete, it means that the data is still missing, and the data product can only be distributed after the data of the observation ID is complete. Here, the data product is received in batches.

[0093] 5) If it is judged that the satellite has not completed the observation mission, the data integrity is also judged. If the data product is not complete, it means that the data product needs to be distributed to the scientific application system in a timely manner and is judged to be ready for distribution. If it is judged to be complete, it means that there is a problem with the processing of the data product and it needs to be repaired before it can be distributed.

[0094] Step 2: Generate distribution jobs according to data product distribution strategies, schedule distribution jobs through satellite data distribution and data aggregation job scheduling, store distribution status in satellite data product distribution library, store data products in satellite data distribution storage directory, and parse data product information into satellite data product distribution library. After data products are stored in the library, obtain the user's distribution strategy and generate a data product distribution list for the user.

[0095] Step 3: After the scientific satellite data distribution and collection unified service system completes data distribution, it sends the user's data product distribution list message to the corresponding scientific satellite data acquisition and collection client, and the scientific satellite data acquisition and collection client acquires data.

[0096] The above process is that the scientific satellite data distribution and collection unified service system monitors and receives data product distribution task requests carrying satellite data products and their information sent by external systems; interprets distribution conditions according to data product distribution strategies, and generates data distribution jobs according to data product distribution strategies when distribution conditions are met; schedules distribution job tasks, stores satellite data product information and distribution status in the satellite data product distribution library, and stores satellite data products in the satellite data product distribution storage directory; generates user data product distribution lists according to user distribution strategies; and after completing data distribution, provides data acquisition interface services for scientific satellite data acquisition and collection clients and sends data product distribution lists.

[0097] The process of acquiring scientific satellite data and collecting data products from clients is as follows: Figure 4 As shown. The client automatic synchronization background service regularly monitors the data product distribution list of the scientific satellite data distribution and collection unified service system. When the data product distribution list is monitored, it is responsible for parsing the data product distribution list, reading the client data product distribution list, and copying the data files to the local user storage directory in priority order according to the data type priority configuration; when the data copying is completed, the synchronization status is recorded in the satellite data product distribution library, and the copied data is used to generate the data product synchronization list file to the local user storage directory.

[0098] Preferably, the scientific satellite data distribution and collection unified service system also calculates MD5 for data products in the data product distribution list that meet the distribution conditions to obtain an MD5 value, saves the MD5 value in the data product distribution list and sends it to the scientific satellite data acquisition and collection client, so that the scientific satellite data acquisition and collection client can verify the integrity of the data to ensure that the file has not been modified or damaged during the transmission process.

[0099] The specific implementation methods for the collection process of scientific satellite data products are as follows:

[0100] 1. Scientific satellite data acquisition and collection clients can be deployed on the local server of the scientific application system. The workflow is as follows: Figure 5 The client receives the collection order created by the user, and performs a preliminary MD5 data check on the files in the list. For those that fail the check, the failure result is fed back to the user. For the data files that pass the check, the MD5 value calculation results are saved, and the data is uploaded to the storage directory of the scientific satellite data distribution and collection unified service system. After the upload is completed, the completion status is sent to the server.

[0101] 2. The workflow of the collection service end of the scientific satellite data distribution and collection unified service system is as follows Figure 6 As shown, the service system monitors and receives the data product collection request sent by the scientific satellite data acquisition and collection client, regularly receives the collection order submitted by the client, parses the order information and data product file information, and saves them to the satellite data product information collection library; performs collection task scheduling, and at the same time establishes a data transmission link between the collection service and the collection client software, receives the data product entity file contained in the collection order, calculates the MD5 of the collected data product according to the collection data list to obtain the MD5 value, and compares it with the MD5 value recorded in the collection data list. When they are consistent, the data product collection storage directory is saved; when the collection data product file is received, the collection product metadata information and product file storage information are saved to the satellite data product information collection library, a data submission list is generated, and the submission status is recorded, the submission status is stored in the satellite data product information collection library, and the submission list is stored in the data product collection storage directory.

[0102] The scientific satellite data distribution and collection platform, scientific satellite data distribution and collection unified service system and method provided by the embodiments of the present invention meet the data distribution and collection requirements, especially the unified data distribution and collection requirements of multiple satellites. A unified scientific satellite data distribution and collection service platform is designed using the C / S (server / client) model, which can simultaneously support the data distribution and collection services of multiple scientific satellite missions; a scientific satellite data product distribution scenario judgment algorithm is designed to meet the data distribution requirements of different scientific application systems, which can judge whether the data product meets the distribution conditions according to the distribution scenario judgment rules, and effectively distribute the scientific satellite data products based on the judgment results; the design supports the deployment of scientific satellite data acquisition and collection servers in multi-location heterogeneous environments, which can distribute scientific data according to the user-defined data acquisition strategy, and collect local advanced scientific data products to the ground system; an MD5 checksum mechanism is added in the data distribution and collection process to ensure the consistency and integrity of the distributed and collected data products.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention is described in detail with reference to the embodiments, it should be understood by those skilled in the art that any modification or equivalent replacement of the technical solutions of the present invention does not depart from the spirit and scope of the technical solutions of the present invention and should be included in the scope of the claims of the present invention.

Claims

1. A scientific satellite data distribution and collection platform, characterized in that: include: A unified service system for distributing and collecting scientific satellite data and a client for acquiring and collecting scientific satellite data; The scientific satellite data distribution and collection unified service system is used to monitor and receive data product distribution task requests carrying satellite data products and their information sent by external systems; to judge distribution conditions according to data product distribution strategies, and when distribution conditions are met, to generate data distribution jobs according to data product distribution strategies; to schedule distribution job tasks, store satellite data product information and distribution status in a satellite data product distribution library, and store satellite data products in a satellite data product distribution storage directory; to generate a user's data product distribution list according to a user distribution strategy; and after completing data distribution, to provide a data acquisition interface service for a scientific satellite data acquisition and collection client and send a data product distribution list; The scientific satellite data acquisition and collection client is used to monitor and receive the data product distribution list of the scientific satellite data distribution and collection unified service system, download the data products from the satellite data product distribution storage directory to the local user storage directory according to the data product distribution list and the provided data acquisition interface, and generate the user's data product synchronization list after completion, and store the data product synchronization list in the local user storage directory.

2. The scientific satellite data distribution and collection platform according to claim 1, characterized in that: The data product distribution task request also carries a data list; The scientific satellite data distribution and collection unified service system is specifically used to obtain and match the observation number ID of the scientific data product and the observation number ID of the engineering data product respectively according to the data list carried in the distribution task request; when matched, if it is judged that the satellite has completed the observation task and the data product is complete, or the satellite has not completed the observation task and the data product is incomplete, it is determined that the received data product meets the distribution conditions.

3. The scientific satellite data distribution and collection platform according to claim 1, characterized in that: The scientific satellite data distribution and collection unified service system is also used to calculate MD5 for data products in the data product distribution list that meet the distribution conditions, obtain MD5 values, record the MD5 values ​​in the data product distribution list and send them to the scientific satellite data acquisition and collection client; The scientific satellite data acquisition and collection client is also used to calculate the MD5 of the data product according to the data product distribution list to obtain the MD5 value when downloading the data product to the local user storage directory, and compare it with the MD5 value recorded in the data product distribution list. When they are consistent and the data product download is completed, the successful distribution status of the data product is fed back to the scientific satellite data distribution and collection unified service system.

4. The scientific satellite data distribution and collection platform according to any one of claims 1 to 3, characterized in that: The scientific satellite data acquisition and collection client is also used to receive a collection order containing a collection data product and a collection data list created by a user, and perform MD5 data verification on the collection data product in the collection order; If the verification is successful, the MD5 value is calculated and recorded in the collection data list, and the collection order is carried in the data product collection request and sent to the scientific satellite data distribution and collection unified service system; The scientific satellite data distribution and collection unified service system is also used to monitor and receive data product collection requests sent by scientific satellite data acquisition and collection clients, generate data product collection tasks according to the requested task type and quantity, perform collection task scheduling, establish a data transmission link with the scientific satellite data acquisition and collection clients, receive the collected data products carried in the data product collection request, calculate MD5 for the collected data products according to the collected data list to obtain the MD5 value, and compare it with the MD5 value recorded in the collected data list. When they are consistent, the collected data list is saved in the satellite data product information collection library; the collected data products are saved in the data product collection storage directory, and when the collected data products are received, the collected product metadata information and product file storage information are saved in the satellite data product information collection library.

5. A scientific satellite data distribution and collection unified service system, characterized in that: include: Data distribution object and distribution strategy management module, satellite data distribution and collection operation scheduling module and satellite data product distribution service module; The data distribution object and distribution strategy management module are used to configure the data product distribution strategy, uniformly manage the data product distribution users, and configure the distribution strategy for each user; The satellite data distribution and collection operation scheduling module is used to monitor and receive data product distribution task requests carrying satellite data products and their information sent by an external system; perform distribution operation task scheduling, store satellite data product information and distribution status in a satellite data product distribution library, and store satellite data products in a satellite data product distribution storage directory; The satellite data product distribution service module is used to judge the distribution conditions according to the data product distribution strategy, generate a data distribution job according to the data product distribution strategy when the distribution conditions are met; generate a user's data product distribution list according to the user distribution strategy; after completing the data distribution, provide a data acquisition interface service for the scientific satellite data acquisition and collection client and send a data product distribution list.

6. The scientific satellite data distribution and collection unified service system according to claim 5, characterized in that: The data product distribution task request also carries a data list; The satellite data product distribution service module is specifically used to obtain and match the observation number ID of the scientific data product and the observation number ID of the engineering data product according to the data list carried in the distribution task request; when matching, if it is judged that the satellite has completed the observation task and the data product is complete, or the satellite has not completed the observation task and the data product is incomplete, it is determined that the received data product meets the distribution conditions.

7. The scientific satellite data distribution and collection unified service system according to claim 5, characterized in that: The satellite data product distribution service module is also used to calculate MD5 for data products in the data product distribution list that meet the distribution conditions, obtain MD5 values, save the MD5 values ​​in the data product distribution list and send them to the scientific satellite data acquisition and collection client.

8. The scientific satellite data distribution and collection unified service system according to any one of claims 5 to 7, characterized in that: The satellite data distribution and collection job scheduling module is also used to monitor and receive data product collection requests sent by scientific satellite data acquisition and collection clients, generate data product collection tasks according to the requested task type and quantity, and perform collection task scheduling. The data product collection request carries a collection order, which includes a collection data product and a collection data list; The scientific satellite data distribution and collection unified service system also includes: a satellite data product collection service module; The satellite data product aggregation service module is used to establish a data transmission link with a scientific satellite data acquisition and aggregation client, receive the aggregated data product carried in the data product aggregation request, calculate the MD5 of the aggregated data product according to the aggregated data list to obtain the MD5 value, and compare it with the MD5 value recorded in the aggregated data list. When they are consistent, the aggregated data list is saved in the satellite data product information aggregation library; call the standardized audit tool to verify the data product, and save the aggregated data product that passes the verification to the data product aggregation storage directory. When the aggregated data product is received, the aggregated product metadata information and product file storage information are saved in the satellite data product information aggregation library.

9. The scientific satellite data distribution and collection unified service system according to any one of claims 5 to 7, characterized in that: Also includes: Satellite data distribution and collection supervision module; The satellite data distribution and collection supervision module is used to supervise the status of the distribution tasks and collection tasks of each scientific satellite product, including the scheduling execution status of the distribution tasks, the scheduling execution status of the collection tasks, and the collection task status feedback sent by the scientific satellite data acquisition and collection client.

10. A method for distributing and collecting scientific satellite data, comprising: Monitor and receive data product distribution task requests sent by external systems that carry satellite data products and their information; Determine the distribution conditions according to the data product distribution strategy, and generate data distribution jobs according to the data product distribution strategy when the distribution conditions are met; Perform distribution task scheduling, store satellite data product information and distribution status into the satellite data product distribution library, and store satellite data products into the satellite data product distribution storage directory; Generate a data product distribution list for the user based on the user distribution strategy; After completing the data distribution, provide data acquisition interface services for scientific satellite data acquisition and aggregation clients and send data product distribution lists.

11. The method for distributing and collecting scientific satellite data according to claim 10, characterized in that: The data product distribution task request carries a data list; The distribution conditions are judged according to the data product distribution strategy. When the distribution conditions are met, a data distribution job is generated according to the data product distribution strategy, including: According to the data list carried in the distribution task request, the observation number ID of the scientific data product and the observation number ID of the engineering data product are obtained and matched respectively; when matched, if it is judged that the satellite has completed this observation task and the data product is complete, or the satellite has not completed this observation task and the data product is incomplete, it is determined that the received data product meets the distribution conditions.

12. The method for distributing and collecting scientific satellite data according to claim 10, characterized in that: Also includes: Calculate MD5 for data products in the data product distribution list that meet the distribution conditions to obtain MD5 values, save the MD5 values ​​in the data product distribution list and send them to the scientific satellite data acquisition and aggregation client; and / or Listen to and receive the data product collection request sent by the scientific satellite data acquisition and collection client, generate data product collection tasks according to the requested task type and quantity, schedule the collection tasks, establish a data transmission link with the scientific satellite data acquisition and collection client, receive the collected data products carried in the data product collection request, calculate the MD5 of the collected data products according to the collection data list to obtain the MD5 value, and compare it with the MD5 value recorded in the collection data list. When they are consistent, save the collection data list to the satellite data product information collection library; save the collected data products to the data product collection storage directory. When the collection data products are received, save the collected product metadata information and product file storage information to the satellite data product information collection library.

Images