A GNSS occultation detection ground application system

Abstract

The present application discloses a GNSS occultation detection ground application system, which relates to the field of data processing systems. The system utilizes a task scheduling module to schedule and allocate tasks, and utilizes a distributed task execution module to execute tasks. For the most common data inversion tasks, after the task scheduling module parses and obtains the positioning observation data and the occultation observation data in the original payload data of the satellite, it generates a corresponding data inversion task only when it detects that the data volume of the positioning observation data matching each group of occultation observation data in the distributed file system reaches the inversion requirement, thereby ensuring the quality of subsequent data inversion. The task scheduling module can flexibly allocate tasks for a large number of tasks with different requirements according to the resource conditions of different task execution modules and the scheduling priorities of different tasks, thereby ensuring the effectiveness of task execution and being conducive to improving task processing efficiency.

Description

Technical Field

[0001] The present application relates to the field of data processing systems, and in particular to a GNSS occultation detection ground application system. Background Art

[0002] GNSS (Global Navigation Satellite System) occultation detection technology is a new remote sensing technology that can detect the atmosphere and ionosphere on a global scale, thereby providing ionosphere and neutral atmosphere profiles, and plays an important role in space environment missions. GNSS occultation detection technology has the advantages of high precision, high vertical resolution, long-term stability, global coverage and all-weather detection, which can make up for the shortcomings of traditional atmospheric detection methods. In addition, its detection payload has the advantages of small size and low power consumption, which is very suitable for carrying small satellites to carry out large-scale constellation detection.

[0003] When applying GNSS occultation detection technology, it is necessary to use ground application systems to receive and process the payload data obtained by satellites and further generate data products at all levels. With the development and gradual promotion of GNSS occultation detection technology, the number of satellites is increasing, and the amount of payload data obtained by each satellite is also becoming more and more abundant, which poses new challenges to the performance of ground application systems. Summary of the invention

[0004] In view of the above problems and technical requirements, this application proposes a GNSS occultation detection ground application system. The technical solution of this application is as follows:

[0005] A GNSS occultation detection ground application system, the GNSS occultation detection ground application system comprising a task scheduling module, a plurality of task execution modules, a ZooKeeper distributed cache and a distributed file system, wherein a list of tasks to be executed and a list of execution module status are maintained in the ZooKeeper distributed cache;

[0006] The task scheduling module is used to receive the original payload data sent by each satellite of the GNSS occultation detection system, and after parsing the original payload data received each time to obtain positioning observation data and occultation observation data, store the parsed positioning observation data in a distributed file system, and when it is detected that the data amount of the positioning observation data matching each set of occultation observation data in the distributed file system reaches the inversion requirement, initialize and generate a data inversion task according to each set of occultation observation data and its matching positioning observation data, and add it to a list of tasks to be executed;

[0007] The task scheduling module is also used to assign each task in the to-be-executed task list to a corresponding task execution module according to the resource occupancy of each task execution module recorded in the execution module status list and the scheduling priority of each task in the to-be-executed task list;

[0008] Each task execution module is used to execute the received task and feed back the task execution status to the task scheduling module; when executing the data inversion task, the task execution module performs data inversion on the occultation observation data and its matching positioning observation data in the data inversion task to obtain the inversion result data, and stores the inversion result data in the distributed file system;

[0009] The task scheduling module is also used to update the resource occupancy of each task execution module recorded in the execution module status list according to the task execution status fed back by each task execution module.

[0010] Its further technical feature is that the task scheduling module is also used to detect whether the data amount of the positioning observation data matching the currently parsed occultation observation data in the distributed file system meets the inversion requirement each time a set of occultation observation data is parsed; when the data amount of the positioning observation data matching the currently parsed occultation observation data in the distributed file system meets the inversion requirement, the data inversion task is initialized and generated according to the currently parsed occultation observation data and the matching positioning observation data, otherwise the currently parsed occultation observation data and the satellite identifier of the satellite that sent the occultation observation data are correspondingly stored in the unprocessed list maintained by the distributed file system;

[0011] The task scheduling module is also used to traverse each group of occultation observation data in the unprocessed list in sequence at predetermined time intervals, and when it is detected that the amount of positioning observation data matching the traversed occultation observation data in the distributed file system reaches the inversion requirement, initialize the generation of the corresponding data inversion task and delete the traversed occultation observation data from the unprocessed list, otherwise the traversed occultation observation data will be retained in the unprocessed list.

[0012] Its further technical feature is that the task scheduling module stores the parsed positioning observation data in the distributed file system, including: correspondingly storing each positioning frame and positioning time in each parsed positioning observation data, and the satellite identifier of the satellite that sends the positioning observation data in the distributed file system;

[0013] The task scheduling module detects whether the amount of positioning observation data matching each set of occultation observation data in the distributed file system meets the inversion requirements, including:

[0014] Detecting the number of positioning frames matching the occultation observation data in the distributed file system, wherein the positioning frames matching the occultation observation data are positioning frames corresponding to the target satellite identifier and corresponding to the positioning time within the duration of the occultation observation data, and the target satellite identifier is the satellite identifier of the satellite that sent the occultation observation data;

[0015] When it is detected that the number of positioning frames matching the occultation observation data reaches a quantity threshold, it is determined that the data amount of the positioning observation data matching the occultation observation data meets the inversion requirement, and all positioning frames matching the occultation observation data are extracted to obtain the positioning observation data matching the occultation observation data; otherwise, it is determined that the data amount of the positioning observation data matching the occultation observation data does not meet the inversion requirement.

[0016] Its further technical features are that each task execution module integrates multiple CPU cores, each task execution module reserves several CPU cores as basic cores for running the operating system of the task execution module, and uses the isolcpus command to isolate other CPU cores except the basic cores;

[0017] Each task execution module is further used to select a CPU core from the isolated CPU cores when receiving a data inversion task, and use a taskset command to control the received data inversion task to be executed on the selected CPU core.

[0018] Its further technical feature is that each task execution module is used to define a unique environment variable for the data inversion task and copy it to each process when starting multiple processes to execute a data inversion task in parallel;

[0019] Each task execution module is also used to close the failed process when it is detected that any process fails during the execution of the data inversion task, and close other processes with the same environment variables as the failed process to close all processes executing the same data inversion task.

[0020] Its further technical features are that the task scheduling module is deployed on two physical nodes respectively, the real IP addresses of the two physical nodes are virtualized into the virtual IP address of the task scheduling module through VRRP technology, and the task scheduling module provides services to the outside through the virtual IP address;

[0021] Each physical node sends its own node attribute information to another physical node when it starts up. Each physical node determines its own control priority based on its own node attribute information, and determines the control priority of another physical node based on the node attribute information received from another physical node. Each physical node determines the physical node with the highest control priority as the primary node, and the other physical node as the backup node.

[0022] The master node has control over the virtual IP address of the task scheduling module and provides external services for the task scheduling module. The master node also performs heartbeat communication with the backup node.

[0023] When any physical node detects that the duration of disconnection of heartbeat communication with another physical node reaches a duration threshold, the physical node detects whether it can perform gateway communication;

[0024] When the physical node determines that it can perform gateway communication and is currently the master node, the physical node continues to provide external services of the task scheduling module until it receives node attribute information sent by another physical node and resumes heartbeat communication with the other physical node;

[0025] When the physical node determines that it can perform gateway communication and is currently a backup node, the physical node switches to the master node to have control over the virtual IP address of the task scheduling module and provide external services of the task scheduling module, and the physical node resumes heartbeat communication with the other physical node after receiving the node attribute information sent by the other physical node;

[0026] When the physical node determines that the gateway communication cannot be performed, it restarts and resends its own node attribute information to another physical node after the restart.

[0027] Its further technical feature is that the node attribute information of each physical node includes the startup time, version number and node host name of the physical node. The earlier the startup time of the physical node, the smaller the version number and the higher the dictionary sorting of the node host name, the higher the control priority of the physical node.

[0028] Its further technical feature is that the task scheduling module assigns each task in the to-be-executed task list to the corresponding task execution module, including:

[0029] The tasks in the task list to be executed are assigned in descending order of scheduling priority. When each task is assigned, the idle resources of each task execution module are determined according to the resource occupancy of each task execution module recorded in the execution module status list, the resource requirements of the task to be assigned are determined, and the task to be assigned is assigned to the task execution module whose idle resources meet the resource requirements of the task;

[0030] Among them, the scheduling priority of each task in the list of tasks to be executed is determined according to the importance of the task, the creation time, and the waiting scheduling time of the task; the higher the importance of the task, the earlier the creation time, and the longer the waiting scheduling time, the higher the scheduling priority of the task.

[0031] Its further technical feature is that the task scheduling module is also used to add the task execution module and its corresponding resource occupancy status to the execution module status list when detecting the access of a new task execution module;

[0032] The task scheduling module is also used to reallocate the task to the task execution module when the waiting time has expired after the task is assigned to the task execution module and the task execution status feedback from the task execution module has not been received; when the number of repetitions of assigning the same task to the same task execution module reaches a threshold, it is determined that the task execution module is abnormal, the task execution module is deleted from the execution module status list and the task is reallocated to other task execution modules.

[0033] Its further technical feature is that the task scheduling module is further used to initialize and generate a scheduled task that meets the scheduled start condition and add it to the list of tasks to be executed when detecting that the scheduled start condition of any scheduled task in the scheduled task list is met; the scheduled task list is stored in a distributed file system, and the scheduled task list includes multiple scheduled tasks and their corresponding scheduled start conditions;

[0034] The task scheduling module is also used to initialize and generate a manual trigger task according to the manual trigger instruction and add it to the list of tasks to be executed when a manual trigger instruction is received.

[0035] The beneficial technical effects of this application are:

[0036] The present application discloses a GNSS occultation detection ground application system, which utilizes a task scheduling module to initialize and schedule tasks, and utilizes a distributed task execution module to execute tasks. For the most common data inversion task, after the task scheduling module parses and obtains the positioning observation data and the occultation observation data in the original payload data of the satellite, the corresponding data inversion task is generated only when it is detected that the data amount of the positioning observation data matching each group of occultation observation data in the distributed file system reaches the inversion requirement, thereby ensuring the quality of subsequent data inversion. In addition, a list of pending tasks maintained in the ZooKeeper distributed cache is used to manage pending tasks, and an execution module status list is used to dynamically manage the resource occupancy of different task execution modules, thereby making it possible to flexibly allocate tasks for a large number of tasks with different requirements according to the resource conditions of different task execution modules and the scheduling priorities of different tasks, thereby ensuring the effectiveness of task execution and facilitating improving task processing efficiency.

[0037] For computing-intensive tasks such as data inversion tasks, controlling them to run on an independent CPU core will result in faster execution and shorter response times than sharing computing resources among multiple CPU cores. In addition, by making full use of Linux's CPU scheduling characteristics and the transitivity of environment variables, CPU utilization is maximized, so that when a data inversion task fails, all processes belonging to the same data inversion task can be completely shut down to avoid the legacy of idle processes. The implementation method is simple and reliable.

[0038] The task scheduling module in the GNSS occultation detection ground application system uses a virtual routing redundancy protocol to achieve dynamic switching of the master and backup nodes, and ensures the continuous and stable operation of the system through the master-backup mechanism.

[0039] The list of tasks to be executed and the list of execution module status are maintained in the ZooKeeper distributed cache, which implements multiple backups of status data and can be restored in time when failures or parameter damage occur. In addition, the ZooKeeper distributed cache avoids single point failures in Zookeeper itself. As long as more than half of the Zookeeper nodes survive, the normal operation of the master-slave hot switch can still be guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the module structure of a GNSS occultation detection ground application system according to an embodiment of the present application.

[0041] Figure 2 It is a flowchart of initializing and generating a data inversion task in one embodiment of the present application. DETAILED DESCRIPTION

[0042] The specific implementation of the present application is further described below in conjunction with the accompanying drawings.

[0043] This application discloses a GNSS occultation detection ground application system, which includes a task scheduling module, multiple task execution modules, ZooKeeper distributed cache and distributed file system. Please refer to Figure 1 ,The task scheduling module establishes communication connections with multiple task execution modules respectively. The task scheduling module also establishes communication connections with ZooKeeper distributed cache and distributed file system. Each task execution module also establishes communication connections with the distributed file system. In addition, the task scheduling module is also connected to multiple satellites in the GNSS occultation detection system, so that it can receive the original payload data sent by each satellite.

[0044] The GNSS occultation detection ground application system is built using multiple servers. The servers are connected to each other through a network switch to form a server cluster. The task scheduling module, task execution module, ZooKeeper distributed cache and distributed file system in the GNSS occultation detection ground application system are deployed on one or more servers respectively.

[0045] The GNSS occultation detection ground application system is used to perform data management tasks for the GNSS occultation detection system, including the following links:

[0046] 1. Task Generation

[0047] Among the data management tasks performed on the GNSS occultation detection system, the most important and most important data management task is the data inversion task. The data inversion task is driven by the original payload data of each satellite. When the original payload data of any satellite reaches the GNSS occultation detection ground application system, the GNSS occultation detection ground application system needs to perform data inversion to obtain the inversion result data, which is introduced as follows:

[0048] 1. Data inversion task

[0049] The task scheduling module receives the original payload data sent by each satellite of the GNSS occultation detection system, and parses the original payload data received each time to obtain positioning observation data (called POD package) and occultation observation data (called OCC package). The parsing process of the original payload data includes error detection, verification, unpacking, subpackaging, quality verification and rearrangement. This part can refer to the existing parsing method for the original payload data, and this embodiment will not be repeated.

[0050] Conventional practice is to directly perform data inversion on the resolved positioning observation data and occultation observation data. However, in reality, due to different sampling frequencies, satellite transit times, and data sizes, the starting and ending times of the positioning observation data and occultation observation data contained in the original payload data sent by the same satellite are often different, that is, there is a time misalignment. When performing data inversion, it is mainly necessary to use the data that is aligned in time between the positioning observation data and the occultation observation data. If there is less time-aligned data, it is easy to cause data inversion failure or low accuracy of the data inversion results.

[0051] Therefore, the present application does not directly perform data inversion on the positioning observation data and occultation observation data obtained by analysis, but first stores the positioning observation data obtained by analysis in a distributed file system. When it is detected that the amount of positioning observation data matching each set of occultation observation data in the distributed file system reaches the inversion requirement, the data inversion task is generated based on each set of occultation observation data and its matching positioning observation data, and then the generated data inversion task is added to the list of tasks to be executed. The list of tasks to be executed includes multiple tasks to be executed, and the list of tasks to be executed is maintained in the ZooKeeper distributed cache.

[0052] In actual execution, after receiving and parsing the original payload data sent by a satellite to obtain a set of occultation observation data, the task scheduling module will detect whether the amount of positioning observation data matching the currently parsed occultation observation data in the distributed file system meets the inversion requirements. There are two cases, please refer to Figure 2 The flowchart shown:

[0053] Case 1: When the amount of positioning observation data matching the currently parsed occultation observation data in the distributed file system reaches the inversion requirement, the positioning observation data matching the currently parsed occultation observation data is directly extracted from the distributed file system, and then the data inversion task is initialized based on the currently parsed occultation observation data and its matching positioning observation data and added to the list of tasks to be executed.

[0054] Case 2: When the amount of positioning observation data matching the currently parsed occultation observation data in the distributed file system does not meet the inversion requirements, the currently parsed occultation observation data and the satellite identifier of the satellite that sent the occultation observation data are stored in an unprocessed list maintained by the distributed file system.

[0055] Then the task scheduling module sequentially traverses each group of occultation observation data in the unprocessed list at a predetermined time interval, and when it detects that the amount of positioning observation data matching the traversed occultation observation data in the distributed file system reaches the inversion requirement, it initializes the generation of the corresponding data inversion task and deletes the traversed occultation observation data from the unprocessed list, otherwise the traversed occultation observation data is retained in the unprocessed list. The predetermined time interval can be customized.

[0056] Regardless of the above situation, when the task scheduling module stores the parsed positioning observation data in the distributed file system, it stores each positioning frame and its positioning time in each parsed positioning observation data, as well as the satellite identifier of the satellite that sent the positioning observation data in the distributed file system. Each satellite has a unique satellite identifier.

[0057] Then the method in which the task scheduling module detects whether the amount of positioning observation data matching each set of occultation observation data in the distributed file system meets the inversion requirements includes: detecting the number of positioning frames matching the occultation observation data in the distributed file system. The positioning frames matching the set of occultation observation data are positioning frames corresponding to the target satellite identifier and the corresponding positioning time within the duration of the set of occultation observation data. The duration of a set of occultation observation data is the period between the start time and the end time of the set of occultation observation data, and the target satellite identifier is the satellite identifier of the satellite that sent the set of occultation observation data. Therefore, the positioning frames matching the set of occultation observation data are positioning frames that come from the same satellite as the set of occultation observation data and whose positioning time is within the duration of the set of occultation observation data.

[0058] When it is detected that the number of positioning frames matching the occultation observation data reaches the quantity threshold, it is determined that the data volume of the positioning observation data matching the occultation observation data meets the inversion requirement, and the positioning frames matching the occultation observation data are extracted as the positioning observation data matching the occultation observation data, otherwise it is determined that the data volume of the positioning observation data matching the occultation observation data does not meet the inversion requirement. The quantity threshold can be set by user, and the quantity threshold is generally positively correlated with the data volume of the occultation observation data. In one embodiment, the total number of positioning moments N contained in the duration of a group of occultation observation data is determined, and N*η is used as the quantity threshold, η is a proportional coefficient less than 1, and the closer the value of η is to 1, the more the number of positioning frames matching the occultation observation data is generated. For example, if the positioning moment is in seconds, the total number of positioning moments N contained in a group of occultation observation data with a duration of 2 hours is 7200, and η=0.9 is taken, then the quantity threshold is taken as 6480.

[0059] 2. Other data management tasks

[0060] In addition to data inversion tasks, data management tasks performed on GNSS occultation detection systems also include other data management tasks. Another common type is scheduled tasks, which are tasks that need to be performed at a scheduled time. According to actual business scenarios, scheduled tasks include data backup tasks, data export tasks, data integrity check tasks, data clearing tasks, data subscription tasks, and data push tasks.

[0061] A scheduled task list is maintained in the distributed file system. The scheduled task list includes multiple scheduled tasks and their corresponding scheduled start conditions. The content and scheduled start conditions of each scheduled task are custom settings.

[0062] When the task scheduling module detects that the timed start condition of any timed task in the timed task list is met, it initializes and generates the timed task that meets the timed start condition and adds it to the list of tasks to be executed.

[0063] In addition to data inversion tasks and scheduled tasks, there is another type of data management task that is manually triggered. When the task scheduling module receives a manual trigger instruction, it initializes and generates a manual trigger task according to the manual trigger instruction and adds it to the list of tasks to be executed. Manually triggered tasks are generally urgent tasks that need to be executed temporarily. Some failed tasks may also be retriggered by the on-duty personnel.

[0064] In actual application, each satellite will send multiple original payload data, so multiple data inversion tasks will be initialized for each satellite. The GNSS occultation detection system includes multiple satellites, so the amount of data inversion tasks is even larger. In addition, there are various scheduled tasks and manually triggered tasks that cannot be determined when they will be triggered. The amount of data management tasks that need to be performed is relatively large, and the requirements for the importance of different tasks are also different. Therefore, it is necessary to schedule each task appropriately. From the above introduction, it can be seen that each task to be executed generated by the task scheduling module is first added to the list of tasks to be executed.

[0065] In addition, each task generated by initialization has its own importance. The higher the importance, the more priority the task needs to be executed. The importance of each task can be customized. Generally, the importance of manually triggered tasks is often the highest.

[0066] 2. Task Allocation

[0067] Then the task scheduling module will assign each task in the task list to the corresponding task execution module according to the resource occupancy of each task execution module recorded in the execution module status list and the scheduling priority of each task in the task list to be executed. The execution module status list is maintained in the ZooKeeper distributed cache. The execution module status list records the resource occupancy of each task execution module. The resource occupancy includes the total resources of the task execution module and the resources that have been occupied. The resources occupied by a task execution module include the resources occupied by the operating system of the task execution module and the resources occupied by other tasks. Each task execution module integrates multiple CPU cores, and the resources of a task execution module include CPU cores and memory.

[0068] The execution module status list is dynamically maintained. When a new task execution module is connected to the GNSS occultation detection ground application system, the task execution module will report the resource occupancy to the task scheduling module, and the task scheduling module will add the newly connected task execution module and its corresponding resource occupancy to the execution module status list. When a task execution module fails or crashes, the task execution module will delete the abnormal task execution module and its resource occupancy from the execution module status list.

[0069] When the task scheduling module assigns tasks in the task list to be executed to the task execution module, the tasks in the task list to be executed are assigned in descending order of scheduling priority. The scheduling priority of each task in the task list to be executed is determined according to the importance of the task, the creation time, and the waiting time for the task to be scheduled. The higher the importance of the task, the earlier the creation time, and the longer the waiting time for scheduling, the higher the scheduling priority of the task. The specific method for determining the scheduling priority can be customized.

[0070] When allocating each task, the idle resources of each task execution module are determined according to the resource occupancy of each task execution module recorded in the execution module status list, the resource requirements of the task to be allocated are determined, and the task to be allocated is allocated to the task execution module whose idle resources meet the resource requirements of the task.

[0071] 3. Execution of Tasks

[0072] Each task execution module is used to execute the received task. When executing the data inversion task, the task execution module performs data inversion on the occultation observation data and its matching positioning observation data in the data inversion task to obtain the inversion result data, and stores the inversion result data in the distributed file system. The specific data inversion method can refer to the existing practice, and this embodiment will not be repeated.

[0073] The data inversion task is a computationally intensive task. In one embodiment, for each task execution module integrated with multiple CPU cores, the task execution module reserves several CPU cores as basic cores for running the operating system of the task execution module, and uses the isolcpus command to isolate the CPU cores other than the basic cores. When receiving the data inversion task, the task execution module selects a CPU core from the isolated CPU cores, and uses the taskset command to control the execution of the received data inversion task on the selected CPU core. This allows the data inversion task to run on an independent CPU core, avoiding frequent context switching between multiple CPU cores. Compared with sharing computing resources between multiple CPU cores, the execution speed will be faster and the response time will be shorter.

[0074] When executing a data inversion task, the task execution module will start multiple processes to execute the data inversion task in parallel. During the execution of the data inversion task, the output information of each process used to execute the data inversion task will be monitored and recorded in the distributed file system. If the output information of the data inversion task is not read for more than the predetermined time, the data inversion task may be stuck or exit unexpectedly, and the data inversion task needs to be closed. Since the data inversion task is processed by multiple processes in parallel, closing a process alone cannot ensure that all processes executing the data inversion task are closed, which will result in idle processes left on the task execution module, resulting in resource waste.

[0075] To solve this problem, when the task execution module starts multiple processes to execute a data inversion task in parallel, it defines a unique environment variable for the data inversion task and copies it to each process. One way to define the environment variable is to assign a task identifier JobId to each data inversion task, and then when starting multiple processes to execute the data inversion task in parallel, pass the task identifier JobId and the task start time startTime of the data inversion task into the command exportSKY_LUCAS_ENV=SKY@${JobId}@${startTime} to define the environment variable and copy it to each process.

[0076] When the task execution module detects that any process fails during the data inversion task, it closes the failed process and other processes with the same environment variables as the failed process, so as to close all processes executing the same data inversion task. The specific operation is that when a process fails, the ps command of the Linux system is called to obtain a list of all processes, traverse each process, execute the "cat / proc / ${pid} / environ" command to obtain its output, and check whether there is a SKY_LUCAS_ENV environment variable containing @${JobId}@. If there is, it means that the traversed process and the failed process are used to execute the same data inversion task, and they are also closed. In this way, all processes started by the same data inversion task can be closed.

[0077] When the task execution module executes other tasks, it operates according to the task content, which will not be described in detail in this embodiment.

[0078] Regardless of the type of task being executed, each task execution module will periodically feed back the task execution status to the task scheduling module. The task scheduling module will update the resource usage of each task execution module recorded in the execution module status list according to the task execution status fed back by each task execution module.

[0079] When the task scheduling module reaches the waiting time after assigning the task to the task execution module and does not receive the task execution status feedback from the task execution module, it is determined that the task execution module has timed out, and the task is reassigned to the task execution module. When the number of repetitions of assigning the same task to the same task execution module reaches the number threshold, it is determined that the task execution module is abnormal, the task execution module is deleted from the execution module status list, and the task is reassigned to other task execution modules.

[0080] It can be seen from the above functional introduction that the task scheduling module is the focus of the GNSS occultation detection ground application system of this application, and the task scheduling module is deployed on the physical node. The physical node here is a single or multiple servers. In the real network environment, once the physical node where the task scheduling module is deployed fails, the function of the GNSS occultation detection ground application system will be seriously damaged. In order to improve the reliability of the GNSS occultation detection ground application system, the task scheduling module adopts a master-slave redundant design. The task scheduling modules are deployed on two physical nodes respectively. The real IP addresses of the two physical nodes are virtualized into the virtual IP address of the task scheduling module through VRRP (Virtual Router Redundancy Protocol) technology. The task scheduling module provides services to the outside world through the virtual IP address. At the same time, only one physical node (this physical node is called the master node) has control over the virtual IP, and the master node provides the services of the task scheduling module to the outside world. The operation process is as follows:

[0081] (1) Each physical node sends its own node attribute information to another physical node when it starts. The node attribute information of each physical node includes the startup time, version number and node host name of the physical node.

[0082] (2) Each physical node determines its own control priority based on its own node attribute information, and determines the control priority of another physical node based on the node attribute information received from another physical node. Each physical node determines the physical node with the highest control priority as the primary node, and the other physical node as the backup node. The earlier the startup time of each physical node, the smaller the version number, and the higher the dictionary sorting of the node host name, the higher the control priority of the physical node. The specific method for determining the control priority can be customized.

[0083] (3) The master node has control over the virtual IP address of the task scheduling module and provides external services for the task scheduling module. In addition, the master node and the backup node perform heartbeat communication, including: the master node sends a heartbeat packet to the backup node, and the backup node returns a heartbeat packet to the master node in response to the heartbeat packet of the master node. The heartbeat packet sent by each physical node includes the role information, node attribute information, and status information of the physical node, etc. The role information indicates whether the physical node is a master node or a backup node.

[0084] (4) When any physical node detects that the duration of the disconnection of heartbeat communication with another physical node reaches the duration threshold, it may be caused by a network failure in the physical node itself or in another physical node on the other end. At this time, the physical node first detects whether it can communicate with the gateway.

[0085] When the physical node determines that it can communicate with the gateway, it can be determined that another physical node has a network failure. When the physical node itself is currently the master node, it continues to provide external services of the task scheduling module until it receives node attribute information sent by another physical node after restoring network connection, and then restores heartbeat communication with the other physical node.

[0086] When the physical node determines that it can communicate with the gateway and is currently a backup node, it can be determined that the current master node has a network failure. In order to ensure that the function of the task scheduling module is not affected, the physical node automatically switches from the backup node to the master node to have control over the virtual IP address of the task scheduling module and provide external services for the task scheduling module. And the physical node will resume the heartbeat communication with another physical node when it receives the node attribute information sent by another physical node after the network connection is restored.

[0087] When the physical node determines that it cannot communicate with the gateway, it means that a network failure occurs in the physical node itself, and then the physical node restarts, and resends its own node attribute information to another physical node after the restart.

[0088] Therefore, by using the virtual routing redundancy protocol, dynamic switching of the master and backup nodes is achieved. Through the master-backup mechanism, the entire GNSS occultation detection ground application system can be ensured to operate continuously and stably.

[0089] The above is only a preferred embodiment of the present application, and the present application is not limited to the above embodiments. It is understood that other improvements and changes directly derived or associated by those skilled in the art without departing from the spirit and concept of the present application should be considered to be included in the protection scope of the present application.

Claims

1. A GNSS occultation detection ground application system, characterized in that: The GNSS occultation detection ground application system includes a task scheduling module, multiple task execution modules, a ZooKeeper distributed cache and a distributed file system, wherein the ZooKeeper distributed cache maintains a list of tasks to be executed and a list of execution module status; The task scheduling module is used to receive the original payload data sent by each satellite in the GNSS occultation detection system, and after parsing the original payload data received each time to obtain positioning observation data and occultation observation data, store each positioning frame and its positioning time in the positioning observation data obtained each time, and the satellite identifier of the satellite that sends the positioning observation data in the distributed file system, and detect the number of positioning frames matching the occultation observation data currently parsed in the distributed file system, the positioning frame matching the occultation observation data is a positioning frame corresponding to the target satellite identifier and the corresponding positioning time within the duration of the occultation observation data, and the target satellite identifier is the satellite identifier of the satellite that sends the occultation observation data; when it is detected that the number of positioning frames matching the occultation observation data reaches a quantity threshold, determine the number of positioning frames matching the occultation observation data. The data amount of the matched positioning observation data meets the inversion requirement, and all positioning frames matching the occultation observation data are extracted to obtain the positioning observation data matching the occultation observation data; otherwise, it is determined that the data amount of the positioning observation data matching the occultation observation data does not meet the inversion requirement; when the data amount of the positioning observation data matching the occultation observation data currently parsed in the distributed file system meets the inversion requirement, a data inversion task is generated based on the occultation observation data currently parsed and its matching positioning observation data, and added to the list of tasks to be executed; when the data amount of the positioning observation data matching the occultation observation data currently parsed in the distributed file system does not meet the inversion requirement, the occultation observation data currently parsed and the satellite identifier of the satellite that sent the occultation observation data are correspondingly stored in the unprocessed list maintained by the distributed file system; The task scheduling module is further used to sequentially traverse each group of occultation observation data in the unprocessed list at predetermined time intervals, and when it is detected that the amount of positioning observation data matching the traversed occultation observation data in the distributed file system reaches the inversion requirement, initialize and generate the corresponding data inversion task and add it to the list of tasks to be executed, and delete the traversed occultation observation data from the unprocessed list, otherwise the traversed occultation observation data is retained in the unprocessed list; The task scheduling module is further used to assign each task in the to-be-executed task list to a corresponding task execution module according to the resource occupancy of each task execution module recorded in the execution module status list and the scheduling priority of each task in the to-be-executed task list; Each task execution module is used to execute the received task and feed back the task execution status to the task scheduling module; When executing the data inversion task, the task execution module performs data inversion on the occultation observation data and the matching positioning observation data in the data inversion task to obtain inversion result data, and stores the inversion result data in the distributed file system; The task scheduling module is further used to update the resource occupancy of each task execution module recorded in the execution module status list according to the task execution status fed back by each task execution module.

2. The GNSS occultation detection ground application system according to claim 1, characterized in that: Each task execution module integrates multiple CPU cores, and each task execution module reserves several CPU cores as basic cores for running the operating system of the task execution module, and uses the isolcpus command to isolate other CPU cores except the basic cores; Each task execution module is further used to select a CPU core from the isolated CPU cores when receiving a data inversion task, and use a taskset command to control the received data inversion task to be executed on the selected CPU core.

3. The GNSS occultation detection ground application system according to claim 1, characterized in that: Each task execution module is used to define a unique environment variable for the data inversion task and copy it to each process when multiple processes are started to execute the data inversion task in parallel; Each task execution module is also used to close the failed process when it is detected that any process fails during the execution of the data inversion task, and close other processes with the same environment variables as the failed process to close all processes executing the same data inversion task.

4. The GNSS occultation detection ground application system according to claim 1, characterized in that: The task scheduling module is deployed on two physical nodes respectively. The real IP addresses of the two physical nodes are virtualized into the virtual IP address of the task scheduling module through VRRP technology. The task scheduling module provides services to the outside through the virtual IP address. Each physical node sends its own node attribute information to another physical node when it starts up. Each physical node determines its own control priority based on its own node attribute information, and determines the control priority of another physical node based on the node attribute information received from another physical node. Each physical node determines the physical node with the highest control priority as the master node and the other physical node as the backup node. The master node has control over the virtual IP address of the task scheduling module and provides external services for the task scheduling module, and the master node performs heartbeat communication with the backup node; When any physical node detects that the duration of disconnection of heartbeat communication with another physical node reaches a duration threshold, the physical node detects whether it can perform gateway communication; When the physical node determines that it can perform gateway communication and is currently the master node, the physical node continues to provide external services of the task scheduling module until it receives node attribute information sent by another physical node and then resumes heartbeat communication with the other physical node; When the physical node determines that it can perform gateway communication and is currently a backup node, the physical node switches to a master node to have control over the virtual IP address of the task scheduling module and provide external services of the task scheduling module, and the physical node resumes heartbeat communication with another physical node after receiving node attribute information sent by another physical node; When the physical node determines that the gateway communication cannot be performed, it restarts, and resends its own node attribute information to another physical node after the restart.

5. The GNSS occultation detection ground application system according to claim 4, The node attribute information of each physical node includes the startup time, version number and node host name of the physical node. The earlier the startup time of the physical node, the smaller the version number and the higher the dictionary sorting of the node host name, the higher the control priority of the physical node.

6. The GNSS occultation detection ground application system according to claim 1, characterized in that: The task scheduling module assigns each task in the to-be-executed task list to a corresponding task execution module, including: The tasks in the to-be-executed task list are sequentially assigned in descending order of scheduling priority. When each task is assigned, the idle resources of each task execution module are determined according to the resource occupancy of each task execution module recorded in the execution module status list, the resource requirements of the to-be-assigned tasks are determined, and the to-be-assigned tasks are assigned to the task execution modules whose idle resources meet the resource requirements of the tasks; Among them, the scheduling priority of each task in the list of tasks to be executed is determined according to the importance of the task, the creation time of the task, and the waiting scheduling time of the task; the higher the importance of the task, the earlier the creation time, and the longer the waiting scheduling time, the higher the scheduling priority of the task.

7. The GNSS occultation detection ground application system according to claim 1, characterized in that: The task scheduling module is further configured to add the task execution module and its corresponding resource occupancy status to the execution module status list when a new task execution module is detected to be connected; The task scheduling module is also used to reallocate the task to the task execution module when a waiting period has expired after the task is assigned to the task execution module and no task execution status is received from the task execution module; when the number of repetitions of assigning the same task to the same task execution module reaches a threshold number, it is determined that the task execution module is abnormal, the task execution module is deleted from the execution module status list, and the task is reallocated to other task execution modules.

8. The GNSS occultation detection ground application system according to claim 1, characterized in that: The task scheduling module is further used to initialize and generate a scheduled task that meets the scheduled start condition and add it to the list of tasks to be executed when it is detected that the scheduled start condition of any scheduled task in the scheduled task list is met; The scheduled task list is stored in the distributed file system, and the scheduled task list includes a plurality of scheduled tasks and their corresponding scheduled start conditions; The task scheduling module is further configured to, when receiving a manual trigger instruction, initialize and generate a manual trigger task according to the manual trigger instruction and add the task to the list of tasks to be executed.

Images