Distributed high-performance resolving method for large-scale GNSS network

Abstract

The invention provides a distributed high-performance resolving method for a large-scale GNSS (Global Navigation Satellite System) network. The distributed high-performance resolving method comprises the following steps of S1, setting project resolving parameters; s2, according to the total number of single-day baseline resolving tasks, selecting to adopt a distributed-single node parallel mode or a distributed-cluster parallel mode; s3, uploading the data to be solved; s4, resolving by adopting a corresponding mode; s5, performing quality inspection on the single-day baseline solution; s6, analyzing a time sequence; s7, carrying out net adjustment analysis; and S8, obtaining a calculation result. According to the distributed high-performance resolving method for the large-scale GNSS network, the computing power of a computer is deeply excavated, the advantages of high efficiency of a shared memory system and high flexibility of a distributed system are fully played, centralized parallel and distributed parallel collaborative operation is realized, and the production efficiency and the computing resource utilization rate are greatly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of high-precision global satellite navigation and positioning, and in particular relates to a distributed high-performance solution method for a large-scale GNSS network. Background Art

[0002] With the full completion of my country's Beidou Navigation Satellite System (BDS) and the mutual compatibility and modernization of multi-system GNSS technology, GNSS has entered the era of multi-system multi-frequency signals with tens of thousands of global reference stations and more than 100 navigation satellites. The total amount of data stored in the CDDIS (Crustal Dynamics Data Information System) data center has exceeded 20TB. In recent years, in order to meet the needs of real-time positioning, disaster prevention, plate movement analysis, ecological environment monitoring and other applications, the scale of continuously operating base station networks in many countries and regions around the world has continued to expand, resulting in a massive increase in the scale of GNSS data. The sharp increase in the amount of multi-frequency and multi-mode GNSS data helps to improve the solution accuracy of the product on the one hand, but also greatly increases the complexity of the model and the calculation time on the other.

[0003] In recent years, with the continuous development of GNSS technology, observation data has shown the characteristics of real-time, high precision, and large time and space span. From the perspective of spatial scale, the data can cover not only local areas, but also a country or the world. From the perspective of time scale, it can range from seconds, hours, days to years or even decades. As the number of base stations continues to rise, the scale of the GNSS network continues to expand, and the amount of data is increasing dramatically, the demand for data processing in large-scale GNSS networks continues to increase, and the timeliness of data processing faces huge challenges. The problems exposed by traditional serial and multi-window baseline solution methods are becoming more and more prominent, and it is becoming increasingly difficult to meet the strong timeliness needs of national and regional modern surveying and mapping benchmarks. There is an urgent need to study new data processing solutions. Specifically, it is manifested in:

[0004] (1) Low timeliness of baseline solution

[0005] At present, the traditional method of large-scale GNSS network baseline solution in data centers is a semi-automatic and fragmented operation method in which a single machine manually opens multiple windows. It is more based on the work experience of technicians rather than scientific high-performance computing theories and models, has certain limitations, and the timeliness of baseline solution is low. On the one hand, it is impossible to fully release the computing power of a single computer. On the other hand, it does not have the ability to work together with multiple computers. With the continuous increase in the amount of GNSS network data and the number of stations, the traditional operation method has become increasingly difficult to meet the strong current needs of baseline solution.

[0006] (2) Low computing resource utilization

[0007] The number of windows opened manually in the traditional way is very blind, and it is impossible to fully tap and repeatedly schedule computer resources, resulting in a large amount of idleness and waste of computer resources in the data center. An unreasonable number of windows will not only make the baseline solution inefficient, but also easily cause jams and crashes.

[0008] (3) Low degree of automation

[0009] Before solving the baseline in the traditional way, the subnet or subtask needs to be manually fragmented to form several isolated sub-projects. After the baseline solution of each sub-project is completed, the solution results are manually merged one by one. Not only is the operation cumbersome and the degree of automation low, but it is also easy to make mistakes and rework. Summary of the invention

[0010] In view of the current massive large-scale GNSS network, the traditional single-machine centralized processing mode and the solution that simply relies on increasing single-node configuration can no longer meet the needs.

[0011] To achieve the above object, the present invention provides a large-scale GNSS network distributed high-performance solution method, comprising the following steps:

[0012] S1. Set project solution parameters;

[0013] S2. According to the total number of baseline solution tasks for a single day, choose to use the distributed-single-node parallel mode or the distributed-cluster parallel mode;

[0014] S3, upload the data to be solved;

[0015] S4, using the corresponding mode to solve;

[0016] S5. Perform quality check on the single-day baseline solution;

[0017] S6, time series analysis;

[0018] S7, network adjustment analysis;

[0019] S8. Obtain solution results.

[0020] Furthermore, in the S1, setting project solution parameters, the parameters include project name, GNSS network level, solution constellation, coordinate system, data sampling rate, subnet division method, baseline solution, network adjustment, etc.

[0021] Furthermore, the specific process of step S2, selecting whether to adopt the distributed-single-node parallel mode or the distributed-cluster parallel mode according to the total number of baseline solution tasks in a single day, is: the distributed-single-node parallel mode is suitable for large-scale GNSS networks that can be supported by the computing power of a single node and whose total number of baseline solution tasks in a single day does not exceed 800, and the distributed-cluster parallel mode is suitable for ultra-large-scale GNSS networks that cannot be supported by the computing power of a single node and whose total number of baseline solution tasks in a single day is greater than 800.

[0022] Furthermore, the S4 is solved using a corresponding mode; if a distributed-single-node parallel mode is used, the specific process is as follows:

[0023] S401, the client configures distributed single-node computing parameters and baseline solution strategy parameters;

[0024] S402, the client uploads the data to be solved to the server based on the TCP protocol network socket technology:

[0025] S403, the server checks the timeliness of the observation data and tables files;

[0026] S404, the server divides the subnets according to the subnet partition list and the corresponding observation data and table files, and generates a parallel computing file system;

[0027] S405, the server determines the number of processes in the initialization process pool according to the number of tasks (i.e., the number of time periods) solved by the daily baseline, the number of logical cores of the server CPU (automatically identified by the program), and the distributed single-node computing parameters;

[0028] S406. The server submits the task of single-day baseline solution and high-performance solution parameters to the dynamically expandable process pool, and performs parallel solution on the baselines of each subnet to obtain a single-day baseline solution for each subnet.

[0029] Further, in S406, the server submits the task of single-day baseline solution and high-performance solution parameters to the dynamically scalable process pool, and performs parallel solution on the baselines of each subnet to obtain the single-day baseline solution of each subnet. The specific steps are as follows:

[0030] S501, initialization process pool:

[0031] The process pool creates a main process (management process) and a corresponding number of child processes (resource processes) according to the preset number of concurrent processes; the main process applies for space to store data structures, manage the status of child processes (such as idle, busy) and related information (such as process ID, pipeline file descriptor, etc.); considering that the local computer may have other task requirements besides large-scale GNSS network baseline solution, in order to take into account the computing power needs of both, the optimal number of processes P is calculated as follows:

[0032]

[0033] Among them, P is the optimal number of processes, C is the number of logical cores of the assumed computer, N is the total number of baseline solution tasks in a single day, and S is the preset number of concurrency;

[0034] S502, monitoring task:

[0035] The main process or a specific task distributor is responsible for monitoring the daily baseline solution task queues of each subnet; when a new task arrives, it is placed in the task queue;

[0036] S503, task allocation:

[0037] The main process (or task distributor) checks the status of the child processes in the process pool, finds the idle child process through inter-process communication (such as pipes, message queues, etc.), and assigns the task to the idle child process;

[0038] S504, subprocess execution task:

[0039] The subprocess receives the single-day baseline solution task information from the main process, and performs file reading and writing, baseline solution and other operations according to the set parameters; the subprocess can update its status (such as from idle to busy) during the execution of the baseline solution task, and return to the idle state after the task is completed;

[0040] S505, task completion and result return:

[0041] After the subprocess completes the task, it returns the single-day baseline solution result to the main process through file writing, message sending, etc., and notifies the main process that it has completed the task and can receive new tasks through pipeline writing operations, signal sending, etc.

[0042] S506, process pool resource recovery and shutdown

[0043] When all single-day baseline solution tasks are completed, the main process is responsible for recovering the sub-process resources of the completed tasks, including closing file descriptors, releasing memory, etc.; the main process closes the process pool and waits for all sub-processes to exit; ensuring that all resources are released correctly to avoid resource leakage.

[0044] Furthermore, the S4 is solved using a corresponding mode; if distributed-cluster parallelism is used, the specific process is as follows:

[0045] S601, the client configures distributed cluster computing parameters and baseline solution strategy parameters;

[0046] S602, the client uploads the data to be solved to the master node of the cluster based on the network socket technology of the TCP protocol:

[0047] S603, the master node checks the timeliness of the observation data and tables files;

[0048] S604, the master node divides the subnets according to the subnet partition list and the corresponding observation data and table files, and generates a parallel computing file system;

[0049] S605, the master node configures the optimal computing throughput of the master and slave nodes of the cluster system according to the number of tasks (i.e., the number of time periods) solved by the single-day baseline and the distributed cluster computing parameters;

[0050] S606: Submit the task of single-day baseline solution and high-performance solution parameters to the high-performance solution cluster. The master node cooperates with the slave nodes to perform distributed parallel solution on the baseline of each subnet to obtain a single-day baseline solution for each subnet.

[0051] Further, in S605, the master node configures the optimal computing throughput P of the master and slave nodes of the cluster system according to the number of tasks (ie, the number of time periods) solved by the single-day baseline and the distributed cluster computing parameters as follows:

[0052]

[0053] Among them, C m is the number of logical cores of the master node (Master) in the cluster system, T is the number of slave nodes (Worker) in the cluster system, C i is the number of logical cores of the ith slave node, C' j is the number of concurrent processes when the jth slave node is not fully loaded. In particular, when j = 1, let N is the total number of baseline solution tasks in a single day, and P is the optimal computing throughput of the cluster system.

[0054] Further, the S606, submitting the task of single-day baseline solution and high-performance solution parameters to the high-performance solution cluster, the master node cooperates with the slave node to perform distributed parallel solution on the baseline of each subnet to obtain the single-day baseline solution of each subnet, the specific steps are:

[0055] S701, master and slave nodes synchronously solve data:

[0056] The master node (Master) sends the solution data to each slave node (Worker) synchronously in a broadcast aggregation communication mode (master-slave replication mode); the operations in this process include: each slave node starts the data transmission service on its own designated port; the master node starts multiple data transmission services on multiple ports in a multi-process parallel manner according to the number of slave nodes; the master node sends the solution data to each slave node on the corresponding port at the same time;

[0057] S702, task monitoring and heartbeat monitoring:

[0058] After the solution data is uploaded to each Worker, the client uses RPC technology to remotely call the baseline solution function of the Master to process the data. At the same time, the Master synchronizes the baseline solution function to each Worker and calls it to achieve collaborative parallel computing between the Master and the Worker. During the collaborative parallel computing process, the Master performs heartbeat monitoring on each Worker, monitors the working status of each Worker in real time, and gives feedback on abnormalities.

[0059] S703, task allocation and load balancing:

[0060] A smooth weighted round-robin load balancing algorithm is used to implement the allocation of solution tasks. The specific implementation method is as follows: the load balancer first assigns a weight value to each computing node (including the master node and the slave node) (the present invention determines the weight according to the number of CPU logical cores of each node), and the weight reflects the processing capacity of the master and slave nodes; computing nodes with higher weights will be assigned more solution tasks, and computing nodes with lower weights will be assigned fewer solution tasks;

[0061] S704, each node performs tasks and dynamic load balancing:

[0062] The process of each computing node receives the single-day baseline solution task information and performs file reading and writing, baseline solution and other operations according to the set parameters. After each node completes part of the solution task, it notifies the load balancer through pipeline write operations, signal sending and other mechanisms that it has completed the task and can receive new tasks. During the execution of the solution task, the load balancer will continuously check the computing resource load status of each node, find an idle node, and dynamically allocate the solution task to the idle node.

[0063] S705, task completion and result return:

[0064] After the master node monitors the completion of all baseline solution tasks of each node through inter-process communication (such as pipelines, message queues, etc.), it calls the data transmission service running on each slave node to download and merge the single-day baseline solution results of each node (including master and slave nodes) in a collected aggregate communication mode;

[0065] S706, process data cleaning:

[0066] After the daily baseline solution is completed, the master node remotely calls the data cleaning module of each slave node through RPC to complete the process data cleaning work.

[0067] Further, the S5, quality check of the single-day baseline solution includes standardized root mean square error NRMS, baseline repeatability R, point position and baseline component mean error and relative mean error;

[0068] Among them, the standardized root mean square error NRMS calculation formula is as follows:

[0069]

[0070] Where Y i is the calculated value for each baseline vector epoch, Y is the true value of the baseline vector, N is the total number of epochs, and n is the number of baselines. is the variance of the solution value at each epoch; if the NRMS value is less than the set threshold, the baseline solution is considered qualified, otherwise it is unqualified;

[0071] The calculation formula of baseline repeatability R is as follows:

[0072]

[0073] Where n is the total number of observation periods of the same baseline; C i is the baseline component or side length of a period; is the C corresponding to the i period i Variance of the component; C m It is the weighted average of each period.

[0074] Furthermore, the specific process of S6, time series analysis, is: coordinate time series analysis is performed on the single-day baseline solution of each subnet, a single-day solution time series diagram is generated, steps are judged and analyzed, and gross errors are eliminated; the specific process of S7, network adjustment analysis, is: network adjustment joint processing is performed on the single-day baseline of each subnet after time series analysis, and the coordinates, speed and baseline results of each measuring station are obtained; according to the project GNSS network level requirements, the chi-square value, each component of the adjustment baseline and the relative mean error and other indicators are checked.

[0075] The advantages of the present invention are as follows: the present invention provides a large-scale GNSS network distributed high-performance solution method, proposes a large-scale GNSS network baseline solution multi-core parallel and network multi-node parallel "time and space integration" double-layer and three-layer data parallel solution, deeply taps the computer computing power, gives full play to the high efficiency of the shared memory system and the high scalability of the distributed system, realizes centralized parallel and distributed parallel collaborative operation, and greatly improves production efficiency and computing resource utilization. At the same time, the solution integrates a powerful quality inspection function, realizing "both fast and accurate calculation".

[0076] The use of distributed and cluster computing technologies breaks through the limitations of traditional single-machine centralized processing and the simple reliance on adding single-node resource configuration solutions, supports remote service calls between different nodes and "load balancing" of computing resources between nodes, effectively avoiding huge waste of computing resources due to "uneven busyness". At the same time, the number of nodes can be flexibly expanded according to business needs, achieving the maximum utilization of existing equipment and reducing and saving investment costs.

[0077] The present invention is described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a schematic diagram of the technical solution for the high-performance computing platform of large-scale GNSS network.

[0079] Figure 2 This is a schematic diagram of the distributed-single-node parallel mode.

[0080] Figure 3 This is a schematic diagram of the distributed-cluster parallel mode. DETAILED DESCRIPTION

[0081] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose, the specific implementation methods, structural features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and embodiments.

[0082] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0083] In the description of the present invention, it should be understood that the terms "center", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "aligned", "overlapping", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. The terms "first" and "second" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features; in the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0084] Example 1

[0085] The purpose of the present invention is to use new high-performance computing technologies such as parallel computing, distributed computing, and cluster computing to overcome the technical pain points of poor timeliness and low computing resource utilization in large-scale GNSS network baseline solution, and propose a "time-space integration" double-layer (DistriSmp) and three-layer (DistriCluster) data parallel solution for multi-core parallel and network multi-node parallel large-scale GNSS network baseline solution, which deeply taps the computing power of computers, gives full play to the high efficiency of shared memory systems and the high scalability advantages of distributed systems, realizes centralized parallel and distributed parallel collaborative operations, and greatly improves production efficiency and computing resource utilization. At the same time, the method integrates a powerful full-process quality inspection function, realizing "both fast and accurate calculations".

[0086] This embodiment provides a Figure 1 The large-scale GNSS network distributed high-performance solution method shown includes the following steps:

[0087] S1. Set project solution parameters;

[0088] S2. According to the total number of baseline solution tasks for a single day, choose to use the distributed-single-node parallel mode or the distributed-cluster parallel mode;

[0089] S3, upload the data to be solved;

[0090] S4, using the corresponding mode to solve;

[0091] S5. Perform quality check on the single-day baseline solution;

[0092] S6, time series analysis;

[0093] S7, network adjustment analysis;

[0094] S8. Obtain solution results.

[0095] Furthermore, in the S1, setting the project solution parameters, the parameters include project name, GNSS network level, solution constellation, coordinate system, data sampling rate, subnet division method, baseline solution, network adjustment, etc. The subnet division method generally adopts the subnet division method in the prior art: three methods: by regional geographic boundary line, global uniform thinning and site list.

[0096] In addition, other related parameter settings are as follows:

[0097] 1) Satellite orbit: using IGS precise ephemeris;

[0098] 2) Satellite cut-off altitude angle: 10 degrees;

[0099] 3) Epoch interval: 30 seconds;

[0100] 4) Observation value: combined observation value after eliminating the ionosphere;

[0101] 5) Tropospheric correction model: Saastamoinen model is used for standard meteorological correction;

[0102] 6) Solar radiation pressure correction: BERNE model;

[0103] 7) Tropospheric delay model: first-order Gauss-Markov;

[0104] 8) Baseline solution coordinate constraints: The starting point is given a constraint of 5cm in the horizontal direction and 10cm in the vertical direction, and the point to be solved is given a constraint of 10m;

[0105] 9) Solid tide model: IERS2010 (IERS2010 solid tide model continues to use IERS2003 standard);

[0106] 10) Tidal model: FES series tidal model (FES2004);

[0107] 11) Tropospheric mapping function model: VMF1;

[0108] 12) Number of zenith delay parameters: 13.

[0109] Furthermore, the specific process of step S2, selecting whether to adopt the distributed-single-node parallel mode or the distributed-cluster parallel mode according to the total number of baseline solution tasks in a single day, is: the distributed-single-node parallel mode is suitable for large-scale GNSS networks that can be supported by the computing power of a single node and whose total number of baseline solution tasks in a single day does not exceed 800, and the distributed-cluster parallel mode is suitable for ultra-large-scale GNSS networks that cannot be supported by the computing power of a single node and whose total number of baseline solution tasks in a single day is greater than 800.

[0110] Further, the S4 is solved by using a corresponding mode;

[0111] Baseline solution mathematical model

[0112] The double difference phase observation equation for baseline solution is:

[0113]

[0114] Where 1 and 2 represent L1 and L2 carrier phase observations, respectively, and the unit of observation is week;

[0115] f1 is the L1 carrier frequency;

[0116] f2 is the L2 carrier frequency;

[0117] τij is the geometric propagation delay time of the carrier signal between the satellite and the receiver;

[0118] φ ij clock (t j ) is the phase change caused by the satellite and receiver clock differences;

[0119] φ ij trop is the tropospheric refraction delay on the carrier signal propagation path;

[0120] k ij It is the influence of ionospheric refraction;

[0121] v ij is the measurement error and the residual error that is not modeled (such as multipath effects);

[0122] b ij =n ij +δφ j -δφ i , where n ij is the integer ambiguity, δφ j is the initial phase deviation of the receiver, δφ i is the initial phase deviation of the satellite.

[0123] The commonly used combination to eliminate the influence of the ionosphere is the LC (ionospheric-free) observation combination, which is expressed as:

[0124] LC=L1-(g / (1-g 2 ))·(L2-gL1); g=1227.6 / 1575.42

[0125] Based on L1 and L2 and ionospheric delay constrained observations k The linear combination of the carrier phase observation data is as follows:

[0126]

[0127] Where D is the double difference operator, which maps all phase one-way observations of the same epoch into independent double difference observations, x a are the unknown parameters to be solved, including unknown station coordinates, unknown orbit parameters, ambiguity parameters, polar motion parameters, and tropospheric zenith delay parameters, etc. c1 、v c1 is the residual of the double-difference observation.

[0128] here:

[0129]

[0130] Ac1 =(1-g 2 ) -1 (A1-gA2)

[0131] A c2 =(1 / 2g)(gA1+A2)

[0132] If the distributed-single-node parallel mode is adopted, the specific process is as follows:

[0133] S401, the client configures distributed single-node computing parameters and baseline solution strategy parameters;

[0134] S402, the client uploads the data to be solved to the server based on the TCP protocol network socket technology:

[0135] S403, the server checks the timeliness of the observation data and tables files;

[0136] S404, the server divides the subnets according to the subnet partition list and the corresponding observation data and table files, and generates a parallel computing file system;

[0137] S405, the server determines the number of processes in the initialization process pool according to the number of tasks (i.e., the number of time periods) solved by the daily baseline, the number of logical cores of the server CPU (automatically identified by the program), and the distributed single-node computing parameters;

[0138] S406. The server submits the task of single-day baseline solution and high-performance solution parameters to the dynamically expandable process pool, and performs parallel solution on the baselines of each subnet to obtain a single-day baseline solution for each subnet.

[0139] Further, in S406, the server submits the task of single-day baseline solution and high-performance solution parameters to the dynamically scalable process pool, and performs parallel solution on the baselines of each subnet to obtain the single-day baseline solution of each subnet. The specific steps are as follows:

[0140] S501, initialization process pool:

[0141] The process pool creates a main process (management process) and a corresponding number of child processes (resource processes) according to the preset number of concurrent processes; the main process applies for space to store data structures, manage the status of child processes (such as idle, busy) and related information (such as process ID, pipeline file descriptor, etc.); considering that the local computer may have other task requirements besides large-scale GNSS network baseline solution, in order to take into account the computing power needs of both, the optimal number of processes P is calculated as follows:

[0142]

[0143] Among them, P is the optimal number of processes, C is the number of logical cores of the assumed computer, N is the total number of baseline solution tasks in a single day, and S is the preset number of concurrency;

[0144] S502, monitoring task:

[0145] The main process or a specific task distributor is responsible for monitoring the daily baseline solution task queues of each subnet; when a new task arrives, it is placed in the task queue;

[0146] S503, task allocation:

[0147] The main process (or task distributor) checks the status of the child processes in the process pool, finds the idle child process through inter-process communication (such as pipes, message queues, etc.), and assigns the task to the idle child process;

[0148] S504, subprocess execution task:

[0149] The subprocess receives the single-day baseline solution task information from the main process, and performs file reading and writing, baseline solution and other operations according to the set parameters; the subprocess can update its status (such as from idle to busy) during the execution of the baseline solution task, and return to the idle state after the task is completed;

[0150] S505, task completion and result return:

[0151] After the subprocess completes the task, it returns the single-day baseline solution result to the main process through file writing, message sending, etc., and notifies the main process that it has completed the task and can receive new tasks through pipeline writing operations, signal sending, etc.

[0152] S506, process pool resource recovery and shutdown

[0153] When all single-day baseline solution tasks are completed, the main process is responsible for recovering the sub-process resources of the completed tasks, including closing file descriptors, releasing memory, etc.; the main process closes the process pool and waits for all sub-processes to exit; ensuring that all resources are released correctly to avoid resource leakage.

[0154] Furthermore, the S4 is solved using a corresponding mode; if distributed-cluster parallelism is used, the specific process is as follows:

[0155] S601, the client configures distributed cluster computing parameters and baseline solution strategy parameters;

[0156] S602, the client uploads the data to be solved to the master node of the cluster based on the network socket technology of the TCP protocol:

[0157] S603, the master node checks the timeliness of the observation data and tables files;

[0158] S604, the master node divides the subnets according to the subnet partition list and the corresponding observation data and table files, and generates a parallel computing file system;

[0159] S605, the master node configures the number of parallel processes of the master and slave nodes of the cluster system according to the number of tasks (i.e., the number of time periods) solved by the single-day baseline and the distributed cluster computing parameters;

[0160] S606: Submit the task of single-day baseline solution and high-performance solution parameters to the high-performance solution cluster. The master node cooperates with the slave nodes to perform distributed parallel solution on the baseline of each subnet to obtain a single-day baseline solution for each subnet.

[0161] Further, in S605, the master node configures the optimal computing throughput P of the master and slave nodes of the cluster system according to the number of tasks (ie, the number of time periods) solved by the single-day baseline and the distributed cluster computing parameters as follows:

[0162]

[0163] Among them, C m is the number of logical cores of the master node (Master) in the cluster system, T is the number of slave nodes (Worker) in the cluster system, C i is the number of logical cores of the ith slave node, C' j is the number of concurrent processes when the jth slave node is not fully loaded. In particular, when j = 1, let N is the total number of baseline solution tasks in a single day. P is the optimal computing throughput of the cluster system.

[0164] Further, the S606, submitting the task of single-day baseline solution and high-performance solution parameters to the high-performance solution cluster, the master node cooperates with the slave node to perform distributed parallel solution on the baseline of each subnet to obtain the single-day baseline solution of each subnet, the specific steps are:

[0165] S701, master and slave nodes synchronously solve data:

[0166] The master node (Master) sends the solution data to each slave node (Worker) synchronously in a broadcast aggregation communication mode (master-slave replication mode); the operations in this process include: each slave node starts the data transmission service on its own designated port; the master node starts multiple data transmission services on multiple ports in a multi-process parallel manner according to the number of slave nodes; the master node sends the solution data to each slave node on the corresponding port at the same time;

[0167] S702, task monitoring and heartbeat monitoring:

[0168] After the solution data is uploaded to each Worker, the Client uses RPC technology to remotely call the baseline solution function of the Master to process the data. At the same time, the Master synchronizes the baseline solution function to each Worker and calls it to achieve collaborative parallel computing between the Master and the Worker. During the collaborative parallel computing process, the Master performs heartbeat monitoring (HeartBeat) on each Worker, monitors the working status of each Worker in real time, and gives feedback on abnormalities.

[0169] S703, task allocation and load balancing:

[0170] A smooth weighted round-robin load balancing algorithm is used to implement solution task allocation; the specific implementation method is: the load balancer first assigns a weight value to each computing node (including master nodes and slave nodes) (the present invention determines the weight according to the number of CPU logical cores of each node), and the weight reflects the processing capacity of the master and slave nodes; computing nodes with higher weights will be assigned more solution tasks, and computing nodes with lower weights will be assigned fewer solution tasks.

[0171] Under certain special weights, the common weighted round-robin scheduling algorithm may cause some nodes to have instantaneous high loads due to uneven loads, resulting in the risk of system downtime. In order to solve this scheduling defect, this method uses the smooth weighted round-robin load balancing algorithm to realize the automatic configuration of baseline solution tasks, which can not only improve the system throughput but also ensure the stability of the system.

[0172] Basic principle: The load balancer first creates a new node S for each computing node. i (including master node and slave node) assign a weight value W i (The present invention determines the weight according to the number of CPU logical cores of each node), and the weight reflects the processing capacity of the master and slave nodes; computing nodes with higher weights will be assigned more solution tasks, and computing nodes with lower weights will be assigned fewer solution tasks.

[0173] The specific implementation is as follows:

[0174] Suppose there are n computer nodes, the corresponding IP address list, configuration weight list, and effective weight list are: S = [S1, S2, ..., S n ],W=[W1,W2,…,W n ],CW=[CW1,CW2,…,CW n ]. Each node i also has 1 configuration weight W i and the current effective weight CWi , and CW i Initialized to W i ; The indicator variable max_idx represents the IP address index of the currently selected node, which is initialized to -1; The sum of the configuration weights of all nodes is The algorithm retrieval result list is DestList. The smooth weighted polling scheduling algorithm process is:

[0175] 1) Initialize the current effective weight CW of each node i i To configure the weight W i , get the initial CW, and find the total configuration weight weightSum;

[0176] 2) Get the node with the largest weight in the current valid weight list CW, and point the indicator variable max_idx to this position, and then select the node S max_idx Append to the search result list DestList;

[0177] 3) Set node S max_idx Current effective weight CW max_idx Subtract the configuration weight sum weightSum and update the current effective weight list CW;

[0178] 4) Set each element CW in the current effective weight list CW i Add the corresponding W in the configuration weight list W i , get the current valid weight list CW after the second update;

[0179] 5) Repeat steps 2) to 4) for each scheduling until all baseline solution tasks are assigned.

[0180] S704, each node performs tasks and dynamic load balancing:

[0181] The process of each computing node receives the single-day baseline solution task information and performs file reading and writing, baseline solution and other operations according to the set parameters. After each node completes part of the solution task, it notifies the load balancer through pipeline write operations, signal sending and other mechanisms that it has completed the task and can receive new tasks. During the execution of the solution task, the load balancer will continuously check the computing resource load status of each node, find an idle node, and dynamically allocate the solution task to the idle node.

[0182] S705, task completion and result return:

[0183] After the master node monitors the completion of all baseline solution tasks of each node through inter-process communication (such as pipelines, message queues, etc.), it calls the data transmission service running on each slave node to download and merge the single-day baseline solution results of each node (including master and slave nodes) in a collected aggregate communication mode;

[0184] S706, process data cleaning:

[0185] After the daily baseline solution is completed, the master node remotely calls the data cleaning module of each slave node through RPC to complete the process data cleaning work.

[0186] Further, the S5, quality check of the single-day baseline solution includes standardized root mean square error NRMS, baseline repeatability R, point position and baseline component mean error and relative mean error;

[0187] The standardized root mean square error NRMS calculation formula is as follows:

[0188]

[0189] Where Y i is the calculated value for each baseline vector epoch, Y is the true value of the baseline vector, N is the total number of epochs, and n is the number of baselines. is the variance of the solution value at each epoch; if the NRMS value is less than the set threshold, the baseline solution is considered qualified, otherwise it is unqualified.

[0190] The calculation formula of baseline repeatability R is as follows:

[0191]

[0192] Where n is the total number of observation periods of the same baseline; C i is the baseline component or side length of a period; is the C corresponding to the i period i Variance of the component; C m It is the weighted average of each period.

[0193] The repeatability of the side length, north-south component, east-west component and vertical component of each baseline is subjected to straight line fitting of fixed error and proportional error as a reference indicator for measuring baseline accuracy. The fixed error and proportional error do not exceed the indicators in Table 1:

[0194] Table 1 Accuracy classification

[0195] level Fixed error α / mm Proportional error coefficient b / (mm / km) B ≤3 ≤1 C ≤5 ≤3 D ≤10 ≤10 E ≤10 ≤20

[0196] The mean error and relative mean error of the point position and baseline component meet the accuracy requirements in the following table:

[0197] The Class A GNSS network is composed of micro navigation and positioning base stations. The mean error of the annual rate of change of coordinates, relative accuracy and the annual average mean error of each component of the geocentric coordinates should not exceed the requirements of Table 2.

[0198] Table 2 Class A GNSS measurement accuracy

[0199]

[0200] The mean error of point position, the mean error accuracy of baseline components of adjacent points and the average distance between adjacent points of Class B, C, D and E GNSS networks should not exceed the requirements of Table 3.

[0201] Table 3 B, C, D, E GNSS measurement level accuracy

[0202]

[0203] Furthermore, the specific process of S6, time series analysis, is: adjusting the daily baseline solutions of each subnet to generate the coordinate time series of each station, and obtaining the clean coordinate time series data of each station through step detection and gross error elimination.

[0204] The specific process is as follows:

[0205] (1) Perform daily adjustment on the baseline solution of each period, calculate the daily coordinates of each station, and generate the station coordinate time series;

[0206] (2) By performing step analysis on the site coordinate time series, the site coordinate time series is segmented;

[0207] (3) Preliminary gross error elimination is performed based on the accuracy level of the single-day solution, and thresholds are set in the horizontal and vertical directions for detection and elimination;

[0208] (4) Least squares residual method is used to detect and eliminate gross errors. A linear fitting model is established for the coordinate time series of each station, and the difference between the single-day solution and the fitting value of each station is calculated. The period with excessive mean error is eliminated with a limit error of 3 times the mean error, and finally a clean time series is obtained.

[0209] Furthermore, the specific process of S7, network adjustment analysis is: perform network adjustment joint processing on the single-day baselines of each subnet after time series analysis, and obtain the coordinates, speed and baseline results of each measuring station; according to the project GNSS network level requirements, check the chi-square value, each component of the adjustment baseline and relative mean error and other indicators.

[0210] Adjustment mathematical model

[0211] Using Kalman filtering method:

[0212]

[0213] σ i 2 =PSD×(t k+1 -t k )

[0214] Where Δt is t k ,t k+1 The difference in time. k is the state transfer matrix. I is the identity matrix. A k is the coefficient matrix of the error equation. k is the difference vector between the observed value and the theoretical calculated value, and the theoretical calculated value is calculated from the prior value of the parameter. is the correction of the station position and velocity, V k It can be obtained from the output result file of the baseline solution. is the value of the i-th parameter in the covariance matrix of the random disturbance. PSD is the power spectral density.

[0215] After adjustment, the chi-square value of each period is less than 10, which is considered qualified, otherwise it is unqualified. The indicators such as the components of the adjusted baseline and the relative mean error are the same as those corresponding to the baseline solution.

[0216] In summary, this large-scale GNSS network distributed high-performance solution method proposes a "time-space integration" double-layer and three-layer data parallel solution for large-scale GNSS network baseline solution multi-core parallel and network multi-node parallel, deeply taps the computing power of computers, gives full play to the high efficiency of shared memory systems and the high scalability of distributed systems, realizes centralized parallel and distributed parallel collaborative operations, and greatly improves production efficiency and computing resource utilization. At the same time, the solution integrates powerful quality inspection functions, realizing "both fast and accurate calculations".

[0217] The use of distributed and cluster computing technologies breaks through the limitations of traditional single-machine centralized processing and the simple reliance on adding single-node resource configuration solutions, supports remote service calls between different nodes and "load balancing" of computing resources between nodes, effectively avoiding huge waste of computing resources due to "uneven busyness". At the same time, the number of nodes can be flexibly expanded according to business needs, achieving the maximum utilization of existing equipment and reducing and saving investment costs.

[0218] Example 2

[0219] A distributed-single-node parallel mode is adopted to collect observation data from 820 reference stations (including IGS stations) from 060 to 090 in 2021, and the GNSS network is divided into 20 subnets, numbered NT01-NT20, constituting a total of 620 time periods of baseline solution tasks.

[0220] Comparison of timeliness with traditional serial mode

[0221]

[0222] The distributed single-node parallel (DistriSmp) mode has the same results as the traditional multi-window serial mode, and its solution timeliness is more than 5 times faster than that of the traditional multi-window serial mode.

[0223] Example 3

[0224] A distributed cluster is used to parallelize the observation data of 3181 reference stations (including IGS stations) in a certain area from 060 to 090 in 2022, and the GNSS network is divided into 69 subnets, numbered NT01-NT69, which constitute a total of 2204 baseline solution tasks.

[0225] Workstation combination

[0226]

[0227] Comparison of timeliness with traditional serial mode

[0228]

[0229] The distributed cluster parallel (DistriSmp) mode has the same results as the traditional multi-window serial mode, and its solution timeliness is improved by more than 20 times compared with the traditional multi-window serial mode.

[0230] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A large-scale GNSS network distributed high-performance solution method, characterized in that: The steps include: S1. Set project solution parameters; S2. According to the total number of baseline solution tasks for a single day, choose to use the distributed-single-node parallel mode or the distributed-cluster parallel mode; S3, upload the data to be solved; S4, using the corresponding mode to solve; S5. Perform quality check on the single-day baseline solution; S6, time series analysis; S7, network adjustment analysis; S8. Obtain solution results.

2. A large-scale GNSS network distributed high-performance solution method as claimed in claim 1, characterized in that: In the S1, setting project solution parameters, the parameters include project name, GNSS network level, solution constellation, coordinate system, data sampling rate, subnet division method, baseline solution, and network adjustment.

3. A large-scale GNSS network distributed high-performance solution method as claimed in claim 1, characterized in that: The specific process of step S2, selecting the distributed-single-node parallel mode or the distributed-cluster parallel mode according to the total number of baseline solution tasks in a single day, is as follows: the distributed-single-node parallel mode is suitable for large-scale GNSS networks where the total number of baseline solution tasks in a single day does not exceed 800 and the computing power of a single node can support it; the distributed-cluster parallel mode is suitable for ultra-large-scale GNSS networks where the total number of baseline solution tasks in a single day is greater than 800 and the computing power of a single node is insufficient to support it.

4. A large-scale GNSS network distributed high-performance solution method as claimed in claim 1, characterized in that: S4, adopts the corresponding mode to solve; if the distributed-single-node parallel mode is adopted, the specific process is as follows: S401, the client configures distributed single-node computing parameters and baseline solution strategy parameters; S402, the client uploads the data to be solved to the server based on the TCP protocol network socket technology: S403, the server checks the timeliness of the observation data and tables files; S404, the server divides the subnets according to the subnet partition list and the corresponding observation data and table files, and generates a parallel computing file system; S405, the server determines the number of processes in the initialization process pool according to the number of tasks (i.e., the number of time periods) solved by the daily baseline, the number of logical cores of the server CPU (automatically identified by the program), and the distributed single-node computing parameters; S406. The server submits the task of single-day baseline solution and high-performance solution parameters to the dynamically expandable process pool, and performs parallel solution on the baselines of each subnet to obtain a single-day baseline solution for each subnet.

5. A large-scale GNSS network distributed high-performance solution method as claimed in claim 4, characterized in that: In S406, the server submits the task of single-day baseline solution and high-performance solution parameters to the dynamically expandable process pool, and performs parallel solution on the baselines of each subnet to obtain the single-day baseline solution of each subnet. The specific steps are as follows: S501, initialization process pool: The process pool creates a main process (management process) and a corresponding number of child processes (resource processes) according to the preset number of concurrent processes; the main process applies for space to store data structures, manage the status of child processes (such as idle, busy) and related information (such as process ID, pipeline file descriptor, etc.); considering that the local computer may have other task requirements besides large-scale GNSS network baseline solution, in order to take into account the computing power needs of both, the optimal number of processes P is calculated as follows: Among them, P is the optimal number of processes, C is the number of logical cores of the assumed computer, N is the total number of baseline solution tasks in a single day, and S is the preset number of concurrency; S502, monitoring task: The main process or a specific task distributor is responsible for monitoring the daily baseline solution task queues of each subnet; when a new task arrives, it is placed in the task queue; S503, task allocation: The main process (or task distributor) checks the status of the child processes in the process pool, finds the idle child process through inter-process communication (such as pipes, message queues, etc.), and assigns the task to the idle child process; S504, subprocess execution task: The subprocess receives the single-day baseline solution task information from the main process, and performs file reading and writing, baseline solution and other operations according to the set parameters; the subprocess can update its status (such as from idle to busy) during the execution of the baseline solution task, and return to the idle state after the task is completed; S505, task completion and result return: After the subprocess completes the task, it returns the single-day baseline solution result to the main process through file writing, message sending, etc., and notifies the main process that it has completed the task and can receive new tasks through pipeline writing operations, signal sending, etc. S506, process pool resource recovery and shutdown When all single-day baseline solution tasks are completed, the main process is responsible for recovering the sub-process resources of the completed tasks, including closing file descriptors, releasing memory, etc.; the main process closes the process pool and waits for all sub-processes to exit; ensuring that all resources are released correctly to avoid resource leakage.

6. A large-scale GNSS network distributed high-performance solution method as claimed in claim 1, characterized in that: S4, adopts the corresponding mode to perform the calculation; if the distributed-cluster parallel is adopted, the specific process is as follows: S601, the client configures distributed cluster computing parameters and baseline solution strategy parameters; S602, the client uploads the data to be solved to the master node of the cluster based on the network socket technology of the TCP protocol: S603, the master node checks the timeliness of the observation data and tables files; S604, the master node divides the subnets according to the subnet partition list and the corresponding observation data and table files, and generates a parallel computing file system; S605, the master node configures the optimal computing throughput of the master and slave nodes of the cluster system according to the number of tasks (i.e., the number of time periods) solved by the single-day baseline and the distributed cluster computing parameters; S606: Submit the task of single-day baseline solution and high-performance solution parameters to the high-performance solution cluster. The master node cooperates with the slave nodes to perform distributed parallel solution on the baseline of each subnet to obtain a single-day baseline solution for each subnet.

7. A large-scale GNSS network distributed high-performance solution method as claimed in claim 6, characterized in that: In S605, the master node configures the optimal computing throughput P of the master and slave nodes of the cluster system according to the number of tasks (i.e., the number of time periods) solved by the single-day baseline and the distributed cluster computing parameters as follows: Among them, C m is the number of logical cores of the master node (Master) in the cluster system, T is the number of slave nodes (Worker) in the cluster system, C i is the number of logical cores of the ith slave node, C' j is the number of concurrent processes when the jth slave node is not fully loaded. In particular, when j = 1, let N is the total number of baseline solution tasks in a single day, and P is the optimal computing throughput of the cluster system.

8. A large-scale GNSS network distributed high-performance solution method as claimed in claim 6, characterized in that: S606, submitting the task of single-day baseline solution and high-performance solution parameters to the high-performance solution cluster, the master node cooperates with the slave node to perform distributed parallel solution on the baseline of each subnet to obtain the single-day baseline solution of each subnet, the specific steps are: S701, master and slave nodes synchronously solve data: The master node (Master) sends the solution data to each slave node (Worker) synchronously in a broadcast aggregation communication mode (master-slave replication mode); The operations in this process include: each slave node starts the data transmission service on its designated port; the master node starts multiple data transmission services on multiple ports in a multi-process parallel manner according to the number of slave nodes; the master node sends the solution data to each slave node on the corresponding port at the same time; S702, task monitoring and heartbeat monitoring: After the solution data is uploaded to each Worker, the client uses RPC technology to remotely call the baseline solution function of the Master to process the data. At the same time, the Master synchronizes the baseline solution function to each Worker and calls it to achieve collaborative parallel computing between the Master and the Worker. During the collaborative parallel computing process, the Master performs heartbeat monitoring on each Worker, monitors the working status of each Worker in real time, and gives feedback on abnormalities. S703, task allocation and load balancing: A smooth weighted round-robin load balancing algorithm is used to implement the allocation of solution tasks. The specific implementation method is as follows: the load balancer first assigns a weight value to each computing node (including the master node and the slave node) (the present invention determines the weight according to the number of CPU logical cores of each node), and the weight reflects the processing capacity of the master and slave nodes; computing nodes with higher weights will be assigned more solution tasks, and computing nodes with lower weights will be assigned fewer solution tasks; S704, each node performs tasks and dynamic load balancing: The process of each computing node receives the single-day baseline solution task information and performs file reading and writing, baseline solution and other operations according to the set parameters. After each node completes part of the solution task, it notifies the load balancer through pipeline write operations, signal sending and other mechanisms that it has completed the task and can receive new tasks. During the execution of the solution task, the load balancer will continuously check the computing resource load status of each node, find an idle node, and dynamically allocate the solution task to the idle node. S705, task completion and result return: After the master node monitors the completion of all baseline solution tasks of each node through inter-process communication (such as pipelines, message queues, etc.), it calls the data transmission service running on each slave node to download and merge the single-day baseline solution results of each node (including master and slave nodes) in a collected aggregate communication mode; S706, process data cleaning: After the daily baseline solution is completed, the master node remotely calls the data cleaning module of each slave node through RPC to complete the process data cleaning work.

9. A large-scale GNSS network distributed high-performance solution method as claimed in claim 1, characterized in that: S5, quality check of the single-day baseline solution including standardized root mean square error NRMS, baseline repeatability R, mean error of point position and baseline component and relative mean error; Among them, the standardized root mean square error NRMS calculation formula is as follows: Where Y i is the calculated value for each baseline vector epoch, Y is the true value of the baseline vector, N is the total number of epochs, and n is the number of baselines. is the variance of the solution value at each epoch; if the NRMS value is less than the set threshold, the baseline solution is considered qualified, otherwise it is unqualified; The calculation formula of baseline repeatability R is as follows: Where n is the total number of observation periods of the same baseline; C i is the baseline component or side length of a period; is the C corresponding to the i period i Variance of the component; C m It is the weighted average of each period.

10. A large-scale GNSS network distributed high-performance solution method as claimed in claim 1, characterized in that: The specific process of S6, time series analysis, is as follows: coordinate time series analysis is performed on the single-day baseline solutions of each subnet, a single-day solution time series diagram is generated, steps are judged and analyzed, and gross errors are eliminated; the specific process of S7, network adjustment analysis, is as follows: network adjustment joint processing is performed on the single-day baselines of each subnet after time series analysis, and the coordinates, speed and baseline results of each measuring station are obtained; according to the project GNSS network level requirements, indicators such as the chi-square value, each component of the adjustment baseline and the relative mean error are checked.

Images