Multi-node publish / subscribe system for spacecraft on-orbit data security transmission

Abstract

This invention discloses a multi-node publish-subscribe system for secure on-orbit data transmission of spacecraft. The system includes a data subscription layer, a data publishing layer, a data processing layer, and a data caching layer. The data subscription and publishing layers utilize distributed cloud service nodes managed by Zookeeper to jointly implement multi-node subscription and publishing of on-orbit data. The data processing layer employs a heartbeat-based priority election algorithm and multi-node data consistency communication technology to process and analyze high-concurrency remote data. The data caching layer stores on-orbit data of spacecraft using distributed fast caching technology. This invention leverages big data and cloud computing platform technologies to overcome several key technical challenges, ensuring data security, consistency, and computational efficiency, providing technical support for the transmission, management, and caching of massive on-orbit experimental data from spacecraft.

Description

Technical Field

[0001] This invention relates to computer communication technology, and more particularly to a multi-node publish-subscribe system for secure on-orbit data transmission of spacecraft. Background Technology

[0002] The space environment in which spacecraft operate in orbit is complex and ever-changing. Various environmental effects, such as particle radiation, electromagnetic radiation, and temperature fluctuations, can interfere with and affect spacecraft, potentially leading to malfunctions and significant losses. Therefore, accurately and intuitively displaying the on-orbit operating conditions of spacecraft using effective methods, promptly identifying potential hazards and providing early warnings of malfunctions, and quickly locating and implementing contingency plans when malfunctions occur are crucial for ensuring the long-term stable operation of spacecraft in orbit. Thus, research on comprehensive detection and monitoring display technologies for spacecraft operational status is urgently needed in the aerospace industry and has high practical value.

[0003] In practical applications, monitoring the on-orbit operational status of spacecraft relies heavily on the analysis of spacecraft telemetry parameters and transmitted images. Spacecraft test data serves as the sole basis for ground monitoring personnel and experts to understand the spacecraft's operational status. With the rapid development of aerospace technology, the types of spacecraft, the frequency of tests, and the number of measurement stations are increasing daily, and more and more new payloads are being deployed, leading to a dramatic increase in the amount of spacecraft test data. During on-orbit monitoring, how to efficiently and reliably receive, manage, and statistically analyze massive amounts of test data in real time using a relatively small number of servers to accurately characterize the spacecraft's operational status, while simultaneously reducing server room deployment space and costs, is a key concern for ground monitoring personnel.

[0004] The emergence and development of distributed cloud computing has enabled the time-consuming processing of massive amounts of data to rely on server clusters with limited computing power. By sharing hardware and software resources, these resources can be allocated on demand to computers and other devices. Hadoop is an open-source cloud computing platform that has been widely used in recent years. It consists of the core component Hadoop Distributed File System (HDFS), the big data computing engine MapReduce, and several sub-projects such as Pig, ZooKeeper, HBase, and Hive. It possesses characteristics such as versatility, high reliability, scalability, and dynamic interactivity. Faced with the continuously growing volume of spacecraft test data, the introduction of the distributed cloud computing platform Hadoop will undoubtedly bring convenience to the analysis and display of spacecraft on-orbit test processes.

[0005] However, while HDFS, the distributed file system used for data transmission between computing nodes during Hadoop platform operation, offers advantages such as high fault tolerance, batch processing, and petabyte-scale data transfer capabilities, it cannot meet the demands for efficient small file storage and multi-threaded concurrent data read / write. Therefore, a high-concurrency data publish-subscribe system is needed in cloud computing platforms. The asynchronous communication paradigm of publish-subscribe systems enables large-scale distributed communication applications: data senders transmit data content to logical channels named by topic, while data receivers freely subscribe to topics of interest and synchronously receive all data sent to these topic channels. This approach features multi-threaded concurrency, loose coupling, and high scalability. Current mainstream publish-subscribe systems include topic-based systems such as iBus, SCRIBE, and CORBA Notification Service, as well as content-based, channel-based, and type-based publish-subscribe model systems.

[0006] Currently, there is some research on the integration of cloud computing platforms with publish-subscribe systems. For example, some literature proposes a suite of publish-subscribe middleware for integrating Wireless Sensor Networks (WSNs) into cloud platforms, providing the community with rich resources, services, and storage related to WSN-driven data. It also proposes a content-based matching algorithm to analyze subscription intent, thus facilitating the delivery of appropriate content to subscribers. However, in practical applications, considering that Hadoop's massively parallel computing relies on offline data stored on local disks, it cannot meet the application requirements of real-time online data acquisition, processing, analysis, and display for spacecraft on-orbit experiments. Therefore, it is necessary to perform additional real-time data interaction processing outside of Hadoop to achieve secure and reliable big data transmission. Since the data processing order of different cloud computing nodes often differs in cloud environments, many classic data security mechanisms have emerged, such as the Paxos data consistency algorithm, the Raft priority election mechanism, and Practical Byzantine Fault Tolerance (PBFT). However, these algorithms have varying degrees of limitations due to their different characteristics. For example, the Paxos and Raft consensus algorithms do not consider the Byzantine node problem in the system, while the PBFT algorithm has poor scalability.

[0007] Therefore, how to transmit massive amounts of data in real time, securely, and stably throughout the entire lifecycle of a spacecraft is a major challenge in the design of publish-subscribe systems in a cloud environment. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the existing technology and to propose a multi-node publish-subscribe system for secure on-orbit data transmission of spacecraft.

[0009] To achieve the above objectives, this invention proposes a multi-node publish-subscribe system for secure on-orbit data transmission of spacecraft. The system includes: a data subscription layer, a data publishing layer, a data processing layer, and a data caching layer; wherein...

[0010] The data subscription layer and data publishing layer are used to jointly realize the multi-node subscription and publishing of spacecraft on-orbit data based on distributed cloud service nodes managed by Zookeeper;

[0011] The data processing layer is used to process and analyze high-concurrency remote data by employing a priority election algorithm based on heartbeat detection and multi-node data consistency communication technology.

[0012] The data caching layer is used to store spacecraft on-orbit data through distributed fast caching technology.

[0013] As an improvement to the aforementioned system, the multi-node subscription and publishing architecture for spacecraft on-orbit data includes several distributed cloud service nodes, several publisher nodes, and several subscriber nodes; wherein,

[0014] One of the distributed cloud service nodes is the master control node, and the rest are service proxy slave nodes;

[0015] The master control node is used to receive on-orbit data sent by each publisher node and forward it to the service agent slave nodes;

[0016] The service agent slave node is used to store the received on-orbit data into memory or disk according to topic partition blocks;

[0017] The publisher node is used to send spacecraft on-orbit data to the master control node;

[0018] The subscriber node is used to subscribe to and consume on-orbit data from the service broker via topics.

[0019] As an improvement to the above system, the publisher node includes a structure port for subscribing to multi-source experimental data of spacecraft and a character port for subscribing to configuration parameters;

[0020] The publisher node employs a processing mechanism consisting of a main thread and a sending thread, wherein...

[0021] The main thread is used to receive user-created on-orbit data, which is then processed by an interceptor, serializer, and partitioner and cached at the tail of the double-ended queue of the data accumulator.

[0022] The sending thread is used to obtain on-orbit data from the head of the double-ended queue of the data accumulator and send the obtained on-orbit data to the service agent slave node through the responder.

[0023] As an improvement to the above system, the subscriber node subscribes to on-orbit data from the service broker through a topic. Each subscriber node has a corresponding subscription group. The publisher node delivers the corresponding on-orbit data to the subscription group that subscribes to the topic. Each subscriber node in the subscription group can repeatedly consume the on-orbit data, and different subscription groups do not affect each other.

[0024] As an improvement to the above system, the subscriber node uses a pull mode to read data between the water level and the consumption offset from the data block of the corresponding topic of the service broker slave node, and saves a consumption offset; wherein, the water level is used to identify the on-orbit data that has been consumed by the subscriber node; the consumption offset is the displacement value of the subscriber node pulling the next on-orbit data, and the consumption rate is determined by the number of data in the service broker slave node and the consumption rate.

[0025] As an improvement to the above system, the priority election algorithm based on heartbeat detection specifically includes:

[0026] All service agent slave nodes are prioritized and a threshold is set according to the current cluster size. Nodes above the threshold are designated as the election zone, and those below the threshold are designated as the waiting zone. Service agent slave nodes in the election zone are eligible to participate in the election of the master node. Service agent slave nodes that have just joined the cluster or have recovered from a failure are placed in the waiting zone.

[0027] When the master node stops working due to an unexpected process interruption or unexpected crash, it initiates an election in the election area. If more than half of the service agent slave nodes vote in favor, the service agent slave node with the highest priority is set as the new master node, and data consistency is checked among the nodes.

[0028] After the election, the priority of service agent slave nodes that participated in the election is increased; at the same time, the threshold is dynamically adjusted according to the actual situation: when the number of service agent slave nodes in the election area exceeds the preset value, the service agent slave node with the lowest priority in the election area is demoted to the waiting area; when the number of service agent slave nodes in the waiting area exceeds the preset value, the service agent slave node with the highest priority in the waiting area is promoted to the election area.

[0029] As an improvement to the aforementioned system, each of the distributed cloud service nodes includes epoch information and a data offset, wherein...

[0030] The epoch information is used to record the number of times the master node changes. The initial value of the epoch information is 0, and the value is incremented by 1 each time the master node changes.

[0031] The recorded data offset is used to synchronize the offset coordinates when the first data is written under the current epoch information. The service agent slave node requests data synchronization from the master node at a fixed frequency, and when the node state changes, it compares the position of the data offset of each node to perform data truncation or padding operations to protect the data integrity of the service agent slave node.

[0032] As an improvement to the above system, the data consistency determination between nodes specifically includes:

[0033] Each service agent sends a separate message to the master node to request synchronization from each node;

[0034] If a service agent restarts from a node that has crashed, it compares its own epoch information with the data offset and performs data truncation or completion operations based on the water level of the master node.

[0035] If the master node crashes and restarts, the newly elected master node will increment its own epoch information by 1 based on the newly written data and shift the record data offset down; the original master node will be downgraded to a service proxy slave node and will perform the corresponding request synchronization processing.

[0036] As an improvement to the above system, the distributed fast caching technology combines a distributed node read and append write mechanism, and completes distributed fast data caching by designing checkpoints and performing snapshot backups at the checkpoints; the checkpoint is formed by caching the state of all tasks after they have finished processing the same input data, thus forming the current consistency checkpoint of the data.

[0037] As an improvement to the above system, the distributed node read and append write mechanism specifically includes: running a data backup daemon thread in each of the distributed cloud service nodes, and using the append write mechanism to write real-time data to the Edits file to disk, so as to ensure data writing efficiency and security;

[0038] The step of performing snapshot backups at checkpoints to complete distributed data fast caching specifically includes:

[0039] When a write operation encounters a checkpoint, a new file is created and the write process continues. At the same time, the daemon thread adds an index to the header of the Edits file and merges it into the source data storage block of the distributed nodes according to the unique identifier and the backup image file, forming a data file consistent with the data source.

[0040] Compared with the prior art, the advantages of the present invention are:

[0041] 1. This invention mainly addresses the enormous pressure faced by massive experimental data in the transmission, sharing, processing, analysis, and storage processes during multiple stages of spacecraft test mission monitoring, such as on-orbit fault diagnosis and status detection, health status assessment, and spacecraft life prediction. It constructs a multi-node publish-subscribe system framework for managing multi-source heterogeneous data of spacecraft.

[0042] 2. This invention leverages big data and cloud computing platform technologies to overcome key technologies such as multi-node publish and subscribe technology, priority election technology based on heartbeat detection, and multi-node data consistency communication, ensuring data security, consistency, and computational efficiency, and providing technical support for the transmission, management, and caching of massive on-orbit experimental data from spacecraft.

[0043] 3. To address the processing pressure of massive amounts of test data during spacecraft test missions in the processes of transmission, sharing, processing analysis, and storage, and to provide support for real-time monitoring of the entire life cycle of spacecraft test missions, health monitoring of operational conditions, and rapid prediction and early warning of risks and faults;

[0044] 4. Promote the development of publish-subscribe systems based on big data technology and communication security technologies to solve problems such as packet loss, redundant calculations, and low system availability in existing big data transmission technologies. Attached Figure Description

[0045] Figure 1 This is a technical roadmap for the multi-node publish-subscribe system for secure on-orbit data transmission of spacecraft, as described in this invention.

[0046] Figure 2 This is a diagram of a multi-node publish / subscribe technology architecture;

[0047] Figure 3 This is the publisher pattern design diagram;

[0048] Figure 4 This is a design diagram of the subscriber pattern;

[0049] Figure 5 This is a diagram illustrating the consumer offset.

[0050] Figure 6 This is a design diagram of a distributed cache storage model;

[0051] Figure 7 This is a schematic diagram of multi-node data consistency communication technology;

[0052] Figure 8 This is a diagram illustrating the deployment of the test hardware environment;

[0053] Figure 9 shows the response latency of each publish / subscribe system under different data packets. Figure 9(a) shows the response latency of Kafka data subscription, Figure 9(b) shows the response latency of ActiveMQ data subscription, Figure 9(c) shows the response latency of RabbitMQ data subscription, Figure 9(d) shows the response latency of RocketMQ data subscription, and Figure 9(e) shows the response latency of BPSS data subscription.

[0054] Figure 10 It is a comparison of the throughput performance of different publish / subscribe systems. Detailed Implementation

[0055] This invention proposes a multi-node publish-subscribe system for secure on-orbit data transmission of spacecraft. The system includes: a data subscription layer, a data publishing layer, a data processing layer, and a data caching layer; wherein,

[0056] The data subscription layer and data publishing layer are used to jointly realize the multi-node subscription and publishing of spacecraft on-orbit data based on distributed cloud service nodes managed by Zookeeper;

[0057] The data processing layer is used to process and analyze high-concurrency remote data by employing a priority election algorithm based on heartbeat detection and multi-node data consistency communication technology.

[0058] The data caching layer is used to store spacecraft on-orbit data through distributed fast caching technology.

[0059] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0060] Example 1

[0061] Embodiments of this invention propose a multi-node publish-subscribe system for secure on-orbit data transmission from spacecraft, the technical approach of which is as follows: Figure 1 As shown.

[0062] The multi-node publish / subscribe system comprises a four-layer architecture: data publishing layer, data subscription layer, data processing layer, and data caching layer. The multi-node publish / subscribe system mainly involves four key technologies: multi-node publish and subscribe technology, priority election technology based on heartbeat detection, multi-node data consistency communication technology, and distributed fast caching technology.

[0063] The data publishing and subscription layers, based on distributed cloud service nodes managed by ZooKeeper, jointly implement multi-node publish-subscribe technology. The data publishing layer records data offsets using timestamp-type watermarks and checkpoints to ensure the sequential nature of data publishing. The data subscription layer partitions data consumption according to spatial environment data types and ensures the sequential nature of data processing based on consumption offsets. The data processing layer ensures data security, consistency, and computational efficiency through heartbeat-based priority election technology and multi-node data consistency communication technology, enabling high-concurrency remote data processing and analysis. The data caching layer achieves secure data storage through distributed fast caching technology, realizing high throughput and strong fault tolerance at the memory level.

[0064] (1) Multi-node publish and subscribe technology

[0065] The main technical architecture of multi-node publish and subscribe technology is as follows: Figure 2 As shown.

[0066] Multi-node publish and subscribe technology includes several publisher nodes, several subscriber nodes, a master node, and several service proxy nodes. The publisher nodes send data to the master node, the master node assigns service proxy nodes to store the received data on disk, and the subscriber nodes are responsible for subscribing to and consuming data from the service proxy nodes.

[0067] The publisher node design consists of two threads working in coordination: a main thread and a sending thread. In the main thread, user-created data is then cached at the tail of the message accumulator's dequeue after passing through possible interceptors, serializers, and partitioners. This allows the sending thread to send data in batches, reducing network transmission resource consumption and improving performance. The sending thread is responsible for retrieving data from the head of the message accumulator's dequeue and sending it to the service broker node via a responder. Figure 3 As shown.

[0068] To ensure high flexibility and usability, the publisher node provides two data subscription interfaces: a structure port for subscribing to multi-source experimental data from spacecraft, and a character port for subscribing to configuration parameters and other data. It also supports customizing port types by implementing event type interfaces, improving generalization performance.

[0069] Subscriber nodes are responsible for subscribing to topics on the service proxy node and pulling data from the subscribed topics. Each subscriber node has a corresponding subscription group. When data is published to a topic, it is delivered to the subscription group that subscribed to it. Each subscriber in the group can consume the data repeatedly, and the two subscription groups do not affect each other. Figure 4 As shown.

[0070] The subscriber node uses a pull model to read data from the service broker node based on the watermark and saves a consumption offset, such as... Figure 5 As shown in the diagram. The watermark defines the visibility of data, indicating which data can be consumed by subscriber nodes. The consumption offset is the offset value at which a subscriber node pulls the next piece of data; therefore, data between the watermark and the consumption offset is considered data that can be pulled.

[0071] The specific consumption rate of subscriber nodes is mainly determined by the amount of data and the consumption rate in the service broker nodes. Meanwhile, the pull mode can consume data in partitions based on the spatial environment and data type. For data with a fixed receiving frequency, a data-driven approach can be adopted, continuously consuming data through long polling, with a unified daemon process determining the offset of various heterogeneous data types. This avoids redundant configuration for multi-source data, effectively reducing server load.

[0072] (2) Distributed fast caching technology

[0073] The data caching layer of a multi-node publish / subscribe system implements distributed fast caching technology. Combining distributed node read and append-write mechanisms, it achieves distributed fast data caching by designing checkpoints and performing snapshot backups at these checkpoints. The overall technical approach is as follows: Figure 6 As shown.

[0074] To prevent data loss during failures, distributed fast caching technology is used to serialize and store data flowing in memory onto disk. In the event of data loss, a data recovery mechanism can restore the data content in memory. Each distributed cloud service node runs a data backup daemon thread, employing an append-only mechanism to write real-time data to edit files on disk, ensuring both data writing efficiency and security.

[0075] Meanwhile, repeated write operations can lead to excessively large files and unnecessary disk operations. To address this, a checkpoint snapshot backup storage mechanism is employed. A checkpoint means that when all tasks have exactly finished processing the same input data, their states are cached to form a current consistency checkpoint. Checkpoints avoid storing additional information besides the state, improving data caching efficiency. Therefore, when a write operation encounters a checkpoint, a new file is created and appending continues. Simultaneously, a daemon thread adds an index to the header of this Edits file. Finally, based on a unique identifier and the backup image file, the data is merged into the source data storage blocks of the distributed nodes, forming a data file consistent with the data source. This ensures both efficiency and data recovery in case of system failure or server power outage.

[0076] (3) Priority election technology based on heartbeat detection

[0077] Multi-channel metadata enters the data processing layer through subscriber nodes and is then processed with high concurrency. To ensure secure data transmission between cloud computing nodes and reduce redundant calculations to improve data processing efficiency, this invention introduces a heartbeat detection mechanism to periodically detect the communication status of each cloud service node.

[0078] There are many heartbeat detection protocols currently available, such as accelerated heartbeat protocols, multi-machine system heartbeat detection mechanisms, and adaptive heartbeat detection. However, most heartbeat detection protocols suffer from the problem of unrecoverable failures after node anomalies and recovery delays caused by single-point failures of the master node. A common method to address single-point failures is dual-machine fault tolerance technology, which uses a backup machine to smoothly take over the work of the failed master node when it fails. However, this reduces the flexibility of the distributed system. Therefore, an election mechanism was developed to address the shortcomings of dual-machine fault tolerance technology: when a slave node detects a failure of the master node, it needs to initiate an election process that requires the participation of other slave nodes to elect a new slave node to take over the work of the master node.

[0079] However, in practical engineering applications, it has been found that the more cloud computing nodes there are, the longer the election mechanism takes to execute. Furthermore, when the cloud platform performs load balancing, new nodes frequently enter the cluster and trigger another election, leading to constant changes in the master node and a significant reduction in system availability. To improve these issues, this invention designs a priority election algorithm based on heartbeat detection, which dynamically adjusts the number of nodes eligible to participate in the election. Its pseudocode is shown in Table 1.

[0080] Table 1 Priority election algorithm based on heartbeat detection

[0081]

[0082]

[0083] As shown in Table 1, the priority election algorithm based on heartbeat detection first prioritizes all nodes and sets a threshold based on the current cluster size. Nodes above the threshold are designated as the election zone, and those below are designated as the waiting zone. Only slave nodes in the election zone are eligible to participate in the master node election. After the election, the priority of slave nodes that participated is further increased to ensure the master node's performance is as optimal as possible within the cluster. The threshold is also dynamically adjusted based on actual conditions: when the number of slave nodes in the election zone is too large, the lowest-priority node in the election zone is moved to the waiting zone; similarly, when the number of slave nodes in the waiting zone is too large, the highest-priority slave node in the waiting zone is promoted to the election zone. Furthermore, newly added nodes or nodes recovering from failures are directly placed in the waiting zone to reduce the number of elections required when a node joins the cluster. If the master node stops working due to an unexpected process interruption or crash, an election is initiated in the election zone. If more than half of the slave nodes vote in favor, the highest-priority slave node takes over the master node's work, and data consistency is checked among the nodes.

[0084] (4) Multi-node data consistency communication technology

[0085] Due to performance differences among service computing nodes and varying degrees of data synchronization among different slave nodes, data inconsistency can occur after a master node change during the election mechanism, leading to data loss. To further improve the reliability of data communication and transmission, this invention designs a multi-node data consistency communication technology within the heartbeat-based priority election algorithm, the pseudocode of which is shown in Table 2.

[0086] Table 2 Multi-node Data Consistency Communication Technology

[0087]

[0088] As shown in Table 2, each node needs to record epoch information indicating the number of times the master node has changed. The initial value of the epoch information is 0. Each time the master node changes, the epoch information value is incremented by 1, which is equivalent to adding a version number to the master node. Simultaneously, each node needs to record the data offset, used to synchronize the offset coordinates when the first piece of data is written under the current epoch information. Slave nodes request data synchronization from the master node at a fixed frequency, and when the node state changes, they compare the positions of their respective node data offsets to perform data truncation or padding operations, thereby protecting the data integrity of each node.

[0089] A visual example of multi-node data consistency communication technology is as follows: Figure 7As shown in the diagram, suppose node B crashes before its water level has been updated. When node B restarts, it needs to send a request to the master node to query the data offset value of master node A. In the example above, this value is 2. Simultaneously, node B will perform a data truncation operation, adjusting the data offset to the previous water level value. Therefore, the data with an offset value of "1" is deleted from node B. If node B then takes over as the master node, this data will be completely lost.

[0090] To address this issue, a data synchronization mechanism is added to multi-node data consistency communication technology: After the truncation operation is completed, slave node B can retrieve the data with an offset value of "1" from master node A to complete the data by comparing its own epoch information with the data offset. After data synchronization, assuming master node A crashes and slave node B becomes master node C through a priority election strategy, the same logical judgment is executed when master node A (which is also the new slave node D) restarts. Thus, the data with an offset value of "1" is retained in both nodes.

[0091] Finally, when the publisher node writes new data to the slave node B (which is also the new master node C), the master node epoch information in the cache of slave node B also needs to be incremented by 1, and the data offset is reset to the watermark position "2". Then, it continues to determine whether to perform data truncation or padding operations. Thus, the data consistency communication mechanism greatly avoids data loss during master-slave node state switching.

[0092] Innovation points:

[0093] (1) Since the current cloud computing platform cannot meet the application requirements of real-time online access, processing, analysis and display of spacecraft on-orbit test data, and the current publish-subscribe system has problems such as data loss and duplicate calculation during the transmission of big data, this invention uses big data and cloud computing platform technology to build a multi-node publish / subscribe system for managing multi-source heterogeneous data of spacecraft, which has the advantages of fast big data subscription response and high throughput stability.

[0094] (2) In practical applications, due to the performance differences of each cloud node in the cloud computing platform, the data is not synchronized when the master and slave nodes change. The current domestic and foreign multi-node election mechanism has limitations such as low election efficiency and frequent changes in network topology, which leads to reduced system availability. This invention designs a priority election algorithm based on heartbeat detection and a multi-node data consistency communication algorithm, which can effectively reduce the number of elections and improve election efficiency, ensure data transmission security, consistency and computational efficiency, and improve system availability.

[0095] Technical effects:

[0096] (1) Preparation of experimental environment

[0097] The distributed publish-subscribe management system BPSS designed in this invention was field-tested in a large-scale spacecraft test simulation system. Its hardware deployment diagram is shown below. Figure 8 As shown, a large-capacity data exchange environment is built using a cloud platform virtual node cluster, a general server cluster, Inspur distributed cloud storage equipment, Galaxy Kylin operating system, and Shenzhou General domestic database.

[0098] (2) Analysis of experimental results

[0099] To meet the needs of multi-source data transmission, interaction, and display in spacecraft on-orbit monitoring, including telemetry, telecontrol, data transmission images, and remote control commands, two types of experiments were designed: one on big data subscription throughput performance and the other on big data transmission security performance, to jointly verify the actual performance of BPSS.

[0100] First, data publishing was conducted using on-orbit transmission data packets from spacecraft with individual packet sizes of 1M and varying quantities of 10, 100, and 1000 packets. The data content included interactive data such as telemetry, telemetry, data transmission, data transmission images, and control commands for communication between space and ground and between spacecraft. The data types included heterogeneous data types such as JSON, TXT, JPEG, and XML. The data subscription response latency of BPSS was compared with that of mainstream open-source publish / subscribe systems at home and abroad, such as Kafka, ActiveMQ, RabbitMQ, and RocketMQ.

[0101] Data subscription response latency refers to the time consumed from the publication of data until the subscriber obtains all the data, as shown in formula (1).

[0102] T delay =T subscribe -T publish (1)

[0103] Where T delay For data subscription response latency, T subscribe T represents the data subscription timestamp. publish This is the timestamp for data publication.

[0104] The experimental results are shown in Table 3. It can be seen that in different scales and types of data transmission, the data subscription response latency is roughly positively correlated with the amount of subscribed data. The average response latency of BPSS data subscription is 0.05s, while the average response latencies of other publish / subscribe systems such as Kafka, ActiveMQ, RabbitMQ, and RocketMQ are 28.64s, 0.07s, 0.14s, and 0.11s, respectively. In comparison, BPSS has a faster data subscription response rate.

[0105] Table 3 Comparison of response latency of different publish / subscribe systems

[0106]

[0107] Subsequently, spacecraft with individual data unit sizes of 1M, 10M, and 100M were used to transmit data packets in orbit to test whether different data unit sizes and different data types (json, txt, jpeg, xml, etc.) affected the subscription rate of various publish / subscribe systems. The experimental results are shown in Figure 9, where Figure 9(a) shows the data subscription response latency of Kafka, Figure 9(b) shows the data subscription response latency of ActiveMQ, Figure 9(c) shows the data subscription response latency of RabbitMQ, Figure 9(d) shows the data subscription response latency of RocketMQ, and Figure 9(e) shows the data subscription response latency of BPSS. Analysis shows that in all types of publish / subscribe systems, the data subscription response latency is positively correlated with the individual data unit size. Furthermore, compared to other data types, the subscription response latency for jpeg image data is the longest in all publish / subscribe systems, but BPSS shows the least difference in response rate for various multi-source heterogeneous data.

[0108] The system's data throughput was then analyzed, and the system's data throughput capacity was calculated as shown in formula (2).

[0109]

[0110] Where Num subscribe This represents the amount of data subscribed under the condition of continuous data release, where t represents the time spent subscribing to this data, in seconds.

[0111] The experiment used a 1M spacecraft to transmit data packets in orbit, continuously publishing at a fixed data transmission frequency of 1000 packets / second for 5 minutes. The throughput performance of BPSS was tested compared to publish-subscribe systems such as Kafka, ActiveMQ, and RabbitMQ. The relevant experimental results are as follows: Figure 10 As shown in Table 3, higher throughput capacity results in a larger Tps value and a more stable curve trend, indicating that BPSS exhibits good throughput performance in various publish / subscribe systems. Furthermore, analysis of Table 3 reveals that data subscription response latency is inversely proportional to the large data throughput performance of the publish / subscribe system; the longer the data subscription response latency, the worse the system's throughput performance.

[0112] Finally, a spacecraft with a single unit capacity of 1M was used to transmit data packets in orbit. Each data packet was assigned an auto-incrementing number and published at a frequency of 1 packet / second. The numbering of missing subscription data numbers and the completeness of data content were statistically analyzed for different subscription durations of 1 hour, 6 hours, 12 hours, and 24 hours, to test the actual data security transmission performance of different publish / subscribe systems.

[0113] The experimental indicators are data transmission frame loss rate and data corruption rate. The indicators are defined as shown in formulas (3) and (4). The experimental results are shown in Table 4.

[0114]

[0115]

[0116] Table 4 Comparison of Data Integrity Transmitted by Different Publish / Subscribe Systems

[0117]

[0118] The results show that when the daily throughput reaches around 80GB, the BPSS data transmission frame loss rate is controlled at 0.025% and the data corruption rate is controlled at 0.018%. Moreover, the BPSS maintains a relatively stable frame loss and corruption rate for data transmission at different times throughout the day, demonstrating its advantage in stable transmission of large data.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A multi-node publish-subscribe system for secure on-orbit data transmission of spacecraft, characterized in that, The system comprises: a data subscription layer, a data publishing layer, a data processing layer, and a data caching layer; wherein, The data subscription layer and data publishing layer are used to jointly realize the multi-node subscription and publishing of spacecraft on-orbit data based on distributed cloud service nodes managed by Zookeeper; The data processing layer is used to process and analyze high-concurrency remote data by employing a priority election algorithm based on heartbeat detection and multi-node data consistency communication technology. The data caching layer is used to store spacecraft on-orbit data through distributed fast caching technology; The priority election algorithm based on heartbeat detection specifically includes: All service agent slave nodes are prioritized and a threshold is set according to the current cluster size. Nodes above the threshold are designated as the election zone, and those below the threshold are designated as the waiting zone. Service agent slave nodes in the election zone are eligible to participate in the election of the master node. Service agent slave nodes that have just joined the cluster or have recovered from a failure are placed in the waiting zone. When the master node stops working due to an unexpected process interruption or unexpected crash, it initiates an election in the election area. If more than half of the service agent slave nodes vote in favor, the service agent slave node with the highest priority is set as the new master node, and data consistency is checked among the nodes. After the election, the priority of service agent slave nodes that participated in the election is increased; at the same time, the threshold is dynamically adjusted according to the actual situation: when the number of service agent slave nodes in the election area exceeds the preset value, the service agent slave node with the lowest priority in the election area is demoted to the waiting area; when the number of service agent slave nodes in the waiting area exceeds the preset value, the service agent slave node with the highest priority in the waiting area is promoted to the election area.

2. The multi-node publish-subscribe system for secure on-orbit data transmission of spacecraft according to claim 1, characterized in that, The architecture for the multi-node subscription and publication of spacecraft on-orbit data includes several distributed cloud service nodes, several publisher nodes, and several subscriber nodes; among them, One of the distributed cloud service nodes is the master control node, and the rest are service proxy slave nodes; The master control node is used to receive on-orbit data sent by each publisher node and forward it to the service agent slave nodes; The service agent slave node is used to store the received on-orbit data into memory or disk according to topic partition blocks; The publisher node is used to send spacecraft on-orbit data to the master control node; The subscriber node is used to subscribe to and consume on-orbit data from the service broker via topics.

3. The multi-node publish-subscribe system for secure on-orbit data transmission of spacecraft according to claim 2, characterized in that, The publisher node includes a structure port for subscribing to multi-source experimental data of spacecraft and a character port for subscribing to configuration parameters; The publisher node employs a processing mechanism consisting of a main thread and a sending thread, wherein... The main thread is used to receive user-created on-orbit data, which is then processed by an interceptor, serializer, and partitioner and cached at the tail of the double-ended queue of the data accumulator. The sending thread is used to obtain on-orbit data from the head of the double-ended queue of the data accumulator and send the obtained on-orbit data to the service agent slave node through the responder.

4. The multi-node publish-subscribe system for secure on-orbit data transmission of spacecraft according to claim 2, characterized in that, The subscriber nodes subscribe to on-orbit data from the service broker nodes through topics. Each subscriber node has a corresponding subscription group. The publisher node delivers the corresponding on-orbit data to the subscription group that subscribes to the topic. Each subscriber node in the subscription group can repeatedly consume on-orbit data, and different subscription groups do not affect each other.

5. The multi-node publish-subscribe system for secure on-orbit data transmission of spacecraft according to claim 2, characterized in that, The subscriber node uses a pull mode to read data between the water level and the consumption offset from the data block of the corresponding topic of the service broker slave node, and saves a consumption offset. The water level is used to identify the on-orbit data that has been consumed by the subscriber node. The consumption offset is the displacement value of the subscriber node when pulling the next on-orbit data. The consumption rate is determined by the number of data in the service broker slave node and the consumption rate.

6. The multi-node publish-subscribe system for secure on-orbit data transmission of spacecraft according to claim 1, characterized in that, Each of the distributed cloud service nodes includes epoch information and data offset, wherein... The epoch information is used to record the number of times the master node changes. The initial value of the epoch information is 0, and the value is incremented by 1 each time the master node changes. The recorded data offset is used to synchronize the offset coordinates when the first data is written under the current epoch information. The service agent slave node requests data synchronization from the master node at a fixed frequency, and when the node state changes, it compares the position of the data offset of each node to perform data truncation or padding operations to protect the data integrity of the service agent slave node.

7. The multi-node publish-subscribe system for secure on-orbit data transmission of spacecraft according to claim 6, characterized in that, The data consistency assessment between nodes specifically includes: Each service agent sends a separate message to the master node to request synchronization from each node; If a service agent restarts from a node that has crashed, it compares its own epoch information with the data offset and performs data truncation or completion operations based on the water level of the master node. If the master node crashes and restarts, the newly elected master node will increment its own epoch information by 1 based on the newly written data and shift the record data offset down; the original master node will be downgraded to a service proxy slave node and will perform the corresponding request synchronization processing.

8. The multi-node publish-subscribe system for secure on-orbit data transmission of spacecraft according to claim 1, characterized in that, The distributed fast caching technology combines distributed node read and append write mechanisms, and completes distributed fast data caching by designing checkpoints and performing snapshot backups at the checkpoints; the checkpoints are formed by caching the states of all tasks after they have finished processing the same input data, thus creating the current consistency checkpoints of the data.

9. The multi-node publish-subscribe system for secure on-orbit data transmission of spacecraft according to claim 8, characterized in that, The distributed node read and append write mechanism specifically includes: running a data backup daemon thread in each of the distributed cloud service nodes, and using the append write mechanism to write real-time data to the Edits file to disk, ensuring data writing efficiency and security; The step of performing snapshot backups at checkpoints to complete distributed data fast caching specifically includes: When a write operation encounters a checkpoint, a new file is created and the write process continues. At the same time, the daemon thread adds an index to the header of the Edits file and merges it into the source data storage block of the distributed nodes according to the unique identifier and the backup image file, forming a data file consistent with the data source.

Images