Multi-task multi-satellite telemetry data efficient processing method, system and electronic device

Abstract

The application provides a kind of multitask multi-star telemetry data efficient processing method, system and electronic equipment, it is related to satellite ground telemetry data processing technical field. Including: based on the resource state evaluation result of each computing node in server cluster, task plug-in is dynamically distributed to optimal computing node;Task registration information is uploaded to configuration center, and the telemetry parameter configuration information contained in information is stored in the form of key-value pair in memory database;According to the assigned task plug-in, task telemetry data is extracted from the specified source, and the data is subdivided level by level based on the data shunt model;Based on the obtained telemetry data stream of each frequency band, telemetry source package is extracted, and telemetry source package analysis task is distributed through thread pool.Based on telemetry parameter configuration information, all parameters in source package are calculated by using parameter calculation virtual thread parallel. Solve the defect that high code rate telemetry data cannot be processed in real time in the scene of multi-task multi-star.

Description

Technical Field

[0001] This invention relates to the field of satellite ground telemetry data processing technology, and in particular to a method, system and electronic device for efficient processing of multi-mission and multi-satellite telemetry data. Background Technology

[0002] Satellites in orbit play a vital role in applications such as navigation, network communication, environmental monitoring, meteorological research, and reconnaissance. Each satellite platform carries various payloads and sensors. During operation, it transmits telemetry data, including satellite attitude, position, payload operating status, fault information, temperature information, and specific mission data, to the ground in binary encrypted stream format. The ground receives the data and performs descrambling, frame format verification, telemetry source packet splitting, parameter extraction and inversion, and formula calculation to obtain the final engineering values ​​of the satellite platform and payload telemetry parameters. This provides strong data support for subsequent mission command and decision-making, satellite health status diagnosis, rapid mission assessment, and three-dimensional situational awareness reconstruction.

[0003] In recent years, satellite communication technology has made significant progress, with improvements in telemetry data downlink rate, telemetry and control channel data transmission bandwidth, and data transmission channel bandwidth. This increased downlink telemetry data transmission rate places higher demands on ground-based analysis and processing software. In real-time applications, the ground telemetry analysis rate must exceed the data acquisition and downlink rate to promptly analyze and confirm the current satellite mission execution status based on the ground telemetry calculation results, providing support for subsequent command and decision-making and preventing missed opportunities for optimal strategy implementation. In post-processing scenarios, processing must also be completed as quickly as possible to avoid impacting subsequent data analysis. Furthermore, in certain space scenarios, multiple satellites may collaborate, with each satellite supporting parallel downlink of space-based and ground-based multi-band telemetry data. When multiple tasks are executed concurrently, the amount of downlink telemetry data increases linearly with the number of satellites. Therefore, ground-based analysis software faces the challenge of high-bit-rate telemetry data analysis efficiency in multi-task, multi-satellite scenarios. The impact of massive amounts of high-bit-rate data not only makes it impossible for ground-based software to meet real-time calculation requirements but also poses risks such as software processing blockage and crashes, affecting command and operation personnel and ultimately hindering mission execution.

[0004] Existing telemetry data processing technologies generally focus on a general processing framework for telemetry data, paying less attention to the processing speed. For the problem of rapid telemetry data processing, current solutions involve splitting telemetry frames to varying degrees and processing the multiple subframes in parallel to achieve fast processing capabilities. However, existing technologies generally have low parallelism in processing telemetry data, thus failing to reduce processing latency to an ideal level and unable to meet the real-time processing requirements of high-bitrate telemetry data. Furthermore, they do not consider multi-task, multi-satellite scenarios, cannot adapt to the telemetry processing needs of such scenarios, and cannot flexibly scale when new tasks are added. Summary of the Invention

[0005] This invention provides a method, system, and electronic device for efficient processing of multi-task, multi-satellite telemetry data, addressing the shortcomings of existing technologies that cannot process high-code-rate telemetry data in real-time and efficiently under multi-task, multi-satellite scenarios. This invention significantly improves telemetry data processing efficiency through higher parallel processing granularity, while also being adaptable to multi-task, multi-satellite processing scenarios and capable of rapid expansion when new tasks are added. The technical solution proposed by this invention is as follows:

[0006] In a first aspect, the present invention provides a method for efficient processing of multi-task, multi-satellite telemetry data, comprising:

[0007] Obtain the task registration information configured in the front-end interface, and dynamically allocate the task plugin to the optimal computing node based on the resource status evaluation results of each computing node in the server cluster.

[0008] The task registration information is uploaded to the configuration center, and the telemetry parameter configuration information contained in the task registration information is stored in the memory database in the form of key-value pairs; the memory database supports millisecond-level data reading.

[0009] According to the assigned task plugin, the task telemetry data is retrieved from the designated information source, and the task telemetry data is subdivided step by step based on the pre-built data splitting model to obtain telemetry data streams for each frequency band.

[0010] Extract telemetry source packets based on telemetry data streams from each frequency band;

[0011] For each extracted telemetry source packet, a telemetry source packet parsing task is allocated through a thread pool, and based on the stored telemetry parameter configuration information, all parameters in the telemetry source packet are calculated in parallel using a parameter calculation virtual thread to obtain the parsing results of each source packet.

[0012] The parsing results of each source packet are merged according to the frame structure, and a telemetry frame processing result structure is generated, serialized, and then sent to the message queue.

[0013] Optionally, the resource status assessment result is an idle resource score, which is calculated based on the remaining memory, number of CPU cores, and CPU utilization of the computing node.

[0014] Optionally, the pre-built data routing model is a three-level data routing model based on mission-satellite-frequency band; the step of receiving mission telemetry data from a designated source according to the assigned mission plugin, and subdividing the mission telemetry data level by level based on the pre-built data routing model to obtain telemetry data streams for each frequency band includes:

[0015] A data access thread is established, and the task plugin is controlled to retrieve task telemetry data from a specified information source based on the established data access thread and the task registration information, and put the task telemetry data into a blocking queue. At the same time, a data splitting thread is established to asynchronously retrieve message data from the blocking queue.

[0016] The message data is processed by splitting it into different levels based on the three-level data splitting model of mission-satellite-frequency band to obtain telemetry data streams for each frequency band.

[0017] Optionally, the method further includes:

[0018] A blocking queue is dynamically established based on the number of satellites and frequency bands in the mission registration information;

[0019] Establish a mapping relationship using mission-satellite-frequency band as the key and the corresponding memory address of the blocked queue as the value;

[0020] Based on the mapping relationship, the telemetry data streams of each frequency band are routed to the corresponding blocking queues;

[0021] Dynamically establish frequency band-level telemetry data stream processing tasks using each blocked queue as input;

[0022] The processing tasks are submitted to the frequency band-level processing thread pool for parallel processing of multi-frequency band telemetry data streams.

[0023] The frequency band-level telemetry data stream processing task includes: asynchronously acquiring the frequency band telemetry data stream from the blocking queue, and performing subsequent processing on the acquired frequency band telemetry data stream. The subsequent processing includes at least data preprocessing, efficient computation, and result return and distribution processing.

[0024] Optionally, each band telemetry data stream consists of a continuous sequence of telemetry frames, and the telemetry frame format includes a frame synchronization word, a frame count, and a master header pointer;

[0025] The extraction of telemetry source packets based on telemetry data streams of each frequency band includes:

[0026] For each frequency band telemetry data stream, frame synchronization word verification, frame length verification, and frame count continuity verification are performed. If any verification fails, the telemetry frame is discarded as an error frame, and the telemetry frame that passes the verification is obtained.

[0027] Specifically, when extracting telemetry source packets, the position of the first source packet header in the telemetry frame is located according to the dominant header pointer, and the source packet is extracted based on the telemetry source packet structure; the size of the telemetry source packet is determined according to the length of the data field, and when the telemetry source packet is stored across frames, the remaining part is obtained from the front end of the data area of ​​the adjacent frame.

[0028] Optionally, for each extracted telemetry source packet, a telemetry source packet parsing task is allocated through a thread pool, and based on the stored telemetry parameter configuration information, all parameters within the telemetry source packet are calculated in parallel using a parameter calculation virtual thread to obtain the parsing results for each source packet, including:

[0029] Extract the packet identification code of the telemetry source packet;

[0030] The corresponding telemetry source packet configuration information is extracted from the memory database using the mission code-satellite code-packet identifier as the key; the telemetry source packet configuration information contains configuration information for each parameter.

[0031] The configuration information of each parameter is encapsulated, and a telemetry source packet parsing auxiliary object is constructed based on the encapsulated parameter configuration information.

[0032] Based on the telemetry source packet parsing auxiliary object and the dynamic generation of telemetry source packet parsing tasks, the telemetry source packet parsing tasks are submitted to the telemetry source packet processing thread pool for processing; in particular, each received telemetry source packet is used to form a parsing task and submit it to the thread pool, and multiple telemetry source packets are parsed in parallel.

[0033] Multiple parameter calculation virtual threads are established in the telemetry source packet parsing task processing thread. One parameter calculation virtual thread is assigned to each parameter in the telemetry source packet, and parameter calculation is completed in parallel based on the encapsulated parameter configuration information.

[0034] Obtain the calculation results of the virtual thread for each parameter, and merge the calculation results of all parameters of the same telemetry source packet into the parsing result of that source packet;

[0035] The telemetry source packet parsing auxiliary object includes at least a parameter source code extraction function, a source code storage variable, and a source code inversion function. The parameter source code extraction function is used to locate and extract the original data of each parameter from the telemetry source packet. The source code storage variable is used to temporarily store the original data of the parameters extracted from the telemetry source packet. The source code inversion function is used to perform format conversion and engineering value calculation on the original data of the parameters according to the parameter data type.

[0036] Secondly, the present invention also provides a high-efficiency processing system for multi-task, multi-satellite telemetry data, comprising the following modules:

[0037] The task load balancing module is used to obtain the task registration information configured in the front-end interface, dynamically allocate the task plugins to the optimal computing node based on the resource status evaluation results of each computing node in the server cluster, and upload the task registration information to the configuration center, and store the telemetry parameter configuration information contained in the task registration information in the form of key-value pairs in the memory database; the memory database supports millisecond-level data reading.

[0038] The data acquisition and splitting module is used to acquire task telemetry data from a specified information source according to the assigned task plug-in, and to subdivide the task telemetry data step by step based on a pre-built data splitting model to obtain telemetry data streams for each frequency band.

[0039] The data preprocessing module is used to extract telemetry source packets based on telemetry data streams from each frequency band.

[0040] The parameter calculation module is used to allocate telemetry source packet parsing tasks through a thread pool for each extracted telemetry source packet, and to use the parameter calculation virtual thread to calculate all parameters in the telemetry source packet in parallel based on the stored telemetry parameter configuration information, so as to obtain the parsing results of each source packet.

[0041] The result merging and distribution module is used to merge the parsing results of each source packet according to the frame structure, generate a telemetry frame processing result structure, serialize it, and send it to the message queue.

[0042] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the efficient processing method for multi-task multi-satellite telemetry data as described in the first aspect above.

[0043] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the efficient processing method for multi-task multi-satellite telemetry data as described in the first aspect above.

[0044] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the efficient processing method for multi-task multi-satellite telemetry data as described in the first aspect above.

[0045] Based on the above technical solution, the beneficial effects of the present invention compared with the prior art are as follows:

[0046] The present invention provides a method, system, and electronic device for efficient processing of multi-task, multi-satellite telemetry data. By acquiring the real-time computing resource status of each computing node in a server cluster, the system dynamically allocates task plugins to the optimal computing node. This mechanism ensures efficient resource utilization and, even with high-bitrate data input, avoids single-point overload through load balancing, thereby improving overall processing efficiency. For each telemetry source packet, a thread pool is used to allocate parsing tasks, and virtual threads are used to solve all parameters within the source packet in parallel. This highly parallel processing method significantly improves data processing speed and enables real-time processing of high-bitrate telemetry data. Telemetry parameter configuration information is stored in an in-memory database, supporting millisecond-level data retrieval. This design ensures rapid access to configuration information during parameter calculation, further improving processing efficiency. Moreover, users can flexibly define task types, priorities, and other parameters by configuring task registration information through a front-end interface. This flexibility allows the system to easily adapt to multi-task, multi-satellite processing scenarios and meet the needs of different tasks. Uploading task registration information to a configuration center enables centralized management and maintenance of tasks. When new tasks are added, they can be quickly expanded and deployed simply by configuring them in the configuration center. Asynchronous communication is achieved through message queues, where data processing results are serialized and sent to the message queue. This design decouples the data processing flow from other systems (such as storage and analysis systems), improving the system's scalability and maintainability. When new tasks or requirements are added, only corresponding adjustments need to be made on the message queue's consumer side, solving the scalability issues in multi-task, multi-star processing scenarios.

[0047] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained through the structures particularly pointed out in the description and the drawings.

[0048] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating the efficient processing method for multi-task, multi-satellite telemetry data provided by the present invention.

[0051] Figure 2 This is a schematic diagram of the processing flow of the task load balancing module provided by the present invention.

[0052] Figure 3 This is a schematic diagram of the processing flow of the data input and output module provided by the present invention.

[0053] Figure 4 This is a schematic diagram of the three-level data splitting model provided by the present invention.

[0054] Figure 5 This is a schematic diagram of the telemetry frame structure provided by the present invention.

[0055] Figure 6 This is a schematic diagram of the telemetry source packet filling provided by the present invention.

[0056] Figure 7 This is a schematic diagram of the telemetry source packet structure provided by the present invention.

[0057] Figure 8 This is a schematic diagram of the processing steps of the solution module provided by the present invention.

[0058] Figure 9 This is a schematic diagram of the structure of the high-efficiency processing system for multi-task, multi-satellite telemetry data provided by the present invention.

[0059] Figure 10 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0061] Existing high-bitrate telemetry data processing methods split telemetry data packets into multiple subframes with the same structure, obtain the frame number and time code of each subframe, and then perform splitting and parsing of each subframe's data. Finally, the parsed telemetry parameters are sorted and merged. This method achieves fast parsing of single-frame data through parallel processing of each subframe.

[0062] The drawbacks of this processing method are as follows: First, this method only performs subframe splitting and processing at the telemetry data packet level, resulting in coarse-grained parallel processing. When dealing with scenarios involving massive, high-speed telemetry data transmission, the processing latency cannot be optimized. Second, this method does not consider telemetry processing methods in multi-task, multi-satellite scenarios, and cannot adapt to the real-time processing requirements of multi-task, multi-satellite telemetry. Third, it lacks universality. Currently, most telemetry data packets are divided into three parts: packet identification code, packet sequence control, and data field. The packet sequence count and timecode are located in a fixed area and exist only once. Therefore, the above method cannot be used to split the telemetry packet into multiple subframes and obtain the frame number and timecode of each subframe.

[0063] To address the challenge of real-time and efficient processing of high-code-rate telemetry data in multi-task, multi-satellite scenarios, this invention proposes a method, system, and electronic device for efficient processing of multi-task, multi-satellite telemetry data. By significantly improving the efficiency of telemetry data processing through higher parallel processing granularity, it is adapted to multi-task, multi-satellite processing scenarios and can be quickly expanded when new tasks are added.

[0064] The following is combined Figures 1-10 This invention describes a method, system, and electronic device for efficient processing of multi-task, multi-satellite telemetry data.

[0065] This invention provides a method for efficient processing of multi-task, multi-satellite telemetry data. It achieves a parallel and efficient processing flow for multi-task, multi-satellite telemetry data by combining task load balancing, a three-level data routing model, fine-grained concurrent processing using thread pools (multithreading technology in Java), Redis caching, and virtual threads (lightweight threads in Java). The method employs a distributed, modular system architecture, mainly comprising five functional modules: task load balancing, data acquisition and routing, data preprocessing, parameter calculation, and result merging and distribution. These modules work collaboratively to achieve efficient processing of multi-task, multi-satellite telemetry data. Here, "multiple satellites" refers to multiple satellites. (See reference...) Figure 1 As shown, the method includes the following:

[0066] S110. Obtain the task registration information configured in the front-end interface, and dynamically allocate the task plugin to the optimal computing node based on the resource status evaluation results of each computing node in the server cluster; upload the task registration information to the configuration center, and store the telemetry parameter configuration information contained in the task registration information in the form of key-value pairs in the memory database; the memory database supports millisecond-level data reading.

[0067] This invention develops a task load balancing module, such as... Figure 2 As shown, task plugins are registered through task registration information configured in the front-end interface, providing high task scalability. However, as the number of tasks increases, the computing resources of a single server cannot meet the demands of multi-task processing. This task load balancing module will handle different task plugins (such as...) Figure 2 Task Plugin 1, Task Plugin 2, Task Plugin 3 The processing of task plugins (n) is dynamically distributed across multiple computing nodes in the server cluster to achieve parallel processing of multiple tasks, thereby realizing load balancing for multi-task processing. After distribution, a mapping relationship between the task plugin and the server address is automatically generated and injected into the task registration information. Task registration information, except for telemetry parameter configurations, is uploaded to the configuration center.

[0068] Specifically, real-time monitoring of each computing node in the server cluster (such as...) Figure 2 The system monitors the computing resource status of compute nodes 1, 2, and 3 in the server cluster, including CPU utilization, memory usage, and disk I / O. The main function of the task load balancing module is to dynamically allocate task plugins across the server cluster to achieve load balancing. This module assesses the computing resource status of each compute node in the cluster and then dynamically assigns different task plugins to the servers with the most available resources based on this assessment. This enables parallel processing of multiple tasks and improves overall processing efficiency.

[0069] The configuration center, based on ZooKeeper (an open-source distributed application coordination service), stores and manages configuration information and notifies all listening nodes of configuration changes. Each task plugin listens to the configuration center upon startup. After updating the task configuration in the configuration center, the new configuration is synchronously distributed to downstream task plugins. Each relevant plugin updates its memory information according to the distributed configuration and uses the new configuration to guide subsequent computation and processing.

[0070] The aforementioned task registration information mainly includes, but is not limited to, source address and port, task code, task name, satellite identifier, satellite code, satellite name, supported frequency bands, frequency band identifiers, and telemetry parameter configuration information. The telemetry parameter configuration information provides a unified Excel template, which users can use to compile and import data in batches. The telemetry parameter configuration information includes fields such as satellite code, source packet identification code, source packet name, subsystem to which the source packet belongs, parameter code, parameter name, parameter byte number, parameter bit number, parameter data type, ground processing formula, unit, and normal value range, which guide subsequent telemetry parameter parsing. An example of one row in the Excel spreadsheet is shown in Table 1.

[0071] Table 1

[0072]

[0073] GNC stands for Guidance, Navigation and Control.

[0074] In Table 1 above, the byte number of the parameter is 0~3, indicating that it occupies the first 4 bytes of the telemetry source packet, a total of 32 binary bits. The bit number of the parameter is 0~31, indicating that it occupies all the binary bits of the 4 bytes. The index number of the highest byte or the highest bit within a byte is recorded as 0. The index numbers increase sequentially from high to low. This parameter is inverted according to Float (single-precision floating-point type). The inverted value is substituted into the ground processing formula to calculate the final engineering value.

[0075] Because the telemetry parameter configuration information is a large dataset, directly storing it in the configuration center and synchronizing it to the task plugin process memory would result in high process memory usage, impacting server processing efficiency and the number of deployable processes, while also degrading the configuration center's performance. Therefore, this task load balancing module uses task code-satellite code-telemetry packet identifier as the key and specific telemetry packet configuration information as the value to form key-value pairs, storing them in an in-memory database (e.g., ...). Figure 2 The Redis cluster (including nodes 1, 2, and 3) is used for subsequent computation and parsing. During this process, the task plugin accesses the in-memory database to dynamically read and parse the source package configuration guide. Task registration information, excluding telemetry parameter configuration information, is uploaded to the configuration center.

[0076] After the configuration is successfully uploaded to the configuration center, the corresponding task plugin is started on the assigned computing node. After the task plugin process starts, it will obtain the relevant configuration of the task from the configuration center and put it into the process memory. Other configuration information required for the calculation of telemetry parameters (thread pool configuration, measurement and control resource information, control and processing switch quantities, etc.) are preheated into the process memory after the task plugin starts.

[0077] This invention, through dynamic task allocation and load balancing algorithms, fully utilizes the computing resources in a server cluster, avoiding resource idleness or overload. The flexibility of task registration information and centralized management in the configuration center enable easy adaptation to the addition of new tasks, achieving rapid expansion. Storing telemetry parameter configuration information in an in-memory database (such as a Redis cache) supports millisecond-level data retrieval, improving the access speed of configuration information during parameter calculation. Through these steps, the task load balancing module achieves dynamic allocation of multi-task plugins in the server cluster, effectively supporting multi-task parallel processing and improving overall processing efficiency and reliability.

[0078] S120. According to the assigned task plugin, the task telemetry data is retrieved from the designated information source, and the task telemetry data is subdivided step by step based on the pre-built data splitting model to obtain telemetry data streams for each frequency band.

[0079] This invention develops a data acquisition and splitting module. Task plugins on each server acquire specific task data from designated sources through task registration information and asynchronously perform data splitting. Specifically, based on the assigned task plugins, task telemetry data is acquired from designated sources (such as satellite ground stations, data relay satellites, etc.). The data acquisition and splitting module supports multiple communication protocols and data formats to ensure accurate data reception.

[0080] Based on a pre-built data routing model, the incoming telemetry data streams are subdivided step by step. For example, they are first routed by satellite, and then further subdivided by frequency band or data type to obtain telemetry data streams for each frequency band. This step facilitates subsequent targeted processing of data from different frequency bands or types.

[0081] S130. Extract telemetry source packets based on telemetry data streams of each frequency band.

[0082] This invention develops a data preprocessing module for preprocessing the split telemetry data stream, such as telemetry frame format verification, error frame removal, source packet splitting, and other related processing, and distributes the split telemetry source packets to the next processing node.

[0083] S140. For each extracted telemetry source packet, a telemetry source packet parsing task is allocated through a thread pool, and based on the stored telemetry parameter configuration information, all parameters in the telemetry source packet are calculated in parallel using a parameter calculation virtual thread to obtain the parsing results of each source packet.

[0084] This invention develops a parameter calculation module to achieve efficient processing and parsing of telemetry source packet parameters. For each extracted telemetry source packet, a parsing task is allocated through a thread pool. The thread pool manages multiple worker threads, each responsible for parsing one or more source packets. Parallel calculation of all parameters within the source packet is achieved using parameter calculation virtual threads (such as coroutines), improving processing efficiency. Based on stored telemetry parameter configuration information, the parameters within the source packet are calculated. The calculation process may include data decoding, unit conversion, numerical calculation, etc., ultimately yielding the parsed results for each parameter.

[0085] In the parameter calculation module, after each telemetry source packet is parsed, all its internal parameters are calculated independently, and the calculation results are encapsulated into a unified structure. These structures are collected into a set of parsed telemetry source packets, representing the parsing results of all source packets within a telemetry frame.

[0086] S150. Merge the parsing results of each source packet according to the frame structure, generate a telemetry frame processing result structure, serialize it, and send it to the message queue.

[0087] This invention develops a result merging and distribution module that merges the parsing results of each source packet in the current frame, encapsulates them into a telemetry frame processing result structure, and distributes it downstream. Specifically, after receiving these source packet parsing results, the result merging and distribution module merges them according to the order of the telemetry source packets in the frame. The merged telemetry frame processing result is encapsulated into a structure containing key information such as the mission name, satellite name, frequency band name, data frame reception time, downlink telemetry and control equipment code, and the set of telemetry source packet parsing results. This information together constitutes a complete description of the telemetry frame processing result. The structure design fully considers data integrity and readability, ensuring that downstream systems can easily parse and use this data.

[0088] To transmit the telemetry frame processing result structure to the message queue, the structure is serialized into a transmittable format. The serialization process converts the data in the structure into a byte stream for transmission over the network. Serialization methods such as JSON and Protobuf can be chosen, depending on system requirements and performance considerations. Serialized data is compact and efficient, making it suitable for network transmission.

[0089] Message queues such as Kafka and RabbitMQ are chosen as the data transmission channel. Message queues provide asynchronous communication mechanisms to ensure reliable data transmission and processing. The serialized telemetry frame processing results are sent to the designated message queue topic. Downstream systems can consume this data from the topic and perform further processing and analysis. For example, the data can be stored in a database for later querying or used for real-time data analysis to provide decision support.

[0090] By merging source packet parsing results according to frame structure, the integrity of telemetry frame processing results is ensured, providing a reliable foundation for subsequent data analysis and decision-making. The serialization process converts structured data into a compact byte stream, improving data transmission efficiency over the network. Data transmission via message queues ensures data reliability and order, guaranteeing data integrity and consistency even under network fluctuations or system failures. The message queue mechanism allows this method to be easily integrated with other systems, enabling data sharing and exchange.

[0091] The present invention provides a high-efficiency processing method for multi-task, multi-satellite telemetry data. By acquiring the real-time computing resource status of each computing node in the server cluster, the method dynamically allocates task plugins to the optimal computing node. This mechanism ensures efficient resource utilization and, even with high bitrate data input, avoids single-point overload through load balancing, thereby improving overall processing efficiency. For each telemetry source packet, a parsing task is allocated through a thread pool, and parameter calculation virtual threads are used to calculate all parameters within the source packet in parallel. This highly parallel processing method significantly improves data processing speed and enables real-time processing of high bitrate telemetry data. Telemetry parameter configuration information is stored in an in-memory database, supporting millisecond-level data retrieval. This design ensures rapid access to configuration information during parameter calculation, further improving processing efficiency.

[0092] Furthermore, by configuring task registration information through the front-end interface, users can flexibly define parameters such as task type and priority. This flexibility allows the system to easily adapt to multi-task, multi-star processing scenarios, meeting the needs of different tasks. Uploading task registration information to the configuration center enables centralized management and maintenance of tasks. When a new task is added, only corresponding configuration in the configuration center is needed to quickly expand and deploy the task. Asynchronous communication is achieved through message queues, serializing data processing results and sending them to the message queue. This design decouples the data processing flow from other systems (such as storage and analysis systems), improving the system's scalability and maintainability. When new tasks or new requirements are added, only corresponding adjustments need to be made on the message queue consumer side, solving the scalability issues in multi-task, multi-star processing scenarios.

[0093] This invention significantly improves the processing efficiency of high-code-rate telemetry data in multi-task, multi-satellite scenarios through dynamic task allocation and load balancing, multi-threaded / virtual-thread parallel processing, and millisecond-level data retrieval from an in-memory database. Simultaneously, through flexible task configuration and registration, centralized management based on a configuration center, and asynchronous communication and decoupling via message queues, it enables rapid system expansion and deployment. These innovations collectively address the shortcomings of existing technologies and meet the real-time, accuracy, and reliability requirements of aerospace, aviation, and other fields for telemetry data processing.

[0094] In some embodiments, the resource status assessment result is an idle resource score, which is calculated based on the remaining memory, number of CPU cores, and CPU utilization of the computing node. The calculation formula is:

[0095]

[0096] In the above formula, Score for idle resources; The remaining memory (in GB) represents the memory resources currently available to the compute node. The number of CPU cores represents the number of physical CPU cores on a compute node. CPU utilization represents the current percentage of CPU resources being used, ranging from 0 to 1 (or 0% to 100%). This formula comprehensively considers three key factors—remaining memory, number of CPU cores, and CPU utilization—to assess the idle resource status of a computing node.

[0097] Real-time acquisition of each computing node , and The values ​​are substituted into the above formula, and the idle resource score of each server (i.e., the computing node) in the server cluster is calculated using the above formula. The task plugin is assigned to the server with the highest idle resource score. If the number of servers with the highest idle resource score is greater than 1 (there is a phenomenon of the same score), one is randomly selected. The random algorithm is as follows: put the n server IPs with the same idle resource score into a one-dimensional array Arr. The length of the array is the number of servers with the same score, that is: Arr[0]=ip1, Arr[1]=ip2, ,Arr[n 1]=ip n Generate a random decimal integer and take the remainder after dividing by the array length. Use the remainder as the index to retrieve the corresponding IP address in the array Arr. IP1, IP2, IP n For the server IP address. Arr[0], Arr[1], , Arr[n Arr[0] represents the element in the array. Arr[0] stores the network address of the first server, Arr[1] stores the network address of the second server, and so on, Arr[n] stores the network address of the second server. 1] Store the network address of the nth server.

[0098] This invention achieves load balancing by dynamically allocating task plugins to the most resource-rich computing nodes through real-time calculation of the idle resource score of each computing node. This prevents performance degradation on some computing nodes due to resource overload while improving resource utilization on other computing nodes. Load balancing ensures that tasks can be executed quickly on resource-rich computing nodes, thereby improving overall processing efficiency. In scenarios requiring real-time processing of high-bitrate telemetry data, avoiding resource overload reduces the risk of task failure due to computing node failure or performance degradation. This improves the overall reliability of the system, ensuring timely and accurate processing of telemetry data. When new tasks are added, the system can dynamically allocate tasks to suitable computing nodes based on the new idle resource score. This allows the system to flexibly adapt to changes in task volume without complex reconfiguration or hardware upgrades. By comprehensively considering memory, CPU core count, and CPU utilization, the system can more accurately assess the idle resource status of computing nodes. This helps optimize resource utilization, ensuring that each computing node can achieve maximum performance within its capabilities.

[0099] In some embodiments, the data acquisition and splitting module described above completes the splitting of multi-task, multi-satellite, and multi-frequency band data based on a three-level data splitting model and a blocking queue, realizing the isolation of multi-task, multi-satellite, and multi-frequency band telemetry data, providing support for parallel processing of each data stream, and increasing the data processing concurrency through two asynchronous processes, effectively avoiding the time loss caused by streaming serial execution. The pre-built data splitting model described above is a three-level data splitting model based on task-satellite-frequency band; the process described in S120 above, which involves acquiring task telemetry data from a specified source according to the assigned task plugin, and subdividing the task telemetry data level by level based on the pre-built data splitting model to obtain telemetry data streams for each frequency band, includes:

[0100] S1201. Establish a data access thread, control the task plugin to retrieve task telemetry data from a specified information source based on the established data access thread and the task registration information, and put the task telemetry data into a blocking queue. At the same time, establish a data splitting thread to asynchronously retrieve message data from the blocking queue.

[0101] First, such as Figure 3 As shown, an independent data access thread is established for each mission plugin. This thread is responsible for establishing a connection with the designated information source (such as a satellite ground station or other data source) and continuously receiving mission telemetry data. The data access thread locates and connects to the correct information source based on the mission registration information (including information source address and port, etc.), thereby receiving specific mission telemetry data.

[0102] Received telemetry data (i.e.) Figure 3The original data frames in the algorithm, including data 1, data 2, data 3... data n, are immediately placed into a blocking queue. A blocking queue is a special type of queue where, when the queue is empty, threads attempting to retrieve data from it are blocked until new data is added. This mechanism ensures the order and integrity of the data while preventing data loss.

[0103] S1202. The three-level data decoupling model based on mission-satellite-frequency band is used to decouple the message data step by step to obtain telemetry data streams for each frequency band.

[0104] In parallel with the data access thread, one or more data routing threads are established. These threads are responsible for asynchronously retrieving message data from the blocking queue and performing subsequent routing processing. The data routing threads employ a three-level data routing model based on mission-satellite-frequency band to progressively subdivide the received telemetry data stream.

[0105] The structure of the three-level data splitting model is as follows: Figure 4 As shown, the top level is the task hierarchy, with different tasks (i.e.) Figure 4 The first level (tasks 1-n) acquires data from different source addresses. The second level is satellite-level demultiplexing, which separates the telemetry data streams of each satellite by associating the satellite identifiers in the mission registration information with the satellite identifiers in the telemetry frames. The lowest level is frequency band-level demultiplexing. Currently, satellites can simultaneously support multiple data downlink frequency bands, and data is split across different frequency bands by associating the frequency band identifiers in the mission registration information with the frequency band identifiers in the telemetry frames. The task plugin, based on a three-level data routing model, ultimately subdivides data into frequency band-level data streams. Simultaneously, it dynamically establishes blocking queues based on the number of satellites and frequency bands in the task registration information. A mapping relationship is established using task-satellite-frequency band as the key and the corresponding blocking queue memory address as the value, and this mapping is loaded into memory. Data from each frequency band is then routed to its corresponding blocking queue according to this mapping. Frequency band-level telemetry data stream processing tasks are dynamically created using each blocking queue as input. These tasks are submitted to a frequency band-level processing thread pool for parallel processing of multi-frequency band data. Each frequency band-level telemetry data stream processing task asynchronously retrieves data from the blocking queue and performs subsequent data preprocessing, efficient computation, result merging, and distribution. (Refer to...) Figure 3 As shown, data from satellite band 1 is placed into blocking queue 1, data from satellite band 2 is placed into blocking queue 2, data from satellite band 1 is placed into blocking queue 3, and data from satellite band 2 is placed into blocking queue 4.

[0106] This invention employs a three-level data splitting model to subdivide telemetry data streams into smaller processing units (frequency band-level data streams), thereby achieving finer-grained parallel processing. This significantly improves processing efficiency, enabling the system to process large amounts of telemetry data more quickly. The three-level data splitting model allows the system to easily adapt to the data processing needs of different tasks, satellites, and frequency bands. When a new task or satellite is added, the system simply updates the task registration information and adjusts the configuration of the data splitting threads. Through blocking queues and asynchronous processing mechanisms, the system can utilize computing resources more effectively. When a processing unit (such as a data splitting thread for a specific frequency band) is temporarily idle, it can immediately retrieve new data from the blocking queue for processing, thus avoiding resource waste. The three-level data splitting model also enhances system reliability. Even if a processing unit fails or experiences performance degradation, it will only affect the data stream processed by that unit, without affecting other units or the operation of the entire system. This reduces the risk of system failure and improves the system's fault tolerance.

[0107] In some embodiments, the method further includes:

[0108] S1203. Dynamically establish a blocking queue based on the number of satellites and frequency bands in the mission registration information.

[0109] Based on the number of satellites and the number of frequency bands supported by each satellite provided in the mission registration information, a blocking queue is dynamically established for each frequency band. The blocking queue is a thread-safe queue implementation. When the queue is empty, threads attempting to retrieve data from the queue are blocked until new data is added. When the queue is full, threads attempting to add data are also blocked until space becomes available. This feature ensures data order and integrity while avoiding data loss and thread contention.

[0110] S1204. Establish a mapping relationship using the mission-satellite-frequency band as the key and the corresponding congestion queue memory address as the value. This mapping relationship enables quick location of the congestion queue corresponding to a specific mission, satellite, and frequency band telemetry data stream. The mapping relationship is stored in the system's memory for fast access. When mission registration information changes (such as adding a new satellite or frequency band), the system updates the mapping relationship accordingly.

[0111] S1205. Based on the mapping relationship, the telemetry data streams of each frequency band are routed to the corresponding blocking queue.

[0112] Based on the mapping relationship, telemetry data streams from each frequency band are routed to their corresponding blocking queues. The data routing process is asynchronous, meaning that after the data access thread places data into the blocking queue, it can continue receiving new data without waiting for the data to be processed. This improves the system's throughput and response speed.

[0113] S1206. Dynamically establish frequency band-level telemetry data stream processing tasks using each blocking queue as input. Each processing task is responsible for processing a telemetry data stream of a specific frequency band. When a processing task is established, it is configured to asynchronously obtain the frequency band telemetry data stream from the corresponding blocking queue and perform subsequent processing.

[0114] S1207. Submit the processing task to the frequency band-level processing thread pool for parallel processing of multi-frequency band telemetry data streams.

[0115] A band-level processing thread pool is a pre-created thread pool used to manage and reuse processing threads. It improves system performance and resource utilization because thread creation and destruction are expensive operations. Band-level telemetry data stream processing tasks are submitted to the band-level processing thread pool for parallel processing. The thread pool dynamically allocates threads to handle these tasks based on the current load.

[0116] The frequency band-level telemetry data stream processing task includes: asynchronously acquiring the frequency band telemetry data stream from the blocking queue, and performing subsequent processing on the acquired frequency band telemetry data stream. This subsequent processing includes at least data preprocessing, efficient computation, and result aggregation and distribution. Specifically, after asynchronously acquiring the frequency band telemetry data stream from the blocking queue, the frequency band-level telemetry data stream processing task first performs data preprocessing, such as frame format verification, erroneous frame removal, and source packet splitting. Next, the processing task uses the parameter computation module to perform parameter computation on the telemetry source packets to obtain the final settlement result. Finally, the processing task merges the computation results, encapsulates them into a telemetry frame processing result structure, serializes it, and sends it to the message queue for consumption by downstream systems.

[0117] This invention, by dynamically establishing blocking queues and mapping relationships, enables the system to quickly route telemetry data streams to corresponding processing units, reducing data transmission latency and overhead. Simultaneously, the use of frequency-band-level processing thread pools improves the parallelism of processing tasks, further enhancing data processing efficiency. The system can dynamically adjust the number of blocking queues and processing tasks based on the number of satellites and frequency bands in the task registration information, thus adapting to the data processing needs of different tasks, satellites, and frequency bands. This flexibility allows the system to easily handle various complex data processing scenarios. The use of blocking queues and thread pools allows the system to utilize computing resources more effectively. When a processing unit is temporarily idle, it can immediately retrieve new data from the blocking queue for processing, thus avoiding resource waste. At the same time, the thread pool's reuse mechanism reduces the overhead of thread creation and destruction. Through the asynchronous processing mechanism of the blocking queues and the load balancing capability of the thread pool, the system can handle large amounts of telemetry data streams more stably. Even if a processing unit fails or experiences performance degradation, it will only affect the data stream processed by that unit, without affecting other units or the operation of the entire system. This reduces the risk of system failure and improves the system's fault tolerance.

[0118] In some embodiments, such as Figure 5 As shown, the telemetry frame format mainly includes a protocol header (containing satellite identifier, frequency band identifier and other related information), frame synchronization word, frame count, master header pointer (indicating the position of the first source packet header in the telemetry frame), data area, checksum and other parts. In order to ensure processing efficiency and reduce the time cost of the data preprocessing module, the frame format verification only performs simple checks such as frame synchronization word verification, frame length verification and frame count continuity judgment. If the verification fails, it is rejected.

[0119] The extraction of telemetry source packets based on telemetry data streams of each frequency band as described in S130 above includes:

[0120] S1301. Perform frame synchronization word verification, frame length verification, and frame count continuity verification on the telemetry data stream of each frequency band. If any verification fails, the telemetry frame is discarded as an error frame, and the telemetry frame that passes the verification is obtained.

[0121] Each frequency band telemetry data stream consists of a continuous sequence of telemetry frames, each starting with a frame synchronization word. First, the received telemetry frames undergo frame synchronization word verification to ensure the correct start position of the frame. If the frame synchronization word verification fails, the telemetry frame is determined to be an erroneous frame and discarded. After the frame synchronization word verification passes, the length of the telemetry frame is verified. The frame length should conform to predefined format requirements; if the frame length does not meet the requirements, it is determined to be an erroneous frame and discarded. The telemetry frame contains frame count information to identify the frame order. The frame count is verified for continuity to ensure the frames are in the correct order. If the frame count is discontinuous, it is determined to be an erroneous frame and discarded.

[0122] S1302. Extract telemetry source packets from the verified telemetry frames; wherein, when extracting telemetry source packets, the position of the first source packet header in the telemetry frame is located according to the dominant header pointer, and the source packet is extracted based on the telemetry source packet structure; the size of the telemetry source packet is determined according to the data field length, and when the telemetry source packet is stored across frames, the remaining part is obtained from the front end of the data area of ​​the adjacent frame.

[0123] Telemetry source packets are located in the telemetry frame data area, such as Figure 6 As shown, the placement rules are as follows: source packets are continuously placed into each data area of ​​the frame data area. If the remaining space in the data area is insufficient to store the next source packet, the current data area is filled with the source packet, and the remaining part of the source packet is placed at the beginning of the next telemetry frame data area. It should be noted that if the remaining space in the data area is no longer used to store source packets after the last source packet is placed, the bytes are filled with a fixed hexadecimal codeword (0xAA).

[0124] The verified telemetry frame contains a master header pointer, which indicates the position of the first source packet header in the telemetry frame. The starting position of the source packet header is located based on the master header pointer. Based on a predefined telemetry source packet structure, the source packet is parsed and extracted starting from the located source packet header position. The telemetry source packet structure is as follows: Figure 7 As shown, the packet consists of a header and a body. The header mainly includes three parts: packet identification code, packet sequence control, and data field length. The body is the packet data area (i.e., the data field). Based on the above telemetry source packet placement rules and source packet structure, telemetry source packets are extracted. The extracted telemetry source packets are then sent to the high-efficiency calculation module for source packet parameter calculation.

[0125] When the size of a telemetry source packet exceeds the capacity of a single frame's data area, the source packet will be stored across frames. The total size of the telemetry source packet is determined based on the data field length, and it is then determined whether cross-frame processing is necessary. If cross-frame processing is required, the remaining portion of the source packet is retrieved from the beginning of the data area of ​​the adjacent frame and appended to the source packet data extracted in the current frame.

[0126] Assuming that, according to the requirements of the telemetry mission, such as Figure 6As shown, this example demonstrates the dynamic generation of 10 telemetry source packets (source packet 1 to source packet 10). Figure 6 Frames n, n+1, and n+2 represent continuously transmitted telemetry data streams, each containing one or more telemetry source packets. Before filling, the frame header area of ​​each source packet is initialized, including a frame synchronization word, a frame length field, a frame counter, and a master header pointer. The frame synchronization word is preset to a fixed value for receiver frame synchronization. The frame length field is initialized to 0, and its actual length is calculated after the source packet data is filled. The frame counter increments from 1 to ensure frame sequence continuity. The master header pointer points to the beginning of the first source packet within the frame. Source packets 1-4 and the first half of source packet 5 are placed in the first telemetry frame data area. The remaining portion of source packet 5, source packets 6, 7, and the first half of source packet 8 are placed in the second telemetry frame data area. The remaining portion of source packet 8, source packet 9, and source packet 10 are placed in the third telemetry frame data area. The remaining space in the data area is filled with bytes 0xAA.

[0127] This invention effectively eliminates erroneous frames through frame synchronization word verification, frame length verification, and frame count continuity verification, ensuring the accuracy of extracted telemetry frame data. This provides a reliable foundation for subsequent data processing and analysis. The cross-frame source packet processing mechanism enables the system to handle situations where source packets are stored across frames, enhancing its adaptability to complex data formats. This improves the system's robustness, allowing it to operate stably in various data environments. By quickly locating the source packet header position using the dominant header pointer, the system can efficiently extract telemetry source packets, reducing data processing latency. Simultaneously, parsing and extraction based on predefined source packet structures also improves the automation and efficiency of data processing. The telemetry source packet extraction mechanism designed for the system can adapt to telemetry source packets of different lengths and structures, supporting flexible data format definitions. This allows the system to be easily integrated into different satellite telemetry and control systems, meeting diverse data processing needs.

[0128] In some embodiments, the parameter resolution module takes the telemetry source package extracted in S130 as input, and resolves all parameters within the source package based on the telemetry parameter resolution configuration. The resolution result of a single parameter is encapsulated into a unified structure, and the resolution result of the telemetry source package is the collection of all parameter resolution results within the source package. In S110, the telemetry parameter configuration information guiding the resolution of the telemetry source package has been pre-warmed to the Redis cluster (including nodes 1, 2, and 3). Task registration information and telemetry source package resolution thread pool configuration, excluding the telemetry parameter configuration information, have also been pre-warmed to the task plugin process memory. As described in S140 above, for each extracted telemetry source package, a telemetry source package resolution task is allocated through a thread pool, and based on the stored telemetry parameter configuration information, all parameters within the telemetry source package are resolved in parallel using parameter resolution virtual threads to obtain the resolution results for each source package, including:

[0129] S1401. Extract the packet identification code of the telemetry source packet, and extract the corresponding telemetry source packet configuration information from the memory database using the mission code-satellite code-packet identification code as the key.

[0130] Reference Figure 8 As shown, for each telemetry source packet in the telemetry source packet source code (i.e. Figure 8 The system identifies source packets (source packets 1, 2, 3...n) and allocates telemetry source packet parsing tasks through a thread pool. The thread pool manages a group of threads to concurrently execute multiple parsing tasks, improving processing efficiency. Before allocating tasks, the Application Packet Identifier (APID) of the telemetry source packets is extracted. The APID is a unique identifier for the telemetry source packets and is used for subsequent matching of configuration information.

[0131] Using the mission code, satellite code, and packet identifier as keys, the corresponding telemetry source packet configuration information is extracted from the in-memory database. This configuration information includes the configuration details of each parameter within the telemetry source packet, such as parameter type, data format, and calculation formula.

[0132] S1402. Encapsulate the configuration information of each parameter and construct a telemetry source packet parsing auxiliary object based on the encapsulated parameter configuration information.

[0133] The extracted parameter configuration information is encapsulated for subsequent processing. The encapsulated parameter configuration information contains all necessary solution parameters and rules. Based on the encapsulated parameter configuration information, a parameter source code extraction function, a source code storage variable, and a source code inversion function are added to form a telemetry source packet parsing auxiliary object, which is then placed into the memory variable M. This telemetry source packet parsing auxiliary object contains basic information about the telemetry source packet, parameter configuration information, and auxiliary data required for solution.

[0134] The parameter source code extraction function is used to locate and extract the raw data of each parameter from the telemetry source package; the source code storage variable is used to temporarily store the raw data of the parameters extracted from the telemetry source package; the source code inversion function is used to perform format conversion and engineering value calculation on the raw data of the parameters according to the parameter data type.

[0135] S1403. Based on the telemetry source packet parsing auxiliary object and the telemetry source packet, a telemetry source packet parsing task is dynamically generated and submitted to the telemetry source packet processing thread pool for processing; wherein, for each received telemetry source packet, a parsing task is generated and submitted to the thread pool, and multiple telemetry source packets are parsed in parallel;

[0136] Variable extraction is performed from memory variable M to obtain a telemetry source packet parsing auxiliary object. Based on the telemetry source packet parsing auxiliary object and the telemetry source packet itself, telemetry source packet parsing tasks are dynamically generated, with each task corresponding to a telemetry source packet. These tasks are then submitted to the telemetry source packet processing thread pool for processing. The telemetry source packet processing thread pool includes multiple tasks, such as... Figure 8 Task 1, Task 2, Task 3, ..., Task n.

[0137] Each received telemetry source packet undergoes the above processing to form a corresponding telemetry source packet parsing task, which is then submitted to the thread pool. Multiple telemetry source packets are parsed in parallel. After a telemetry source packet parsing task is submitted to the thread pool, a Future object is returned (Future is a Java concurrency module interface; the future processing results of a thread can be automatically placed into a Future instantiated object). Then, exception handling results are submitted, and Future objects are placed into a Future object collection in the order they are returned.

[0138] S1404. In the telemetry source packet parsing task processing thread, establish multiple parameter calculation virtual threads (such as...). Figure 8 The parameter 1-n processing virtual thread allocates a parameter calculation virtual thread for each parameter in the telemetry source packet, and completes parameter calculation in parallel based on the encapsulated parameter configuration information.

[0139] A single telemetry data frame can contain up to 10 telemetry source packets, and each source packet can contain up to several hundred parameters. Therefore, a single telemetry frame can contain up to several thousand telemetry parameters. If multiple tasks, multiple satellites, and multiple frequency bands of telemetry frames are parsed simultaneously, the concurrency of telemetry parameter calculation can reach tens of thousands. Creating tens of thousands of threads for parallel processing would pose a risk of server crashes due to insufficient computing resources. Virtual threads, on the other hand, have the characteristics of low memory consumption, high concurrency, and low computing resource consumption, making them suitable for processing tasks with small computational loads and high concurrency. With current mainstream server configurations, hundreds of thousands or even millions of virtual threads can be created, with extremely low creation and destruction costs. Therefore, to efficiently calculate each telemetry parameter within the source packet, a parameter calculation virtual thread is allocated to each parameter in the telemetry source packet parsing task processing thread. These virtual threads are lightweight and can be created and destroyed quickly, used for parallel execution of parameter calculation tasks.

[0140] Based on the encapsulated parameter configuration information, each parameter calculation virtual thread performs parameter calculation independently. Since the calculation process is parallel, the calculation speed can be significantly improved.

[0141] S1405. Obtain the solution results of the virtual thread for each parameter solution, and merge the solution results of all parameters of the same telemetry source packet into the parsing result of that source packet.

[0142] Retrieve the solution results from the virtual threads for each parameter. These results contain the solution value of each parameter in the telemetry source packet. Merge the solution results of all parameters from the same telemetry source packet into the parsed results of that source packet. The parsed results contain the solution values ​​of all parameters in the telemetry source packet, as well as possible solution status or error information. Merge the solution results from the virtual threads for each parameter into the source packet telemetry parameter parsed results set, which contains the return results of each virtual thread in the telemetry source packet solution thread pool.

[0143] Because all parameters within the source packet are parsed in parallel, the processing time of this source packet is the longest processing time for a single parameter. Simultaneously, all telemetry source packet parsing tasks are submitted to the telemetry source packet processing thread pool for parallel parsing. Therefore, the processing time of all source packets within a single telemetry frame can be recorded as the maximum processing time of all parameters within that frame; that is: the processing time of a single telemetry frame is denoted as... The telemetry frame contains m parameters, and the calculation time for a single parameter is [time value missing]. Where i is the telemetry parameter number, the formula for calculating the solution time of a single telemetry frame is:

[0144] m

[0145] By leveraging thread pools and virtual threads, fine-grained, high-concurrency computation is achieved, reducing telemetry frame-level computation time to that of a single parameter level. This significantly improves telemetry frame computation efficiency, and under current mainstream server configurations, a single frame processing time of approximately 1 millisecond is expected. Currently, the highest real-time telemetry download speed for high bitrates is only tens of frames per second. The telemetry frame processing speed far exceeds the data acquisition and download speed, meeting the real-time computation requirements for high bitrate telemetry. For delayed telemetry transmitted via high-bandwidth data transmission channels, data parsing can also be completed in a short time, facilitating post-analysis by relevant personnel.

[0146] For the Future object set generated by S1403, each Future object is traversed to obtain the telemetry source packet parsing result, which is then placed into the telemetry source packet parsing result set. After traversal, data such as task name, satellite name, frequency band name, data frame reception time, downlink telemetry and control equipment code, and telemetry source packet parsing result set are encapsulated into a telemetry frame processing result structure. This structure is serialized and sent to the Kafka message queue for downstream consumption. The result merging and distribution module only performs simple merging and distribution operations. The processing logic is simple, the time complexity is extremely low, and it will not affect the overall processing efficiency.

[0147] This invention significantly improves the parsing speed and processing efficiency of telemetry source packets by allocating parsing tasks through a thread pool and utilizing virtual threads for parallel parameter calculation. The system can dynamically generate parsing tasks based on the configuration information of the telemetry source packets, adapting to the parsing requirements of different tasks, satellites, and packet identification codes. This flexibility allows the system to easily handle various complex telemetry data parsing scenarios. The use of thread pools and virtual threads enables the system to utilize computing resources more effectively. When a parsing or parameter calculation task is temporarily idle, its occupied thread can be immediately reused by other tasks, thus avoiding resource waste. By performing parameter calculation based on stored telemetry parameter configuration information, the system ensures the accuracy and consistency of the calculation results. Simultaneously, error handling and status feedback mechanisms during the calculation process help to promptly identify and resolve problems.

[0148] The task load balancing module of this invention dynamically allocates multi-task plugins to different computing nodes and can dynamically expand with the addition of new tasks. Each plugin updates its configuration in real time by monitoring the configuration center, effectively supporting multi-task parallel processing. The data acquisition and splitting module implements multi-task, multi-satellite telemetry data splitting and parallel acquisition processing based on a three-level data splitting model of task-satellite-frequency band. The parameter calculation module uses a thread pool to achieve parallel parsing of telemetry source packets. Virtual threads for telemetry parameter calculation are dynamically generated in each telemetry source packet parsing thread to achieve parallel parsing of each telemetry parameter in the source packet, greatly increasing the parallelism of processing and reducing the processing time of a single frame of telemetry data to the single parameter level.

[0149] This invention dynamically allocates multiple task plugins to corresponding computing nodes for parallel processing based on the hardware resource usage of each computing node in a server cluster, achieving load balancing. All task plugins monitor the configuration center for real-time configuration updates, allowing for rapid expansion when new tasks are added. It also proposes a three-level data routing model (task-satellite-frequency band) to subdivide and isolate data streams according to task-satellite-frequency band. For each subdivided multi-frequency band telemetry data stream, a frequency band-level telemetry data stream processing task is established, enabling parallel processing of multi-task, multi-satellite, and multi-frequency band telemetry data. Therefore, it effectively adapts to multi-task, multi-satellite telemetry data processing scenarios. This invention uses a telemetry source packet processing thread pool to achieve parallel processing of multiple telemetry source packets within a single frame. Each parameter within a telemetry source packet is processed in parallel using virtual thread technology, resulting in extremely high concurrency. The telemetry parsing time at the single-frame level can be reduced to the single-parameter level, thus enabling efficient and rapid telemetry data processing.

[0150] The following describes the efficient multi-task multi-satellite telemetry data processing system provided by the present invention. The efficient multi-task multi-satellite telemetry data processing system described below and the efficient multi-task multi-satellite telemetry data processing method described above can be referred to in correspondence.

[0151] The multi-task, multi-satellite telemetry data high-efficiency processing system provided by this invention refers to... Figure 9 As shown, it includes:

[0152] The task load balancing module 210 is used to obtain the task registration information configured in the front-end interface, dynamically allocate the task plugin to the optimal computing node based on the resource status evaluation results of each computing node in the server cluster, and upload the task registration information to the configuration center, and store the telemetry parameter configuration information contained in the task registration information in the form of key-value pairs in the memory database; the memory database supports millisecond-level data reading.

[0153] The data acquisition and splitting module 220 is used to acquire task telemetry data from a specified information source according to the assigned task plug-in, and to subdivide the task telemetry data step by step based on a pre-built data splitting model to obtain telemetry data streams for each frequency band.

[0154] Data preprocessing module 230 is used to extract telemetry source packets based on telemetry data streams of each frequency band;

[0155] The parameter calculation module 240 is used to allocate telemetry source packet parsing tasks through a thread pool for each extracted telemetry source packet, and to use the parameter calculation virtual thread to solve all parameters in the telemetry source packet in parallel based on the stored telemetry parameter configuration information, so as to obtain the parsing results of each source packet.

[0156] The result merging and distribution module 250 is used to merge the parsing results of each source packet according to the frame structure, generate a telemetry frame processing result structure, serialize it, and send it to the message queue.

[0157] Figure 10 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 10 As shown, the electronic device may include a processor 310, a communications interface 320, a memory 330, and a communication bus 340. The processor 310, communications interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions from the memory 330 to execute a multi-task, multi-satellite telemetry data efficient processing method.

[0158] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0159] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the efficient processing method for multi-task multi-satellite telemetry data provided by the above methods.

[0160] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the efficient processing method for multi-task multi-satellite telemetry data provided by the methods described above.

[0161] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0162] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for efficient processing of multi-task, multi-satellite telemetry data, characterized in that, include: Obtain the task registration information configured in the front-end interface, and dynamically allocate the task plugin to the optimal computing node based on the resource status evaluation results of each computing node in the server cluster. The task registration information is uploaded to the configuration center, and the telemetry parameter configuration information contained in the task registration information is stored in the memory database in the form of key-value pairs; The in-memory database supports millisecond-level data reading; According to the assigned task plugin, the task telemetry data is retrieved from the designated information source, and the task telemetry data is subdivided step by step based on the pre-built data splitting model to obtain telemetry data streams for each frequency band. Extract telemetry source packets based on telemetry data streams from each frequency band; For each extracted telemetry source packet, a telemetry source packet parsing task is allocated through a thread pool, and based on the stored telemetry parameter configuration information, all parameters in the telemetry source packet are calculated in parallel using a parameter calculation virtual thread to obtain the parsing results of each source packet. The parsing results of each source packet are merged according to the frame structure, and the telemetry frame processing result structure is generated, serialized, and then sent to the message queue. The pre-built data routing model is a three-level data routing model based on mission-satellite-frequency band; the process involves receiving mission telemetry data from a designated source according to the assigned mission plugin, and then subdividing the mission telemetry data level by level based on the pre-built data routing model to obtain telemetry data streams for each frequency band, including: A data access thread is established, and the task plugin is controlled to retrieve task telemetry data from a specified information source based on the established data access thread and the task registration information, and put the task telemetry data into a blocking queue. At the same time, a data splitting thread is established to asynchronously retrieve message data from the blocking queue. The message data is processed by splitting it into different levels based on the three-level data splitting model of mission-satellite-frequency band to obtain telemetry data streams for each frequency band.

2. The efficient processing method for multi-task, multi-satellite telemetry data according to claim 1, characterized in that, The resource status assessment result is the idle resource score, which is calculated based on the remaining memory, number of CPU cores, and CPU utilization of the computing node.

3. The efficient processing method for multi-task, multi-satellite telemetry data according to claim 1, characterized in that, The method further includes: A blocking queue is dynamically established based on the number of satellites and frequency bands in the mission registration information; Establish a mapping relationship using mission-satellite-frequency band as the key and the corresponding memory address of the blocked queue as the value; Based on the mapping relationship, the telemetry data streams of each frequency band are routed to the corresponding blocking queues; Dynamically establish frequency band-level telemetry data stream processing tasks using each blocked queue as input; The processing tasks are submitted to the frequency band-level processing thread pool for parallel processing of multi-frequency band telemetry data streams. The frequency band-level telemetry data stream processing task includes: asynchronously acquiring the frequency band telemetry data stream from the blocking queue, and performing subsequent processing on the acquired frequency band telemetry data stream. The subsequent processing includes at least data preprocessing, efficient computation, and result return and distribution processing.

4. The efficient processing method for multi-task, multi-satellite telemetry data according to claim 1, characterized in that, Each band of telemetry data stream consists of a continuous sequence of telemetry frames. The telemetry frame format includes a frame synchronization word, a frame count, and a master header pointer. The extraction of telemetry source packets based on telemetry data streams of each frequency band includes: For each frequency band telemetry data stream, frame synchronization word verification, frame length verification, and frame count continuity verification are performed. If any verification fails, the telemetry frame is discarded as an error frame, and the telemetry frame that passes the verification is obtained. The telemetry source packet is extracted from the verified telemetry frame. During the extraction, the header position of the first source packet in the telemetry frame is located according to the dominant header pointer, and the source packet is extracted based on the telemetry source packet structure. The size of the telemetry source packet is determined according to the length of the data field. When the telemetry source packet is stored across frames, the remaining part is obtained from the front end of the data area of ​​the adjacent frame.

5. The efficient processing method for multi-task, multi-satellite telemetry data according to claim 1, characterized in that, For each extracted telemetry source packet, a telemetry source packet parsing task is allocated through a thread pool. Based on the stored telemetry parameter configuration information, a parameter calculation virtual thread is used to parallelly calculate all parameters within the telemetry source packet to obtain the parsing results for each source packet, including: Extract the packet identification code of the telemetry source packet; The corresponding telemetry source packet configuration information is extracted from the memory database using the mission code-satellite code-packet identifier as the key; the telemetry source packet configuration information contains configuration information for each parameter. The configuration information of each parameter is encapsulated, and a telemetry source packet parsing auxiliary object is constructed based on the encapsulated parameter configuration information. Based on the telemetry source packet parsing auxiliary object and the dynamic generation of telemetry source packet parsing tasks, the telemetry source packet parsing tasks are submitted to the telemetry source packet processing thread pool for processing; in particular, each received telemetry source packet is used to form a parsing task and submit it to the thread pool, and multiple telemetry source packets are parsed in parallel. Multiple parameter calculation virtual threads are established in the telemetry source packet parsing task processing thread. One parameter calculation virtual thread is assigned to each parameter in the telemetry source packet, and parameter calculation is completed in parallel based on the encapsulated parameter configuration information. Obtain the calculation results of the virtual thread for each parameter, and merge the calculation results of all parameters of the same telemetry source packet into the parsing result of that source packet; The telemetry source packet parsing auxiliary object includes at least a parameter source code extraction function, a source code storage variable, and a source code inversion function. The parameter source code extraction function is used to locate and extract the original data of each parameter from the telemetry source packet. The source code storage variable is used to temporarily store the original data of the parameters extracted from the telemetry source packet. The source code inversion function is used to perform format conversion and engineering value calculation on the original data of the parameters according to the parameter data type.

6. A high-efficiency processing system for multi-task, multi-satellite telemetry data, characterized in that: include: The task load balancing module is used to obtain the task registration information configured in the front-end interface, dynamically allocate the task plugin to the optimal computing node based on the resource status evaluation results of each computing node in the server cluster, and upload the task registration information to the configuration center, and store the telemetry parameter configuration information contained in the task registration information in the form of key-value pairs in the memory database. The in-memory database supports millisecond-level data reading; The data acquisition and splitting module is used to acquire task telemetry data from a designated source according to the assigned task plugin, and to subdivide the task telemetry data step by step based on a pre-built data splitting model to obtain telemetry data streams for each frequency band. The pre-built data splitting model is a three-level data splitting model based on task-satellite-frequency band. The process of acquiring task telemetry data from a designated source according to the assigned task plugin and subdividing the task telemetry data step by step based on the pre-built data splitting model to obtain telemetry data streams for each frequency band includes: establishing a data access thread, controlling the task plugin to acquire task telemetry data from the designated source based on the established data access thread and the task registration information, and placing the task telemetry data into a blocking queue, while simultaneously establishing a data splitting thread to asynchronously obtain message data from the blocking queue; performing step-by-step splitting processing on the message data based on the three-level data splitting model of task-satellite-frequency band to obtain telemetry data streams for each frequency band; and a data preprocessing module is used to extract telemetry source packets based on the telemetry data streams for each frequency band. The parameter calculation module is used to allocate telemetry source packet parsing tasks through a thread pool for each extracted telemetry source packet, and to use the parameter calculation virtual thread to calculate all parameters in the telemetry source packet in parallel based on the stored telemetry parameter configuration information, so as to obtain the parsing results of each source packet. The result merging and distribution module is used to merge the parsing results of each source packet according to the frame structure, generate a telemetry frame processing result structure, serialize it, and send it to the message queue.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the efficient processing method for multi-task multi-satellite telemetry data as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the efficient processing method for multi-task multi-satellite telemetry data as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the efficient processing method for multi-task multi-satellite telemetry data as described in any one of claims 1 to 5.

Images