A Performance Evaluation Method for Low-Earth Orbit Internet Constellations Based on the XGBoost Model

Abstract

This invention provides a method for evaluating the performance of low-Earth orbit (LEO) internet constellations based on the XGBoost model. The method includes establishing a constellation evaluation index model and constellation scenarios; simulating and obtaining coverage and communication performance indicators, and standardizing these indicators to obtain an index dataset; constructing and pre-training an XGBoost model; using the pre-trained XGBoost model to predict and generate pseudo-labels on the index dataset; iteratively updating the pre-trained XGBoost model using samples with high-confidence scores; and inputting the constellation's key indicators into the finally trained XGBoost model to score the overall constellation performance. This invention constructs 11 indicators to achieve a comprehensive evaluation of the constellation's overall performance; iteratively optimizing the model using the pseudo-labeled expanded dataset yields a highly reliable XGBoost model, effectively reducing reliance on human scoring and significantly improving the objectivity and credibility of the evaluation results.

Description

Technical Field

[0001] This invention relates to the field of constellation performance evaluation technology, and in particular to a method for evaluating the performance of low-Earth orbit (LEO) internet constellations based on the XGBoost model. Background Technology

[0002] Low-Earth orbit (LEO) internet constellations offer wide coverage, low transmission latency, excellent signal strength, and high-capacity communication capabilities. By effectively integrating satellite and terrestrial networks to form a new "space-to-ground" communication architecture, they can effectively supplement and expand traditional terrestrial communication networks, addressing many challenges in deploying and maintaining terrestrial networks in remote and harsh environments.

[0003] However, due to the massive scale and complex performance indicators of internet constellations, it is currently difficult to grasp the overall performance of low-Earth orbit (LEO) internet constellations. There is a lack of means to reflect the comprehensive performance of these constellations, which inconveniences technical personnel in operating, maintaining, and analyzing them. Therefore, comprehensive performance evaluation is particularly important. A comprehensive performance evaluation can provide guidance for the design, construction, and performance optimization of constellations, provide effective judgment criteria for actual operation, and ensure their adaptability and continuous stability in dynamic environments.

[0004] Existing technologies primarily employ high-performance satellite orbit simulation software (such as Satellite Tool Kit, STK) and virtualization-based evaluation methods (such as utilizing the OpenStack cloud platform) to evaluate Low Earth Orbit (LEO) internet constellations. However, these methods have limitations that restrict their ability to comprehensively analyze and optimize constellation performance. For example, while simulation software offers significant advantages in orbit analysis, it has limitations in simulating and evaluating network communication performance. Consequently, these methods fail to comprehensively consider multi-dimensional influencing factors such as user behavior and real-time data flow, making it difficult for the evaluation results to fully reflect the overall performance of the actual network. This limits their applicability and accuracy in evaluating the performance of LEO internet constellations.

[0005] Secondly, while evaluation methods based on virtualization technologies (such as those using the OpenStack cloud platform) demonstrate good simulation results in network communication, they have significant shortcomings in physical topology modeling of LEO internet constellations. These platforms struggle to accurately simulate the interactions and physical connections between satellites, resulting in low reliability of simulations for inter-satellite and satellite-to-ground links. Furthermore, these methods fail to effectively reflect key performance indicators such as constellation coverage capabilities, limiting a comprehensive understanding of the performance of LEO internet constellations in complex environments, thus negatively impacting the scientific validity and effectiveness of the evaluation results.

[0006] Therefore, existing evaluation methods lack systematic and comprehensive selection of indicators, and the evaluation process relies too heavily on subjective judgment. These methods fail to establish a sound quantitative indicator system, leading to questions about the objectivity and reliability of the evaluation results. Some studies rely excessively on subjective expert ratings, resulting in evaluation results that are biased towards subjectivity and cannot fully reflect the overall performance of the network, making it difficult to provide a basis for the optimized design of low-Earth orbit internet constellations. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a method for evaluating the performance of low-Earth orbit (LEO) internet constellations based on the XGBoost model. This method summarizes and organizes the complex indicators of constellations, establishes a comprehensive performance evaluation system, and uses a semi-supervised self-training machine learning method to transform the characteristics of massive constellation data and numerous indicator types into a data-driven advantage, thereby achieving a comprehensive performance evaluation of LEO internet.

[0008] The technical solution of this invention is: a method for evaluating the performance of low-Earth orbit internet constellations based on the XGBoost model, comprising the following steps:

[0009] S1) Establish a constellation assessment index model;

[0010] S2) Construct a constellation scene;

[0011] S3) Obtain coverage indicators, set the target coverage area and calculation step size, import the coverage capability indicator template from step S1) into the track simulation software for calculation, and export the coverage capability indicators.

[0012] S4) Import the constellation scenario data constructed in step S2) into the network communication simulation platform to generate a constellation communication network and simulate inter-satellite and satellite-to-ground communication connections.

[0013] S5) Calculate communication performance indicators through a network communication simulation platform;

[0014] S6) Standardize the coverage capability index in step S3) and the communication performance index in step S5) to obtain the index dataset;

[0015] S7) Construct an XGBoost model and pre-train it using a small number of labeled data samples;

[0016] S8) Use the pre-trained XGBoost model to predict the unlabeled index dataset obtained in step S6) and generate the estimated performance value and corresponding confidence score for each sample.

[0017] S9) Expand the training samples using reliability scores and perform final training on the pre-trained XGBoost model to obtain the final XGBoost model.

[0018] S10) Input the key indicators of the constellation into the XGBoost model trained in step S9) and output the overall performance score of the constellation.

[0019] Preferably, in step S1), the evaluation index model includes four indicators: coverage capability, communication capacity, communication capability, and service capability.

[0020] Preferably, in step S1), the coverage capability includes area coverage rate A, time coverage rate B, coverage overlap C, and minimum elevation angle D.

[0021] Preferably, in step S1), the communication capacity includes constellation capacity E and single-satellite capacity F.

[0022] Preferably, in step S1), the communication capability includes end-to-end latency G and end-to-end packet loss rate H.

[0023] Preferably, in step S1), the service capabilities include communication duration I, interruption duration J, and number of interruptions K.

[0024] Preferably, in step S2), constructing the constellation scene specifically includes the following steps:

[0025] S21) Set the realistic simulation duration and geographical environment;

[0026] S22), Add constellation satellite nodes and configure orbital parameters;

[0027] S23) Add a satellite sensor module to simulate data acquisition and transmission;

[0028] S24) Add ground station nodes and user terminal nodes, which are used for communication connection and to simulate actual needs, respectively.

[0029] Preferably, in step S3), the coverage capability indicators include area coverage rate A, time coverage rate B, coverage overlap C, and minimum elevation angle D.

[0030] Preferably, in step S5), the communication performance indicators include communication capacity, communication capability, and service capability.

[0031] Preferably, in step S6), the standardization process includes positive index standardization and negative index standardization; wherein:

[0032] If the i-th indicator a i If it is a positive indicator, then the standardized representation of the positive indicator is:

[0033]

[0034] Among them, A i It is indicator a i The result of standardization, a max This refers to the largest value among this type of indicator;

[0035] If the i-th indicator a i If it is a negative indicator, then the standardized representation of the negative indicator is:

[0036]

[0037] Among them, a min It refers to the smallest value among this type of indicator.

[0038] Preferably, in step S9), sample data with confidence scores greater than a preset confidence threshold are selected and used as "pseudo-labels" to expand the dataset space; and the pre-trained XGBoost model is iteratively updated using the expanded dataset until the prediction performance is stable and reaches the preset standard, and then the XGBoost model training is stopped.

[0039] The beneficial effects of this invention are as follows:

[0040] 1. This invention constructs 11 indicators including coverage capability, communication capability, and service capability, and achieves objectivity, consistency, and comparability of data through standardization processing; in order to achieve a comprehensive evaluation of the overall performance of the constellation;

[0041] 2. This invention achieves a comprehensive simulation of the real physical topology environment and data packet-level communication performance of low-Earth orbit constellations through the combined use of a high-performance orbit simulation platform and a communication simulation platform.

[0042] 3. This invention constructs an XGBoost model, trains an initial model with a small amount of reliable labeled data, and uses the pre-trained XGBoost model to predict and generate pseudo-labels on unlabeled indicator datasets. The dataset is continuously expanded and the model is iteratively optimized using high-confidence samples, and finally a highly reliable XGBoost model is obtained. This effectively reduces the reliance on human scoring and significantly improves the objectivity and credibility of the evaluation results. Attached Figure Description

[0043] Figure 1 This is a schematic flowchart of the method of the present invention;

[0044] Figure 2 This is a schematic diagram of the constellation evaluation index model constructed in this invention. Detailed Implementation

[0045] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0046] like Figure 1As shown, this embodiment provides a method for evaluating the performance of low-Earth orbit internet constellations based on the XGBoost model, including the following steps:

[0047] S1) Establish a constellation assessment index model;

[0048] In this embodiment, as Figure 2 As shown, the evaluation index model includes four indicators: coverage capability, communication capacity, communication capability, and service capability.

[0049] The coverage capability includes area coverage (A), time coverage (B), coverage density (C), and minimum elevation angle (D).

[0050] Wherein, the area coverage rate A is expressed as the ratio of the area covered by the constellation to the total area of ​​the target area:

[0051]

[0052] In the formula, A A For the cumulative covered area, A T The total area of ​​the target region;

[0053] The time coverage rate B is represented by the proportion of the constellation's coverage time of the target area to the total statistical time:

[0054]

[0055] Among them, B i Let B be the coverage time of the i-th sub-region. T Where N is the total statistical time, and N is the number of sub-regions;

[0056] The coverage multiplicity C is represented by the average number of times multiple satellites in the constellation simultaneously cover the target area:

[0057]

[0058] Among them, C i The number of satellites covering the i-th sub-region;

[0059] The minimum elevation angle D mentioned above represents the minimum elevation angle between a ground object and a satellite in the constellation:

[0060]

[0061] Where, is D L Propagation distance, D H D is the orbital height. R The radius is the Earth's radius.

[0062] The communication capacity includes constellation capacity E and single-satellite capacity F; wherein:

[0063] The constellation capacity E represents the maximum data transmission rate of the entire constellation system.

[0064]

[0065] Among them, E D T represents the total data volume. Total This refers to the total statistical period;

[0066] The single-satellite capacity F represents the maximum transmission rate of a single satellite.

[0067]

[0068] Among them, F N This represents the total number of satellites.

[0069] The communication capabilities include end-to-end latency G and end-to-end packet loss rate H; wherein:

[0070] The end-to-end delay G is represented by the average data transmission time from the data initiation point to the target receiving point:

[0071]

[0072] Among them, G i Let M be the delay of the i-th transmission, and M be the number of transmission rounds;

[0073] The end-to-end packet loss rate H represents the proportion of data packets lost during communication:

[0074]

[0075] Among them, H I For input messages, H O To output the message.

[0076] The service capabilities include communication duration I, interruption duration J, and number of interruptions K; wherein:

[0077] The communication duration I represents the cumulative duration for which the constellation maintains effective communication:

[0078]

[0079] Among them, I j The state of the j-th communication is represented by 1 for normal and 0 for interruption; t is the duration of a single communication.

[0080] The interruption duration J represents the cumulative time of the communication interruption.

[0081] J = T Toal -I;

[0082] The interruption count K represents the number of communication interruptions that occurred.

[0083]

[0084] Among them, com(I j ,I j+1 ) represents comparing the states of two adjacent communications. When I j =1, I j+1 A value of 0 indicates an interrupt.

[0085] S2) Constructing a constellation scene, specifically including the following steps:

[0086] S21) Set the realistic simulation duration and geographical environment;

[0087] S22), Add constellation satellite nodes and configure orbital parameters;

[0088] S23) Add a satellite sensor module to simulate data acquisition and transmission;

[0089] S24) Add ground station nodes and user terminal nodes, which are used for communication connection and to simulate actual needs, respectively.

[0090] S3) Obtain coverage indicators, set the target coverage area and calculation step size, import the coverage capability indicator template from step S1) into the track simulation software STK to calculate and export the coverage capability indicators.

[0091] In this embodiment, the coverage capability indicators include area coverage rate (A), time coverage rate (B), coverage weight (C), and minimum elevation angle (D).

[0092] S4) Import the constellation scenario data constructed in step S2) into the network communication simulation platform Hypatia to generate a constellation communication network, simulate inter-satellite and satellite-to-ground communication connections, and configure user requirements and network load.

[0093] S5) Communication performance indicators are obtained through simulation calculation using the Hypatia network communication simulation platform. In this embodiment, the communication performance indicators include seven indicators: communication capacity, communication capability, constellation capacity E, single-satellite capacity F, end-to-end latency G, end-to-end packet loss rate H, communication duration I, interruption duration J, and number of interruptions K.

[0094] S6) Standardize the 11 indicators in step S3) and step S5) to obtain the indicator dataset.

[0095] In this embodiment, the standardization process includes positive index standardization and negative index standardization; wherein:

[0096] If the i-th indicator ai If it is a positive indicator, then the standardized representation of the positive indicator is:

[0097]

[0098] Among them, A i It is indicator a i The result of standardization, a max This refers to the largest value among this type of indicator;

[0099] If the i-th indicator a i If it is a negative indicator, then the standardized representation of the negative indicator is:

[0100]

[0101] Among them, a min It refers to the smallest value among this type of indicator.

[0102] S7) Construct an XGBoost model and pre-train it using a small number of labeled data samples (pre-set data samples for performance evaluation);

[0103] In this embodiment, the maximum tree depth is set to 6, the learning rate to 0.1, the subsample ratio to 0.8, and the column sampling ratio to 0.8. The XGBoost model is pre-trained using a small amount of labeled metric dataset. Specifically:

[0104] The labeled data samples were divided into training and validation sets in an 8:2 ratio.

[0105] The mean squared error (MSE) is chosen as the loss function, and the model is optimized based on this loss function.

[0106] Cross-validation is used to optimize the hyperparameters of the XGBoost model, and the best combination of hyperparameters is determined by grid search, thereby improving the performance of the XGBoost model.

[0107] In each round of training, the XGBoost model calculates the loss function (MSE) based on the data in the training set and updates the model parameters through the gradient boosting algorithm;

[0108] During training, the XGBoost model generates several decision trees, each of which is adjusted based on the error of the current sample. At the end of each training cycle, the performance of the current XGBoost model is evaluated using validation set data to avoid overfitting.

[0109] During training, the MSE value of the XGBoost model on the validation set is monitored by adjusting hyperparameters and training epochs. If signs of overfitting are detected, L2 regularization is applied to mitigate overfitting. The pre-training process ends when the validation set error of the XGBoost model stabilizes at 0.15.

[0110] S8) Use the pre-trained XGBoost model to predict the unlabeled index dataset obtained in step S6) and generate the estimated performance value and corresponding confidence score for each sample.

[0111] S9) In this embodiment, the preset confidence threshold is 0.8. Sample data with confidence scores greater than the preset confidence threshold are selected and used as "pseudo-labels" to expand the dataset space. The pre-trained XGBoost model is iteratively updated using the expanded dataset until the accuracy reaches more than 90%, at which point the XGBoost model training is stopped.

[0112] S10) Input the key metrics of the Starlink Internet constellation (coverage capability, communication capability, communication capacity, service capability) into the XGBoost model trained in step S9) and output the overall performance score of the constellation.

[0113] In this embodiment, the performance evaluation results of the Starlink low-Earth orbit internet constellation are as follows:

[0114] 1. Coverage capability module: Area coverage rate is 85.76%, time coverage rate is 23.64%, coverage layer count is 2, and minimum elevation angle is 40°;

[0115] 2. Communication performance module: End-to-end latency is 61.25ms, and end-to-end packet loss rate is 71.31%;

[0116] 3. Service Capability Module: Communication duration 6 hours and 53 minutes, interruption duration 17 hours and 7 minutes, number of interruptions 50.

[0117] 4. Overall performance score: The final overall score of the XGBoost model evaluation is 93.6 out of 100.

[0118] The embodiments and descriptions above are merely illustrative of the principles and preferred embodiments of the present invention. Various changes and modifications may be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed.

Claims

1. A method for evaluating the performance of low-Earth orbit internet constellations based on the XGBoost model, characterized in that, Includes the following steps: S1) Establish a constellation assessment index model; S2) Construct a constellation scene; S3) Obtain coverage capability indicators, set the target coverage area and calculation step size, import the coverage capability indicator template from step S1) into the track simulation software for calculation, and export the coverage capability indicators. S4) Import the constellation scenario data constructed in step S2) into the network communication simulation platform to generate a constellation communication network and simulate inter-satellite and satellite-to-ground communication connections. S5) Calculate communication performance indicators through a network communication simulation platform; S6) Standardize the coverage capability index in step S3) and the communication performance index in step S5) to obtain the index dataset; S7) Construct an XGBoost model and pre-train it using a small number of labeled data samples; S8) Use the pre-trained XGBoost model to predict the unlabeled index dataset obtained in step S6) and generate the estimated power value and corresponding confidence score for each sample. S9) Expand the training samples using the confidence score from step S8) and iteratively update the pre-trained XGBoost model to obtain the final trained XGBoost model. S10) Input the key indicators of the constellation into the XGBoost model trained in step S9) and output the overall performance score of the constellation.

2. The method for evaluating the performance of low-Earth orbit internet constellations based on the XGBoost model according to claim 1, characterized in that: In step S1), the evaluation index model includes four indicators: coverage capability, communication capacity, communication capability, and service capability.

3. The method for evaluating the performance of low-Earth orbit internet constellations based on the XGBoost model according to claim 2, characterized in that: In step S1), the coverage capability includes area coverage rate A, time coverage rate B, coverage weight C, and minimum elevation angle D.

4. The method for evaluating the performance of low-Earth orbit internet constellations based on the XGBoost model according to claim 2, characterized in that: In step S1), the communication capacity includes constellation capacity E and single-satellite capacity F.

5. The method for evaluating the performance of low-Earth orbit internet constellations based on the XGBoost model according to claim 2, characterized in that: In step S1), the communication capabilities include end-to-end latency G and end-to-end packet loss rate H.

6. The method for evaluating the performance of low-Earth orbit internet constellations based on the XGBoost model according to claim 2, characterized in that: In step S1), the service capabilities include communication duration I, interruption duration J, and number of interruptions K.

7. The method for evaluating the performance of low-Earth orbit internet constellations based on the XGBoost model according to claim 1, characterized in that: In step S2), the constellation scene is constructed, which specifically includes the following steps: S21) Set the realistic simulation duration and geographical environment; S22), Add constellation satellite nodes and configure orbital parameters; S23) Add a satellite sensor module to simulate data acquisition and transmission; S24) Add ground station nodes and user terminal nodes, which are used for communication connection and to simulate actual needs, respectively.

8. The method for evaluating the performance of low-Earth orbit internet constellations based on the XGBoost model according to claim 1, characterized in that: In step S5), the communication performance indicators include communication capacity, communication capability, and service capability.

9. The method for evaluating the performance of low-Earth orbit internet constellations based on the XGBoost model according to claim 1, characterized in that: In step S6), the standardization process includes positive index standardization and negative index standardization; wherein: If the i-th indicator a i If it is a positive indicator, then the standardized representation of the positive indicator is: Among them, A i It is indicator a i The result of standardization, a max This refers to the largest value among this type of indicator; If the i-th indicator a i If it is a negative indicator, then the standardized representation of the negative indicator is: Among them, a min It refers to the smallest value among this type of indicator.

10. The method for evaluating the performance of low-Earth orbit internet constellations based on the XGBoost model according to claim 1, characterized in that: In step S9), sample data with high confidence scores are selected using a preset confidence threshold and used as "pseudo-labels" to expand the dataset space; The pre-trained XGBoost model is then iteratively updated using the expanded dataset until the prediction performance stabilizes and reaches the preset standard, at which point the XGBoost model training is stopped.

Images