In-band measurement method and device, electronic equipment and storage medium

Abstract

The invention discloses an in-band measurement method and device, electronic equipment and a storage medium, and the method comprises the steps: firstly transmitting a first probe message to at least one first satellite in a satellite network, and obtaining the first network state information of the at least one first satellite based on the first probe message; and based on the first network state information of the at least one first satellite, generating a second probe message corresponding to the at least one second satellite. And sending a corresponding second probe message to the at least one second satellite to obtain second network state information of the at least one second satellite. The invention provides a scheme for realizing in-band measurement by combining two different probe messages, and the second probe message is generated based on the first network state information of at least one first satellite acquired by the first probe message. Compared with the prior art that fixed probe messages are sent regularly for the satellite nodes, the in-band measurement scheme of the satellite nodes is realized, and the flexibility and pertinence of in-band measurement are improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to an in-band measurement method, device, electronic device, and storage medium. Background Art

[0002] In-band measurement is a hybrid measurement method that enables more detailed measurements of network topology and performance. It collects network status by sequentially inserting metadata into packets at intermediate switching nodes along the path. An in-band network telemetry system consists of a telemetry server and switches equipped with in-band network telemetry capabilities. The packet processing flow for in-band network telemetry is as follows: When a data packet arrives at a switching node, the in-band network telemetry module mirrors the packet and inserts an in-band network telemetry (INT) header as needed for the telemetry task. When the packet is forwarded to the last hop of the in-band network telemetry system, the switching device matches the INT header and extracts all telemetry information. The telemetry server parses the telemetry information in the telemetry packet and reports it to the upper-layer telemetry application for network monitoring, performance analysis, and fault location.

[0003] In related technologies, an in-band measurement solution generally involves periodically sending fixed probe messages to satellite nodes to implement in-band measurement of satellite nodes. However, in-band measurement has poor flexibility. Summary of the Invention

[0004] The present application provides an in-band measurement method, apparatus, electronic device, and storage medium to solve the problem of poor flexibility of in-band measurement solutions in related technologies.

[0005] In a first aspect, the present application provides an in-band measurement method, the method comprising: Sending a first probe message to at least one first satellite to obtain first network status information of the at least one first satellite; generating, based on the first network status information of the at least one first satellite, a second probe message corresponding to at least one second satellite; A corresponding second probe message is sent to the at least one second satellite to obtain second network status information of the at least one second satellite.

[0006] In an optional embodiment, the method includes: The first network status information of the at least one first satellite is input into a reinforcement learning model, and a second probe message corresponding to the at least one second satellite is generated based on the reinforcement learning model.

[0007] In an optional embodiment, the method further includes: inputting first network status information of the at least one first satellite into a reinforcement learning model, and determining at least one detection path based on the reinforcement learning model; The method comprises: A corresponding second probe message is sent to at least one second satellite in the at least one detection path.

[0008] In an optional embodiment, the method includes: Sending a first probe message to the at least one first satellite based on a first frequency; Sending a corresponding second probe message to the at least one second satellite based on the second frequency; The second frequency is lower than the first frequency.

[0009] In an optional embodiment, the method further includes: The first network status information of the at least one first satellite is input into a reinforcement learning model, and the second frequency corresponding to the at least one second satellite is determined based on the reinforcement learning model.

[0010] In an optional embodiment, the method includes: Determining bitmap information corresponding to the network status information of the at least one second satellite based on the reinforcement learning model; A corresponding second probe message carrying the bitmap information is sent to the at least one second satellite; wherein the bitmap information is used to indicate measurement network status information.

[0011] In an optional implementation, the sending the first probe message to at least one first satellite includes: For the at least one first satellite, determining a target gateway station that is closest to the first satellite; The first probe message is sent to the first satellite through the target gateway.

[0012] In an optional implementation, acquiring the first network status information of the at least one first satellite includes: For the at least one first satellite, first network status information of at least one first satellite on a path from the target gateway to the first satellite is obtained.

[0013] In an optional implementation, acquiring the second network status information of the at least one second satellite includes: For the at least one detection path, second network status information of at least one second satellite on the detection path is determined; wherein the second frequencies corresponding to the second satellites on the detection path are the same.

[0014] In an optional embodiment, determining at least one detection path based on the reinforcement learning model includes: The reinforcement learning model is used to determine the next-hop satellite of a satellite, and the at least one detection path includes the satellite and the next-hop satellite; wherein the reinforcement learning model is used to determine, for the satellite, the conditional probability of the satellite and at least one candidate satellite connected to the next hop; and determine the candidate satellite with the largest conditional probability as the next-hop satellite of the satellite.

[0015] In an optional implementation, the process of determining the at least one candidate satellite for the next hop connection includes: At least one satellite adjacent to the satellite and in normal communication with the satellite is determined according to the first network status information of the satellite, and the at least one satellite is determined as at least one candidate satellite.

[0016] In an optional embodiment, the training process of the reinforcement learning model includes: For at least one first satellite in at least one sample path, determine a first reward value corresponding to the first satellite based on bitmap information and a weight value corresponding to network status information of the first satellite during a current iterative training process, as well as a frequency of sending probe messages; determine a second reward value corresponding to the sample path based on the first reward value corresponding to the at least one first satellite in the sample path; update the gradient of the reinforcement learning model, the bitmap information and the weight value corresponding to the network status information, and the frequency of sending probe messages based on the minimum second reward value corresponding to the at least one sample path; and repeat iterative training until training of the reinforcement learning model is completed.

[0017] In an optional embodiment, determining the first reward value corresponding to the first satellite according to the bitmap information and weight value corresponding to the network status information of the first satellite in the current iteration and the frequency of sending probe messages includes: Determine the first sub-reward value and resource information occupied by the probe message according to the bitmap information and weight value corresponding to the network status information of the first satellite in the current iteration process; determining a second sub-reward value according to the resource information occupied by the probe message and the sending frequency of the probe message; A first reward value corresponding to the first satellite is determined according to the first sub-reward value and the second sub-reward value.

[0018] In an optional embodiment, the method further includes: Determine the corresponding associated network status information based on the telemetry requirements of the in-band measurement; Initial bitmap information and an initial weight value corresponding to the network status information of the first satellite are determined according to the associated network status information.

[0019] In a second aspect, the present application provides an in-band measurement device, the device comprising: A first acquisition module is configured to send a first probe message to at least one first satellite to acquire first network status information of the at least one first satellite; a determining module, configured to generate a second probe message corresponding to at least one second satellite based on the first network status information of the at least one first satellite; The second acquisition module is configured to send a corresponding second probe message to the at least one second satellite to acquire second network status information of the at least one second satellite.

[0020] In an optional implementation, the determination module is specifically configured to input the first network status information of the at least one first satellite into a reinforcement learning model, and generate a second probe message corresponding to the at least one second satellite based on the reinforcement learning model.

[0021] In an optional embodiment, the determining module is further configured to input the first network status information of the at least one first satellite into a reinforcement learning model, and determine at least one detection path based on the reinforcement learning model; The determining module is specifically configured to send a corresponding second probe message to at least one second satellite in the at least one detection path.

[0022] In an optional implementation, the first acquisition module is specifically configured to send a first probe message to the at least one first satellite based on a first frequency; The second acquisition module is specifically configured to send a corresponding second probe message to the at least one second satellite based on a second frequency; The second frequency is lower than the first frequency.

[0023] In an optional implementation, the determination module is further configured to input the first network status information of the at least one first satellite into a reinforcement learning model, and determine the second frequency corresponding to the at least one second satellite based on the reinforcement learning model.

[0024] In an optional implementation, the determining module is further configured to determine bitmap information corresponding to the network status information of the at least one second satellite based on the reinforcement learning model; The second acquisition module is specifically configured to send a corresponding second probe message carrying the bitmap information to the at least one second satellite; wherein the bitmap information is used to indicate measurement network status information.

[0025] In an optional implementation, the first acquisition module is specifically configured to determine, for the at least one first satellite, a target gateway station that is closest to the first satellite; and send the first probe message to the first satellite through the target gateway station.

[0026] In an optional implementation, the first acquisition module is specifically configured to acquire, for the at least one first satellite, first network status information of at least one first satellite on a path from the target gateway to the first satellite.

[0027] In an optional embodiment, the second acquisition module is specifically configured to determine, for the at least one detection path, second network status information of at least one second satellite on the detection path; wherein the second frequencies corresponding to the second satellites on the detection path are the same.

[0028] In an optional embodiment, the reinforcement learning model is used to determine the next-hop satellite of a satellite, and the at least one detection path includes the satellite and the next-hop satellite; wherein the reinforcement learning model is used to determine, for the satellite, the conditional probability of the satellite and at least one candidate satellite connected to the next hop; and the candidate satellite with the largest conditional probability is determined as the next-hop satellite of the satellite.

[0029] In an optional implementation, the determination module is specifically configured to determine, based on the first network status information of the satellite, at least one satellite that is adjacent to the satellite and in normal communication with the satellite, and determine the at least one satellite as at least one candidate satellite.

[0030] In an optional embodiment, the device further includes: The training module is configured to determine, for at least one first satellite in at least one sample path, a first reward value corresponding to the first satellite based on bitmap information and a weight value corresponding to network status information of the first satellite during a current iterative training process, as well as a frequency of sending probe messages; determine a second reward value corresponding to the sample path based on the first reward value corresponding to the at least one first satellite in the sample path; update the gradient of the reinforcement learning model, the bitmap information and weight value corresponding to the network status information, and the frequency of sending probe messages based on the minimum second reward value corresponding to the at least one sample path; and repeat iterative training until training of the reinforcement learning model is completed.

[0031] In an optional embodiment, the training module is specifically configured to determine a first sub-reward value and resource information occupied by a probe message based on bitmap information and a weight value corresponding to the network status information of the first satellite during a current iteration; determine a second sub-reward value based on the resource information occupied by the probe message and the sending frequency of the probe message; and determine a first reward value corresponding to the first satellite based on the first sub-reward value and the second sub-reward value.

[0032] In an optional embodiment, the training module is further used to determine corresponding associated network status information based on the telemetry requirements of the in-band measurement; and determine the initial bitmap information and initial weight value corresponding to the network status information of the first satellite based on the associated network status information.

[0033] In a third aspect, the present application provides an electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; Memory for storing computer programs; The processor is used to implement the method when executing the program stored in the memory.

[0034] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described is implemented.

[0035] In a fifth aspect, the present application provides a computer program product, which includes an executable program, and the executable program is executed by a processor to implement the described method.

[0036] In the present application, a first probe message is first sent to at least one first satellite in a satellite network, and first network status information of at least one first satellite is obtained based on the first probe message. The first network status information includes performance indicator information such as satellite load, queue length, congestion status, port packet loss rate, port timestamp, and port packet count. Then, based on the first network status information of at least one first satellite, a second probe message corresponding to at least one second satellite is generated. Then, a corresponding second probe message is sent to at least one second satellite to obtain second network status information of at least one second satellite. The present application provides a solution for implementing in-band measurement by combining two different probe messages, and the second probe message is generated based on the first network status information of at least one first satellite obtained by the first probe message. Compared with the related art of periodically sending fixed probe messages to satellite nodes to implement in-band measurement of satellite nodes, the flexibility and pertinence of in-band measurement are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0038] Figure 1 Schematic diagram of the first in-band measurement process provided by this application; Figure 2 Schematic diagram of the second in-band measurement process provided by this application; Figure 3 Schematic diagram of the third in-band measurement process provided by this application; Figure 4 Schematic diagram of the training process of the reinforcement learning model provided in this application; Figure 5 A diagram showing the in-band measurement framework for a deep learning-based mega-constellation network provided in this application; Figure 6 The format diagram of the basic probe message provided by this application; Figure 7 The format diagram of the dynamic probe message provided by this application; Figure 8 Schematic diagram of the deep reinforcement learning model provided for this application; Figure 9 A schematic diagram of the structure of the in-band measurement device provided in this application; Figure 10 This is a schematic diagram of the electronic device structure provided in this application. DETAILED DESCRIPTION

[0039] In order to make the purpose and implementation of this application clearer, the exemplary implementation of this application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only part of the embodiments of this application, not all of the embodiments.

[0040] It should be noted that the brief descriptions of terms in this application are only for the purpose of facilitating the understanding of the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0041] In the specification and claims of this application and the accompanying drawings, the terms "first," "second," "third," etc. are used to distinguish similar or similar objects or entities, and are not necessarily intended to limit a particular order or sequence, unless otherwise noted. It should be understood that the terms used in this manner are interchangeable under appropriate circumstances.

[0042] The terms "comprise," "include," and "have," and any variations thereof, are intended to cover but not exclude inclusion; for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0043] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functionality associated with that element.

[0044] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

[0045] For ease of explanation, the above description has been made with reference to specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations are possible. The above embodiments are selected and described to better explain the principles and practical applications, so that those skilled in the art can better utilize the embodiments and various different variations of the embodiments suitable for specific use considerations.

[0046] In related technologies, the packet processing flow for in-band network telemetry is as follows: 1. When a normal data packet arrives at the first switching node of the in-band network telemetry system, the in-band network telemetry module matches and mirrors the packet using the sampling method configured on the switch. Based on the telemetry task requirements, the in-band network telemetry (INT) header is inserted after the Layer 4 header. The telemetry information specified in the INT header is encapsulated into metadata (MD) and inserted after the INT header. 2. When the packet is forwarded to the intermediate node, the device matches the INT header and inserts the MD. 3. When the message is forwarded to the last hop of the in-band network telemetry system, the switching device matches the INT header, inserts the last MD, extracts all telemetry information, and forwards it to the telemetry server via remote procedure call (gRPC). 4. The telemetry server parses the telemetry information in the telemetry message and reports it to the upper-layer telemetry application for network monitoring, performance analysis, and fault location.

[0047] In related technical solutions, network telemetry is primarily categorized into two main categories: passive network telemetry (PNT) and active network telemetry (ANT). Passive network telemetry relies on network devices to automatically embed internal information, such as link utilization and queuing delay, during packet transmission for analysis by a centralized controller. Active network telemetry uses source routing to proactively generate and send telemetry probes to probe user-specified network paths. ANT systems in related technologies can optimize and flexibly plan probe paths based on fixed network environments and performance metrics to meet diverse telemetry needs.

[0048] Active network telemetry methods in related technologies lack the ability to quickly adapt to new telemetry requirements and self-adjust in dynamic network environments. While passive network telemetry methods provide detailed network status data, they struggle to achieve full network coverage due to uncontrolled probe paths and can increase network overhead. These methods primarily employ two methods: periodically sending network-wide probes and manually setting probe parameters, resulting in weak adaptability. In the case of periodically sending network-wide probes, excessively long probe transmission periods can reduce network measurement efficiency and prevent timely detection of network issues. However, too short a period can lead to a sharp increase in network overhead and prevent the ability to target probes locally based on the actual state of the constellation network, resulting in poor flexibility. Manually setting probe parameters offers operational flexibility compared to periodically sending network-wide probes, but its effectiveness relies heavily on the measurement operator's experience and subjective judgment, often requiring repeated trial-and-error adjustments. When applied to large-scale constellation networks, the system's complexity and dynamic nature increase dramatically, making it difficult for pre-set parameters and rules to fully adapt to changing conditions. In practice, setting thresholds too high can miss key signal features, while setting them too low can expose the results to excessive noise and interference, reducing measurement accuracy.

[0049] Traditional in-band measurement methods are primarily designed for terrestrial mobile networks, where the scenarios are relatively simple. The performance metrics they can measure, such as bit error rate, cannot meet the requirements of satellite communication networks. Furthermore, they cannot adaptively adjust measurement metrics based on measurement needs.

[0050] Furthermore, related technologies for network fault location based on in-band measurements primarily rely on periodically sending network-wide probes, which is costly and inflexible. Furthermore, they lack autonomous learning mechanisms, preventing them from fully tapping into the deep insights of historical measurement data to dynamically guide network fault discovery and location.

[0051] In order to solve the above technical problems, the present application provides an in-band measurement solution that combines static probe messages and dynamic probe messages. Figure 1 The first in-band measurement process provided in this application includes the following steps: S101: Send a first probe message to at least one first satellite to obtain first network status information of the at least one first satellite; S102: Generate a second probe message corresponding to at least one second satellite based on the first network status information of the at least one first satellite; S103: Send a corresponding second probe message to the at least one second satellite to obtain second network status information of the at least one second satellite.

[0052] The in-band measurement method provided in this application is applied to an electronic device, which can be a network-side device such as an eNB (Evolved NodeB, wireless base station) on the network side; it can also be a terminal device such as a mobile phone, a computer; or it can be a server.

[0053] In a satellite network, there are multiple satellites, and an electronic device sends a first probe message to at least one first satellite in the satellite network. Optionally, the electronic device may send the first probe message to all first satellites in the satellite network; or send the first probe message to all first satellites in a specified satellite sub-domain of the satellite network according to the satellite sub-domain specified in the telemetry requirement; or send the first probe message to at least one satellite according to the at least one satellite specified in the telemetry requirement. The first probe message, also known as the basic probe message, is sent by the electronic device to at least one first satellite to obtain first network status information of the at least one first satellite. Optionally, the first network status information includes performance indicator information such as satellite load, queue length, congestion status, port packet loss rate, port timestamp, and port packet count. In addition, the first network status information may also include topology information of the satellite network. Based on the topology information of the satellite network, for each first satellite, the adjacent satellite that is in normal communication with the first satellite can be determined.

[0054] After sending a first probe message to at least one first satellite to obtain the first network status information of the at least one first satellite, a second probe message corresponding to at least one second satellite is generated based on the first network status information of the at least one first satellite. The second probe message can be understood as a dynamic probe message. The first network status information of at least one first satellite is different, and the second probe message corresponding to the at least one second satellite generated may also be different. Then, a corresponding second probe message is sent to at least one second satellite, and for at least one second satellite, the second network status information of the second satellite is obtained based on the second probe message corresponding to the second satellite.

[0055] In the present application, a first probe message is first sent to at least one first satellite in a satellite network, and first network status information of at least one first satellite is obtained based on the first probe message. The first network status information includes performance indicator information such as satellite load, queue length, congestion status, port packet loss rate, port timestamp, and port packet count. Then, based on the first network status information of at least one first satellite, a second probe message corresponding to at least one second satellite is generated. Then, a corresponding second probe message is sent to at least one second satellite to obtain second network status information of at least one second satellite. The present application provides a solution for implementing in-band measurement by combining two different probe messages, and the second probe message is generated based on the first network status information of at least one first satellite obtained by the first probe message. Compared with the related art of periodically sending fixed probe messages to satellite nodes to implement in-band measurement of satellite nodes, the flexibility and pertinence of in-band measurement are improved.

[0056] In an optional embodiment, the method includes: The first network status information of the at least one first satellite is input into a reinforcement learning model, and a second probe message corresponding to the at least one second satellite is generated based on the reinforcement learning model.

[0057] A pre-trained reinforcement learning model is deployed in the electronic device. It should be noted that the reinforcement learning model can be trained in the main electronic device that executes the in-band measurement method; or the reinforcement learning model can be trained in other electronic devices, and then the trained reinforcement learning model is deployed in the main electronic device that executes the in-band measurement method. The reinforcement learning model is used to generate a second probe message corresponding to at least one second satellite. Optionally, after determining the first network status information of at least one first satellite, the first network status information of at least one first satellite is input into the trained reinforcement learning model, and based on the reinforcement learning model, a second probe message corresponding to at least one second satellite is generated.

[0058] In an optional embodiment, the method further includes: inputting first network status information of the at least one first satellite into a reinforcement learning model, and determining at least one detection path based on the reinforcement learning model; The method comprises: A corresponding second probe message is sent to at least one second satellite in the at least one detection path.

[0059] The reinforcement learning model is also used to predict the in-band measurement detection path. Optionally, a second satellite is randomly initialized in each first satellite, and then the first network status information of the second satellite and the first network status information of each satellite connected to the second satellite in one hop are input into the reinforcement learning model. Based on the reinforcement learning model, the next-hop second satellite of the initialized second satellite is screened from each satellite connected to the second satellite in one hop. Then, the first network status information of the next-hop second satellite and the first network status information of each satellite connected to the next-hop second satellite in one hop are input into the reinforcement learning model. Based on the reinforcement learning model, the next-hop second satellite of the next-hop second satellite is screened from each satellite connected to the next-hop second satellite in one hop. This process is repeated until the number of second satellites determined in the detection path reaches a preset threshold, at which point the detection path is obtained.

[0060] Then, a second satellite is randomly initialized from among the first satellites other than the second satellite included in the determined detection path. The above steps are repeated to obtain another detection path. In this way, at least one detection path can be determined based on the reinforcement learning model. The number of satellites included in the detection path does not exceed a preset threshold, and the detection direction of the detection path is the direction of the next hop connection between the second satellites.

[0061] For example, a detection path includes five second satellites: second satellite A, second satellite B, second satellite C, second satellite D, and second satellite E. Second satellite B is the next-hop satellite of second satellite A, second satellite C is the next-hop satellite of second satellite B, second satellite D is the next-hop satellite of second satellite C, and second satellite E is the next-hop satellite of second satellite D. Therefore, the detection direction of the detection path is from second satellite A to second satellite B to second satellite C to second satellite D to second satellite E.

[0062] A corresponding second probe message is sent to at least one second satellite in at least one detection path to obtain the second network status information of the at least one second satellite. Optionally, the first second satellite of the detection path is determined according to the detection direction of the detection path, and then a second probe message is sent to the first second satellite, and the second network status information of the first second satellite is obtained based on the second probe message. Optionally, if it is necessary to measure the network status information of a second satellite in the middle of the detection path, a second probe message may be sent to the first second satellite first, and then the second probe message is forwarded to the network status information of a second satellite to be measured according to the detection direction of the detection path, and then the second network status information of the second satellite is obtained based on the second probe message. Optionally, a second probe message may be sent to the first second satellite, and then the second probe message is forwarded to all the second satellites included in the detection path in sequence according to the detection direction of the detection path, and then the second network status information of all the second satellites included in the detection path is obtained. The second probe message can be understood as a dynamic probe message.

[0063] In this application, a first probe message is first sent to at least one first satellite in a satellite network, and first network status information of the at least one first satellite is obtained based on the first probe message. The first network status information includes performance indicators such as satellite load, queue length, congestion status, port packet loss rate, port timestamp, and port packet count. The first network status information of the at least one first satellite is then input into a reinforcement learning model, and at least one detection path is determined based on the reinforcement learning model, and a second probe message corresponding to at least one second satellite is generated. A corresponding second probe message is then sent along the at least one detection path to at least one second satellite in the at least one detection path to obtain second network status information of the at least one second satellite. This application provides an in-band measurement solution based on deep learning. Specifically, after obtaining the first network status information of the at least one first satellite, at least one detection path is determined based on a reinforcement learning model derived from deep learning, and then the second network status information of the at least one second satellite in the at least one detection path is obtained. Compared to the related art solution of periodically sending fixed probe messages to satellite nodes, implementing in-band measurement of satellite nodes further improves the flexibility and targetedness of in-band measurement.

[0064] In an optional embodiment, the method includes: Sending a first probe message to the at least one first satellite based on a first frequency; Sending a corresponding second probe message to the at least one second satellite based on the second frequency; The second frequency is lower than the first frequency.

[0065] The method further comprises: The first network status information of the at least one first satellite is input into a reinforcement learning model, and the second frequency corresponding to the at least one second satellite is determined based on the reinforcement learning model.

[0066] The first frequency may be a preset frequency. The second frequency may be a frequency for sending the second probe message corresponding to the second satellite determined based on the reinforcement learning model.

[0067] Optionally, if the second network status information of each second satellite in the detection path is obtained separately, the second frequencies corresponding to the second satellites are the same or different.

[0068] If a second probe message is sent to the first second satellite, and then the second probe message is forwarded sequentially to all second satellites included in the detection path according to the detection direction of the detection path, and then the second network status information of all second satellites included in the detection path is obtained, then the second frequencies corresponding to the second satellites included in the detection path are the same.

[0069] Figure 2 The second in-band measurement process provided in this application includes the following steps: S201: Sending a first probe message to at least one first satellite based on a first frequency to obtain first network status information of the at least one first satellite; S202: Inputting first network status information of the at least one first satellite into a reinforcement learning model, determining at least one detection path based on the reinforcement learning model, and generating a second probe message corresponding to at least one second satellite; S203: Send a second probe message to at least one second satellite in the at least one detection path based on a second frequency to obtain second network status information of the at least one second satellite; wherein the second frequency corresponding to the at least one second satellite is determined based on the reinforcement learning model.

[0070] In an optional embodiment, the method includes: Determining bitmap information corresponding to the network status information of the at least one second satellite based on the reinforcement learning model; A corresponding second probe message carrying the bitmap information is sent to the at least one second satellite; wherein the bitmap information is used to indicate measurement network status information.

[0071] The bitmap information is used to indicate which network status information needs to be measured. For example, network status information includes six types of performance indicator information, including satellite load, queue length, congestion status, port packet loss rate, port timestamp, and port packet count. The first to sixth bits in the bitmap information respectively represent the above six types of performance indicator information. For each bit, if the value in the bitmap information is 1, it means that the performance indicator parameter of the bit is measured; if the value in the bitmap information is 0, it means that the performance indicator parameter of the bit is not measured. For example, for a second satellite, the bitmap information corresponding to the network status information to be measured for the second satellite is 1, 1, 1, 0, 0, 0, which means that the satellite load, queue length, and congestion status of the second satellite are measured, while the port packet loss rate, port timestamp, and port packet count are not measured.

[0072] The second probe message sent by the electronic device to at least one second satellite carries corresponding bitmap information, thereby acquiring network status information indicated by the corresponding bitmap information of the second satellite.

[0073] Figure 3 The third in-band measurement process provided in this application includes the following steps: S301: Sending a first probe message to at least one first satellite based on a first frequency to obtain first network status information of the at least one first satellite; S302: Inputting first network status information of the at least one first satellite into a reinforcement learning model, determining at least one detection path based on the reinforcement learning model, generating a second probe message corresponding to at least one second satellite, and determining bitmap information corresponding to the network status information of the at least one second satellite; S303: Sending a corresponding second probe message carrying the bitmap information to at least one second satellite in the at least one detection path based on a second frequency; obtaining second network status information of the at least one second satellite; wherein the second frequency corresponding to the at least one second satellite is determined based on the reinforcement learning model.

[0074] In an optional implementation, the sending the first probe message to at least one first satellite includes: For the at least one first satellite, determine a target gateway station that is closest to the first satellite; and send the first probe message to the first satellite through the target gateway station.

[0075] To improve the efficiency of in-band measurement, when the electronic device sends a first probe message to at least one first satellite, the electronic device first determines a target gateway station closest to the at least one first satellite. The electronic device first sends the first probe message to the target gateway station, and then the target gateway station sends the first probe message to the first satellite.

[0076] In one scenario, for at least one first satellite, the target gateway closest to the first satellite communicates directly with the first satellite. In this case, the electronic device first sends a first probe message to the target gateway, which then sends the first probe message to the first satellite, thereby obtaining first network status information of the first satellite. In another scenario, for at least one first satellite, the target gateway closest to the first satellite communicates with the first satellite in multi-hop fashion. In this case, the electronic device first sends a first probe message to the target gateway, which then sends the first probe message to at least one intermediate routing device. The first probe message is then forwarded by the at least one routing device and sent to the first satellite, thereby obtaining first network status information of the first satellite. The at least one routing device may be another gateway or another satellite.

[0077] In the present application, for each first satellite, a target gateway station closest to the first satellite can be determined; a first probe message is sent to the first satellite through the target gateway station to obtain first network status information of the first satellite.

[0078] Preferably, obtaining the first network status information of the at least one first satellite includes: For the at least one first satellite, first network status information of at least one first satellite on a path from the target gateway to the first satellite is obtained.

[0079] In the present application, for at least one first satellite, the target gateway closest to the first satellite communicates with the first satellite in multi-hop communication. In this case, the electronic device first sends a first probe message to the target gateway, which then sends the first probe message to at least one intermediate routing device. The first probe message is then forwarded by the at least one routing device and sent to the first satellite. At this time, the path from the target gateway to the first satellite includes at least one other first satellite. In this application, the first network status information of at least one first satellite on the path from the target gateway to the first satellite can be obtained. Preferably, the first network status information of all first satellites on the path from the target gateway to the first satellite can be obtained. In this way, for the intermediate first satellites whose first network status information has already been determined, it is no longer necessary to determine their corresponding target gateway, thereby saving resources for determining the first network status information of each first satellite.

[0080] In an optional implementation, acquiring the second network status information of the at least one second satellite includes: For the at least one detection path, second network status information of at least one second satellite on the detection path is determined; wherein the second frequencies corresponding to the second satellites on the detection path are the same.

[0081] In the present application, for at least one detection path, the electronic device sends a second probe message to the first second satellite of the detection path, and then forwards the second probe message to all second satellites included in the detection path according to the detection direction of the detection path, and then obtains the second network status information of at least one second satellite included in the detection path. Optionally, the second network status information of all second satellites included in the detection path can be obtained. At this time, since the second network status information of all second satellites included in the detection path is obtained simultaneously, the second frequencies corresponding to the second satellites included in the detection path are the same.

[0082] In an optional embodiment, determining at least one detection path based on the reinforcement learning model includes: The reinforcement learning model is used to determine the next-hop satellite of a satellite, and the at least one detection path includes the satellite and the next-hop satellite; wherein the reinforcement learning model is used to determine, for the satellite, the conditional probability of the satellite and at least one candidate satellite connected to the next hop; and determine the candidate satellite with the largest conditional probability as the next-hop satellite of the satellite.

[0083] A satellite is randomly initialized as a candidate satellite for processing. The candidate satellite and at least one candidate satellite for the next hop connection are then determined. The first network state information of the candidate satellite and the first network state information of the at least one candidate satellite for the next hop connection are then input into a reinforcement learning model. The conditional probability of the at least one candidate satellite for the next hop connection is determined based on the reinforcement learning model.

[0084] Among them, the weight values ​​corresponding to the various performance indicators included in the network status information are obtained based on the reinforcement learning model training, and then for at least one candidate satellite for the next hop connection of the satellite to be processed, according to the formula Determine the conditional probability of the candidate satellite, and then determine the candidate satellite with the largest conditional probability as the next hop satellite of the satellite to be processed. , represents n performance indicators. , represents the weight value corresponding to each of the n performance indicators, obtained through training.

[0085] In this application, the process of determining the candidate satellite for the at least one next-hop connection includes: At least one satellite adjacent to the satellite and in normal communication with the satellite is determined according to the first network status information of the satellite, and the at least one satellite is determined as at least one candidate satellite.

[0086] Based on the first network status information of each first satellite, a satellite that is in normal communication with the first satellite can be determined for each first satellite. Normal communication means that the communication function is normal. Adjacent means that the satellites are physically adjacent, including left-right and top-bottom proximity, or may also include diagonal proximity.

[0087] Figure 4 The training process diagram of the reinforcement learning model provided in this application includes the following steps: S401: For at least one first satellite in at least one sample path, determine a first reward value corresponding to the first satellite based on bitmap information and a weight value corresponding to network status information of the first satellite during a current iterative training process, as well as a probe message sending frequency; S402: Determine a second reward value corresponding to the sample path according to a first reward value corresponding to at least one first satellite in the sample path; S403: Update the gradient of the reinforcement learning model, the bitmap information and weight value corresponding to the network status information, and the sending frequency of probe messages according to the minimum second reward value corresponding to the at least one sample path; repeat the iterative training until the reinforcement learning model training is completed.

[0088] In the present application, after obtaining the first network status information of each first satellite, the reinforcement learning model is trained based on the first network status information of each first satellite. When the first network status information of each first satellite is determined again next time, the reinforcement learning model is trained again based on the first network status information of each first satellite, and so on.

[0089] The following describes the training process of the reinforcement learning model. First, it should be noted that each time the first network status information of each first satellite is obtained, the reinforcement learning model training needs to be completed through multiple iterative parameter updates.

[0090] First, at least one sample path is obtained using a conditional probability calculation formula based on the first network status information of each first satellite and its weight value during the current iterative training process. For at least one first satellite in the at least one sample path, a first reward value corresponding to the first satellite is determined based on the bitmap information and weight value corresponding to the network status information of the first satellite during the current iterative training process, as well as the frequency of dynamic probe messages sent by the first satellite during the current iterative training process.

[0091] Determining the first reward value corresponding to the first satellite according to the bitmap information and the weight value corresponding to the network status information of the first satellite in the current iteration process, and the sending frequency of the probe message includes: Determine the first sub-reward value and resource information occupied by the probe message according to the bitmap information and weight value corresponding to the network status information of the first satellite in the current iteration process; determining a second sub-reward value according to the resource information occupied by the probe message and the sending frequency of the probe message; A first reward value corresponding to the first satellite is determined according to the first sub-reward value and the second sub-reward value.

[0092] The bitmap information and weight values ​​corresponding to the network status information of the first satellite are different, and the resource information occupied by the probe message corresponding to the first satellite is also different. The electronic device can pre-save the bitmap information and weight values ​​corresponding to the network status information, and the corresponding relationship with the resource information occupied by the probe message. In this way, the resource information occupied by the probe message is determined based on the bitmap information and weight values ​​corresponding to the network status information of the first satellite in the current iteration. In addition, the first sub-reward value is determined based on the bitmap information and weight values ​​corresponding to the network status information of the first satellite in the current iteration. Optionally, it can be determined according to the formula Determine the first sub-reward value.

[0093] The second sub-reward value is determined based on the resource information occupied by the probe message and the frequency of sending the probe message. Optionally, the second sub-reward value is determined as the ratio of the resource information occupied by the probe message and the frequency of sending the probe message.

[0094] A first reward value corresponding to the first satellite is determined based on the first sub-reward value and the second sub-reward value. Optionally, the sum of the first sub-reward value and the second sub-reward value is determined as the first reward value corresponding to the first satellite.

[0095] According to the first reward value corresponding to at least one first satellite in the sample path, the second reward value corresponding to the sample path is determined. Optionally, the sum of the first reward values ​​corresponding to at least one first satellite in the sample path is determined as the second reward value corresponding to the sample path. In this way, each sample path corresponds to a second reward value, and the smallest second reward value is selected to update the gradient of the reinforcement learning model, the bitmap information and weight value corresponding to the network status information, and the frequency of sending probe messages; repeat the iterative training until the reinforcement learning model training is completed. Optionally, when the number of iterative training meets the requirements, the reinforcement learning model training is determined to be completed; or, when the difference between the smallest second reward values ​​obtained from two adjacent trainings is less than a set threshold, the reinforcement learning model training is determined to be completed.

[0096] In this application, the method further includes: Determine the corresponding associated network status information based on the telemetry requirements of the in-band measurement; Initial bitmap information and an initial weight value corresponding to the network status information of the first satellite are determined according to the associated network status information.

[0097] The telemetry requirements for in-band measurement carry the objectives of the in-band measurement. For example, the objectives of in-band measurement may be measuring congestion status or satellite load. The electronic device pre-stores associations between different telemetry requirements and network status information. Association can be understood as network status information that affects the objectives of the telemetry requirements being associated network status information. Network status information that does not affect the objectives of the telemetry requirements being unassociated network status information. The initial bitmap value corresponding to associated network status information is 1, while the initial bitmap value corresponding to unassociated network status information is 0. Furthermore, different network status information has different degrees of impact on the objectives of the telemetry requirements; network status information with greater impact has a greater initial weight. Based on the above strategy, initial bitmap information and initial weight values ​​corresponding to the network status information of the first satellite are determined based on the associated network status information. Subsequently, the gradient of the reinforcement learning model, the bitmap information and weight values ​​corresponding to the network status information, and the frequency of probe message transmission are updated based on the determined minimum second reward value. Iterative training is repeated until the reinforcement learning model is trained.

[0098] The in-band measurement method based on deep learning provided by this application is described in detail below with reference to the accompanying drawings.

[0099] Figure 5 The deep learning-based giant constellation network provides an in-band measurement framework diagram for this application, such as Figure 5 As shown, this application proposes a deep learning-based in-band measurement method for a mega-constellation network. The method consists of three main components: a basic in-band measurement module, a dynamic in-band measurement module, and a telemetry analyzer module. Users can specify telemetry tasks through a web application, provide telemetry requirements to the telemetry analyzer, and distribute them to the basic in-band measurement module. The basic in-band measurement module sends basic measurement probes (first probe messages) to satellite nodes in the constellation network to obtain basic network status information (first network status information). This information then serves as the initial input for the passive in-band measurement module, driving the dynamic in-band measurement module to generate dynamic measurement probes (second probe messages). The dynamic measurement probe results also serve as input for the dynamic in-band measurement module to perform reinforcement learning and select risky satellite nodes or paths for further dynamic telemetry. The basic and dynamic measurement probe results are simultaneously input into the telemetry analyzer module for network fault analysis and location, as well as adaptive adjustment of alarm thresholds and rules.

[0100] Performance metrics collected from dynamic probes can be categorized into three levels: node-level, port-level, and bitmap-level. Node-level metrics include signal strength, link quality, and congestion. Port-level metrics include data rate, bandwidth utilization, bit error rate, packet loss rate, latency, and jitter. Bitmap-level metrics focus on signal strength, signal-to-noise ratio, link quality, and queue length.

[0101] The basic in-band measurement module uses basic probes to collect essential network parameters. This information is used to understand the current network status and provides foundational data for subsequent dynamic probe path planning. By using basic probe path selection methods and rationally planning paths, basic probes can traverse all links in the network, achieving full network coverage. Basic probes can also be used to locate network faults. When a link fails, the fault can be quickly located by identifying which basic probes are unable to reach the controller.

[0102] Figure 6 The format diagram of the basic probe message provided by this application is as follows: Figure 6 As shown, the header of a base probe is similar to a typical data packet header. It consists of an Ethernet header (ETH), an Internet Protocol (IP) header, and a TCP / UDP header. TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) are the two primary transport layer protocols in the Internet protocol suite. The source port is the Source Port, and the destination port is set to "Base_probe" to ensure that programmable satellite routing nodes can parse the base probe. Furthermore, the destination address of the base probe is set to the controller's IP address to forward the base probe to the controller. The base probe also contains a source routing stack (SR stack, i.e., the INT header) and an information stack (INFO stack, i.e., metadata). The SR stack is a fixed-length stack consisting of a series of port tags. Each port tag records a hop on the base probe's path. Satellite routing switching nodes forward the base probe based on these port tags. The INFO stack stores network information. Each INFO tag contains a switch ID and a value for each piece of network metadata.

[0103] The primary task of basic probes is to collect basic network status information, which is crucial for dynamic probe path planning. Basic probes use the shortest geodesic distance as a matching rule. A fixed-length SR stack ensures that algorithm execution is not interrupted by stack capacity issues, thereby ensuring the effectiveness and reliability of path planning. Because basic probes collect data relatively infrequently and carry a small amount of data, a fixed-length SR stack does not overconsume network resources. Using a variable-length SR stack can lead to unnecessary resource waste due to an excessively large stack capacity, or it can be too small to meet path planning requirements.

[0104] The basic probe path selection is used for the routing path through which the probe is sent and received. In order to ensure minimum overhead and real-time performance, this application uses the distance to the nearest gateway as the path selection criterion. Set the sampling interval, traverse the entire measurement time period, calculate the geodesic distance from the target satellite node to different gateways at different times, and use the shortest geodesic distance best matching rule to match satellite nodes and gateways. The probe message is sent by the network controller and sent directly to the target satellite node via the gateway. In order to ensure that they can traverse the entire network topology and collect all network status information, the propagation path may involve multiple network nodes (including satellites and possible gateways). Prioritize gateways. If the gateway cannot reach a certain node, it will be forwarded through other satellites.

[0105] The satellite node-gateway matching calculation process is as follows:

[0106] Where satId is the selected satellite node, trajectory is the available path for the selected node, step is the sampling interval with a default value of 60s, samplePoints is the trajectory sampling result, gwsList is the gateway list, gwId is the gateway ID closest to the satellite node at the current moment, distance is the corresponding shortest geodesic distance, and satGwMap is the satellite node-gateway matching result.

[0107] The dynamic in-band measurement module uses dynamic probes, building upon basic probes, to collect specific network information required by network applications. This information includes, but is not limited to, node load, queue length, congestion status, port packet loss rate, port timestamp, and port packet count. By adjusting the paths of dynamic probes and the types of information collected, diverse telemetry requirements can be met. Compared to basic probes, dynamic probes are injected into the network at a higher frequency, enabling them to collect network status information more frequently. This high-frequency data collection facilitates more granular network monitoring. Dynamic probe paths are dynamically planned based on real-time network status. The dynamic probe path discovery method, based on deep reinforcement learning, adjusts probe paths in real time based on factors such as network load and topology changes to optimize telemetry performance. By adjusting the reward function, it supports multi-objective optimization. When network topology or load changes, dynamic probes can rapidly adjust paths and locate faults to ensure accurate and timely telemetry information.

[0108] Figure 7 The format diagram of the dynamic probe message provided by this application is as follows: Figure 7 As shown in Figure 1, dynamic probes are responsible for dynamically collecting specific network information required by applications. The format of dynamic probes is similar to that of basic probes. Since the length of dynamic probe paths varies more widely, using a fixed-length SR stack may result in waste. In addition, given the diverse types of network metadata collected by dynamic probes, the dynamic probe format introduces two new fields: Length and Bitmap. Figure 7 As shown in the figure, in the dynamic probe format, the SR stack is a variable-length stack, and the "length" field determines the length of the SR stack. The "bitmap" field determines what types of metadata the INFO tag should collect, and each bit corresponds to a type of metadata. If a bit is set to 1, it means that the DP needs to collect this type of metadata when passing through the corresponding node; if it is set to 0, it means that it does not need to be collected. The destination port in the dynamic probe is set to "Dynamic_probe" so that the satellite routing node can parse the dynamic probe. Through the Bitmap field in the INFO stack, the system can selectively collect specific types of metadata at different ports, further improving the pertinence and efficiency of network telemetry.

[0109] Figure 8 This application uses the basic network status information of the entire network collected by the basic probes for path planning to ensure that DP can effectively cover and meet specific telemetry requirements.

[0110] Network topology: Represented as an undirected graph G = (V, E), where V is the set of nodes and E is the set of edges. Each node represents a device in the network (such as a switch), and each edge represents a connection between devices.

[0111] Node information: This includes each node's connection status (i.e., its connection relationship with other nodes) and port delay information. This information is used to evaluate the performance of different paths.

[0112] The output of this application covers dynamic node selection, dynamic node parameter mask (bitmap) prediction, and transmission cycle and interval prediction. The dynamic probe path consists of a series of node sequences, which clearly stipulates the transmission route of the probe in the network and collects necessary network status information when passing through each node. Bitmap prediction generates a binary bitmap that matches the number of metadata types that can be collected in the network, and uses bit values ​​of 0 and 1 to determine whether the dynamic probe needs to collect specific metadata at each node in the transmission path, thereby achieving refined control of data collection. Finally, the transmission cycle prediction outputs a time interval value (representing frequency), which is dynamically adjusted according to network load fluctuations, changes in telemetry requirements and network environment. It represents the time interval between two consecutive dynamic probes injected into the network, thereby effectively managing network overhead while ensuring data timeliness.

[0113] In the dynamic path discovery method, the model adaptively selects nodes and ports (i.e., dynamic node-port selection) and determines the information metrics to be collected (i.e., metric selection-info bitmap). The model can measure network performance parameters at the node, port, and bitmap levels. The model dynamically adjusts the frequency of dynamic probe transmission and path planning strategies based on real-time network information and telemetry requirements. This adaptive capability enables the model to quickly respond to changes in network status and optimize the timing and path of probe transmission, thereby improving telemetry accuracy and efficiency. The model calculates an appropriate transmission interval and number of probes based on the network topology, link status, and telemetry task requirements, and adjusts the probe transmission interval and number of probes based on bandwidth utilization. If a network failure occurs, the model immediately stops sending dynamic probes.

[0114] In the network telemetry system, the jth dynamic detection path is represented as , where p j,i Represents path p j The i-th node passed, N p is the total number of nodes in the path. p The set of links covered is denoted as l p. There are a total of P paths in the network telemetry system. For the jth path, the node passed through is represented as pj. The set of links connecting these nodes is called lq. The concept of lq is only for a single dynamic detection path and serves as a symbolic identifier. When planning this path, it is necessary to comprehensively consider a variety of performance indicators, including signal strength, link quality and congestion status at the node level, data transmission rate, bandwidth utilization, bit error rate, packet loss rate, delay and jitter at the port level, and signal strength, signal-to-noise ratio, link quality and queue length at the bitmap level. All telemetry task performance indicators are summarized as a set , where m i is the performance index m i Given the different importance of each performance indicator, for each indicator m i Assign weight w i , the weight set is To ensure full coverage of the service network, the dynamic detection path planning problem can be expressed as minimizing the sum C of the products of each performance indicator and its weight, which can be expressed as: .

[0115] The input set contains the connection status of node i with other nodes and the delay information of each port. X0 contains all the initial information required to generate the sequence. At each step t (from 0 to T), it is necessary to calculate the current state (i.e. the part of the sequence that has been generated) and the current input set X t ) to determine the next element y t+1 The probability sequence calculation formula can be expressed as: ; It is known that the sequence Y generated in the previous t steps t and the current input set X t Under the condition, select the next element y t+1 The conditional probability of is calculated as follows.

[0116] Initialization: Set t=0, Y0=0 (empty sequence), and the initial input set X0.

[0117] X i It can be understood as the network status information of the i-th node, which nodes it is connected to; Y can be understood as the generated path, T is the length of the path, and t is each step taken. Each step determines a node.

[0118] Recursive calculation: For each step t (from 0 to T-1): according to the current sequence Y t and input set X t , calculate the conditional probability . Select a The largest y t+1 , and add it to the sequence, i.e. , to obtain the best choice. Update the input set X t+1 , to reflect the latest sequence status.

[0119] Termination condition: The calculation is terminated when t = T. T is the number of satellites included in the preset path.

[0120] Provides two-way capabilities for path planning: through probability sequence Select nodes; by minimizing Select Performance.

[0121] State: At the end of any time point, the current decision environment state can be expressed as s t Status t Contains two pieces of information: the current node index and the set of links for which telemetry is remaining. Node indexes are divided into two states: state_index, which indexes a node in the network, and state_create, or the special value 0, which indicates that a new path is being considered.

[0122] Action: Based on the current state t , this method selects an action to perform. If the current state is state_index , this method selects a node from the nodes connected to the current node based on the probability sequence and adds it to the current path. If the current state is state_create , this method creates a new path and selects a starting point from all links with telemetry requirements.

[0123] Reward: In the algorithm, the reward mechanism is a key factor in guiding the model to make optimal decisions. The goal is to minimize weighted telemetry benefit, meaning that the dynamic probe path meets the most telemetry requirements with the least resource consumption. The reward function calculates a reward value based on how well the current path deployment plan meets various telemetry requirements. This reward value is composed of the weighted sum of multiple telemetry metrics (such as latency and bandwidth utilization). Each metric has a corresponding weight, which can be adjusted based on actual user needs. The algorithm simulates different path deployment plans and calculates the weighted telemetry benefit under each plan. The plan with the lowest weighted telemetry benefit is then selected as the optimal solution and awarded a corresponding reward. This reward value is fed back to the model to update its parameters, enabling it to make better choices in the next decision. Dynamic probes can have multiple possible paths to collect network status information. Each path deployment plan has different resource consumption and telemetry performance. The algorithm simulates these different path deployment plans and selects the optimal path deployment plan by evaluating the network performance and resource consumption under each plan. Weighted telemetry benefit is a comprehensive metric used to evaluate the overall performance of each path deployment solution. It considers multiple performance metrics, such as control overhead, latency, and bandwidth utilization. Each metric is assigned a weight, reflecting its importance in the overall evaluation, to find the optimal solution.

[0124] The cycle calculation model aims to model network performance indicators, network topology and other data and predict the sending cycle of dynamic probes.

[0125] Model structure: Input layer: Receives time series data, including network performance indicators (latency, bandwidth utilization, packet loss rate, etc.), targets, and network topology.

[0126] LSTM layer: A layer of LSTM units, used to capture long-term dependencies in time series. Each LSTM layer contains multiple units and uses dropout to prevent overfitting.

[0127] Dense layer: After the LSTM layer, add a Dense layer to map the output of the LSTM layer to the final predicted value: the number of transmissions and the period.

[0128] Output layer: outputs the predicted number of transmissions and intervals.

[0129] This application uses the Deep Reinforcement Learning (DRL) framework to train the reinforcement learning model. The training process is as follows: Initialize the global actor network and the global critic network: Initialize the global actor network and the global critic network with random weights. The global actor network is responsible for predicting the probability distribution of the next action. The global critic network is responsible for estimating the value function for a given state.

[0130] Create multiple task workers: Each worker has its own Actor network and Critic network, which are copies of the global network. Each agent runs in an independent training environment.

[0131] Worker training process (asynchronous): Sampled network topology instance: Samples the network topology from the training set, including the specific network topology and related node information.

[0132] Generate Path: Generate possible dynamic probe paths based on the current policy (output by the agent's Actor network). Path generation is a sequential decision-making process where each step selects the next node based on the current state.

[0133] Calculate rewards: Calculate rewards based on the generated path and the user's optimization goal. The reward function is calculated based on the performance indicators and weights of the path.

[0134] When the value of the reward function increases, the system will update the bitmap according to the new optimization target, reduce the number and frequency of transmissions, and ensure that DPs can collect more data related to key performance indicators.

[0135] When the value of the reward function becomes smaller, the model explores different actions and paths more actively, updates more indicators that need to be covered, and increases the number and frequency of transmissions.

[0136] Calculate gradients and update the local network: Use the policy gradient algorithm to calculate the gradient of the Actor network. Use the Temporal Difference Error to calculate the gradient of the Critic network. Update the parameters of the agent's Actor network and Critic network through backpropagation.

[0137] Pushing gradients to the global network: The agent pushes the gradients it computes to the global network. The global network uses these gradients to update its own parameters.

[0138] Pull the latest parameters from the global network: The worker pulls the latest parameters from the global network to update its own Actor network and Critic network.

[0139] Repeat the above steps: All agents asynchronously repeat the above steps until the model converges.

[0140] Model convergence: When the model converges, the global actor network and the global critic network are used as the final model.

[0141] The loss function of the reinforcement learning model is as follows: ; In the formula, N is the number of satellite nodes included in the path, L s Indicates the resource information occupied by the probe message corresponding to node s, P s is the frequency corresponding to the satellite node; n is the number of performance indicators included in the network status information, w i Indicates the weight value corresponding to the i-th performance indicator, m i Represents the i-th performance indicator value.

[0142] Taking the conditional probability training instructions as an example, the following is explained: Assume that the current satellite node is and is connected to three network satellite nodes A, B, and C. The network status information of these three is contained in X t In, X t The status information of the current node includes the connection status with adjacent nodes, link delay information, etc.

[0143] The n in the formula refers to the number of samples in the conditional probability model training, which is set manually.

[0144] Conditional probability formula part: .

[0145] u t is the unnormalized score, where w c is a trainable weight matrix that performs a linear transformation on the concatenation of x and h; v c is a trainable variable used to calculate the unnormalized score.

[0146] , c t Used to calculate the hidden state. θ is the updated parameter in the actor network. When θ is updated, w c and v c Update together.

[0147] The Actor network is used to calculate conditional probabilities, and the Critic network is used to evaluate rewards.

[0148] Initialize the global Actor network parameters θ and Critic network parameters φ (both parameters are randomly generated). Initialize the global gradient △θ=0,△φ=0.

[0149] Parallel training: Create multiple workers, each with its own Actor network and Critic network, which are copies of the global network. Each worker runs in an independent training environment.

[0150] The training process of each worker: Initialization: Each worker obtains the latest parameters and from the global network.

[0151] Sampling: Sampling in the current environment, for each sampling, through conditional probability Select the path to get a new one. Calculate the reward based on the selected path .

[0152] Update gradient: ;

[0153] Synchronize gradients to the global network: Synchronize the calculated gradients △θ, △φ to the global network.

[0154] Update global network parameters: ; Update the worker's network parameters: Get the latest parameters θ and φ from the global network and update the worker's network.

[0155] Termination condition: Stop training when the maximum number of training rounds is reached or the performance of the global network no longer improves.

[0156] When the electronic device collects probe data, the telemetry analyzer parses each probe into a series of dictionaries or tuples. These dictionaries or tuples contain specific telemetry information collected from various network nodes (such as switches and satellites), such as link latency, queue length, and port throughput. This information can be used for network monitoring, troubleshooting, and traffic control. For basic measurement data, the analyzer provides basic network status for the dynamic probe routing system. For the data plane (DP), the analyzer stores network information in a database accessible to other network applications. To ensure structured storage, the storage format is designed based on the switch ID and metadata type. To conserve storage space, the telemetry analyzer updates existing dictionaries or tuples with the latest network information whenever new probe data is received. Finally, the telemetry analyzer calculates high-level metadata based on queries. Network applications can periodically obtain network telemetry reports based on their telemetry needs. High-level metadata refers to higher-level network performance metrics or status information derived from the raw data collected from network nodes. High-level link quality metadata is derived by calculating the average link latency over a period of time.

[0157] Another task of telemetry analysis is to locate network faults. When a link fails, all dynamic probes that pass through the faulty link cannot be collected by the controller. To minimize the number of dynamic probes required, it is necessary to ensure that only one dynamic probe passes through each link. By utilizing non-overlapping dynamic probe paths, the fault path can be quickly found and the fault location workload can be reduced. This can be achieved by ensuring that a unique probe path is assigned to each link when generating the shortest geodesic.

[0158] This application proposes an adaptive in-band network telemetry method assisted by dual-time-scale probes, including basic probes and dynamic probes. From a technical perspective, the basic probe is set to a long period to collect basic network status information, while the dynamic probe is set to a dynamic period to adaptively and targetedly collect network status information based on the telemetry task input and network status.

[0159] The dynamic in-band measurement module in this application adopts an actor-critic mechanism based on deep reinforcement learning. It can optimize future path planning strategies based on previously collected network information and path planning results, and can automatically complete these tasks, reducing dependence on human intervention.

[0160] The telemetry analysis module in this application proposes a fault location mechanism. When a link fails, all dynamic probes that pass through the failed link are not collected by the controller. To minimize the number of dynamic probes that need to be checked, it is necessary to ensure that only one dynamic probe passes through each link. By utilizing non-overlapping dynamic probe paths, the fault path can be quickly found, reducing the workload of fault location.

[0161] This application proposes an adaptive in-band network telemetry method assisted by dual-time-scale probes. Basic network status information is collected through basic probes, and network status information is adaptively and targetedly collected through dynamic probes based on telemetry task input and network status self-learning results. This method can effectively improve the adaptability and flexibility of in-band measurements, ensure the in-band measurement effect, and reduce the in-band measurement overhead.

[0162] The dynamic in-band measurement module in this application uses an Actor-Critic mechanism based on deep reinforcement learning. It can automatically adjust the deployment plan according to the different requirements of users. It is more robust than traditional heuristic algorithms, making the telemetry system adaptable and no longer subject to the experience accumulation and subjective judgment of measurement personnel. It can also quickly solve the same type of combinatorial optimization problems and adapt to the frequent fluctuations in network load and dynamic changes in the network environment such as link disconnection or congestion, adjust system parameters in a timely manner, and optimize the quality and efficiency of network telemetry. At the same time, its self-learning ability can effectively extract key features from the collected data, guide the analysis results to be free from noise and interference, and improve the accuracy of measurement and analysis.

[0163] The telemetry analysis module in this application proposes a fault location mechanism that leverages the autonomous learning mechanism of the dynamic in-band measurement module to fully tap into the deep information of historical measurement data and dynamically guide network fault discovery and location. By utilizing non-overlapping dynamic probe paths, the fault path is quickly found, reducing the workload and measurement overhead of fault location.

[0164] The measurement method proposed in this application can collect data at the node, port, and bitmap levels. At the node level, it can measure signal strength, link quality, congestion, and other factors. At the port level, it can measure data transmission rate, bandwidth utilization, bit error rate, packet loss rate, latency, and jitter. At the bitmap level, it focuses on signal strength, signal-to-noise ratio, link quality, and queue length. This method fills the gap in the performance requirements of satellite communication networks that traditional telemetry methods cannot meet, while also adaptively adjusting measurement metrics based on measurement needs.

[0165] When telemetry requirements change or the network environment changes significantly (for example, the network size changes), the algorithm proposed by the dynamic in-band measurement module can use transfer learning methods, that is, setting different reward functions and training sets to assist in training the model to solve different problems and be applied to various network environments.

[0166] Figure 9 The schematic diagram of the in-band measurement device provided for this application includes: A first acquisition module 11 is configured to send a first probe message to at least one first satellite to acquire first network status information of the at least one first satellite; A determination module 12 is configured to generate a second probe message corresponding to at least one second satellite based on the first network status information of the at least one first satellite; The second acquisition module 13 is configured to send a corresponding second probe message to the at least one second satellite to acquire second network status information of the at least one second satellite.

[0167] The determining module 12 is specifically configured to input the first network status information of the at least one first satellite into a reinforcement learning model, and generate a second probe message corresponding to the at least one second satellite based on the reinforcement learning model.

[0168] The determining module 12 is further configured to input the first network status information of the at least one first satellite into a reinforcement learning model, and determine at least one detection path based on the reinforcement learning model; The determining module 12 is specifically configured to send a corresponding second probe message to at least one second satellite in the at least one detection path.

[0169] The first acquisition module 11 is specifically configured to send a first probe message to the at least one first satellite based on a first frequency; The second acquisition module 13 is specifically configured to send a corresponding second probe message to the at least one second satellite based on the second frequency; The second frequency is lower than the first frequency.

[0170] The determination module 12 is further configured to input the first network status information of the at least one first satellite into a reinforcement learning model, and determine the second frequency corresponding to the at least one second satellite based on the reinforcement learning model.

[0171] The determining module 12 is further configured to determine bitmap information corresponding to the network status information of the at least one second satellite based on the reinforcement learning model; The second acquisition module 13 is specifically configured to send a corresponding second probe message carrying the bitmap information to the at least one second satellite; wherein the bitmap information is used to indicate measurement network status information.

[0172] The first acquisition module 11 is specifically configured to determine, for the at least one first satellite, a target gateway station that is closest to the first satellite; and send the first probe message to the first satellite through the target gateway station.

[0173] The first acquisition module 11 is specifically configured to acquire, for the at least one first satellite, first network status information of at least one first satellite on a path from the target gateway to the first satellite.

[0174] The second acquisition module 13 is specifically configured to determine, for the at least one detection path, second network status information of at least one second satellite on the detection path; wherein the second frequencies corresponding to the second satellites on the detection path are the same.

[0175] The reinforcement learning model is used to determine the next-hop satellite of a satellite, and the at least one detection path includes the satellite and the next-hop satellite; wherein the reinforcement learning model is used to determine, for the satellite, the conditional probability of the satellite and at least one candidate satellite connected to the next hop; and determine the candidate satellite with the largest conditional probability as the next-hop satellite of the satellite.

[0176] The determining module 12 is specifically configured to determine, based on the first network status information of the satellite, at least one satellite that is adjacent to the satellite and in normal communication with the satellite, and determine the at least one satellite as at least one candidate satellite.

[0177] The device further comprises: The training module 14 is configured to determine, for at least one first satellite in at least one sample path, a first reward value corresponding to the first satellite based on the bitmap information and weight value corresponding to the network status information of the first satellite during the current iterative training process, as well as the frequency of sending probe messages; determine a second reward value corresponding to the sample path based on the first reward value corresponding to the at least one first satellite in the sample path; update the gradient of the reinforcement learning model, the bitmap information and weight value corresponding to the network status information, and the frequency of sending probe messages based on the minimum second reward value corresponding to the at least one sample path; and repeat iterative training until the reinforcement learning model training is completed.

[0178] The training module 14 is specifically configured to determine a first sub-reward value and information about resources occupied by probe messages based on bitmap information and weight values ​​corresponding to the network status information of the first satellite during a current iteration; determine a second sub-reward value based on the information about resources occupied by the probe messages and the frequency of sending the probe messages; and determine a first reward value corresponding to the first satellite based on the first sub-reward value and the second sub-reward value.

[0179] The training module 14 is further configured to determine corresponding associated network status information according to telemetry requirements of the in-band measurement; and determine initial bitmap information and initial weight values ​​corresponding to the network status information of the first satellite according to the associated network status information.

[0180] The present application also provides an electronic device, such as Figure 10 As shown, it includes: a processor 21, a communication interface 22, a memory 23 and a communication bus 24, wherein the processor 21, the communication interface 22, and the memory 23 communicate with each other through the communication bus 24; The memory 23 stores a computer program, and when the program is executed by the processor 21 , the processor 21 performs at least one of the above method steps.

[0181] The communication bus mentioned in the electronic devices mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into address buses, data buses, control buses, etc. For ease of illustration, only a single thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0182] The communication interface 22 is used for communication between the electronic device and other devices.

[0183] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0184] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0185] The present application also provides a computer storage readable storage medium, which stores a computer program that can be executed by an electronic device. When the program runs on the electronic device, the electronic device implements at least one of the above method steps when executing.

[0186] The present application provides a computer program product, which includes an executable program. When the executable program is executed by a processor, the method described above is implemented.

[0187] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0188] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. An in-band measurement method, characterized in that: Applied to electronic equipment, the method comprises: Sending a first probe message to at least one first satellite to obtain first network status information of the at least one first satellite; Based on the first network status information of the at least one first satellite, generating a second probe message corresponding to at least one second satellite; A corresponding second probe message is sent to the at least one second satellite to obtain second network status information of the at least one second satellite.

2. The method according to claim 1, characterized in that The method comprises: The first network status information of the at least one first satellite is input into a reinforcement learning model, and based on the reinforcement learning model, a second probe message corresponding to the at least one second satellite is generated.

3. The method according to claim 1, characterized in that The method further comprises: Inputting the first network status information of the at least one first satellite into a reinforcement learning model, and determining at least one detection path based on the reinforcement learning model; The method comprises: A corresponding second probe message is sent to at least one second satellite in the at least one detection path.

4. The method according to claim 3, characterized in that The method comprises: Sending a first probe message to the at least one first satellite based on a first frequency; Sending a corresponding second probe message to the at least one second satellite based on the second frequency; The second frequency is smaller than the first frequency.

5. The method according to claim 4, characterized in that The method further comprises: The first network status information of the at least one first satellite is input into a reinforcement learning model, and the second frequency corresponding to the at least one second satellite is determined based on the reinforcement learning model.

6. The method according to claim 2, characterized in that The method comprises: Determine bitmap information corresponding to the network status information of the at least one second satellite based on the reinforcement learning model; A corresponding second probe message carrying the bitmap information is sent to the at least one second satellite; wherein the bitmap information is used to indicate measurement network status information.

7. The method according to claim 1, characterized in that The sending of a first probe message to at least one first satellite includes: For the at least one first satellite, determining a target gateway station that is closest to the first satellite; The first probe message is sent to the first satellite through the target gateway.

8. The method according to claim 7, characterized in that Acquiring first network status information of the at least one first satellite includes: For the at least one first satellite, first network status information of at least one first satellite on a path from the target gateway to the first satellite is obtained.

9. The method according to claim 4, characterized in that Acquiring second network status information of the at least one second satellite includes: For the at least one detection path, second network status information of at least one second satellite on the detection path is determined; wherein the second frequencies corresponding to the second satellites on the detection path are the same.

10. The method according to claim 3, characterized in that Determining at least one detection path based on the reinforcement learning model includes: The reinforcement learning model is used to determine the next-hop satellite of the satellite, and the at least one detection path includes the satellite and the next-hop satellite; wherein the reinforcement learning model is used to determine, for the satellite, the conditional probability of the candidate satellite connected to the satellite and at least one next-hop; and determine the candidate satellite with the largest conditional probability as the next-hop satellite of the satellite.

11. The method according to claim 10, characterized in that The process of determining the candidate satellite for the at least one next hop connection includes: At least one satellite adjacent to the satellite and communicating normally is determined according to the first network status information of the satellite, and the at least one satellite is determined as at least one candidate satellite.

12. The method according to any one of claims 2 to 11, characterized in that The training process of the reinforcement learning model includes: For at least one first satellite in at least one sample path, a first reward value corresponding to the first satellite is determined according to the bitmap information and weight value corresponding to the network status information of the first satellite in the current iterative training process, and the sending frequency of the probe message; a second reward value corresponding to the sample path is determined according to the first reward value corresponding to the at least one first satellite in the sample path; the gradient of the reinforcement learning model, the bitmap information and weight value corresponding to the network status information, and the sending frequency of the probe message are updated according to the minimum second reward value corresponding to the at least one sample path; and the iterative training is repeated until the training of the reinforcement learning model is completed.

13. The method according to claim 12, characterized in that Determining the first reward value corresponding to the first satellite according to the bitmap information and the weight value corresponding to the network status information of the first satellite in the current iteration process, and the sending frequency of the probe message includes: Determine the first sub-reward value and the resource information occupied by the probe message according to the bitmap information and the weight value corresponding to the network status information of the first satellite in the current iteration process; Determining a second sub-reward value according to the resource information occupied by the probe message and the sending frequency of the probe message; A first reward value corresponding to the first satellite is determined according to the first sub-reward value and the second sub-reward value.

14. The method according to claim 12, characterized in that The method further comprises: Determine corresponding associated network status information according to the telemetry requirements of the in-band measurement; According to the associated network status information, initial bitmap information and initial weight values ​​corresponding to the network status information of the first satellite are determined.

15. An in-band measurement device, characterized in that: Applied to electronic equipment, the device comprises: A first acquisition module, configured to send a first probe message to at least one first satellite to acquire first network status information of the at least one first satellite; A determination module, configured to generate a second probe message corresponding to at least one second satellite based on the first network status information of the at least one first satellite; The second acquisition module is used to send a corresponding second probe message to the at least one second satellite to acquire second network status information of the at least one second satellite.

16. The device according to claim 15, characterized in that The determination module is specifically configured to input the first network status information of the at least one first satellite into a reinforcement learning model, and generate a second probe message corresponding to the at least one second satellite based on the reinforcement learning model.

17. The device according to claim 15, characterized in that The determination module is further configured to input the first network status information of the at least one first satellite into a reinforcement learning model, and determine at least one detection path based on the reinforcement learning model; The determining module is specifically configured to send a corresponding second probe message to at least one second satellite in the at least one detection path.

18. The device according to claim 17, characterized in that The first acquisition module is specifically configured to send a first probe message to the at least one first satellite based on a first frequency; The second acquisition module is specifically configured to send a corresponding second probe message to the at least one second satellite based on a second frequency; The second frequency is smaller than the first frequency.

19. The device according to claim 18, characterized in that The determination module is further used to input the first network status information of the at least one first satellite into a reinforcement learning model, and determine the second frequency corresponding to the at least one second satellite based on the reinforcement learning model.

20. The device according to claim 16, characterized in that The determination module is further used to determine the bitmap information corresponding to the network status information of the at least one second satellite based on the reinforcement learning model; The second acquisition module is specifically configured to send a corresponding second probe message carrying the bitmap information to the at least one second satellite; wherein the bitmap information is used to indicate measurement network status information.

21. The device according to claim 15, characterized in that The first acquisition module is specifically configured to determine, for the at least one first satellite, a target gateway station that is closest to the first satellite; and send the first probe message to the first satellite through the target gateway station.

22. The device according to claim 21, characterized in that The first acquisition module is specifically configured to acquire, for the at least one first satellite, first network status information of at least one first satellite on a path from the target gateway to the first satellite.

23. The device according to claim 18, characterized in that The second acquisition module is specifically used to determine, for the at least one detection path, second network status information of at least one second satellite on the detection path; wherein the second frequencies corresponding to the second satellites on the detection path are the same.

24. The device according to claim 17, characterized in that The reinforcement learning model is used to determine the next-hop satellite of the satellite, and the at least one detection path includes the satellite and the next-hop satellite; wherein the reinforcement learning model is used to determine, for the satellite, the conditional probability of the candidate satellite connected to the satellite and at least one next-hop; and determine the candidate satellite with the largest conditional probability as the next-hop satellite of the satellite.

25. The device according to claim 24, characterized in that The determination module is specifically configured to determine, based on the first network status information of the satellite, at least one satellite that is adjacent to the satellite and in normal communication with the satellite, and determine the at least one satellite as at least one candidate satellite.

26. The device according to any one of claims 16 to 25, characterized in that The device also includes: A training module is used to determine, for at least one first satellite in at least one sample path, a first reward value corresponding to the first satellite according to bitmap information and a weight value corresponding to network status information of the first satellite in a current iterative training process, and a sending frequency of a probe message; determine a second reward value corresponding to the sample path according to the first reward value corresponding to at least one first satellite in the sample path; update the gradient of the reinforcement learning model, the bitmap information and the weight value corresponding to the network status information, and the sending frequency of the probe message according to the minimum second reward value corresponding to the at least one sample path; and repeat the iterative training until the training of the reinforcement learning model is completed.

27. The device according to claim 26, characterized in that The training module is specifically used to determine the first sub-reward value and the resource information occupied by the probe message according to the bitmap information and the weight value corresponding to the network status information of the first satellite in the current iteration process; Determining a second sub-reward value according to the resource information occupied by the probe message and the sending frequency of the probe message; A first reward value corresponding to the first satellite is determined according to the first sub-reward value and the second sub-reward value.

28. The device according to claim 26, characterized in that The training module is further used to determine corresponding associated network status information according to the telemetry requirements of the in-band measurement; and to determine initial bitmap information and initial weight values ​​corresponding to the network status information of the first satellite according to the associated network status information.

29. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method according to any one of claims 1 to 14 when executing a program stored in a memory.

30. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 14 is implemented.

31. A computer program product, characterized in that The computer program product comprises an executable program, which is executed by a processor to implement the method according to any one of claims 1 to 14.

Images