A space-air-ground collaborative scene-oriented digital twin semantic information synchronization method

Abstract

The application belongs to the field of space-air-ground integrated network and edge computing technology, and particularly relates to a digital twin semantic information synchronization method for space-air-ground collaborative scenarios, comprising the following steps: establishing a three-level space-air-ground digital twin network system composed of ground mobile devices, unmanned aerial vehicles and satellites; constructing various computing models of the three-level space-air-ground digital twin network system; constructing a joint optimization problem model based on the three-level space-air-ground digital twin network system; modeling the joint optimization problem model as a Markov decision process with a discrete-continuous hybrid action space; solving the Markov decision process by using a constraint-aware virtual action compressed proximal policy optimization method, and outputting discrete device correlation strategies and continuous semantic, communication and computing resource allocation strategies; and the application effectively solves the problem of neglecting reconstruction accuracy in digital twin semantic information synchronization under a space-air-ground network.

Description

Technical Field

[0001] This invention belongs to the fields of integrated air-space-ground networks and edge computing technology, and specifically relates to a method for synchronizing digital twin semantic information for air-space-ground collaborative scenarios. Background Technology

[0002] The integrated air-space-ground network, by combining ground mobile devices, drones, and satellites, constructs a layered network architecture with wide coverage and flexible deployment methods. It can provide reliable communication and computing services for scenarios where ground communication infrastructure is severely limited, such as remote mountainous areas, forests, deserts, islands, and disaster areas.

[0003] Digital twin technology creates virtual mapping models of physical devices to simulate and analyze device behavior, and can be applied to real-time monitoring of device status in integrated space-air-ground networks. To ensure that the Age of Information (AoI) between the digital twin and the real device remains within a sufficiently small range, the real device needs to frequently synchronize its status information to the corresponding digital twin.

[0004] However, traditional digital twin synchronization schemes typically transmit raw data directly. As the number of devices increases and wireless link bandwidth becomes limited, the transmission of raw data leads to significant communication latency and energy consumption, causing not only delays in twin state updates but also potential severe congestion of the wireless link. To address this issue, the task information required for twin state reconstruction can be semantically encoded before transmission, effectively removing irrelevant and redundant data and achieving the goal of reducing communication latency and energy consumption.

[0005] Nevertheless, existing digital twin semantic synchronization strategies still have two significant shortcomings: First, current semantic information synchronization methods usually only focus on information age (AoI) and ignore the accuracy of semantic reconstruction, thus making it difficult to truly meet the synchronization quality requirements in dynamic environments; Second, most existing methods are limited to edge scenarios, and the algorithms used are difficult to adapt to the highly complex and heterogeneous application environment of space-air-ground networks. Summary of the Invention

[0006] This invention aims to address the quality bottleneck of semantic synchronization in digital twins within space-air-ground networks. To overcome the shortcomings of existing methods, it proposes a digital twin semantic information synchronization method for space-air-ground collaborative scenarios. This method can simultaneously meet the synchronization quality requirements of information age and semantic reconstruction accuracy, adapt to the complex application scenarios of layered space-air-ground networks, and achieve joint optimization of semantic processing strategies and communication computing resources.

[0007] This plan includes the following steps:

[0008] S1. Establish a three-tiered air-space-ground digital twin network system consisting of ground mobile devices, drones, and satellites;

[0009] S2. Construct various computational models for a three-tiered space-air-ground digital twin network system;

[0010] S3. Based on a three-level space-air-ground digital twin network system, a joint optimization problem model is constructed. The optimization objective is to minimize the long-term cumulative expectation of the sum of the accurate semantic twin synchronization age of all time slots and the system energy consumption, under the condition that the total number of time slots tends to infinity.

[0011] S4. Model the joint optimization problem as a Markov decision process with a discrete-continuous mixed action space;

[0012] S5. The Constraint-Aware Virtual-Action-Compressed Proximal Policy Optimization (CAVAC-PPO) method is used to solve the Markov decision process, outputting discrete device association policies and continuous semantic, communication and computing resource allocation policies.

[0013] The beneficial effects of this invention are:

[0014] This invention effectively addresses the problem of neglecting reconstruction accuracy in digital twin semantic information synchronization under space-air-ground networks, as well as the limitations of existing algorithms in adapting to complex hierarchical network scenarios, by using a virtual action compression near-end policy optimization algorithm. While improving the accuracy and real-time performance of digital twin state synchronization, this algorithm reduces the dimensionality and decision complexity of the joint action space through a virtual action compression mechanism, accelerating the convergence process of policy learning, thereby ensuring the stable and efficient operation of digital twin semantic information synchronization in space-air-ground networks. Attached Figure Description

[0015] Figure 1 This is a flowchart of a digital twin semantic information synchronization method for a space-air-ground collaborative scenario according to the present invention.

[0016] Figure 2 This is a schematic diagram of the three-level air-space-ground digital twin network system of the present invention;

[0017] Figure 3 This is a flowchart of the constraint-aware virtual action compression proximal strategy optimization method of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] To overcome the problems of neglecting the coordinated optimization of information freshness and semantic reconstruction accuracy in existing technologies, and the problem of action space explosion easily caused by high-dimensional hybrid action spaces, this invention provides a digital twin semantic information synchronization method for space-air-ground collaborative scenarios, such as... Figure 1 As shown, it includes the following steps:

[0020] S1. Establish a three-tiered air-space-ground digital twin network system consisting of ground mobile devices, drones, and satellites, and determine the deployment locations of the global controller and inter-domain controllers.

[0021] In some embodiments, the three-level air-space-ground digital twin network system constructed by the present invention, such as Figure 2 As shown, the system includes A ground mobile device, The system consists of a drone and a satellite, with a length of... The discrete time slots are used as the basic unit of operation. Let the set of mobile devices be... drone collection Time slot set They are respectively: , , , Indicates the number of time slots.

[0022] In this system, each ground mobile device corresponds to a unique digital twin instance, and all digital twin instances are deployed in the digital twin resource pool of a single satellite. The ground mobile device generates a certain number of bits of raw state data in each time slot. Due to the limited computing resources of the ground mobile device, this raw state data is first partially semantically encoded locally on the ground mobile device. Subsequently, the ground mobile device selects a UAV to supplement the semantic encoding of the remaining raw state data. Finally, the UAV uploads the encoded data to the satellite, where the satellite performs semantic decoding and state reconstruction, thereby achieving complete digital twin synchronization. A single digital twin state synchronization task is defined as follows:

[0023] (1)

[0024] In the formula, Indicates the first The ground mobile device in the first A digital twin state synchronization task in one time slot; Indicates the first The ground mobile device in the first The amount of raw state data for each time slot; Indicates the first The ground mobile device in the first The number of CPU cycles required to process each bit of data in each time slot; This represents the number of CPU cycles required to complete the supplementary semantic encoding for each bit of the drone corresponding to the kth ground mobile device. Indicates the satellite's position on the first The number of CPU cycles required per bit when performing semantic data decoding and digital twin reconstruction on a ground mobile device; Indicates the first The maximum end-to-end state synchronization delay allowed for a single ground mobile device; Indicates the first The upper limit of the reconstruction error allowed by the digital twin state of a ground mobile device.

[0025] Multiple drones are divided into several drone clusters based on their geographic service area or mission area, with an inter-domain controller set up within each cluster. The inter-domain controller is responsible for collecting resource status data, managing drone resources, and executing global decisions within its cluster. A global controller is deployed on a single satellite to centrally manage the inter-domain controllers of all drone clusters and maintain the status of all network devices, drone computing and communication resources, satellite resources, and digital twin synchronization.

[0026] The policy network and value network, trained using the constraint-aware virtual action compression proximal policy optimization method, are deployed on the global controller on the satellite. During online operation, the global controller inputs the current network state, generates joint virtual actions for all devices, and distributes the actual execution parameters for the corresponding clusters to the relevant inter-domain controllers.

[0027] The controller is only used to describe the deployment location and resource management level of the algorithm. It is not used as an action variable for joint optimization, nor is a separate mathematical model of the controller established.

[0028] S2. Construct various computational models for a three-tiered space-air-ground digital twin network system, including:

[0029] The model includes a local semantic coding model for the device, a transmission model from the device to the drone, a supplementary semantic coding model for the drone, a semantic transmission model from the drone to the satellite, a satellite decoding and digital twin state reconstruction model, a system energy consumption model, and an accurate semantic twin synchronization age model.

[0030] In some embodiments, due to the limited communication capabilities of ground mobile devices, it is impossible to directly and efficiently transmit all raw state data to the UAV. Therefore, local semantic encoding processing of the collected raw state data is required first. For any remaining data that cannot be processed due to computing power or time constraints, it is uploaded to the UAV, which then undertakes the subsequent supplementary semantic encoding. Based on this, the device's local semantic encoding model is defined as follows:

[0031] For each ground mobile device Set the corresponding local semantic processing ratio. This distinguishes between the raw state data that the ground mobile device itself needs to process and the raw state data that needs to be sent to the UAV for further processing, specifically as follows:

[0032] , (2)

[0033] In the formula, Indicates the first The ground mobile device in the first The amount of raw state data processed locally per time slot, Indicates the first The ground mobile device in the first The amount of remaining raw state data for each time slot. The higher the value, the more likely the ground mobile equipment is to be affected. The greater the computational cost, the less the amount of original state data remains.

[0034] Therefore, via ground mobile equipment The amount of local semantic data obtained by semantic encoding is:

[0035] (3)

[0036] In the formula, Indicates the first The ground mobile device in the first The amount of local semantic data per time slot; This represents the local semantic preservation coefficient; the smaller the value, the greater the compression strength.

[0037] At this point, the data transmitted via the air-to-ground link includes local semantic data and the remaining raw state data, represented as follows:

[0038] (4)

[0039] In the formula, Indicates the first The ground mobile device in the first Data is transmitted via a ground-to-air link in each time slot.

[0040] In addition, for each ground mobile device Configure the corresponding device local computing frequency. Therefore, the local encoding delay is:

[0041] (5)

[0042] In the formula, Indicates the first The ground mobile device in the first Local encoding delay for each time slot; Indicates the first The ground mobile device in the first The upper limit of the device's local calculation frequency for each time slot.

[0043] In some embodiments, the device-to-drone transmission model is defined as follows:

[0044] (6)

[0045] (7)

[0046] In the formula, Indicates the first The first time slot, the first Ground mobile devices and drones The transmission rate between them; Indicates the ground-to-air bandwidth; Indicates the first The transmission power of a ground mobile device; Indicates the first Ground mobile devices and drones Channel power gain between; Indicates the first Ground mobile devices and drones The time required for data transmission between them; This represents the power spectral density of the equivalent Gaussian white noise. Represents a binary variable, when the first... The ground mobile device in the first Select drone for each time slot When undertaking the semantic encoding of the remaining original state data =1, otherwise =0. Each ground mobile device can only select one drone per time slot, i.e. , .

[0047] It should be noted that the air-to-ground bandwidth must meet the following constraints:

[0048] , (8)

[0049] In the formula, Indicates drone The maximum total bandwidth available for the air-to-ground link to the ground mobile devices it serves.

[0050] In some embodiments, the UAV supplemental semantic coding model is defined as follows:

[0051] After receiving data from ground-based mobile devices, the drone only performs semantic encoding on the remaining raw state data. Therefore, compared to the first... A drone corresponding to a ground mobile device Local encoding latency for:

[0052] (9)

[0053] In the formula, This indicates the computing frequency allocated by the drone to ground-based mobile devices.

[0054] At this point, the amount of semantic data supplemented by the UAV semantic encoding is represented as follows:

[0055] (10)

[0056] In the formula, Indicates the first Each ground mobile device corresponds to a drone. Supplemental semantic data volume This represents the semantic preservation coefficient for drones.

[0057] Ultimately, ground mobile equipment via drone Complete semantic data volume sent to satellite Represented as:

[0058] (11)

[0059] It should be noted that the computational resource constraints of the drone satisfy:

[0060] , (12)

[0061] In the formula, Indicates drone It can be used to supplement the maximum computational frequency limit for semantic encoding tasks.

[0062] In some embodiments, the UAV-to-satellite semantic transmission model is defined as follows:

[0063] (13)

[0064] (14)

[0065] In the formula, Indicates the first The first time slot, the first Each ground mobile device corresponds to a drone. Transmission rate with satellite; Indicates drone Bandwidth allocated for space-air links; Indicates drone The transmission power; Indicates drone Channel gain with satellites; Indicates drone The time required for data transmission between satellites; Indicates drone Distance to satellites This indicates the speed of electromagnetic wave propagation.

[0066] It should be noted that the bandwidth corresponding to the space-air link must meet the following constraints:

[0067] , (15)

[0068] In the formula, Indicates drone The maximum total bandwidth that can be used for semantic data transmission in the space-air link.

[0069] In some embodiments, the satellite decoding and digital twin state reconstruction model is defined as follows:

[0070] The satellite only performs semantic decoding and digital twin state reconstruction, thus:

[0071] (16)

[0072] In the formula, Indicates the satellite's position on the first Processing latency of semantic data from a ground mobile device Indicates the satellite is assigned to the first The computation frequency of a digital twin instance corresponding to a ground mobile device.

[0073] When performing calculations, satellites must meet the following constraints:

[0074] , (17)

[0075] In the formula, This indicates the maximum computing power that the satellite can use for semantic decoding and digital twin reconstruction; Indicates the first The resources occupied by a digital twin instance corresponding to a ground mobile device, namely the model, state and cache resources it needs to occupy; This indicates the capacity of the satellite's digital twin resource pool.

[0076] Based on the foregoing analysis, the first The ground mobile device in the first The latency of a single digital twin state update in a time slot Represented as

[0077] (18)

[0078] in, , This indicates the upper limit of the latency for updating the state of the digital twin.

[0079] In some embodiments, the system energy consumption model is defined as follows:

[0080] (19)

[0081] (20)

[0082] (twenty one)

[0083] (twenty two)

[0084] (twenty three)

[0085] (twenty four)

[0086] In the formula, Indicates the first The ground mobile device in the first Total energy consumption per time slot; Indicates the first The ground mobile device in the first Energy consumption for local semantic computation per time slot; Indicates the first The ground mobile device in the first Energy consumption for ground-to-air transmission per time slot; Indicates the first The first time slot, the first Each ground mobile device corresponds to a drone. The computational energy consumption of semantic encoding; Indicates the first Each ground mobile device corresponds to a drone. Energy consumption for semantic transmission to satellites; Indicates the satellite's position on the first Energy consumption for processing semantic data of a ground mobile device; , , These represent the energy consumption coefficients of ground equipment, drones, and satellite processors, respectively.

[0087] In some embodiments, the accurate semantic twin synchronized age model is defined as follows:

[0088] (25)

[0089] In the formula, Indicates the first The ground mobile device in the first The accurate semantic twin synchronization age (AoASTS) of each time slot. Indicates the first The ground mobile device in the first Age information of the digital twin updated in each time slot; Indicates the first The ground mobile device in the first The degree of semantic mismatch between digital twins in each time slot.

[0090] Specifically,

[0091] (26)

[0092] In the formula, This represents the update time factor, used to determine whether the update is completed within the preset time. Specifically, , when the condition At the time of its establishment, =1, otherwise =0, 1, {·} represents the characteristic function.

[0093] (27)

[0094] in, Indicates the state-validation gating factor. , when the condition and At the time of its establishment, =1, otherwise =0. This indicates the permissible age threshold for digital twin information. This indicates the upper limit of the allowed reconstruction error; This indicates the accuracy of the reconstruction.

[0095] Based on the saturated multiplicative fitting relationship between semantic computing resources and reconstruction accuracy, this invention defines reconstruction accuracy as follows:

[0096] (28)

[0097] In the formula, Represents the scaling factor. , The fitting index represents the semantic fidelity term and the satellite computation frequency term. These three parameters are calibrated through offline testing based on the specific semantic encoder and digital twin task. Semantic fidelity is represented as:

[0098] , (29)

[0099] In the formula, This indicates the importance of local semantic data in the reconstruction of the digital twin's state. This indicates the importance of the amount of semantic data supplemented by drones for the reconstruction of the state of the digital twin.

[0100] S3. Based on a three-level space-air-ground digital twin network system, a joint optimization problem model is constructed. The optimization objective is to minimize the long-term cumulative expectation of the sum of the accurate semantic twin synchronization age of all time slots and the system energy consumption, under the condition that the total number of time slots tends to infinity.

[0101] In some embodiments, the joint optimization problem model is defined as follows:

[0102] (30)

[0103] (30a)

[0104] (30b)

[0105] (30c)

[0106] (30d)

[0107] (30e)

[0108] (30f)

[0109] (30g)

[0110] In the formula, Represents the set of optimization variables. This indicates the time domain length of the constructed long-term average target. , These represent the non-negative weights of AoASTS and system energy consumption, respectively. Indicates the first A ground mobile device in a time slot Upload accurate semantic twin synchronized age information. Indicates the first A ground mobile device in a time slot The total energy consumption of uploading semantic information to the satellite semantic reconstruction process. This represents the expectation of randomness induced by random network states and random policies. Specifically, this invention requires joint optimization of the actual actions. Includes: binary variables Local semantic processing ratio Local semantic preservation coefficient UAV semantic preservation coefficient Transmission power Transmission power Ground-to-air bandwidth Space-air bandwidth Device local calculation frequency UAV calculation frequency Satellite calculation frequency .

[0111] As shown above, problem P has three significant characteristics: First, the network state changes dynamically with the amount of device data, the wireless channel, the drone load, and the digital twin state, belonging to a sequential stochastic decision-making problem; second, the association decisions between devices and drones belong to a discrete action space; and third, the allocation of semantic ratios, transmit power, bandwidth, and CPU resources belongs to a continuous action space. In summary, this problem is essentially a sequential optimization problem involving large-scale discrete-continuous mixed actions.

[0112] S4. Model the joint optimization problem as a Markov decision process with a discrete-continuous mixed action space.

[0113] In some embodiments, constructing a Markov decision process includes:

[0114] Define the state space:

[0115] Set time slot status for:

[0116] (31)

[0117] in, Indicates time slot The amount of raw state data for all ground mobile devices; Indicates time slot The age of all digital twin information for terrestrial mobile devices; Indicates time slot Reconstruction accuracy of all ground mobile devices; Indicates time slot The corresponding channel state matrix between ground mobile devices and drones; Indicates time slot The corresponding channel state matrix between the UAV and the satellite; Indicates time slot Available computing resources for drones corresponding to all ground mobile devices; Indicates the satellite's remaining computing resources; This indicates the association results between ground mobile devices and drones in the previous time slot.

[0118] Define the discrete-continuous hybrid action space:

[0119] Set time slot actual actions for:

[0120] (32)

[0121] in, Indicates time slot The proportion of local semantic processing for all ground mobile devices. Indicates time slot Local semantic preservation coefficients for all ground mobile devices Indicates time slot The semantic preservation coefficients of drones for all ground mobile devices Indicates time slot Transmission power of all ground mobile devices; Indicates time slot The corresponding drone transmission power for all ground mobile devices; Indicates time slot Ground-to-air bandwidth for all ground mobile devices; Indicates time slot The corresponding space-air bandwidth for all ground-based mobile devices; Indicates time slot The device-local calculation frequency of all ground mobile devices; Indicates time slot The drone computing frequency corresponding to all ground mobile devices; Indicates time slot Satellite calculation frequencies corresponding to all ground mobile devices.

[0122] Define reward function for:

[0123] ,

[0124] The designed reward function comprehensively considers the accurate semantic twin synchronization age and system energy consumption. In the subsequent constraint-aware virtual action compression near-end strategy optimization algorithm, the total constraints of ground-to-air bandwidth, air-to-space bandwidth, UAV CPU, and satellite CPU are automatically satisfied during the action mapping process through equations (39) and (40), so there is no need to set additional reward penalties for these total resource constraints. After all actual actions are recovered from virtual actions, the system calculates the accurate semantic twin synchronization age (AoASTS) and system energy consumption for this time slot.

[0125] S5. The constrained virtual action compression proximal policy optimization method is used to solve the Markov decision process, and the optimal discrete device association policy and continuous semantic, communication and computing resource allocation policy are output.

[0126] In some embodiments, step S5 specifically includes:

[0127] Step 1: By uniformly representing the discrete-continuous hybrid action space as a low-dimensional continuous virtual action space, the original Markov decision process is equivalently transformed into a continuous action Markov decision process.

[0128] In some embodiments, representing the discrete-continuous hybrid action space as a low-dimensional continuous virtual action space specifically includes:

[0129] S511. The discrete association decision between ground mobile devices and UAVs is compressed into an association virtual sub-vector segment, which serves as the first component of a low-dimensional continuous virtual action space; wherein each ground mobile device... from Discrete selection of one UAV from a set of UAVs is represented as a length of The continuous virtual action sub-vectors of all ground mobile devices constitute an associated virtual sub-vector segment.

[0130] Specifically, when a ground-based mobile device performs a digital twin state synchronization task, it needs to select one drone from multiple drones as a relay. If traditional joint discrete action representation is used, the selection of each device involves... One possibility is that the joint choice space size of all ground mobile devices is... With the exponential growth in the number of ground-based mobile devices and drones, the problem of an explosion in action space is highly likely to occur.

[0131] To address this problem, this invention employs binary encoding to compress and represent discrete selections. The length of the binary virtual action required for each ground mobile device to complete one selection is defined as:

[0132] (33)

[0133] Ready to use Each binary bit can uniquely represent The selection of drones. Based on this, all The association decision for each ground mobile device only requires A single, continuous virtual action dimension can fully express this. Compared to the original... Compared to joint discrete actions, the representation dimension of this binary virtual action only increases with the number of devices. Linear growth, with the number of drones It exhibits logarithmic growth, significantly reducing the complexity of the action space.

[0134] Based on the above compression method, in the constraint-aware virtual action compression proximal strategy optimization method, the Actor network is optimized according to the current state. Output continuous virtual actions :

[0135] (34)

[0136] in, This represents the average value of consecutive virtual actions output by the Actor network. θ represents the standard deviation of continuous virtual actions, and both are determined by the Actor network parameter θ. This represents a Gaussian distribution.

[0137] Continuous virtual actions Total dimensions are Segmented according to the order of ground mobile devices, ground mobile devices Corresponding to the first to Dimension, length is subvectors Continuous virtual actions That is, the corresponding associated virtual sub-vector segment.

[0138] The above process only completes the compressed representation of discrete association decisions into a continuous virtual action subspace, without involving specific numerical generation and interpretation. During the policy execution phase, after the Actor network outputs the total continuous virtual actions containing continuous virtual action subvectors, these subvectors need to be decoded into real UAV selections. The specific process is as follows:

[0139] Pair vectors Each dimension takes a sign function, therefore the ground mobile device Corresponding to the first continuous virtual action Dimension (i.e., the first) (a discrete virtual action) The binary bits obtained after sign quantization for:

[0140] (35)

[0141] This yields the binary virtual code of the ground mobile device k. However, when the number of drones... When the binary virtual code is not a power of 2, invalid combinations may exist, meaning some binary virtual codes do not correspond to any actual drone. Therefore, this invention predefines a code of length [length missing] for each drone. A valid binary virtual code, setting up a virtual code set. ,in Indicates drone The corresponding valid binary virtual code. Based on the binary virtual code generated by the Actor. Mapped by minimum Hamming distance:

[0142] (36)

[0143] in, This indicates that the legal UAV code with the smallest Hamming distance is selected. This represents the Hamming distance.

[0144] This leads to the recovery of the true discrete relational actions:

[0145] (37)

[0146] Equations (34) to (37) show that the random policy action generated by the Actor network is still the true determinant of which drone the ground mobile device chooses. The mapping mechanism between the binary virtual code and the most recently valid binary virtual code is only responsible for interpreting low-dimensional continuous actions as valid discrete choices, while ensuring that when the number of drones... When the value is not a power of 2, the virtual binary combination will not correspond to a non-existent drone, thus ensuring the effectiveness and stability of the associated decision.

[0147] S512. Each ground mobile device The independent continuous control actions are directly represented as unbounded continuous virtual values, forming independent continuous control virtual sub-vector segments, which serve as the second component of the low-dimensional continuous virtual action space. Each independent continuous control variable corresponds to a one-dimensional independent continuous control virtual value, and all independent continuous control virtual values ​​of all ground mobile devices constitute independent continuous control virtual sub-vector segments. Independent continuous control actions include local semantic processing ratio Local semantic preservation coefficient UAV semantic preservation coefficient Equipment transmission power UAV launch power Device local calculation frequency .

[0148] Furthermore, step S512 specifies the equipment. The local semantic processing ratio, local semantic preservation coefficient, UAV semantic preservation coefficient, device transmit power, UAV transmit power, and device local computing frequency are all entered into the total continuous virtual action vector as unbounded continuous virtual values, without any scaling during construction. During the policy execution phase, after the Actor outputs unbounded continuous virtual values, they need to be translated into physically meaningful real continuous control quantities through a unified interval mapping.

[0149] (38)

[0150] In the formula, Indicates time slot Next The final value of a real action, Indicates the first The lower bound of the value of a real action corresponds to the minimum allowable value of the corresponding physical parameter. Indicates the first The upper bound of the value of a real action corresponds to the maximum allowed value of the corresponding physical parameter. Indicates time slot The output of the next Actor policy network is the first Each original continuous virtual action is an unbounded continuous random variable, directly generated by the policy network. This mapping only applies to the independent continuous control virtual sub-vector segments defined by S512. It is not part of the construction process of the virtual action space, but rather a decoding stage during policy execution. It forms a complete action interpretation chain alongside the associated decoding of continuous virtual action sub-vectors and subsequent shared resource decoding.

[0151] S513. Each ground mobile device The competitive actions for each shared resource are represented by virtual competition scores, forming a shared resource competition virtual sub-vector segment as the third component of the low-dimensional continuous virtual action space. Each shared resource competition action corresponds to a one-dimensional virtual competition score, and the total virtual competition scores of all ground mobile devices constitute the shared resource competition virtual sub-vector segment. Shared resources competing for resources include air-to-ground bandwidth. Space-air bandwidth UAV calculation frequency Satellite calculation frequency .

[0152] In the aforementioned S513, the ground-to-air bandwidth, air-to-space bandwidth, and UAV computing frequency belong to the category of UAVs. For shared resources within the resource domain, allocation is only made in association with drones. The calculations are performed between devices; the satellite computing frequency is a satellite-side computing resource shared by all devices in the network. In this invention, ground mobile devices... The competition for ground-to-air bandwidth, space-to-air bandwidth, UAV computing frequency, and satellite computing frequency is incorporated into the shared resource competition virtual sub-vector segment in the form of unbounded continuous competition virtual scores, and no resource allocation calculation is performed during the construction phase.

[0153] During the strategy execution phase, to avoid invalid actions where the sum exceeds the total limit due to the Actor directly outputting the absolute value of resources, this invention does not use the method of correcting through optimization algorithms after the action is executed. Instead, it maps the competition virtual score defined in S513 to the actual resource allocation amount that satisfies the respective total constraints through Softmax normalization. For any type of shared resource Define its total resource volume as ,equipment The corresponding virtual competitive score is The actual amount of resources allocated for:

[0154] (39)

[0155] In the formula, This represents a very small positive real constant, avoiding a denominator of 0. Specifically, These correspond to ground-to-air bandwidth, air-to-space bandwidth, and UAV computing frequency, respectively.

[0156] Softmax normalization converts the competition intensity learned by the Actor into a real resource share that satisfies the total resource limit.

[0157] The allocation of satellite computing frequencies is also obtained by normalizing the competitive virtual score output by the Actor using equation (40), which naturally satisfies the total constraint of satellite computing resources.

[0158] (40)

[0159] This decoding step belongs to the action interpretation stage of the policy execution period, and is strictly distinguished from the virtual space construction of S513 in terms of timing and function.

[0160] S514. The space-based transmission power of each UAV is represented by an unbounded continuous virtual value, forming a virtual sub-vector segment of UAV transmission power, which serves as the fourth component of the low-dimensional continuous virtual action space. Each UAV corresponds to a one-dimensional virtual value of space-based transmission power, and the virtual values ​​of the space-based transmission power of all UAVs constitute the virtual sub-vector segment of UAV transmission power.

[0161] S515. The associated virtual sub-vector segments, independent continuous control virtual sub-vector segments, shared resource competition virtual sub-vector segments, and UAV launch power virtual sub-vector segments are sequentially spliced ​​together to form a total continuous virtual action in a low-dimensional continuous virtual action space.

[0162] In this embodiment of the invention, the virtual action dimensions required for each device include: Each drone has one discrete-related virtual action, one local processing ratio, one local semantic preservation coefficient, one UAV semantic preservation coefficient, one device transmit power, one device CPU, one ground-to-air bandwidth contention score, one space-to-air bandwidth contention score, one UAV CPU contention score, and one satellite CPU contention score. In addition, each UAV also has one space-to-air transmit power virtual action. Therefore, the total dimension of continuous virtual actions is:

[0163] (41)

[0164] As the network size increases, the output dimension of the Actor generally maintains an approximately linear growth, rather than an exponential growth. This is the core reason why this invention can handle large-scale devices and multidimensional hybrid actions.

[0165] Step 2: Perform time-slot iterative sampling, decoding, and policy updates through a low-dimensional continuous virtual action space to minimize the sum of the long-term accurate semantic twin synchronization age and system energy consumption.

[0166] In some embodiments, such as Figure 3 As shown, step two includes:

[0167] The S521.Actor network is based on the state obtained from the global controller. Output total continuous virtual motion .

[0168] S522. Decode the total continuous virtual actions to obtain the real mixed actions that can be executed in the environment, including: restoring the associated virtual sub-vector segments to discrete UAV selection by symbol quantization and mapping with a legal codebook; obtaining the real continuous control quantity by mapping the independent continuous control virtual sub-vector segments to a unified interval; and obtaining the real resource allocation quantity by normalizing the shared resource contention virtual sub-vector segments through Softmax.

[0169] S523. The environment performs a state transition based on the actual mixed actions, and obtains the next state. And calculate instant rewards ;Will Store the trajectory in the current round's cache.

[0170] S524. Determine the time slot Has the single-round time slot limit been reached? If so, proceed to step S525; otherwise, let... Then proceed with step S521.

[0171] S525. Calculate the temporal difference error and generalized dominance estimation based on the current round trajectory cache.

[0172] Specifically, the objective of constraint-aware virtual action compression proximal policy optimization is to maximize long-run discount rewards:

[0173] (42)

[0174] In the formula, Indicates a long-term discount return. Representation strategy Expected trajectory This represents the discount factor. The objective is that while the Actor network only outputs the current action in each time slot, the training objective focuses on the long-term impact of that action on the future synchronization state of the digital twin.

[0175] To achieve this goal, the timing difference error is first calculated. :

[0176] (43)

[0177] In the formula, This indicates a state value estimate.

[0178] Then, the generalized advantage estimation (GAE) is used:

[0179] (44)

[0180] In the formula, This represents the generalized advantage estimation. This represents the GAE smoothing parameter, and l represents l time slots.

[0181] By leveraging discount rewards and GAE, constrained-aware virtual action compression proximal policy optimization can learn the long-term impact of current resource actions on future digital twin synchronization quality. That is, even if the current action only applies to the current time slot, its merits or demerits will be reflected in the advantage estimation through future cumulative rewards, thereby guiding the policy network to update in a direction that is beneficial to long-term synchronization quality.

[0182] S526. Calculate the total continuous virtual actions of the old and new policy networks. The probability ratio.

[0183] Specifically, probability ratio for:

[0184] (45)

[0185] In the formula, This represents a new strategy with network parameters θ. The network parameter is θ old The old strategy.

[0186] S527. Construct constraint-aware virtual action compression proximal policy optimization pruning objective, value network loss, virtual action policy entropy, and total training objective.

[0187] Specifically, the constraint-aware virtual action compression proximal strategy optimizes the pruning target. for:

[0188] (46)

[0189] In the formula, This indicates the time slot obtained from sampling. Calculate the mathematical expectation of the trajectory samples. This represents the pruning function, whose purpose is to prevent excessively large step sizes in a single policy update, thus ensuring training stability. The pruning threshold is a fixed hyperparameter used to limit the difference between the old and new policies, preventing overly aggressive policy updates from causing training failure.

[0190] Value network loss for:

[0191]

[0192] In the formula, This represents the value of the target state.

[0193] To maintain the strategy exploration capability, virtual action strategy entropy is introduced. for:

[0194]

[0195] Overall training objective :

[0196]

[0197] In the formula, This represents the value loss weight coefficient, used to balance the update magnitude of policy loss and value loss, controlling the training step size of the value network, and is an adjustable hyperparameter. This represents the entropy regularization weight coefficient, used to control the exploration intensity of the strategy. The larger the value, the stronger the strategy's exploration; the smaller the value, the more the strategy tends to utilize existing experience. It is an adjustable hyperparameter.

[0198] S528. Divide the current round trajectory cache into several small batches and update the policy network parameters and value network parameters in multiple epochs; after the update is complete, copy the current policy network parameters to the old policy network parameters;

[0199] S529. Determine if the current round has reached the training round limit. If yes, training is complete, and the trained policy network is deployed to the global controller; otherwise, clear the trajectory cache for the current round and increment the round index by 1. Reset to 0 and proceed to step S521.

[0200] It is important to note that in the constraint-aware virtual action compression proximal strategy optimization algorithm used in this invention, the constraint specifically refers to the boundary restrictions of the continuous actions mapped by the algorithm. That is, the continuous actions output in steps 512 and 513 must not exceed their physical upper and lower bounds, such as the upper and lower limits of UAV computing resources. Ordinary PPO algorithms often violate these constraints in the initial stage and need to be gradually corrected through continuous iteration. The core advantage of the constraint-aware mechanism is that it directly ensures that the output always meets the constraint conditions through the action mapping process, thereby ensuring that the algorithm does not violate the constraints throughout the entire training and execution process.

[0201] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for synchronizing digital twin semantic information in a space-air-ground collaborative scenario, characterized in that, Includes the following steps: S1. Establish a three-tiered air-space-ground digital twin network system consisting of ground mobile devices, drones, and satellites, wherein: The system includes A ground mobile device, A drone and a satellite are deployed, along with a collection of mobile devices. drone collection They are respectively: , The system is based on a length of The operation is based on discrete time slots as the basic unit, and the time slot set is defined as follows. , Indicates the number of time slots; each ground mobile device corresponds to a unique digital twin instance, and all digital twin instances are deployed in the satellite's digital twin resource pool; multiple drones are divided into several drone clusters according to the geographic service area or mission area, and each drone cluster is equipped with an inter-domain controller; a global controller is set up on the satellite to centrally manage the inter-domain controllers of all drone clusters; S2. Construct various computational models for a three-tiered space-air-ground digital twin network system, including: The model includes: local semantic coding model for the device, transmission model from the device to the drone, supplementary semantic coding model for the drone, semantic transmission model from the drone to the satellite, satellite decoding and digital twin state reconstruction model, system energy consumption model, and accurate semantic twin synchronization age model. S3. Based on a three-level space-air-ground digital twin network system, a joint optimization problem model is constructed. The optimization objective is to minimize the long-term cumulative expectation of the sum of the accurate semantic twin synchronization age of all time slots and the system energy consumption, under the condition that the total number of time slots tends to infinity. S4. Model the joint optimization problem as a Markov decision process with a discrete-continuous mixed action space; S5. The constrained virtual action compression proximal policy optimization method is used to solve the Markov decision process, and output the discrete device association policy and the continuous semantic, communication and computing resource allocation policy.

2. The digital twin semantic information synchronization method for a space-air-ground collaborative scenario according to claim 1, characterized in that, In a three-tiered space-air-ground digital twin network system, the first... The ground mobile device in the first A digital twin state synchronization task in one time slot. Represented as: In the formula, Indicates the first The ground mobile device in the first The amount of raw state data for each time slot; Indicates the first The ground mobile device in the first The number of CPU cycles required to process each bit of data in each time slot; Indicates the first The number of CPU cycles required to complete the supplementary semantic encoding for each bit of the drone corresponding to each ground mobile device; Indicates the satellite's position on the first The number of CPU cycles required per bit when performing semantic data decoding and digital twin reconstruction on a ground mobile device; Indicates the first The maximum end-to-end state synchronization delay allowed for a single ground mobile device; Indicates the first The upper limit of the reconstruction error allowed by the digital twin state of a ground mobile device.

3. The digital twin semantic information synchronization method for a space-air-ground collaborative scenario according to claim 1, characterized in that, The device's local semantic coding model is defined as follows: ， ， ， ， ， ， In the formula, This indicates that the k-th ground mobile device is in the... The amount of raw state data for each time slot; Indicates the first The ground mobile device in the first The amount of raw state data processed locally per time slot, Indicates the first The ground mobile device in the first The amount of remaining raw state data in each time slot, Indicates the proportion of local semantic processing; Indicates the first The ground mobile device in the first The amount of local semantic data per time slot; Indicates the local semantic preservation coefficient. Indicates the first The ground mobile device in the first Data transmitted via the ground-to-air link in each time slot. Indicates the first The ground mobile device in the first The number of CPU cycles required to process each bit of data in each time slot; Indicates the first The ground mobile device in the first The upper limit of the device's local calculation frequency per time slot; Indicates the first The ground mobile device in the first Local encoding delay for each time slot; Indicates the device's local calculation frequency; The device-to-drone transmission model is defined as follows: ， ， In the formula, Indicates the first The first time slot, the first Ground mobile devices and drones The transmission rate between them; Indicates the ground-to-air bandwidth; Indicates the first The transmission power of a ground mobile device; Indicates the first Ground mobile devices and drones Channel power gain between; Indicates the first Ground mobile devices and drones The time required for data transmission between them; Represents a binary variable, when the first... The ground mobile device in the first Select drone for each time slot When undertaking the semantic encoding of the remaining original state data =1, otherwise =0; This represents the power spectral density of the equivalent Gaussian white noise; The supplementary semantic coding model for drones is defined as follows: ， ， ， In the formula, Indicates the first Each ground mobile device corresponds to a drone. Local encoding latency; Indicates the first The number of CPU cycles required to complete the supplementary semantic encoding for each bit of the drone corresponding to each ground mobile device; Indicates drone For the first The computing frequency allocated to each ground mobile device; Indicates the first Each ground mobile device corresponds to a drone. Supplemental semantic data volume; Indicates the first A ground mobile device via drone The amount of complete semantic data sent to the satellite; Indicates the semantic preservation coefficient of the drone; The semantic transmission model from UAV to satellite is defined as follows: ， ， In the formula, Indicates the first The first time slot, the first Each ground mobile device corresponds to a drone. Transmission rate with satellite; Indicates drone Bandwidth allocated for space-air links; Indicates drone The transmission power; Indicates drone Channel gain with satellites; Indicates drone The time required for data transmission between satellites; Indicates drone Distance to satellites Indicates the speed of electromagnetic wave propagation; The satellite decoding and digital twin state reconstruction model is defined as follows: ， In the formula, Indicates the satellite's position on the first Processing latency of semantic data from a ground mobile device Indicates the satellite is assigned to the first The computation frequency of a digital twin instance corresponding to a ground mobile device; Indicates the satellite's position on the first The number of CPU cycles required per bit when performing semantic data decoding and digital twin reconstruction on a ground mobile device; The system energy consumption model is defined as follows: ， ， ， ， ， ， In the formula, Indicates the first Total energy consumption of a ground mobile device in the t-th time slot; Indicates the first The ground mobile device in the first Energy consumption for local semantic computation per time slot; Indicates the first The ground mobile device in the first Energy consumption for ground-to-air transmission per time slot; Indicates the first The first time slot, the first Each ground mobile device corresponds to a drone. The computational energy consumption of semantic encoding; Indicates the first Each ground mobile device corresponds to a drone. Energy consumption for semantic transmission to satellites; Indicates the satellite's position on the first Energy consumption for processing semantic data of a ground mobile device; , , These represent the equivalent dynamic energy consumption coefficients of ground mobile devices, drones, and satellites, respectively. The accurate semantic twin synchronized age model is defined as follows: ， ， ， ， ， ， In the formula, Indicates the first The ground mobile device in the first The accurate semantic twin synchronization age of each time slot; Indicates the first The ground mobile device in the first Age information of the digital twin updated in each time slot; Indicates the first The ground mobile device in the first The degree of semantic mismatch between digital twins in each time slot; Indicates the update time factor. 1{·} denotes an indicator function; Indicates the state-validation gating factor. Indicates the first The ground mobile device in the first The latency of a single digital twin state update in each time slot. Indicates the first The maximum end-to-end state synchronization delay allowed for a single ground mobile device; ; This indicates the permissible age threshold for digital twin information. Indicates the upper limit of reconstruction error; Indicates the reconstruction accuracy; Represents the scaling factor. , These represent the fit indices for the semantic fidelity term and the satellite computation frequency term, respectively. Indicates semantic fidelity; This indicates the importance of local semantic data in the reconstruction of the digital twin's state. This indicates the importance of the amount of semantic data supplemented by drones for the reconstruction of the state of the digital twin.

4. The digital twin semantic information synchronization method for a space-air-ground collaborative scenario according to claim 1, characterized in that, The joint optimization problem model is defined as follows: ， ， ， ， ， ， ， ， In the formula, Let N represent the set of optimization variables, and let N represent the time domain length for constructing the long-term average objective. , Indicates non-negative weights. Indicates the first A ground mobile device in a time slot Upload accurate semantic twin synchronized age information. Indicates the first A ground mobile device in a time slot The total energy consumption of uploading semantic information to the satellite semantic reconstruction process. Expressing expectations; Indicates the first The ground mobile device in the first The upper limit of the device's local calculation frequency per time slot; Indicates the device's local calculation frequency; Represents a binary variable; Indicates the ground-to-air bandwidth; Indicates drone For the first The computing frequency allocated to each ground mobile device; Indicates drone Bandwidth allocated for space-air links; Indicates drone Maximum total bandwidth of the air-to-ground link used for the ground mobile devices it serves. Indicates drone This can be used to supplement the upper limit of the maximum computational frequency for semantic coding tasks. Indicates drone The maximum total bandwidth available for semantic data transmission in the space-air link. Indicates the satellite is assigned to the first The computation frequency of a digital twin instance corresponding to a ground mobile device; This indicates the maximum computing power that the satellite can use for semantic decoding and digital twin reconstruction; Indicates the first The resources occupied by each digital twin instance corresponding to a ground mobile device; This indicates the capacity of the satellite's digital twin resource pool; Indicates the first The ground mobile device in the first The latency of a single digital twin state update in each time slot; This indicates the upper limit of the latency for updating the state of the digital twin.

5. A digital twin semantic information synchronization method for a space-air-ground collaborative scenario according to claim 1, characterized in that, Constructing a Markov decision process includes: Set time slot status , in, Indicates time slot The amount of raw state data for all ground mobile devices; Indicates time slot The age of all digital twin information for terrestrial mobile devices; Indicates time slot Reconstruction accuracy of all ground mobile devices; Indicates time slot The corresponding channel state matrix between ground mobile devices and drones; Indicates time slot The corresponding channel state matrix between the UAV and the satellite; Indicates time slot Available computing resources for drones corresponding to all ground mobile devices; Indicates the satellite's remaining computing resources; This indicates the association results between ground mobile devices and drones in the previous time slot; Set time slot actual actions for: ， in, Indicates time slot The proportion of local semantic processing for all ground mobile devices. Indicates time slot Local semantic preservation coefficients for all ground mobile devices Indicates time slot The semantic preservation coefficients of drones for all ground mobile devices Indicates time slot Transmission power of all ground mobile devices; Indicates time slot The corresponding drone transmission power for all ground mobile devices; Indicates time slot Ground-to-air bandwidth for all ground mobile devices; Indicates time slot The corresponding space-air bandwidth for all ground-based mobile devices; Indicates time slot The device-local calculation frequency of all ground mobile devices; Indicates time slot The drone computing frequency corresponding to all ground mobile devices; Indicates time slot Satellite calculation frequencies corresponding to all ground mobile devices; Set reward function , in, , Indicates non-negative weights. Indicates the first A ground mobile device in a time slot Upload accurate semantic twin synchronized age information. Indicates the first A ground mobile device in a time slot Total energy consumption for uploading semantic information to the satellite semantic reconstruction process.

6. The digital twin semantic information synchronization method for a space-air-ground collaborative scenario according to claim 1, characterized in that, Step S5 specifically includes: Step 1: By uniformly representing the discrete-continuous hybrid action space as a low-dimensional continuous virtual action space, the original Markov decision process is equivalently transformed into a continuous action Markov decision process. Step 2: Perform time-slot iterative sampling, decoding, and policy updates based on a low-dimensional continuous virtual action space to minimize the sum of the long-term accurate semantic twin synchronization age and system energy consumption.

7. A digital twin semantic information synchronization method for a space-air-ground collaborative scenario according to claim 6, characterized in that, The discrete-continuous hybrid action space is uniformly represented as a low-dimensional continuous virtual action space, specifically including: S511. The discrete association decision between ground mobile devices and UAVs is compressed into an association virtual sub-vector segment, which serves as the first component of a low-dimensional continuous virtual action space; wherein each ground mobile device... from Discrete selection of one UAV from a set of UAVs is represented as a length of The continuous virtual action sub-vectors of all ground mobile devices constitute an associated virtual sub-vector segment; S512. Each ground mobile device The independent continuous control actions are directly represented as unbounded continuous virtual values, forming independent continuous control virtual sub-vector segments, which serve as the second component of the low-dimensional continuous virtual action space. Each independent continuous control variable corresponds to a one-dimensional independent continuous control virtual value, and all independent continuous control virtual values ​​of all ground mobile devices constitute independent continuous control virtual sub-vector segments. The independent continuous control actions include the local semantic processing ratio, the local semantic preservation coefficient, the UAV semantic preservation coefficient, the device transmission power, the UAV transmission power, and the device local computing frequency; S513. Each ground mobile device The competitive actions for each shared resource are represented by virtual competition scores, forming a shared resource competition virtual sub-vector segment as the third component of the low-dimensional continuous virtual action space. Each shared resource competition action corresponds to a one-dimensional virtual competition score, and all virtual competition scores of all ground mobile devices constitute the shared resource competition virtual sub-vector segment. The shared resources involved in the competition include ground-to-air bandwidth, space-to-air bandwidth, UAV computing frequency, and satellite computing frequency; S514. The space-time transmission power of each UAV is formed as an unbounded continuous virtual value, constituting a virtual sub-vector segment of UAV transmission power, which serves as the fourth component of the low-dimensional continuous virtual action space; each UAV corresponds to a one-dimensional virtual value of space-time transmission power, and the virtual values ​​of the space-time transmission power of all UAVs constitute a virtual sub-vector segment of UAV transmission power. S515. Sequentially concatenate the associated virtual sub-vector segments, independent continuous control virtual sub-vector segments, shared resource contention virtual sub-vector segments, and UAV launch power virtual sub-vector segments to form a total continuous virtual action in a low-dimensional continuous virtual action space; the dimension of the total continuous virtual action... .

8. A digital twin semantic information synchronization method for a space-air-ground collaborative scenario according to claim 6, characterized in that, Step two includes: The S521.Actor network is based on the state obtained from the global controller. Output total continuous virtual motion ; S522. For total continuous virtual actions Decoding is performed to obtain the actual mixed actions that can be executed in the environment, including: restoring the discrete UAV selection by symbolically quantizing and mapping the associated virtual sub-vector segments to a legal codebook; obtaining the actual continuous control quantity by mapping the independent continuous control virtual sub-vector segments to a unified interval; and obtaining the actual resource allocation quantity by normalizing the shared resource contention virtual sub-vector segments using Softmax. S523. The environment performs a state transition based on the actual mixed actions, and obtains the next state. And calculate instant rewards ;Will Store in the current round's trajectory cache; S524. Determine the time slot Has the single-round time slot limit been reached? If so, proceed to step S525; otherwise, let... And proceed to step S521; S525. Calculate the temporal difference error and generalized dominance estimation based on the current round trajectory cache; S526. Calculate the probability ratio of the old and new policy networks for the total continuous virtual actions; S527. Based on probability ratio and generalized advantage estimation, construct constraint-aware virtual action compression proximal policy optimization pruning target, value network loss, virtual action policy entropy and total training target; S528. Divide the current round trajectory cache into several small batches and update the policy network parameters and value network parameters in multiple epochs; after the update is complete, copy the current policy network parameters to the old policy network parameters; S529. Determine if the current round has reached the training round limit. If yes, training is complete, and the trained policy network is deployed to the global controller; otherwise, clear the trajectory cache for the current round and increment the round index by 1. Reset to 0 and proceed to step S521.

Images