Digital twin driven multi-modal scenario prediction and resource pre-scheduling method

Abstract

The application discloses a kind of digital twin drives multimodal scene prediction and resource pre-scheduling method, and multimodal data is collected by integrated base station or edge control node, and is mapped as twin state tensor uniformly;When state deviation, perception uncertainty, link risk or calculation congestion is over threshold, call generative scene completion model, predict multi-period scene, scatterer, channel, blind area and resource load trend;In digital twin, beam, perception sampling, power candidate action are carried out multi-step counterfactual deduction, and pre-scheduling strategy is selected and instruction is issued in combination with throughput, beam mismatch, perception completion rate, delay, energy consumption and tail risk;Through execution feedback write-back update twin, model parameter and risk threshold value, blind area, channel degradation and power congestion can be identified in advance, reduce dynamic occlusion, high-speed moving scene task failure rate, realize the closed-loop control of sensing and calculation resources from passive response to prediction and pre-scheduling.

Description

Technical Field

[0001] This invention relates to the field of communication-sensing computing fusion technology, and in particular to a digital twin-driven method for multimodal scene prediction and resource pre-scheduling. Background Technology

[0002] With the development of 5G-A and 6G technologies, base stations are gradually evolving from single communication access nodes into integrated communication, sensing, and computing nodes that combine communication transmission, environmental perception, and edge computing. In dynamic scenarios such as low-altitude economy, vehicle-to-everything (V2X) networks, smart parks, industrial inspection, and urban governance, base stations need to simultaneously perceive the target's location and motion status, maintain reliable communication links, and provide edge computing support for artificial intelligence model inference, target recognition, and task scheduling.

[0003] Existing integrated sensing solutions typically utilize radio frequency echoes, channel state information, visual images, or point cloud data to perform target detection, angle estimation, beam prediction, or sensing-assisted communication. They primarily focus on sensing results and resource allocation within the current moment or a short time window. Existing edge computing resource scheduling solutions mostly offload or allocate computing power based on current queue length, task load, and link status. These methods can improve communication quality and task latency to some extent, but their scheduling process is often reactive, adjusting only when beam mismatch, sensing blind spots, link degradation, or computing power congestion have already occurred or are about to occur.

[0004] In situations involving low-altitude drones, high-speed vehicle movement, or frequent changes in obstruction in industrial settings, relying solely on the current observation status for resource allocation can easily lead to the following technical problems: First, when targets are obstructed by buildings, vehicles, or equipment, base stations struggle to identify potential scatterers and blind spots in advance, resulting in delayed selection of sensing scanning areas. Second, channel conditions and beam quality are highly time-varying; if future link changes cannot be predicted in advance, beam switching may be delayed and communication throughput may decrease. Third, since sensing and computing tasks share edge computing resources, adjusting computing power only after queue congestion may cause critical tasks to miss deadlines. Fourth, existing digital twin networks are mostly used for offline simulation, network planning, or macro-level situational awareness, and have not yet fully formed a closed loop for generative prediction and counterfactual inference of resource actions during base station operation.

[0005] Therefore, it is necessary to develop a digital twin-driven method for multimodal scenario prediction and resource pre-scheduling to solve the above problems. Summary of the Invention

[0006] The purpose of this invention is to design a digital twin-driven multimodal scene prediction and resource pre-scheduling method to solve the above problems.

[0007] The present invention achieves the above objectives through the following technical solutions: A digital twin-driven method for multimodal scenario prediction and resource pre-scheduling, executed by a sensory-computing integrated base station, a base station-side edge server, or a collaborative control node of a sensory-computing integrated base station and a base station-side edge server, includes the following steps: S1. Collect multimodal sensory state data generated by the communication transceiver module, the sensing acquisition module, and the edge computing module within the target service area, and form the current twin state tensor according to a unified timestamp; S2. Update the synesthetic computing digital twin based on the current twin state tensor. The synesthetic computing digital twin includes the physical space state, wireless channel state, perception credibility state, and computing resource state. S3. Determine whether the generative prediction triggering condition is met based on the state deviation between the predicted state and the measured state of the synesthetic computing digital twin, the perception uncertainty, the communication link risk, and the degree of computing resource congestion. When the generative prediction triggering condition is met, call the generative scene completion model to generate the future twin state tensor for multiple future synesthetic computing cycles. S4. Generate a resource action candidate set based on the future twin state tensor, and perform multi-step counterfactual resource deduction on the resource action candidate set in the synesthetic computing digital twin to obtain the communication benefit, perception benefit, computational benefit and risk score corresponding to each resource action candidate. S5. Select a resource pre-scheduling strategy based on communication benefits, sensing benefits, computing benefits and risk scores, and convert the resource pre-scheduling strategy into beam control commands, sensing sampling control commands and edge computing scheduling commands; S6. Send beam control commands, sensing sampling control commands, and edge computing scheduling commands to the communication transmission module, sensing acquisition module, and edge computing module, respectively; receive actual execution feedback and update the syn-sensory computing digital twin, generative scene completion model, and risk threshold based on the actual execution feedback.

[0008] Preferably, in step S1, the multimodal sensing state data includes at least two of the following: visual images, video frames, laser point clouds, millimeter-wave radar echoes, radio frequency sensing echoes, channel state information, reference signal received power, signal-to-noise ratio, positioning coordinates, motion trajectory, task type, task deadline, edge server queue length, processor or graphics processor margin, model inference depth, power consumption budget, and historical resource execution results.

[0009] Preferably, in step S1, the current twin state tensor includes a physical space tensor, a channel state tensor, a perception confidence tensor, and a computational resource vector; the physical space tensor is used to represent the target location, velocity, occlusion relationship, and regional topology; the channel state tensor is used to represent path loss, scatterer distribution, beam quality, and channel change trends; the perception confidence tensor is used to represent the perception blind zone, modal quality, target recognition confidence, and uncertainty; and the computational resource vector is used to represent the edge task queue, processor margin, power budget, and task priority.

[0010] Furthermore, in step S3, the generative prediction triggering conditions include at least one of the following: the state deviation is greater than the deviation threshold, the target speed or trajectory change is greater than the motion threshold, the perception credibility is lower than the credibility threshold, the future channel state prediction error is greater than the error threshold, the edge computing queue length is greater than the congestion threshold, the task deadline is less than the available simulation time, or the probability of synesthetic task failure caused by historical resource actions is greater than the risk threshold.

[0011] Specifically, in step S3, the generative scene completion model is a conditional diffusion model, a generative Transformer model, a spatiotemporal graph generation network, or a combination thereof. The generative scene completion model takes the current twin state tensor, the historical twin state sequence, the occlusion mask, the task constraints, and the resource constraints as conditional inputs, generates a future twin state tensor that includes the distribution of occlusion targets, the distribution of potential scatterers, the future channel state, the distribution of perception blind spots, and the trend of computational resource load, and outputs the uncertainty score corresponding to the future twin state tensor.

[0012] Preferably, in step S4, the resource action candidate set includes at least one of the following: communication beam index, precoding parameters, transmit power, time-frequency resource block, sensing scanning angle domain, sensing sampling period, point cloud or radio frequency echo processing accuracy, edge computing frequency, model inference depth, batch size, task offloading ratio, and task degradation level; the resource action candidate set is jointly constrained by the minimum communication throughput, sensing task deadline, edge computing resource limit, and power consumption budget.

[0013] Specifically, in step S4, the multi-step counterfactual resource extrapolation includes: replicating multiple future state branches in the synesthetic computing digital twin; applying different resource action candidates to each future state branch; calculating the communication throughput, beam mismatch probability, perception completion rate, perception blind zone area, calculation queuing delay, energy consumption, task success rate, and tail risk of each resource action candidate within multiple consecutive synesthetic computing cycles; determining the target resource action according to the evaluation function J(a)=w1T(a)+w2S(a)+w3C(a)-w4D(a)-w5E(a)-w6R(a), where T(a) represents communication gain, S(a) represents perception gain, C(a) represents computation gain, D(a) represents delay, E(a) represents energy consumption, R(a) represents risk, and w1 to w6 are weight parameters.

[0014] Preferably, in step S6, the actual execution feedback includes the real target trajectory, actual communication throughput, actual beam quality, actual sensing results, actual computational latency, model inference results, energy consumption, and task completion status. The integrated sensing and computing base station calibrates the physical space state, wireless channel state, sensing credibility state, and computing resource state based on the difference between the actual execution feedback and the future twin state tensor, and updates the conditional input weights of the generative scenario completion model, the evaluation function weights of the resource pre-scheduling strategy, and the risk threshold for the next sensing and computing cycle based on the difference.

[0015] The beneficial effects of this invention are as follows: First, by constructing a multi-layered digital twin that includes physical space, wireless channels, perception credibility, and computing resources, base stations can uniformly express the operational status of communication, perception, and computing. Second, by using a generative scene completion model to predict occlusion targets, potential scatterers, future channel states, and perception blind spots, resource scheduling can be transformed from responding to the current state to predicting the future state. Third, by using counterfactual resource simulation to compare the consequences of multiple resource actions before actual execution, we can reduce beam mismatch, expanded perception blind spots, and edge computing congestion caused by passive adjustments. Fourth, by converting the pre-scheduling results into beam control, sensing sampling, and edge computing scheduling commands, the digital twin prediction results can be directly applied to the base station's physical communication, sensing, and computing processes. Fifth, by continuously calibrating the twin and risk threshold through execution feedback, the robustness of synesthetic computing tasks in scenarios of low-altitude high-speed movement, dynamic occlusion, and long-tail latency is improved. Attached Figure Description

[0016] Figure 1 The flowchart of the pre-scheduling process for synesthetic computing resources driven by digital twin provided by this invention; Figure 2This is a layered architecture diagram of the integrated sensing and computing base station digital twin provided by the present invention; Figure 3 This is a diagram of the generative scene completion and counterfactual resource deduction module provided by the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0018] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0019] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0020] In the description of this invention, it should be understood that the terms "upper," "lower," "inner," "outer," "left," "right," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are only used to facilitate the description of this invention and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0021] Furthermore, the terms "first," "second," etc., are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0022] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, terms such as "set" and "connection" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0023] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0024] like Figure 1 As shown, the digital twin-driven multimodal scene prediction and resource pre-scheduling method of the present invention includes six stages: multimodal state acquisition, construction of a synesthetic digital twin, generative scene completion prediction, multi-step counterfactual resource inference, output of resource pre-scheduling strategy, and execution feedback and model update.

[0025] Figure 1 The predictive closed loop is represented by a vertical process, in which the actual execution feedback returns from step S6 to step S1, and the prediction error feedback returns from the generative scenario completion prediction stage to the digital twin update stage.

[0026] The six stages are as follows: S1. Multimodal State Acquisition: The integrated sensing and computing base station acquires at least two types of multimodal sensing and computing state data from the communication transceiver module, sensing acquisition module, and edge computing module, including visual images, video frames, laser point clouds, millimeter-wave radar echoes, radio frequency sensing echoes, channel state information, reference signal received power, signal-to-noise ratio, positioning coordinates, motion trajectory, task type, task deadline, edge server queue length, processor or graphics processor margin, model inference depth, power budget, and historical resource execution results, and forms the current twin state tensor according to a unified timestamp.

[0027] In step S1, the integrated sensing and computing base station can acquire radio frequency echoes and channel state information through its own radio frequency transceiver array; it can acquire visual images, point clouds, millimeter-wave radar echoes, radio frequency sensing echoes, and target motion states through a sensing acquisition module, which includes co-located or non-co-located cameras, lidar, millimeter-wave radar, radio frequency sensing units, or other sensing nodes; and it can acquire task queues, inference model occupancy, processor or graphics processor margin, and power consumption budget through the edge server's operating status. All of the above data is aligned under a unified timestamp and forms the current twin state tensor X_t.

[0028] The current twin state tensor includes a physical space tensor, a channel state tensor, a perception confidence tensor, and a computational resource vector. The physical space tensor represents the target location, velocity, occlusion relationship, and regional topology. The channel state tensor represents path loss, scatterer distribution, beam quality, and channel variation trends. The perception confidence tensor represents the perception blind zone, modal quality, target recognition confidence, and uncertainty. The computational resource vector represents the edge task queue, processor margin, power budget, and task priority.

[0029] S2. Construction of the Synesthesia Computing Digital Twin: The base station-side edge control node maps the current twin state tensor into physical space state, wireless channel state, perception credibility state, and computing resource state, forming a Synesthesia Computing digital twin synchronized with the actual base station operation state.

[0030] In step S2, the current twin state tensor X_t can be represented as X_t={P_t,H_t,M_t,Q_t}, where P_t is the physical space tensor, H_t is the wireless channel state tensor, M_t is the perception confidence tensor, and Q_t is the computational resource vector. P_t describes the target, obstacles, trajectory, and region topology; H_t describes path loss, scatterers, beam quality, and channel variation trends; M_t describes the perception blind zone, modal quality, uncertainty, and target recognition confidence; and Q_t describes the edge task queue, model inference depth, processor or graphics processor margin, power budget, and task priority.

[0031] S3. Generative Scene Completion Prediction: Based on the state deviation D_t between the predicted state and the measured state of the synesthetic computing digital twin, the perception uncertainty U_t, the communication link risk L_t, and the computational congestion level Q_t^c, it is determined whether the generative prediction trigger condition is met. When the generative prediction trigger condition is met, the generative scene completion model is invoked to generate future twin state tensors for multiple synesthetic computing cycles. When any of the indicators D_t, U_t, L_t, or Q_t^c exceeds the corresponding threshold, the base station invokes the generative scene completion model. The generative scene completion model takes the current twin state tensor, historical twin state sequence, occlusion mask, task constraints, and resource constraints as inputs, and outputs the future twin state tensors for the next K synesthetic computing cycles and their uncertainty scores.

[0032] In step S3, the generative prediction triggering conditions include at least one of the following: the state deviation is greater than the deviation threshold, the target speed or trajectory change is greater than the motion threshold, the perception credibility is lower than the credibility threshold, the future channel state prediction error is greater than the error threshold, the edge computing queue length is greater than the congestion threshold, the task deadline is less than the available simulation time, or the probability of the synesthetic task failure caused by historical resource actions is greater than the risk threshold.

[0033] In step S3, the generative scene completion model is a conditional diffusion model, a generative Transformer model, a spatiotemporal graph generation network, or a combination thereof. The generative scene completion model takes the current twin state tensor, the historical twin state sequence, the occlusion mask, the task constraints, and the resource constraints as conditional inputs to generate a future twin state tensor that includes the distribution of occlusion targets, the distribution of potential scatterers, the future channel state, the distribution of perception blind spots, and the trend of computational resource load. It also outputs the future twin state tensor and its uncertainty score for the next K synesthetic computing cycles.

[0034] S4. Multi-step counterfactual resource extrapolation: Multiple future state branches are replicated in the synesthetic computing digital twin, and candidate beams, sensory sampling, and computational resource actions are applied respectively. The communication benefits, sensory benefits, computational benefits, and risk scores of each action are evaluated over multiple synesthetic computing cycles.

[0035] In step S4, the counterfactual resource inferrer constructs multiple candidate state branches within the digital twin. For example, one branch employs early beam pre-switching, another increases the sensing sampling frequency, another increases the edge computing quota, and another reduces the inference depth of the non-critical task model. The counterfactual resource inferrer calculates the communication throughput, beam mismatch probability, sensing completion rate, sensing blind zone area, computing queuing delay, energy consumption, task success rate, and tail risk for each branch over multiple consecutive sensing computing cycles. The tail risk includes at least one of tail delay, task failure probability, and high-priority task default probability, and participates in the evaluation function calculation as a component of the risk score R(a).

[0036] In step S4, the evaluation function of resource action candidate a can be expressed as J(a) = w1T(a) + w2S(a) + w3C(a) - w4D(a) - w5E(a) - w6R(a), where T(a) represents communication throughput or link uptime gain, S(a) represents perception completion gain, C(a) represents computation completion gain, D(a) represents latency, E(a) represents energy consumption, and R(a) represents risk score. The risk score is determined by at least one of tail latency, task failure probability, and high-priority task default probability, and w1 to w6 are weight parameters set according to the task type. The base station selects the target action that satisfies the minimum throughput, perception deadline, computational resource limit, power budget, and risk threshold.

[0037] S5. Resource pre-scheduling strategy output: Select a resource pre-scheduling strategy that meets the constraints of communication, perception, computing and risk based on the counterfactual inference results, and output at least one of the following: beam pre-switching, perception scanning area, sampling period, edge computing quota and task degradation strategy.

[0038] In step S5, the resource pre-scheduling strategy prioritizes actions that are high-risk in the future but still resource-feasible. When it is predicted that a low-altitude target will enter the obstruction area in the next cycle, the base station expands the sensing scanning angle domain in advance or switches to a more robust beam; when it is predicted that the edge queue will be congested in the next cycle, the base station reduces the model depth of non-critical tasks in advance or reserves computing quotas for high-priority tasks; when it is predicted that the sensing blind zone will expand, the base station increases the sampling frequency of that area in advance or shortens the sampling period, and requests neighboring base stations to cooperate in sensing.

[0039] S6. Execution Feedback and Model Update: The base station converts the resource pre-scheduling strategy into beam control commands, perception sampling control commands, and edge computing scheduling commands, and sends them to the corresponding hardware or software modules. It also writes the actual execution feedback back to the digital twin to calibrate the twin state, generative model condition weights, evaluation function weights, and risk thresholds.

[0040] In step S6, the actual execution feedback includes the real target trajectory, actual communication throughput, actual beam quality, actual sensing results, actual computational latency, model inference results, energy consumption, and task completion status. The base station corrects the digital twin parameters based on the difference between the actual feedback and the prediction results, and uses the prediction error as the basis for updating the generative scene completion model, evaluation function weights, and risk thresholds.

[0041] In this embodiment, the integrated sensing and computing base station is deployed in scenarios such as low-altitude UAV monitoring, vehicle-to-everything (V2X) communication, or smart park applications. The base station acquires the multimodal status of the target area through communication arrays, radio frequency sensing modules, visual or point cloud sensing nodes, and edge computing servers. For low-altitude UAVs, visual or radar modes provide the target's appearance and spatial location, radio frequency echo and channel state information provide the propagation path and beam quality, the positioning trajectory provides the motion trend, and the task queue and computing load provide the edge computing status.

[0042] like Figure 2 As shown, the edge control node on the base station side constructs a digital twin of the sensing computation. This digital twin operates in a periodic update manner, mapping the real physical space, wireless channel, sensing credibility, and computing resource status to the twin space in each period. To improve runtime availability, the digital twin can be jointly represented by a lightweight state diagram, grid map, channel heatmap, resource queue list, and task priority table, without requiring the construction of a complete 3D model.

[0043] When the target's movement speed increases, observation modes are missing, channel prediction errors increase, or edge computing queues become congested, the base station triggers generative scene completion prediction. The generative scene completion model can employ a conditional diffusion model, a generative Transformer, or a spatiotemporal graph generation network. The model's conditional inputs include the current visible target state, partial point cloud or radar observations, current CSI segments, resource load snapshots, historical twin states, and task constraints. The model outputs include future scene states, occluded target predictions, potential scatterer predictions, future CSI predictions, and perception blind zone predictions.

[0044] like Figure 3As shown, the output of the generative scene completion model enters the counterfactual resource inference engine. Instead of directly executing all candidate actions on the real base station, the counterfactual resource inference engine conducts virtual experiments on different actions within a digital twin. For example, it infers actions such as switching to a narrow beam in advance, maintaining the current beam but increasing transmit power, expanding the sensing scanning angle domain, increasing the sensing sampling frequency, increasing edge computing quotas, and reducing the accuracy of non-critical tasks for the same future scenario.

[0045] The counterfactual resource inference engine calculates multidimensional evaluation metrics for each candidate action. Communication evaluation metrics include throughput, link interruption probability, and beam mismatch probability; perception evaluation metrics include target detection confidence, perception blind zone area, and perception completion rate; computation evaluation metrics include queuing latency, inference latency, energy consumption, and resource utilization; and risk evaluation metrics include tail latency, task failure probability, and high-priority task default probability.

[0046] Based on the above evaluation results, the base station generates a resource pre-scheduling strategy. In one example, when the digital twin predicts that a drone will enter the edge of a building obstruction in the next cycle and the current beam quality degrades, the base station performs beam pre-switching in advance and expands the sensing scanning area. In another example, when the digital twin predicts that the edge server will experience queuing congestion in the next cycle, the base station reserves computing quotas for high-priority sensing tasks in advance and switches non-critical environment identification tasks to low-depth inference. In yet another example, when the generative scene completion model predicts a sensing blind spot in a certain area, the base station increases the sampling frequency of that area in advance or requests neighboring base stations to cooperate in filling the blind spot.

[0047] After the resource pre-scheduling strategy is executed, the base station writes back the actual communication, sensing, and computing feedback to the digital twin. If there is a deviation between the actual target trajectory and the predicted trajectory, the physical space state is calibrated; if there is a deviation between the actual beam quality and the predicted channel state, the wireless channel state is calibrated; if the actual detection confidence is lower than the predicted value, the sensing confidence state is calibrated; if the actual queuing delay is higher than the predicted value, the computing resource state is calibrated. Through the above feedback calibration, the system can develop adaptive prediction capabilities for specific scenarios during long-term operation.

[0048] Through this embodiment, the present invention enables the integrated sensing and computing base station to predict future states and extrapolate the consequences of actions before actual resource actions are executed, thereby avoiding passive responses based solely on the current observation state. This method is particularly suitable for scenarios involving low-altitude high-speed movement, frequent traffic obstruction, dynamic operation of industrial equipment, and multi-task sharing of edge computing power.

[0049] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for digital twin driven multi-modal scenario prediction and resource pre-scheduling, characterized in that, The method is executed by an integrated sensing and computing base station, a base station-side edge server, or a collaborative control node of the integrated sensing and computing base station and the base station-side edge server, and includes the following steps: S1. Collect multimodal sensory state data generated by the communication transceiver module, the sensing acquisition module, and the edge computing module within the target service area, and form the current twin state tensor according to a unified timestamp; S2. Update the synesthetic computing digital twin based on the current twin state tensor. The synesthetic computing digital twin includes the physical space state, wireless channel state, perception credibility state, and computing resource state. S3. Determine whether the generative prediction triggering condition is met based on the state deviation between the predicted state and the measured state of the synesthetic computing digital twin, the perception uncertainty, the communication link risk, and the degree of computing resource congestion. When the generative prediction triggering condition is met, call the generative scene completion model to generate the future twin state tensor for multiple future synesthetic computing cycles. S4. Generate a resource action candidate set based on the future twin state tensor, and perform multi-step counterfactual resource deduction on the resource action candidate set in the synesthetic computing digital twin to obtain the communication benefit, perception benefit, computational benefit and risk score corresponding to each resource action candidate. S5. Select a resource pre-scheduling strategy based on communication benefits, sensing benefits, computing benefits and risk scores, and convert the resource pre-scheduling strategy into beam control commands, sensing sampling control commands and edge computing scheduling commands; S6. Send beam control commands, sensing sampling control commands, and edge computing scheduling commands to the communication transmission module, sensing acquisition module, and edge computing module, respectively; receive actual execution feedback and update the syn-sensory computing digital twin, generative scene completion model, and risk threshold based on the actual execution feedback.

2. The digital twin driven multi-modal scenario prediction and resource pre-scheduling method of claim 1, wherein, In step S1, the multimodal sensing state data includes at least two of the following: visual images, video frames, laser point clouds, millimeter-wave radar echoes, radio frequency sensing echoes, channel state information, reference signal received power, signal-to-noise ratio, positioning coordinates, motion trajectory, task type, task deadline, edge server queue length, processor or graphics processor margin, model inference depth, power budget, and historical resource execution results.

3. The digital twin-driven multimodal scene prediction and resource pre-scheduling method according to claim 2, characterized in that, In step S1, the current twin state tensor includes a physical space tensor, a channel state tensor, a perception confidence tensor, and a computational resource vector. The physical space tensor is used to represent the target location, velocity, occlusion relationship, and regional topology. The channel state tensor is used to represent path loss, scatterer distribution, beam quality, and channel variation trends. The perception confidence tensor is used to represent the perception blind zone, modal quality, target recognition confidence, and uncertainty. The computational resource vector is used to represent the edge task queue, processor margin, power budget, and task priority.

4. The digital twin-driven multimodal scene prediction and resource pre-scheduling method according to claim 1, characterized in that, In step S3, the generative prediction triggering conditions include at least one of the following: the state deviation is greater than the deviation threshold, the target speed or trajectory change is greater than the motion threshold, the perception credibility is lower than the credibility threshold, the future channel state prediction error is greater than the error threshold, the edge computing queue length is greater than the congestion threshold, the task deadline is less than the available simulation time, or the probability of the synesthetic task failure caused by historical resource actions is greater than the risk threshold.

5. The digital twin-driven multimodal scene prediction and resource pre-scheduling method according to claim 4, characterized in that, In step S3, the generative scene completion model is a conditional diffusion model, a generative Transformer model, a spatiotemporal graph generation network, or a combination thereof. The generative scene completion model takes the current twin state tensor, the historical twin state sequence, the occlusion mask, the task constraints, and the resource constraints as conditional inputs, generates a future twin state tensor that includes the distribution of occlusion targets, the distribution of potential scatterers, the future channel state, the distribution of perception blind spots, and the trend of computational resource load, and outputs the uncertainty score corresponding to the future twin state tensor.

6. The digital twin-driven multimodal scene prediction and resource pre-scheduling method according to claim 1, characterized in that, In step S4, the resource action candidate set includes at least one of the following: communication beam index, precoding parameters, transmit power, time-frequency resource block, sensing scanning angle domain, sensing sampling period, point cloud or radio frequency echo processing accuracy, edge computing frequency, model inference depth, batch size, task offloading ratio, and task degradation level; the resource action candidate set is jointly constrained by the minimum communication throughput, sensing task deadline, edge computing resource limit, and power consumption budget.

7. The digital twin-driven multimodal scene prediction and resource pre-scheduling method according to claim 6, characterized in that, In step S4, the multi-step counterfactual resource extrapolation includes: replicating multiple future state branches in the synesthetic computing digital twin; applying different resource action candidates to each future state branch; calculating the communication throughput, beam mismatch probability, perception completion rate, perception blind zone area, computation queuing delay, energy consumption, task success rate, and tail risk of each resource action candidate within multiple consecutive synesthetic computing cycles; determining the target resource action according to the evaluation function J(a)=w1T(a)+w2S(a)+w3C(a)-w4D(a)-w5E(a)-w6R(a), where T(a) represents communication gain, S(a) represents perception gain, C(a) represents computation gain, D(a) represents delay, E(a) represents energy consumption, R(a) represents risk, and w1 to w6 are weight parameters.

8. The digital twin-driven multimodal scene prediction and resource pre-scheduling method according to claim 1, characterized in that, In step S6, the actual execution feedback includes the real target trajectory, actual communication throughput, actual beam quality, actual sensing results, actual computational latency, model inference results, energy consumption, and task completion status. The integrated sensing and computing base station calibrates the physical space state, wireless channel state, perception credibility state, and computing resource state based on the difference between the actual execution feedback and the future twin state tensor. It also updates the conditional input weights of the generative scenario completion model, the evaluation function weights of the resource pre-scheduling strategy, and the risk threshold for the next sensing and computing cycle based on the difference.

Images