A virtual engine driven application management system and method

Abstract

The application provides a virtual engine driven application management system and method, and belongs to the technical field of virtual engines. The system comprises a space-time virtualization engine, a semantic rendering decision maker, a multi-modal signal processor, a GPU rendering unit, an intelligent interaction predictor and a metadata hub. Through multi-dimensional technical innovation, the application systematically solves the core problems of existing virtual engines in resource scheduling, rendering efficiency, audio and video processing and interaction response, significantly improves the intelligent level of application management and user experience, and has significant industrial application value.

Description

Technical Field

[0001] This invention relates to the field of virtual engine technology, and in particular to a virtual engine-driven application management system and method. Background Technology

[0002] With the development of mobile internet and cloud computing technologies, virtual engines are increasingly being used in games, industrial simulations, and metaverse applications. However, existing virtual engine-driven application management systems generally suffer from the following technical bottlenecks:

[0003] Resource scheduling is crude: Traditional two-dimensional sandbox environments (such as Android's native VirtualApp) only support spatial isolation and cannot achieve fine-grained resource allocation in the time dimension, resulting in uneven CPU / memory utilization and easy stuttering in high-load scenarios (such as frame rate fluctuations exceeding 20% ​​when switching between multiple tasks).

[0004] Imbalance between rendering quality and performance: Low-end devices struggle to support high-resolution rendering, while high-end devices waste resources in simple scenes. Traditional LOD technology switches between levels of detail based solely on geometric distance, without incorporating semantic information (such as object importance), resulting in no difference in rendering priority between key elements and background objects, impacting user experience.

[0005] Insufficient precision in multimodal signal processing: Audio and video synchronization relies on a fixed PTS (Presentation Time Stamp) mechanism, with synchronization errors typically exceeding 25ms, easily leading to audio-visual misalignment in fast-moving scenes. Traditional audio noise reduction algorithms (such as Wiener filtering) perform poorly in handling non-steady-state noise (such as sudden human voices), with signal-to-noise ratio improvements of less than 10dB.

[0006] Delayed interaction response: User operation hotspot prediction relies on heuristic rules (such as a fixed area at the bottom of the screen) without combining historical behavior data, resulting in interaction latency generally exceeding 100ms, which is difficult to meet real-time interaction requirements (such as aiming operations in FPS games). Summary of the Invention

[0007] This invention provides a virtual engine-driven application management system and method. Through multi-dimensional technological innovation, it systematically solves the core problems of existing virtual engines in resource scheduling, rendering efficiency, audio and video processing, and interactive response, significantly improving the intelligence level of application management and user experience, and has significant industrial application value.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A virtual engine-driven application management system includes a spatiotemporal virtualization engine, a semantic rendering decision-maker, a multimodal signal processor, a GPU rendering unit, an intelligent interaction predictor, and a metadata hub.

[0010] The spatiotemporal virtualization engine is used to create a four-dimensional isolated environment containing a three-dimensional spatial grid and time slices, generate spatiotemporal units and allocate initial resource allocation tables, and output them to the metadata central storage; at the same time, it receives the optimized resource allocation table from the metadata central and dynamically adjusts the spatiotemporal unit resources.

[0011] The semantic rendering decision-maker connects to the spatiotemporal virtualization engine, receives the spatiotemporal environment handle from the spatiotemporal virtualization engine and the initial resource allocation table from the metadata hub, performs semantic segmentation and instance detection on the scene, constructs an octree spatial index, and generates a list of visible objects by combining the view frustum culling algorithm. The list of visible objects contains semantic labels and spatial coordinates, and is output to the metadata hub and GPU rendering unit. Based on the semantic labels of the visible objects and their distance from the viewpoint, a LOD level configuration table is generated and input into the GPU rendering unit.

[0012] A multimodal signal processor, connected to the GPU rendering unit and the spatiotemporal virtualization engine, acquires the low-resolution video stream from the rendering output and the original audio stream within the four-dimensional isolated environment. It calculates the time offset and synchronizes the audio and video through a cross-correlation algorithm. It extracts video semantic features, audio Mel spectrum, and motion vectors, and fuses them into a multimodal feature vector, which is then input into the metadata center. It receives dynamic bitrate control parameters generated by the metadata center based on the initial resource allocation table and scene complexity, and adjusts the H.265 encoding bitrate accordingly.

[0013] The intelligent interaction predictor connects to the spatiotemporal virtualization engine and the metadata hub, collects user interaction data, generates interaction feature vectors and inputs them into the metadata hub; it receives predictive interaction instructions generated by the metadata hub based on the initial resource allocation table and user profile, and triggers the spatiotemporal virtualization engine to increase resource quotas for spatiotemporal units in high-frequency interaction areas.

[0014] The metadata hub is bidirectionally connected to the spatiotemporal virtualization engine, semantic rendering decision-maker, multimodal signal processor, and intelligent interaction predictor. It stores the initial resource allocation table, model parameters, and user profile data. It receives the visible object list from the semantic rendering decision-maker, the multimodal feature vectors from the multimodal signal processor, and the interaction feature vectors from the intelligent interaction predictor. It combines these with the initial resource allocation table to train and update the scene complexity model and behavior prediction model. It outputs an optimized resource allocation table based on user behavior to the spatiotemporal virtualization engine, provides dynamic bitrate control parameters to the multimodal signal processor, and outputs predicted interaction instructions to the intelligent interaction predictor.

[0015] In this specification, the metadata hub calculates the importance weight of objects in each spatiotemporal unit based on the initial resource allocation table and the visible object list of the semantic rendering decision-maker. Spatiotemporal units containing highly important objects are marked as high priority, and their CPU cycle quota is increased by 20%-40% in the optimized resource allocation table.

[0016] In this specification, when the multimodal signal processor receives the dynamic bit rate control parameters from the metadata hub, if the memory bandwidth quota of the corresponding spatiotemporal unit in the initial resource allocation table is lower than the threshold, it will automatically reduce the encoding resolution to 720p to match the resource limit.

[0017] In this specification, the intelligent interaction predictor queries the initial resource allocation table through the metadata hub to obtain the resource usage history data of the spatiotemporal unit corresponding to the current interaction hot zone. If the utilization rate of three consecutive time slices exceeds 85%, the spatiotemporal virtualization engine is triggered to reclaim resources from low-priority units and reallocate them.

[0018] In this specification, the spatiotemporal virtualization engine requests an initial resource allocation table template from the metadata hub during the system initialization phase, generates an instantiated initial resource allocation table based on device hardware parameters and application type, and stores it in the non-volatile storage area of ​​the metadata hub.

[0019] A virtual engine-driven application management method, comprising:

[0020] S1: Spatiotemporal environment modeling and resource pre-allocation, creating a four-dimensional isolated environment containing a three-dimensional spatial grid and time slices, generating a spatiotemporal environment handle and an initial resource allocation table; the spatiotemporal environment handle is used to identify the four-dimensional isolated environment, and the initial resource allocation table contains the CPU cycle and memory bandwidth allocation parameters for each spatiotemporal unit;

[0021] S2: Scene semantic parsing and dynamic occlusion removal. The Mask R-CNN model is used to perform semantic segmentation and instance detection on the scene in the four-dimensional isolated environment generated by S1. Based on the spatiotemporal environment handle of S1, the spatial coordinate reference is obtained, an octree index is constructed and invisible objects outside the view frustum are removed, and a list of visible objects is generated. The list of visible objects contains semantic labels and spatial coordinates.

[0022] S3: Content-aware super-resolution and dynamic LOD rendering. Based on the semantic labels of objects in the visible object list of S2 and their distance from the viewpoint, the LOD level is dynamically selected. The ESRGAN model is used to generate a super-resolution image from the low-resolution image processed by S2, and the rendering result is obtained.

[0023] S4: Spatiotemporal synchronization and feature extraction of audio and video signals. The video stream in the rendering result of S3 and the original audio stream in the four-dimensional isolation environment are synchronized through cross-correlation algorithm. The time offset is calculated and the synchronized audio and video stream is generated. Video semantic features, audio Mel spectrum and motion vector are extracted and fused into multimodal feature vectors, which are then input into the metadata hub.

[0024] S5: Dynamic bitrate control and intelligent noise reduction. The ResNet-50 model is used to calculate the scene complexity score of the multimodal feature vector of S4 and dynamically adjust the H.265 encoding bitrate. The WaveNet model is used to perform noise reduction on the synchronous audio stream of S4. Combined with the semantic tags in the visible object list of S2, the encoded audio and video stream is obtained.

[0025] S6: User behavior prediction and interaction optimization. The LSTM-Attention model is used to analyze the multimodal feature vectors of S4 and the user historical interaction data stored in the metadata hub to predict future user actions and generate predicted interaction instructions. The interface layout is adjusted according to the prediction results, and a resource preloading request is sent to the four-dimensional isolation environment of S1.

[0026] S7: Dynamic adjustment and feedback loop of spatiotemporal resources. Based on the bitrate fluctuation data of the encoded audio and video streams in S5 and the prediction interaction instructions in S6, the resource utilization deviation of each spatiotemporal unit is calculated, and an optimized resource allocation table is generated. The initial resource allocation table of S1 is updated through the exponential moving average mechanism.

[0027] In this specification, the division of the three-dimensional spatial grid in S1 is calculated by measuring the number of grids based on the physical space range and grid precision, and the time slice uses a circular buffer to store resource allocation records.

[0028] In this specification, an initial resource allocation table is generated based on viewpoint prediction and historical data, which comes from the resource usage patterns of the previous 100 user sessions stored in the metadata central storage.

[0029] In this specification, the scene complexity score range in S5 is [0, 100]. When the score is >80, the bit rate is increased to 20Mbps, and when the score is <30, the bit rate is reduced to 1Mbps. The WaveNet model improves the audio signal-to-noise ratio by 18dB by separating the environmental noise in the noisy frequency. The model training data includes 10,000 noisy environmental audio samples from historical data.

[0030] In this specification, in step S6, based on the hot zone coordinates in the predictive interaction command and the three-dimensional spatial grid mapping in S1, a set of spatiotemporal units corresponding to the hot zone is generated. The set of spatiotemporal units corresponding to the hot zone includes the hot zone center unit and its adjacent 3×3×3 grid units. In step S7, based on the bitrate fluctuation data of the encoded audio and video stream in S5 and the set of spatiotemporal units corresponding to the hot zone, the resource utilization deviation of each spatiotemporal unit is calculated, and an optimized resource allocation table is generated. A resource reclamation strategy is executed: for spatiotemporal units with a utilization rate of <30% and a distance of >10 meters from the viewpoint, 20% of their CPU cycles and memory bandwidth are reclaimed and redistributed to the set of spatiotemporal units corresponding to the hot zone.

[0031] In summary, this invention solves the core problems of traditional virtual engines by using technologies such as spatiotemporal four-dimensional isolation and semantic-aware rendering, thereby improving the intelligence level of application management and user experience. The specific technical effects are as follows:

[0032] Refined resource scheduling leads to a significant improvement in utilization:

[0033] Spatiotemporal four-dimensional isolation mechanism: Breaking through the limitations of traditional two-dimensional sandbox, it realizes fine-grained allocation of resources in the time dimension, greatly reduces stuttering in high-load scenarios, and can reduce frame rate fluctuations when switching between multiple tasks.

[0034] Priority dynamic marking: Increases the CPU cycle quota of high-priority space-time units, improving overall resource utilization by approximately [percentage missing].

[0035] Intelligent resource recycling and allocation: resource quotas are increased in high-frequency interaction areas, resources are recycled and redistributed in low-priority units, and the system's ability to dynamically allocate resources is enhanced.

[0036] Rendering quality and performance are balanced for an optimized user experience.

[0037] Semantic fusion LOD technology: Combines semantic tags and distance to generate LOD levels, giving higher priority to rendering key objects and improving the rendering frame rate of complex scenes under the same hardware.

[0038] Content-aware super-resolution rendering: The ESRGAN model enhances image clarity and significantly improves the richness of details in high-resolution scenes.

[0039] Precise audio and video processing enhances immersion:

[0040] Spatiotemporal synchronization technology: effectively controls audio and video synchronization errors, reduces audio-visual misalignment in fast-moving scenes, and enhances immersion.

[0041] Intelligent noise reduction and dynamic bitrate control: The WaveNet model improves the audio signal-to-noise ratio, dynamically adjusts the bitrate to save bandwidth, and provides better audio and video transmission performance in different network environments.

[0042] The interaction is responsive and the operation is smooth.

[0043] User behavior prediction technology: reduces interaction latency, meets real-time interaction needs, and ensures timely operation response.

[0044] Resource preloading and layout adjustment: Prepare resources for high-frequency interaction areas in advance to reduce loading delays and make operations smoother.

[0045] The system features intelligent closed-loop optimization and strong adaptability.

[0046] The metadata hub forms a feedback loop, dynamically adjusting resource allocation and processing strategies, improving the level of intelligence, enhancing adaptability to diverse scenarios and management efficiency, and possessing industrial application value. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a schematic diagram of the framework of the virtual engine-driven application management system involved in this invention.

[0049] Figure 2 This is a flowchart illustrating the virtual engine-driven application management method involved in this invention.

[0050] Figure 3 This is a schematic diagram of the process for generating a list of visible objects involved in this invention.

[0051] Figure 4 This is a schematic diagram of the process for obtaining multimodal feature vectors involved in this invention. Detailed Implementation

[0052] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the embodiments of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0053] The following disclosure provides many different implementations or examples for carrying out different structures of the embodiments of the present invention. To simplify the disclosure of the embodiments of the present invention, specific examples of components and arrangements are described below. Of course, these are merely examples and are not intended to limit the embodiments of the present invention. Furthermore, reference numerals and / or reference letters may be repeated in different examples of the embodiments of the present invention; such repetition is for simplification and clarity and does not in itself indicate a relationship between the various implementations and / or arrangements discussed.

[0054] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0055] like Figure 1 As shown, this embodiment provides a virtual engine-driven application management system, including a spatiotemporal virtualization engine, a semantic rendering decision-maker, a multimodal signal processor, a GPU rendering unit, an intelligent interaction predictor, and a metadata hub;

[0056] The spatiotemporal virtualization engine is used to create a four-dimensional isolated environment containing a three-dimensional spatial grid and time slices, generate spatiotemporal units and allocate initial resource allocation tables, and output them to the metadata central storage; at the same time, it receives the optimized resource allocation table from the metadata central and dynamically adjusts the spatiotemporal unit resources.

[0057] The semantic rendering decision-maker connects to the spatiotemporal virtualization engine, receives the spatiotemporal environment handle from the spatiotemporal virtualization engine and the initial resource allocation table from the metadata hub, performs semantic segmentation and instance detection on the scene, constructs an octree spatial index, and generates a list of visible objects by combining the view frustum culling algorithm. The list of visible objects contains semantic labels and spatial coordinates, and is output to the metadata hub and GPU rendering unit. Based on the semantic labels of the visible objects and their distance from the viewpoint, a LOD level configuration table is generated and input into the GPU rendering unit.

[0058] A multimodal signal processor, connected to the GPU rendering unit and the spatiotemporal virtualization engine, acquires the low-resolution video stream from the rendering output and the original audio stream within the four-dimensional isolated environment. It calculates the time offset and synchronizes the audio and video through a cross-correlation algorithm. It extracts video semantic features, audio Mel spectrum, and motion vectors, and fuses them into a multimodal feature vector, which is then input into the metadata center. It receives dynamic bitrate control parameters generated by the metadata center based on the initial resource allocation table and scene complexity, and adjusts the H.265 encoding bitrate accordingly.

[0059] The intelligent interaction predictor connects to the spatiotemporal virtualization engine and the metadata hub, collects user interaction data, generates interaction feature vectors and inputs them into the metadata hub; it receives predictive interaction instructions generated by the metadata hub based on the initial resource allocation table and user profile, and triggers the spatiotemporal virtualization engine to increase resource quotas for spatiotemporal units in high-frequency interaction areas.

[0060] The metadata hub is bidirectionally connected to the spatiotemporal virtualization engine, semantic rendering decision-maker, multimodal signal processor, and intelligent interaction predictor. It stores the initial resource allocation table, model parameters, and user profile data. It receives the visible object list from the semantic rendering decision-maker, the multimodal feature vectors from the multimodal signal processor, and the interaction feature vectors from the intelligent interaction predictor. It then uses these data, combined with the initial resource allocation table, to train and update the scene complexity model and behavior prediction model. It outputs an optimized resource allocation table based on user behavior to the spatiotemporal virtualization engine, provides dynamic bitrate control parameters to the multimodal signal processor, and outputs predicted interaction commands to the intelligent interaction predictor. By training the scene complexity model and behavior prediction model, it dynamically adjusts resource allocation strategies (e.g., increasing the CPU quota of high-priority units by 20%-40%).

[0061] In some embodiments, the metadata hub calculates the importance weight of objects in each spatiotemporal unit based on the initial resource allocation table and the visible object list of the semantic rendering decision-maker. Spatiotemporal units containing highly important objects are marked as high priority, and their CPU cycle quota is increased by 20%-40% in the optimized resource allocation table.

[0062] In some embodiments, when the multimodal signal processor receives the dynamic bitrate control parameters from the metadata hub, if the memory bandwidth quota of the corresponding spatiotemporal unit in the initial resource allocation table is lower than the threshold, it automatically reduces the encoding resolution to 720p to match the resource limit.

[0063] In some embodiments, the intelligent interaction predictor queries the initial resource allocation table through the metadata hub to obtain the resource usage history data of the spatiotemporal unit corresponding to the current interaction hot zone. If the utilization rate of three consecutive time slices exceeds 85%, the spatiotemporal virtualization engine is triggered to reclaim resources from low-priority units and reallocate them.

[0064] In some embodiments, the spatiotemporal virtualization engine requests an initial resource allocation table template from the metadata hub during the system initialization phase, generates an instantiated initial resource allocation table based on device hardware parameters and application type, and stores it in the non-volatile storage area of ​​the metadata hub.

[0065] In some embodiments, the spatiotemporal virtualization engine:

[0066] Construction of a four-dimensional isolation environment:

[0067] 3D spatial mesh generation: The number of meshes is calculated to be 100×100×100 based on the physical space range (e.g., 100m×100m×100m) and the mesh precision (e.g., 1m / grid). Each mesh corresponds to one spatial unit. The time slice uses a circular buffer, with each slice lasting 50ms, and stores the resource allocation records of the most recent 10 slices.

[0068] Initial resource allocation table generation:

[0069] During the system initialization phase, templates are instantiated based on device hardware parameters (such as the number of CPU cores and memory capacity) and application type (such as game / industrial simulation).

[0070] Example: In a game scene, the initial allocation of CPU cycles of 500ms / slice and memory bandwidth of 200MB / s is given to the 3×3×3 grid cell at the center of the viewpoint; in an industrial simulation, the initial allocation of CPU cycles of 800ms / s and memory bandwidth of 300MB / s is given to the cell containing the core equipment.

[0071] Dynamic resource adjustment: Receives the optimization table from the metadata hub and updates resource allocation every 100ms. Supports dynamic adjustment of CPU cycle step size of 100ms / slice and memory bandwidth adjustment step size of 50MB / s.

[0072] In some embodiments, the semantic rendering decision-maker:

[0073] Semantic segmentation and instance detection:

[0074] The Mask R-CNN model is used, with an input image size of 640×480 and an output of 80 common object categories (such as people and mechanical parts). It is pre-trained using the COCO dataset and has an inference speed of 25 FPS.

[0075] Construct an octree spatial index with a maximum depth of 6. The leaf nodes store the list of objects. The view frustum culling uses the OpenGL Frustum algorithm to calculate the intersection of the bounding box and the 6 planes of the view frustum.

[0076] Dynamic LOD generation:

[0077] Semantic tag priority mapping: The game protagonist (label "Player") has a LOD level 2 levels higher than the background objects; the industrial simulation core equipment (label "CoreDevice") is forced to maintain the highest LOD.

[0078] Distance threshold settings: Close-up (<5m), medium-range (5-15m), and distant (>15m) correspond to LOD levels 3, 2, and 1, respectively, with the number of triangles reduced by 50% for each level.

[0079] In some embodiments, the multimodal signal processor:

[0080] Audio and video synchronization:

[0081] The cross-correlation algorithm uses normalized cross-correlation (NCC) with a window size of 500ms and a step size of 10ms to calculate the time offset between video frames and audio frames, with the synchronization error controlled within 8ms.

[0082] Intelligent noise reduction and bitrate control:

[0083] WaveNet model: 10-layer network, 128 filters per layer, input audio Mel spectrum (80-dimensional), training data includes 10,000 noisy environment samples (such as factory noise, crowd noise), and the signal-to-noise ratio is improved by 18dB after noise reduction.

[0084] Dynamic bitrate strategy:

[0085] Scene complexity score > 80 (complex scene): H.265 bitrate 20Mbps, encoding resolution 1080p;

[0086] Score < 30 (simple scenario): Bitrate 1Mbps, resolution automatically reduced to 720p (if memory bandwidth < threshold 100MB / s).

[0087] In some embodiments, the intelligent interaction predictor:

[0088] User behavior prediction:

[0089] LSTM-Attention model: Input multimodal feature vector (128-dimensional video semantic features + 80-dimensional audio features + 32-dimensional motion vector = 240-dimensional), LSTM layer with 256 units, attention mechanism weight matrix dimension 256×240, predicts actions in the next 200ms, latency <50ms.

[0090] Hot zone mapping: Predict the coordinates of hot zones and map them to a 3D mesh to generate a set of central cells + adjacent 3×3×3 cells (a total of 3×3×3×10 slices = 270 spatiotemporal cells).

[0091] Resource preloading: A preloading request is sent to the spatiotemporal virtualization engine 100ms in advance to load hotspot-related model resources (such as button icons and interactive interface data). The amount of preloaded data is limited to 10% of the memory capacity.

[0092] In some embodiments, the metadata hub:

[0093] Model training and updates:

[0094] Scene complexity model: Based on ResNet-50, input multimodal feature vectors, fully connected layers output scores, using mean squared error (MSE) loss function, and updating model parameters every second with new data through exponential moving average (EMA, smoothing factor 0.9).

[0095] Behavior prediction model: LSTM-Attention weights are fine-tuned every 5 seconds based on the latest 100 interaction data, using the Adam optimizer with a learning rate of 0.001.

[0096] Resource allocation optimization:

[0097] High-priority units see a 30% increase in CPU cycle time (e.g., from 500ms to 650ms per slice) and a 20% increase in memory bandwidth (e.g., from 200MB to 240MB / s).

[0098] Resource recycling strategy: For cells with utilization rate <30% and distance from viewpoint >10m, 20% of resources are recycled (e.g., CPU cycle time changes from 400ms to 320ms / slice) and redistributed to hot zone cells.

[0099] like Figure 2 As shown, a virtual engine-driven application management method includes:

[0100] S1: Spatiotemporal environment modeling and resource pre-allocation, creating a four-dimensional isolated environment containing a three-dimensional spatial grid and time slices, generating a spatiotemporal environment handle and an initial resource allocation table; the spatiotemporal environment handle is used to identify the four-dimensional isolated environment, and the initial resource allocation table contains the CPU cycle and memory bandwidth allocation parameters for each spatiotemporal unit;

[0101] S2: Scene semantic parsing and dynamic occlusion removal. The MaskR-CNN model is used to perform semantic segmentation and instance detection on the scene in the four-dimensional isolated environment generated by S1. Based on the spatiotemporal environment handle of S1, the spatial coordinate reference is obtained, an octree index is constructed and invisible objects outside the view frustum are removed, and a list of visible objects is generated. The list of visible objects contains semantic labels and spatial coordinates.

[0102] S3: Content-aware super-resolution and dynamic LOD rendering. Based on the semantic labels of objects in the visible object list of S2 and their distance from the viewpoint, the LOD level is dynamically selected. The ESRGAN model is used to generate a super-resolution image from the low-resolution image processed by S2, and the rendering result is obtained.

[0103] S4: Spatiotemporal synchronization and feature extraction of audio and video signals. The video stream in the rendering result of S3 and the original audio stream in the four-dimensional isolation environment are synchronized through cross-correlation algorithm. The time offset is calculated and the synchronized audio and video stream is generated. Video semantic features, audio Mel spectrum and motion vector are extracted and fused into multimodal feature vectors, which are then input into the metadata hub.

[0104] S5: Dynamic bitrate control and intelligent noise reduction. The ResNet-50 model is used to calculate the scene complexity score of the multimodal feature vector of S4 and dynamically adjust the H.265 encoding bitrate. The WaveNet model is used to perform noise reduction on the synchronous audio stream of S4. Combined with the semantic tags in the visible object list of S2, the encoded audio and video stream is obtained.

[0105] S6: User behavior prediction and interaction optimization. The LSTM-Attention model is used to analyze the multimodal feature vectors of S4 and the user historical interaction data stored in the metadata hub to predict future user actions and generate predicted interaction instructions. The interface layout is adjusted according to the prediction results, and a resource preloading request is sent to the four-dimensional isolation environment of S1.

[0106] S7: Dynamic adjustment and feedback loop of spatiotemporal resources. Based on the bitrate fluctuation data of the encoded audio and video streams in S5 and the prediction interaction instructions in S6, the resource utilization deviation of each spatiotemporal unit is calculated, and an optimized resource allocation table is generated. The initial resource allocation table of S1 is updated through the exponential moving average mechanism.

[0107] In some embodiments, the division of the three-dimensional spatial grid in S1 is calculated by measuring the number of grids based on the physical spatial extent and grid precision, and the time slice uses a circular buffer to store resource allocation records.

[0108] In some embodiments, an initial resource allocation table is generated based on viewpoint prediction and historical data, the historical data being the resource usage patterns of the previous 100 user sessions stored in the metadata central storage.

[0109] In some embodiments, the scene complexity score range in S5 is [0, 100]. When the score is >80, the bit rate is increased to 20Mbps, and when the score is <30, the bit rate is reduced to 1Mbps. The WaveNet model improves the signal-to-noise ratio of the denoised audio by 18dB by separating the environmental noise in the noisy frequency. The model training data includes 10,000 noisy environmental audio samples from historical data.

[0110] In some embodiments, in step S6, a set of spatiotemporal units corresponding to the hot zone is generated based on the mapping between the hot zone coordinates in the predicted interactive instruction and the three-dimensional spatial grid in step S1. The set of spatiotemporal units corresponding to the hot zone includes the hot zone center unit and its adjacent 3×3×3 grid units. In step S7, based on the bitrate fluctuation data of the encoded audio and video stream in step S5 and the set of spatiotemporal units corresponding to the hot zone, the resource utilization deviation of each spatiotemporal unit is calculated, and an optimized resource allocation table is generated. A resource reclamation strategy is executed: for spatiotemporal units with a utilization rate of <30% and a distance of >10 meters from the viewpoint, 20% of their CPU cycles and memory bandwidth are reclaimed and redistributed to the set of spatiotemporal units corresponding to the hot zone.

[0111] In some embodiments, spatiotemporal environment modeling and resource pre-allocation

[0112] Create a four-dimensional isolated environment containing a 3D spatial mesh and time slices. First, divide the 3D spatial mesh according to the physical spatial extent (e.g., length, width, and height) and mesh precision, and calculate the total number of meshes. The time dimension is divided into slices of fixed duration, and a circular buffer is used to store the resource allocation records of the most recent slices. An initial resource allocation table is generated based on device hardware parameters (e.g., number of CPU cores, memory capacity), application type (e.g., game / industrial simulation), and historical data (hotspot distribution of the previous 100 user sessions), containing the CPU cycles and memory bandwidth quotas for each spatiotemporal unit, and stored in the metadata hub.

[0113] Calculation of the number of grid cells in three-dimensional space:

[0114] ;

[0115] W, H, D: Length, width, and height of the physical space (unit: m); , , : Grid accuracy for each axis (unit: m / grid), typically taken as = = ; Total number of grid cells.

[0116] Time slice circular buffer:

[0117] Slicing time: =50ms; Buffer capacity: stores the most recent M slices (e.g., M=20, corresponding to 1 second of data); Data structure: Buffer[M][spatial unit ID, CPU cycle quota, memory bandwidth quota, utilization].

[0118] Initial resource allocation formula:

[0119] ;

[0120] C: Number of CPU cores in the device. Single-core baseline period (e.g., 1000ms); Priority coefficient (thermal zone unit) =1.5, ordinary unit =1); Initial CPU cycle quota for a single spatiotemporal unit (unit: ms / slice).

[0121] In some embodiments, scene semantic parsing and dynamic occlusion culling are performed:

[0122] The Mask R-CNN model is used to perform semantic segmentation and instance detection on scene images within a four-dimensional isolated environment, outputting semantic labels (e.g., "Player", "CoreDevice"), spatial coordinates, and confidence scores for objects. An octree spatial index is constructed, recursively dividing the scene space until a maximum depth (e.g., 6 layers) is reached, with each leaf node storing a list of objects. Combined with a view frustum culling algorithm, the positional relationship between the object's bounding box and the six planes of the view frustum is calculated, completely invisible objects are culled, and a list of visible objects containing semantic labels and spatial coordinates is generated. Figure 3 As shown.

[0123] Mask R-CNN inference process:

[0124] ;

[0125] N: The total number of objects detected. : The three-dimensional center coordinates of the i-th object

[0126] Criteria for removing cones:

[0127] Equation of the plane of the view cone: ; The corresponding left, right, bottom, top, near, and far planes of the visual cone. , , : The normal vector component (unit vector) of the j-th plane of the view frustum. : The offset of the j-th plane of the view frustum, used to determine the distance between the plane and the origin.

[0128] Distance from the center of the object's bounding box to the plane:

[0129] ;

[0130] : The 3D center coordinates of the object's bounding box; bounding box radius: r = diagonal length / 2; culling condition: if dist > r, the object is completely invisible.

[0131] In some embodiments, content-aware super-resolution and dynamic LOD rendering:

[0132] Based on the semantic labels of visible objects and their distance from the viewpoint, the Level of Detail (LOD) level is dynamically selected: key objects (such as the main character or core equipment) are prioritized by +1, near-field objects use a higher LOD level, and distant objects use a lower LOD level. For low-resolution images, an ESRGAN model is used to generate super-resolution images, improving detail clarity.

[0133] Calculation of distance between object and viewpoint:

[0134] ;

[0135] Viewpoint coordinates;

[0136] LOD level mapping rules:

[0137] ;

[0138] The number of triangles decreases as the LOD level decreases (e.g., LOD3→2 decreases by 50%).

[0139] ESRGAN model architecture:

[0140] Input: Low-resolution image (Dimension H multiplied by W);

[0141] Output: Super-resolution image (Dimension sH multiplied by sW, s=4 is the scaling factor);

[0142] Loss function: ;

[0143] : Counteracting loss, output by the discriminator; Pixel-level L1 loss; Feature loss based on VGG network; Weighting coefficient in total loss =0.1, Weighting coefficient in total loss =0.01. > This indicates that pixel-level precision has a higher priority than semantic feature consistency (because super-resolution tasks are more concerned with detail sharpness); smaller To avoid the perceptual loss from over-constraining the generator (high-level feature optimization is more difficult, and excessive weights can easily lead to training instability).

[0144] In some embodiments, spatiotemporal synchronization and feature extraction of audio and video signals:

[0145] The time offset between the video and audio streams is calculated using a cross-correlation algorithm, with the synchronization error controlled within 10ms. Video semantic features (such as the ROI pooling output of Mask R-CNN), audio Mel-spectral density (80-dimensional), and motion vectors (32-dimensional) are extracted and fused into a 240-dimensional multimodal feature vector, which is then input into the metadata hub. Figure 4 As shown.

[0146] Normalized cross-correlation (NCC) calculation:

[0147] ;

[0148] : Video frame luminance value sequence (normalized to [0,1]). : Audio frame energy sequence (extracted via short-time Fourier transform), T: Window length (e.g., 500ms). Time offset (unit: ms), synchronization offset: ;

[0149] Multimodal feature vectors:

[0150] Video semantic features (128 dimensions) Audio Mel spectrum (80 dimensions). Motion vector (32-dimensional, such as mouse / touch coordinate changes).

[0151] In some embodiments, dynamic bitrate control and intelligent noise reduction are employed.

[0152] The ResNet-50 model was used to calculate the scene complexity score (range [0, 100]) of the multimodal feature vectors, and the H.265 encoding bitrate was dynamically adjusted according to the score: 20 Mbps for complex scenes (score > 80) and 1 Mbps for simple scenes (score < 30). Meanwhile, the WaveNet model was used to denoise the audio stream, and key sounds (such as human voices) were preserved by combining semantic tags, resulting in an 18 dB improvement in signal-to-noise ratio.

[0153] Scene complexity score: ;

[0154] Dynamic bitrate control rules:

[0155] ;

[0156] If the memory bandwidth quota is less than 100MB / s, the resolution will be forcibly reduced to 720p.

[0157] WaveNet noise reduction model: ;

[0158] Noisy frequency time-domain signal : Noise-reduced audio signal;

[0159] Training data: 10,000 noisy frequency samples (signal-to-noise ratio 5-10dB);

[0160] Signal-to-noise ratio improvement: The signal-to-noise ratio of the output audio after noise reduction by the WaveNet model = the original signal-to-noise ratio of the input audio + 18dB.

[0161] In some embodiments, user behavior prediction and interaction optimization:

[0162] The LSTM-Attention model analyzes multimodal feature vectors and user history interaction data to predict user actions (such as click locations) in the next 200ms. The predicted hotspot coordinates are mapped to a 3D mesh, generating a set of hotspots consisting of a central cell and adjacent 3×3×3 cells. Resources are preloaded and the interface layout is adjusted accordingly.

[0163] LSTM-Attention prediction process: ;

[0164] M: Input sequence length (e.g., 20, corresponding to 1 second of data). Predict the probability distribution of actions (such as the probability of click coordinates);

[0165] Hot zone coordinate mapping: ;

[0166] : Three-dimensional coordinates of the hot zone Predict the coordinates of the three-dimensional mesh element corresponding to the center of the hot zone;

[0167] Hot zone unit set:

[0168] ;

[0169] t: Current time slice index (e.g., the t-th slice in a circular buffer). : Coordinates of the 3D mesh elements after the thermal expansion, ranging from the central element. And its adjacent front, back, left, right, top, and bottom units (a total of 3 x 3 x 3 = 27 spatial units); The hot zone involves time slice indexes, including the current slice t and the next future slice (t+1), for a total of 2 time dimension slices.

[0170] In some embodiments, the spatiotemporal resource dynamic adjustment and feedback closed loop are as follows:

[0171] Calculate the resource utilization deviation (the difference between the actual utilization rate and the target 70%) for each spatiotemporal unit, and trigger an adjustment when it exceeds ±15%. Update resource quotas using the exponential moving average (EMA), and reclaim 20% of resources from low-priority units (utilization rate <30% and distance >10m) and redistribute them to hot zone units.

[0172] Resource utilization rate calculation: ;

[0173] : Actual CPU cycles or memory bandwidth used : Allocated CPU cycles or memory bandwidth;

[0174] Quota target utilization rate: =70%, allowable deviation ±15%;

[0175] Exponential moving average update: ;

[0176] Updated resource quotas (new quotas); Resource quota before update (old quota), smoothing factor =0.9, Current demand quota (calculated based on hot zones or complexity);

[0177] Resource recycling strategies: ;condition: ;

[0178] The amount of resources recovered.

[0179] In industrial simulation scenarios:

[0180] Resource scheduling: The spatiotemporal unit where the core equipment (such as machine tools) is located is marked as high priority, and the CPU cycle is increased by 40% (from 800ms → 1120ms / slice) to ensure that real-time dynamic calculations are not interrupted.

[0181] Rendering optimization: Machine tool parts are identified through semantic tags, maintaining the highest LOD level, and ESRGAN super-resolution rendering of details such as screw textures improves the operator's observation accuracy.

[0182] Interaction optimization: Predict the location of the "Parameter Adjustment" button that engineers frequently click and preload relevant interface resources in advance, reducing the interaction latency from 120ms to 45ms.

[0183] In multiplayer online game scenarios:

[0184] Audio and video processing: In fast-moving gunfight scenes, the cross-correlation algorithm has a synchronization error of 8ms, reducing the audio-visual misalignment rate from 15% in traditional solutions to 3%; WaveNet removes environmental noise other than gunshots, improving the clarity of voice chat.

[0185] Resource recycling: If the utilization rate of units located in grass and trees far away from players (>10m) is less than 25%, 20% of the resources will be recycled and allocated to player gathering areas, and the frame rate in complex team battle scenes will be increased from 45FPS to 60FPS.

[0186] The embodiments described above are for illustrative purposes only and are not intended to limit the invention. Therefore, any changes in numerical values ​​or substitutions of equivalent elements should still fall within the scope of this invention.

[0187] The above detailed description will enable those skilled in the art to understand that the present invention can indeed achieve the aforementioned objectives and has complied with the provisions of the Patent Law.

[0188] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention. The above descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. It should be noted that any modifications, equivalent substitutions, and improvements made within the spirit and principles of the invention should be included within the scope of protection of the invention.

[0189] It should be noted that the above description of the process is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to the process under the guidance of this specification. However, these modifications and changes remain within the scope of this specification.

[0190] The basic concepts have been described above. Obviously, for those skilled in the art who have read this application, the above disclosure is merely illustrative and does not constitute a limitation of this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, and therefore, such modifications, improvements, and corrections still fall within the spirit and scope of the exemplary embodiments of this application.

[0191] Furthermore, this application uses specific terms to describe its embodiments. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that "an embodiment," "one embodiment," or "an alternative embodiment" mentioned twice or more in different positions in this specification do not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application can be appropriately combined.

[0192] Furthermore, those skilled in the art will understand that aspects of this application can be described and illustrated through several patentable types or situations, including any new and useful combination of processes, machines, products, or substances, or any new and useful improvements thereof. Therefore, aspects of this application can be implemented entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. All of the above hardware or software can be referred to as a “unit,” “module,” or “system.” Furthermore, aspects of this application can take the form of a computer program product embodied in one or more computer-readable media, wherein computer-readable program code is contained therein.

[0193] The computer program code required for the operation of each part of this application can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, and Python; general programming languages ​​such as C; Visual Basic, Fortran2103, Perl, COBOL2102, PHP, and ABAP; dynamic programming languages ​​such as Python, Ruby, and Groovy; or other programming languages. This program code can run entirely on the user's computer, or as a standalone software package on the user's computer, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any network, such as a local area network (LAN) or wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as Software as a Service (SaaS).

[0194] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this application are not intended to limit the order of the processes and methods of this application. Although some currently considered useful embodiments of the invention have been discussed in the foregoing disclosure by way of various examples, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the substance and scope of the embodiments of this application. For example, although the implementation of the various components described above can be embodied in a hardware device, it can also be implemented as a purely software solution, such as an installation on an existing server or mobile device.

[0195] Similarly, it should be noted that, in order to simplify the description of the present application and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of the embodiments of the present application sometimes combines multiple features into a single embodiment, drawing, or description thereof. However, this approach of the present application should not be construed as reflecting an intention that the claimed subject matter requires more features than expressly recited in each claim. Rather, the subject of the invention should possess fewer features than in any single embodiment described above.

Claims

1. A virtual engine-driven application management system, characterized in that, It includes a spatiotemporal virtualization engine, a semantic rendering decision-maker, a multimodal signal processor, a GPU rendering unit, an intelligent interaction predictor, and a metadata hub; The spatiotemporal virtualization engine is used to create a four-dimensional isolated environment containing a three-dimensional spatial grid and time slices, generate spatiotemporal units and allocate initial resource allocation tables, and output them to the metadata central storage; at the same time, it receives the optimized resource allocation table from the metadata central and dynamically adjusts the spatiotemporal unit resources. The semantic rendering decision-maker connects to the spatiotemporal virtualization engine, receives the spatiotemporal environment handle from the spatiotemporal virtualization engine and the initial resource allocation table from the metadata hub, performs semantic segmentation and instance detection on the scene, constructs an octree spatial index, and generates a list of visible objects by combining the view frustum culling algorithm. The list of visible objects contains semantic labels and spatial coordinates, and is output to the metadata hub and GPU rendering unit. Based on the semantic labels of the visible objects and their distance from the viewpoint, a LOD level configuration table is generated and input into the GPU rendering unit. A multimodal signal processor connects to the GPU rendering unit and the spatiotemporal virtualization engine to acquire low-resolution video streams from the rendering output and raw audio streams from the four-dimensional isolated environment. It calculates the time offset and synchronizes the audio and video through a cross-correlation algorithm. Extract video semantic features, audio Mel spectrum, and motion vectors, and fuse them into a multimodal feature vector, which is then input into the metadata hub. The metadata hub receives dynamic bitrate control parameters generated based on the initial resource allocation table and scene complexity, and adjusts the H.265 encoding bitrate accordingly. The intelligent interaction predictor connects to the spatiotemporal virtualization engine and the metadata hub, collects user interaction data, generates interaction feature vectors and inputs them into the metadata hub; it receives predictive interaction instructions generated by the metadata hub based on the initial resource allocation table and user profile, and triggers the spatiotemporal virtualization engine to increase resource quotas for spatiotemporal units in high-frequency interaction areas. The metadata hub is bidirectionally connected to the spatiotemporal virtualization engine, semantic rendering decision-maker, multimodal signal processor, and intelligent interaction predictor. It stores the initial resource allocation table, model parameters, and user profile data. It receives the visible object list from the semantic rendering decision-maker, the multimodal feature vectors from the multimodal signal processor, and the interaction feature vectors from the intelligent interaction predictor. It combines these with the initial resource allocation table to train and update the scene complexity model and behavior prediction model. It outputs an optimized resource allocation table based on user behavior to the spatiotemporal virtualization engine, provides dynamic bitrate control parameters to the multimodal signal processor, and outputs predicted interaction instructions to the intelligent interaction predictor.

2. The virtual engine-driven application management system according to claim 1, characterized in that, The metadata hub calculates the importance weight of objects in each spatiotemporal unit based on the initial resource allocation table and the visible object list of the semantic rendering decision-maker. Spatiotemporal units containing highly important objects are marked as high priority, and their CPU cycle quota is increased by 20%-40% in the optimized resource allocation table.

3. The virtual engine-driven application management system according to claim 1, characterized in that, When the multimodal signal processor receives the dynamic bit rate control parameters from the metadata center, if the memory bandwidth quota of the corresponding spatiotemporal unit in the initial resource allocation table is lower than the threshold, it will automatically reduce the encoding resolution to 720p to match the resource limit.

4. The virtual engine-driven application management system according to claim 1, characterized in that, The intelligent interaction predictor queries the initial resource allocation table through the metadata hub to obtain the resource usage history data of the spatiotemporal unit corresponding to the current interaction hot zone. If the utilization rate of three consecutive time slices exceeds 85%, the spatiotemporal virtualization engine is triggered to reclaim resources from low-priority units and reallocate them.

5. The virtual engine-driven application management system according to claim 1, characterized in that, During the system initialization phase, the spatiotemporal virtualization engine requests an initial resource allocation table template from the metadata hub, generates an instantiated initial resource allocation table based on device hardware parameters and application type, and stores it in the non-volatile storage area of ​​the metadata hub.

6. A virtual engine-driven application management method, characterized in that, include: S1: Spatiotemporal environment modeling and resource pre-allocation, creating a four-dimensional isolated environment containing a three-dimensional spatial grid and time slices, generating a spatiotemporal environment handle and an initial resource allocation table; the spatiotemporal environment handle is used to identify the four-dimensional isolated environment, and the initial resource allocation table contains the CPU cycle and memory bandwidth allocation parameters for each spatiotemporal unit; S2: Scene semantic parsing and dynamic occlusion removal. The Mask R-CNN model is used to perform semantic segmentation and instance detection on the scene in the four-dimensional isolated environment generated by S1. Based on the spatiotemporal environment handle of S1, the spatial coordinate reference is obtained, an octree index is constructed and invisible objects outside the view frustum are removed, and a list of visible objects is generated. The list of visible objects contains semantic labels and spatial coordinates. S3: Content-aware super-resolution and dynamic LOD rendering. Based on the semantic labels of objects in the visible object list of S2 and their distance from the viewpoint, the LOD level is dynamically selected. The ESRGAN model is used to generate a super-resolution image from the low-resolution image processed by S2, and the rendering result is obtained. S4: Spatiotemporal synchronization and feature extraction of audio and video signals. The video stream in the rendering result of S3 and the original audio stream in the four-dimensional isolation environment are synchronized through cross-correlation algorithm. The time offset is calculated and the synchronized audio and video stream is generated. Video semantic features, audio Mel spectrum and motion vector are extracted and fused into multimodal feature vectors, which are then input into the metadata hub. S5: Dynamic bitrate control and intelligent noise reduction. The ResNet-50 model is used to calculate the scene complexity score of the multimodal feature vector of S4 and dynamically adjust the H.265 encoding bitrate. The WaveNet model is used to perform noise reduction on the synchronous audio stream of S4. Combined with the semantic tags in the visible object list of S2, the encoded audio and video stream is obtained. S6: User behavior prediction and interaction optimization. The LSTM-Attention model is used to analyze the multimodal feature vectors of S4 and the user historical interaction data stored in the metadata hub to predict future user actions and generate predicted interaction instructions. The interface layout is adjusted according to the prediction results, and a resource preloading request is sent to the four-dimensional isolation environment of S1. S7: Dynamic adjustment and feedback loop of spatiotemporal resources. Based on the bitrate fluctuation data of the encoded audio and video streams in S5 and the prediction interaction instructions in S6, the resource utilization deviation of each spatiotemporal unit is calculated, and an optimized resource allocation table is generated. The initial resource allocation table of S1 is updated through the exponential moving average mechanism.

7. The application management method driven by a virtual engine according to claim 6, characterized in that, The division of the three-dimensional spatial mesh in S1 is calculated by measuring the number of meshes based on the physical spatial range and mesh precision, and the time slice uses a circular buffer to store resource allocation records.

8. The application management method driven by a virtual engine according to claim 6, characterized in that, An initial resource allocation table is generated based on viewpoint prediction and historical data, which is derived from resource usage patterns of the previous 100 user sessions stored in the metadata hub.

9. The application management method driven by a virtual engine according to claim 6, characterized in that, The scene complexity score range in S5 is [0, 100]. When the score is >80, the bit rate is increased to 20Mbps, and when the score is <30, the bit rate is reduced to 1Mbps. The WaveNet model improves the audio signal-to-noise ratio by 18dB by separating the environmental noise in the noisy frequency. The model training data includes 10,000 noisy environmental audio samples from historical data.

10. The application management method driven by a virtual engine according to claim 6, characterized in that, In step S6, based on the mapping between the hot zone coordinates in the predictive interaction command and the three-dimensional spatial grid in S1, a set of spatiotemporal units corresponding to the hot zone is generated. The set of spatiotemporal units corresponding to the hot zone includes the hot zone center unit and its adjacent 3×3×3 grid units. In step S7, based on the bitrate fluctuation data of the encoded audio and video stream in S5 and the set of spatiotemporal units corresponding to the hot zone, the resource utilization deviation of each spatiotemporal unit is calculated, and an optimized resource allocation table is generated. A resource reclamation strategy is executed: for spatiotemporal units with a utilization rate of <30% and a distance of >10 meters from the viewpoint, 20% of their CPU cycles and memory bandwidth are reclaimed and redistributed to the set of spatiotemporal units corresponding to the hot zone.

Images