High-throughput large-scale remote sensing ai inference system and method based on asynchronous decoupling architecture and space life cycle management

Abstract

The application discloses a high-throughput large-scale remote sensing AI inference system and method based on an asynchronous decoupling architecture and space life cycle management, which comprises four asynchronous subsystems connected through a back pressure queue: a disorderly preparation and orderly submission subsystem, a GPU calculation and data transmission decoupling subsystem, a space life cycle memory management subsystem and a lock-free local fusion subsystem. The disorderly preparation and orderly submission subsystem reorganizes high-concurrency disorderly I / O data into a spatially ordered tensor stream through a sequential feeding controller; the GPU calculation and data transmission decoupling subsystem adopts a zero-wait submission mechanism to realize pipeline parallelism of GPU calculation and PCIe transmission. The application realizes deep parallelization of each stage of data acquisition, GPU calculation, data transmission, memory management and post-processing, and introduces a dynamic memory life cycle management mechanism based on spatial dependence, which significantly improves system throughput and resource utilization efficiency while ensuring the correctness of the processing result.

Description

Technical Field

[0001] This invention relates to the technical field of remote sensing image processing and artificial intelligence inference, specifically to a high-throughput large-scale remote sensing AI inference system and method based on asynchronous decoupling architecture and space lifecycle management. Background Technology

[0002] Remote sensing Earth observation technology has undergone several generations of leapfrog development, and the current ability to acquire high-resolution (sub-meter level) imagery covers national and even global scales. The explosive growth of massive amounts of remote sensing data has spurred the rapid development of remote sensing artificial intelligence technology. Deep learning models based on convolutional neural networks or visual Transformers have become the core technical means for automated interpretation tasks such as land cover classification, accurate building extraction, and dynamic change detection.

[0003] Unlike traditional geographic information systems that only need to process static vector or lightweight raster data, remote sensing AI inference is essentially a high-intensity, computationally intensive, and I / O-intensive hybrid task. A single remote sensing orthophoto often contains billions of pixels, with data volumes reaching hundreds of gigabytes or even terabytes, far exceeding the capacity limits of current top-tier AI computing cards and general-purpose computer memory. The industry commonly adopts a block-sliding window strategy, cutting ultra-large images into tens of thousands of tensor blocks (such as a 512×512 pixel sliding window) for pipelined inference, and requires high-precision mathematical fusion of probabilistic maps of overlapping areas in memory to eliminate edge effects.

[0004] Existing remote sensing AI inference frameworks typically employ a synchronous blocking architecture. A single iteration of this architecture includes: the CPU loading remote sensing image tiles from disk or network and decoding them into a matrix; the CPU copying the matrix data to GPU memory; the GPU core performing forward propagation of the model; the CPU suspending and waiting for the GPU to complete its computation before synchronously copying the result tensor from GPU memory back to system memory; and finally, using a mutex lock to protect a large global array and complete the write of the result. This traditional architecture suffers from severe performance bottlenecks when handling large-scale remote sensing AI tasks.

[0005] First, there's the issue of intermittent GPU starvation. Because data transfer from device to host is a highly blocking operation limited by PCIe bus bandwidth, expensive GPU computing units are forced to idle during data transmission. Real-world testing shows that this serial mode of computation and transmission often results in GPU time utilization below 60%, severely restricting AI inference throughput. Second, there's the memory explosion problem caused by AI tensors. Intermediate results from remote sensing AI models, such as multi-class probability maps, are typically floating-point high-dimensional tensors, with a data volume far exceeding that of the original image. Current technologies lack fine-grained management of spatial dependencies, tending to cache the entire image result in memory for stitching, which easily exhausts server memory when processing city-level imagery, leading to out-of-memory crashes. Third, there's the issue of global lock contention during multi-core parallel processing. Multiple threads competing to write to the global result map during post-processing leads to severe lock contention. According to Amdahl's Law, the CPU parallel speedup decreases marginally with increasing core count. Finally, there's the issue of uncontrollable data flow timing. Network downloads and disk reads have random latency characteristics, and processing them directly out of order leads to extremely complex memory management and memory fragmentation. In summary, current technologies lack a systematic solution that can coordinate the asynchronous execution of each processing stage at the architectural level and effectively manage the memory lifecycle throughout the entire processing flow. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the aforementioned background technology and provide a high-throughput, large-scale remote sensing AI inference system and method based on an asynchronous decoupled architecture and spatial lifecycle management. This system achieves deep parallelism in data acquisition, GPU computing, data transmission, memory management, and post-processing stages through the collaborative work of four asynchronously decoupled subsystems. Furthermore, it introduces a dynamic memory lifecycle management mechanism based on spatial dependencies, significantly improving system throughput and resource utilization efficiency while ensuring the correctness of processing results.

[0007] To achieve the above objectives, the present invention provides a high-throughput large-scale remote sensing AI inference system based on asynchronous decoupling architecture and space lifecycle management, comprising four asynchronous subsystems connected by a backpressure queue: out-of-order preparation and ordered submission subsystem, GPU computing and data transmission decoupling subsystem, space lifecycle memory management subsystem, and lock-free local fusion subsystem. The out-of-order preparation and ordered submission subsystem includes a concurrent acquisition pool, a ready buffer, and a sequential feed controller. The concurrent acquisition pool has a shared concurrent thread pool that reads remote sensing tile data in parallel from heterogeneous data sources. The ready buffer adopts a thread-safe hash table structure to temporarily store tensor data that has been downloaded but is not arranged in spatial order. The sequential feed controller has a global sequence pointer with an initial value of zero. The controller monitors the ready buffer in a high-frequency loop. When the tile data corresponding to the global sequence pointer is ready, it is retrieved and pushed into the batch aggregator. Subsequently, the pointer is incremented. The GPU computing and data transmission decoupling subsystem includes a GPU master thread, an unloading queue, and a multi-threaded unloading pool. The GPU master thread is the only thread in the system that holds the GPU context. This thread executes the forward propagation of the deep learning model in a loop. After obtaining the result tensor, it encapsulates the pointer reference of the tensor in the video memory into a task object and pushes it into the unloading queue. The unloading queue is a bounded queue with a maximum capacity limit. The multi-threaded unloading pool includes multiple CPU threads. Each thread obtains a task object from the unloading queue and performs a blocking Device-to-Host data copy. The spatial lifecycle memory management subsystem includes a spatial dependency graph, a reference counting table, and a global cache. Before the inference task starts, the spatial dependency graph uses spatial indexes to pre-calculate the spatial topological relationships between each sliding window and identifies a set of neighboring windows with physical overlap for each window. The reference counting table maintains an atomic counter for each window, and the initial value of the counter is the number of downstream neighboring windows that depend on the data of that window for fusion processing. The global cache is used to store the probability graph data returned from the unloading thread. In the lock-free local fusion subsystem, each post-processing thread applies for a local canvas in its private space. Each thread obtains the data of the center window and its overlapping neighbor windows from the global cache in read-only mode. Weighted fusion is performed on the local canvas according to the Gaussian weight matrix. After fusion, normalization processing, classification mask generation and contour tracking are performed sequentially on the local canvas to extract vector polygons.

[0008] Furthermore, the sequential feed controller executes blocking and releasing logic: when the tile data pointed to by the global sequence pointer has not yet been stored in the ready buffer, even if there is subsequent tile data with a larger sequence number in the buffer, the controller still blocks the submission operation and enters a spin-waiting state; when the tile data corresponding to the global sequence pointer is ready, the controller takes it out of the buffer and pushes it into the batch aggregator, then increments the pointer by 1, and the output data is arranged in row priority order to form a spatially continuous tensor stream.

[0009] Furthermore, the GPU master thread adopts a zero-wait submission mechanism: after obtaining the ordered batch from the batch aggregator, the thread calls the asynchronous memory copy interface to upload the data to the video memory and execute the model forward propagation. After obtaining the result tensor, it does not call the synchronous data return interface, but only obtains the pointer reference of the result tensor in the video memory address space and encapsulates it into a task object and pushes it into the unloading queue. The threads in the multi-threaded unloading pool run in parallel with the GPU master thread. When the unloading thread performs the blocking return of the Nth batch of data through the PCIe bus, the GPU master thread simultaneously executes the model forward propagation calculation of the N+1th batch. The two form a pipeline-like parallel overlap on the time axis.

[0010] Furthermore, the construction of the spatial dependency graph includes: the system receiving the target geographical range and model input size, calculating the number of grid rows and columns required to cover the target survey area according to the sliding step size, generating a list of globally unique window identifiers arranged in row priority order, using R-Tree spatial index to pre-calculate the spatial relationship between each window and surrounding windows and identify a set of neighboring windows with physical overlap, the physical overlap area being the scope of subsequent fusion processing; in the initialization of the reference count table, for each window, all its downstream dependencies are traversed and the counter of that window is incremented by the corresponding value.

[0011] Furthermore, the spatial lifecycle memory management subsystem executes wavefront destruction logic: after the post-processing thread completes the vectorization extraction of the current window, the system queries the spatial dependency graph to obtain all upstream neighbor windows that have provided fusion data for the current window, performs an atomic decrement operation on the reference count of each upstream window, and then checks the count value of each upstream window. The data of windows with counts reaching zero is immediately physically released, and the data of the currently computed wavefront region and its related windows are retained in the system memory; the weighted fusion in the lock-free local fusion subsystem adopts a Gaussian weight matrix, and the fusion formula is: In the formula, V local V represents the fused value at coordinates (x, y) on the local canvas. center V represents the probability value of the center window at that coordinate. neighbor G represents the probability value of the neighboring window. center and G neighborHere, (x, y) represents the Gaussian weight coefficients at the corresponding positions, (x, y) represents the position in the center window coordinate system, (x′, y′) represents the mapping position of the neighbor window data in the local canvas coordinate system, N(center) represents the set of all neighbor windows that physically overlap with the center window, and k represents the index of the neighbor window. After fusion, normalization and Argmax operations are performed sequentially to generate a classification mask. Then, the contour tracking algorithm is called to extract the vector polygon boundaries. The generated vector object is sent to the main thread for storage through a thread-safe channel, and the local canvas memory is automatically released when the function returns.

[0012] Furthermore, the heterogeneous data sources include network WMS tile services, network TMS tile services, local GeoTIFF files, HDF5 files, or NumPy array formats. The system adapts to different data formats through an abstract data source interface layer. The spatial index includes R-Tree indexes, quadtree indexes, or adjacency matrices based on grid partitioning. The deep learning model includes semantic segmentation models, object detection models, or change detection models that require dual-temporal image input. The asynchronous decoupling architecture remains transparent to the network structure inside the model.

[0013] This invention also provides a high-throughput, large-scale remote sensing AI inference method based on asynchronous decoupling architecture and space lifecycle management, comprising the following steps: Spatial planning and dependency graph initialization: The system receives the boundary parameters of the target geographic area and the input size required by the deep learning model. Based on the set sliding step size, it calculates the number of grid rows and columns required to cover the target survey area, generates a list of globally unique window identifiers arranged in row priority, uses spatial index to pre-calculate the spatial relationship between each sliding window and surrounding windows, and identifies the set of neighboring windows with physical overlap to construct a spatial dependency graph. It synchronously initializes the reference count table and sets the initial value of the atomic counter of each window to the number of downstream neighboring windows that depend on the data of that window for fusion processing. Out-of-order data acquisition and ordered batch submission: A concurrent thread pool is started to acquire remote sensing tile data from heterogeneous data sources in parallel. All acquired tile data is stored in a thread-safe ready buffer. The sequential feed controller maintains a global sequence pointer with an initial value of zero and monitors the buffer status in a high-frequency loop. When the tile data corresponding to the global sequence pointer is ready, it is taken out of the buffer and pushed into the batch aggregator. Then the pointer is incremented and the output data is arranged in row priority order. Decoupled parallel inference between GPU and PCIe: The GPU master thread, as the only thread in the system holding the GPU context, obtains ordered batches from the batch aggregator, calls the asynchronous memory copy interface to upload the data to the video memory, and performs the forward propagation of the deep learning model. After obtaining the result tensor, it does not call the synchronous data return interface, but only obtains the pointer reference of the tensor in the video memory address space and encapsulates it into a task object and pushes it into the bounded unloading queue. Each thread in the multi-threaded unloading pool obtains the task object from the unloading queue and performs a blocking Device-to-Host data copy. The GPU master thread and the unloading thread form a pipeline-like parallel operation on the time axis. Dynamic memory lifecycle management based on reference counting: the probabilistic graph data returned to system memory from the unloading thread is stored in the global cache. After the post-processing thread completes the vectorization extraction of the current window, the system queries the spatial dependency graph to obtain all upstream neighbor windows that have provided fused data for the current window, performs an atomic decrement operation on the reference count of each upstream window, and the window data whose count reaches zero is immediately physically released. Lock-free local fusion and vectorization output: The post-processing thread requests a local canvas in its private space, retrieves the probabilistic map data of the center window and its overlapping neighbor windows from the global cache in read-only mode, performs weighted fusion on the local canvas according to the Gaussian weight matrix, and performs normalization and Argmax operations in sequence to generate a classification mask after fusion. It then calls the contour tracking algorithm to extract the vector polygon boundaries and sends the generated vector object to the main thread for storage through a thread-safe channel.

[0014] Furthermore, the sequential feed controller executes blocking and releasing logic: when the tile data pointed to by the global sequence pointer has not yet been stored in the ready buffer, even if there is subsequent tile data with a larger sequence number in the buffer, the controller still blocks the submission operation and enters a spin-wait state. When the tile data corresponding to the global sequence pointer is ready, it is retrieved and pushed into the batch aggregator, and then the pointer is incremented by 1. Due to the random delay in network transmission, tiles with larger spatial indices may be downloaded before tiles with smaller indices. The blocking and releasing logic reassembles the out-of-order I / O data stream in the time domain into a tensor stream arranged in row priority order in the spatial domain.

[0015] Furthermore, the GPU master thread employs a zero-wait submission mechanism. This mechanism requires that after completing the model forward propagation, the master thread does not call methods to convert GPU tensors to CPU tensors or to convert GPU tensors to array objects. Instead, it only obtains a pointer reference to the result tensor in the video memory address space and encapsulates this reference into a task object and pushes it into the unloading queue. The bounded unloading queue has a capacity limit. When the queue reaches its maximum capacity, the GPU master thread's push operation is blocked until a free position appears in the queue. This backpressure control mechanism prevents video memory resources from being exhausted due to insufficient downstream processing power. When the unloading thread performs the blocking backhaul of the Nth batch of data through the PCIe bus, the GPU master thread simultaneously performs the model forward propagation calculation of the N+1th batch.

[0016] Furthermore, after the post-processing thread completes the vectorization extraction of the current window, the system queries the spatial dependency graph to obtain a list of all upstream neighbor windows, performs an atomic decrement operation on the reference count entries of each upstream window, and then checks the count value of each upstream window. The data of windows whose counts reach zero are immediately physically cleared by calling the memory release interface, and the data of the currently computed wavefront region and its related windows are retained in the system memory.

[0017] Furthermore, the Gaussian weight matrix assigns higher weights to the central region of the window and decreasing weights to the edge regions. Data from multiple windows in the overlapping region are accumulated according to their respective weights. After accumulation, normalization is performed, and the accumulated probability value is divided by the sum of the weights at the corresponding positions to obtain the normalized probability distribution. The local canvas memory is automatically released when the processing function returns. Each post-processing thread independently completes the fusion and vectorization processing in its private space without performing write operations on the global data structure.

[0018] Compared with the prior art, the present invention has the following advantages: Firstly, this invention achieves physical separation of computation and data flow through a decoupling subsystem for GPU computation and data transmission. The GPU master thread focuses on issuing inference instructions, while time-consuming data return tasks are offloaded to an independent thread pool. This pipeline masking mechanism ensures that data transmission time is completely covered by computation time, eliminating the idle period of GPU waiting for data return in traditional synchronous architectures, thus achieving extremely fast inference performance and very high throughput. Real-world testing shows that GPU load can be increased from 40%-60% in traditional solutions to over 95%, and the overall AI inference throughput is increased by 2-3 times. For large-scale remote sensing tasks requiring the processing of tens of thousands of tiles, this performance improvement directly translates into a significantly shortened task delivery cycle, transforming the technological advantage of extremely fast inference performance into practical engineering value, and possessing significant engineering application value.

[0019] Secondly, this invention establishes a deterministic memory reclamation system through a lifecycle memory management mechanism based on spatial dependency graphs and reference counting. The system completes topology analysis and pre-sets reference counts before inference, and during runtime, a wavefront-based destruction strategy ensures that each data block is released immediately after completing all its downstream tasks. This precise, use-and-discard management ensures that only a limited amount of data near the current computational wavefront remains in memory, and the peak memory usage is no longer proportional to the processing area, achieving excellent scalability and unlimited area processing capabilities. Verified, it can continuously and stably process TB-level national-level imagery on edge devices equipped with only 16GB of memory without the risk of memory overflow, fully demonstrating its excellent scalability. Simultaneously, memory usage is only related to the "width of the scan line" and is independent of the total map area, completely breaking the limitations of physical memory and overcoming the rigid dependence of traditional solutions on hardware configuration. This achieves unlimited area processing capabilities, significantly lowering the deployment threshold for large-scale remote sensing AI applications and realizing a constant upper bound on memory usage and unlimited area processing capabilities.

[0020] Third, this invention completely reconstructs the traditional result splicing mode based on global locks through a lock-free local fusion subsystem. Each processing thread independently completes the entire process from probabilistic fusion to vector extraction in its private space, only needing to access the shared cache in read-only mode, completely avoiding write conflicts and realizing the release of multi-core computing power and linear scalability. This thread isolation design eliminates the serialization overhead caused by lock contention, enabling post-processing performance to grow almost linearly with the number of available CPU cores, truly releasing multi-core computing power. In modern multi-core server environments, this mechanism can fully unleash the potential of hardware parallel computing, transforming post-processing from a performance bottleneck in traditional solutions into a scalable processing stage, further enhancing linear scalability and achieving linear scaling of post-processing performance with the number of cores.

[0021] Fourth, this invention establishes a bridge between random I / O and deterministic computation flow through out-of-order preparation and ordered submission subsystems. The sequential feed controller employs a blocking-and-allowing mechanism based on global pointers, ensuring that downstream processing receives data batches strictly arranged in spatial order even in the face of out-of-order data arrival caused by network latency, thus achieving system robustness and determinism. This timing reordering capability not only simplifies the complexity of subsequent memory management but also ensures the logical determinism and predictability of the entire AI inference pipeline. It effectively solves the problem of out-of-order data flow caused by network jitter, avoids logical errors or memory fragmentation caused by disordered data arrival order, significantly improves the system's stable operation capability in complex network environments, and enhances the system's robustness to network I / O fluctuations.

[0022] Fifth, the decoupled architecture of this invention is transparent to the internal structure of the AI ​​model, supporting seamless switching between various remote sensing intelligent interpretation models such as semantic segmentation, object detection, and change detection. Simultaneously, the system supports multiple input formats through an abstract data source interface layer, including network TMS tile services, local GeoTIFF files, HDF5 format, and custom NumPy arrays. The spatial index implementation can also be flexibly selected between R-Tree, QuadTree, or grid indexes depending on the application scenario. This modular design gives this invention a wide range of applications and good engineering scalability, enabling it to adapt to application scenarios with different hardware configurations and task requirements. This further supports the implementation of the advantages mentioned above, such as ultra-fast inference, unlimited region processing, and linear expansion, and demonstrates good model and data source versatility. Attached Figure Description

[0023] Figure 1 This is a diagram illustrating the overall asynchronous pipeline architecture of the system provided in this embodiment of the invention. Figure 2 This is a schematic diagram of spatial dependencies and mesh partitioning provided for an embodiment of the present invention; Figure 3 A flowchart of the out-of-order preparation and ordered submission feed controller provided in this embodiment of the invention; Figure 4 The interaction timing diagram of the decoupled inference engine provided in the embodiments of the present invention; Figure 5 A memory lifecycle state transition diagram based on reference counting provided in an embodiment of the present invention; Figure 6 This is a schematic diagram illustrating the principle of lock-free local canvas blending provided in an embodiment of the present invention. Detailed Implementation

[0024] The following examples illustrate the implementation of the present invention in detail, but they do not constitute a limitation on the invention and are merely illustrative. Furthermore, the advantages of the present invention will become clearer and easier to understand by explaining them.

[0025] This invention proposes a high-throughput, large-scale remote sensing AI inference system based on an asynchronous decoupling architecture and spatial lifecycle management, comprising four asynchronous subsystems connected by a backpressure queue: an out-of-order preparation and ordered submission subsystem, a GPU computing and data transmission decoupling subsystem, a spatial lifecycle memory management subsystem, and a lock-free local fusion subsystem. The out-of-order preparation and ordered submission subsystem includes a concurrent acquisition pool, a ready buffer, and a sequential feed controller. The concurrent acquisition pool has a shared concurrent thread pool that reads remote sensing tile data in parallel from heterogeneous data sources. The ready buffer adopts a thread-safe hash table structure to temporarily store tensor data that has been downloaded but is not arranged in spatial order. The sequential feed controller has a global sequence pointer with an initial value of zero. The controller monitors the ready buffer in a high-frequency loop. When the tile data corresponding to the global sequence pointer is ready, it is retrieved and pushed into the batch aggregator. Subsequently, the pointer is incremented. The GPU computing and data transmission decoupling subsystem includes a GPU master thread, an unloading queue, and a multi-threaded unloading pool. The GPU master thread is the only thread in the system that holds the GPU context. This thread executes the forward propagation of the deep learning model in a loop. After obtaining the result tensor, it encapsulates the pointer reference of the tensor in the video memory into a task object and pushes it into the unloading queue. The unloading queue is a bounded queue with a maximum capacity limit. The multi-threaded unloading pool includes multiple CPU threads. Each thread obtains a task object from the unloading queue and performs a blocking Device-to-Host data copy. The spatial lifecycle memory management subsystem includes a spatial dependency graph, a reference counting table, and a global cache. Before the inference task starts, the spatial dependency graph uses spatial indexes to pre-calculate the spatial topological relationships between each sliding window and identifies a set of neighboring windows with physical overlap for each window. The reference counting table maintains an atomic counter for each window, and the initial value of the counter is the number of downstream neighboring windows that depend on the data of that window for fusion processing. The global cache is used to store the probability graph data returned from the unloading thread. In the lock-free local fusion subsystem, each post-processing thread applies for a local canvas in its private space. Each thread obtains the data of the center window and its overlapping neighbor windows from the global cache in read-only mode. Weighted fusion is performed on the local canvas according to the Gaussian weight matrix. After fusion, normalization processing, classification mask generation and contour tracking are performed sequentially on the local canvas to extract vector polygons.

[0026] Specifically, the out-of-order preparation and ordered submission subsystem addresses the high-concurrency, out-of-order characteristics of remote sensing tile network downloads or disk reads, ensuring strict spatial determinism of data entering the AI ​​computation stream. This subsystem comprises three core components: a concurrent acquisition pool, a ready buffer, and a sequential feed controller. The concurrent acquisition pool maintains a pool of concurrent threads, reading data in parallel from heterogeneous data sources, supporting multiple formats including WMS services, local GeoTIFF files, HDF5 files, and custom NumPy arrays. The ready buffer employs a thread-safe hash table structure to temporarily store downloaded but out-of-order tensor data. The sequential feed controller maintains a global sequence pointer, monitoring the buffer status in a high-frequency loop. When the tile pointed to by the pointer is not ready, it forcibly blocks the submission and enters a spin-wait state. Only when the corresponding data is ready is it retrieved, pushed into the batch aggregator, and the pointer incremented. Through this blocking and releasing mechanism, the subsystem reassembles the random I / O stream in the time domain into a tensor stream strictly arranged in row-major order in the spatial domain, avoiding memory fragmentation and ensuring the correctness of subsequent memory reclamation algorithms.

[0027] Specifically, the GPU computation and data transmission decoupling subsystem is the core engine for improving inference speed, and its design philosophy is pipeline masking. This subsystem comprises a single GPU master thread and a multi-threaded offload pool. The GPU master thread is the only thread in the system that holds the GPU context and is responsible for cyclically executing model inference. After issuing a computation instruction, this thread never performs any data return waiting operations; instead, it only obtains the pointer reference of the result tensor in GPU memory, encapsulates it into a task object, pushes it into the offload queue, and then begins the next batch of computation. The multi-threaded offload pool consists of a group of CPU threads, specifically responsible for retrieving tasks from the offload queue and performing blocking device-to-host data copying. Through this separation of responsibilities, while the offload thread is moving the current batch of data through the PCIe bus, the GPU master thread can simultaneously compute the next batch at full speed, achieving physical-level parallelism between computation and transmission. The offload queue has a maximum length limit, and a backpressure control mechanism prevents GPU memory overflow due to slow CPU processing.

[0028] Specifically, the spatial lifecycle memory management subsystem implements precise, disposable memory management for the massive amounts of intermediate data generated by remote sensing AI inference. Before task startup, this subsystem pre-calculates the topology of all sliding windows using R-Tree spatial indexing, constructing a spatial dependency graph. For each window, it identifies all subsequent neighboring windows that depend on its data for edge fusion. The system maintains an atomic reference counter for each window, initially set to the number of its downstream dependencies. After post-processing completes the vectorization of a window, the system triggers wavefront destruction logic, recursively decrementing the count of its upstream data source. Once the count reaches zero, the memory block is immediately physically released. This mechanism ensures that only a narrow computational wavefront is always retained in system memory, preventing memory usage from increasing linearly with map area, thus supporting the processing of geographically arbitrarily sized regions with limited hardware resources.

[0029] Specifically, the lock-free local fusion subsystem utilizes thread isolation technology to accelerate post-processing and integrates a core mathematical fusion algorithm. Each processing thread allocates a temporary array in its private space, covering only the current window size, as a local canvas. It retrieves data from the global cache for the current window and adjacent windows in read-only mode and performs weighted fusion calculations on the private canvas. The fusion process uses a Gaussian weight matrix to weight and accumulate overlapping regions, followed by vectorization processes such as normalization, binarization, and contour extraction directly on the private canvas. Since the entire process does not involve writing to the global array, lock contention is completely eliminated, allowing post-processing performance to scale linearly with the number of CPU cores.

[0030] In some implementations, the sequential feed controller executes blocking and releasing logic: when the tile data pointed to by the global sequence pointer has not yet been stored in the ready buffer, even if there is subsequent tile data with a larger sequence number in the buffer, the controller still blocks the submission operation and enters a spin-wait state; when the tile data corresponding to the global sequence pointer is ready, the controller takes it out of the buffer and pushes it into the batch aggregator, then increments the pointer by 1, and the output data is arranged in row priority order to form a spatially continuous tensor stream.

[0031] In some implementations, the GPU master thread employs a zero-wait submission mechanism: after obtaining ordered batches from the batch aggregator, the thread calls the asynchronous memory copy interface to upload the data to the video memory and executes model forward propagation. After obtaining the result tensor, it does not call the synchronous data return interface, but only obtains the pointer reference of the result tensor in the video memory address space and encapsulates it into a task object and pushes it into the unloading queue. The threads in the multi-threaded unloading pool run in parallel with the GPU master thread. When the unloading thread performs the blocking return of the Nth batch of data through the PCIe bus, the GPU master thread simultaneously executes the model forward propagation calculation of the N+1th batch. The two form a pipeline-like parallel overlap on the time axis.

[0032] In some implementations, the construction of the spatial dependency graph includes: the system receiving the target geographic range and model input size, calculating the number of grid rows and columns required to cover the target survey area based on the sliding step size, generating a list of globally unique window identifiers arranged in row priority order, using R-Tree spatial indexing to pre-calculate the spatial relationship between each window and surrounding windows and identify a set of neighboring windows with physical overlap, wherein the physically overlapping area is the scope of subsequent fusion processing; in the initialization of the reference count table, for each window, all its downstream dependencies are traversed and the counter of that window is incremented by the corresponding value.

[0033] In some implementations, the spatial lifecycle memory management subsystem executes wavefront destruction logic: after the post-processing thread completes the vectorization extraction of the current window, the system queries the spatial dependency graph to obtain all upstream neighbor windows that have provided fusion data for the current window, performs an atomic decrement operation on the reference count of each upstream window, and then checks the count value of each upstream window. The data of windows with counts reaching zero is immediately physically released, and the data of the currently computed wavefront region and its related windows are retained in the system memory; the weighted fusion in the lock-free local fusion subsystem uses a Gaussian weight matrix, and the fusion formula is: In the formula, V local V represents the fused value at coordinates (x, y) on the local canvas. center V represents the probability value of the center window at that coordinate. neighbor G represents the probability value of the neighboring window. center and G neighbor Here, (x, y) represents the Gaussian weight coefficients at the corresponding positions, (x, y) represents the position in the center window coordinate system, (x′, y′) represents the mapping position of the neighbor window data in the local canvas coordinate system, N(center) represents the set of all neighbor windows that physically overlap with the center window, and k represents the index of the neighbor window. After fusion, normalization and Argmax operations are performed sequentially to generate a classification mask. Then, the contour tracking algorithm is called to extract the vector polygon boundaries. The generated vector object is sent to the main thread for storage through a thread-safe channel, and the local canvas memory is automatically released when the function returns.

[0034] In some implementations, the heterogeneous data source includes network WMS tile service, network TMS tile service, local GeoTIFF file, HDF5 file, or NumPy array format, and the system adapts to different data formats through an abstract data source interface layer; the spatial index includes R-Tree index, quadtree index, or adjacency matrix based on grid partitioning; the deep learning model includes semantic segmentation model, object detection model, or change detection model that requires dual-temporal image input, and the asynchronous decoupling architecture remains transparent to the network structure inside the model.

[0035] The present invention provides a high-throughput, large-scale remote sensing AI inference method based on asynchronous decoupling architecture and space lifecycle management, comprising the following steps: Spatial planning and dependency graph initialization: The system receives the boundary parameters of the target geographic area and the input size required by the deep learning model. Based on the set sliding step size, it calculates the number of grid rows and columns required to cover the target survey area, generates a list of globally unique window identifiers arranged in row priority, uses spatial index to pre-calculate the spatial relationship between each sliding window and surrounding windows, and identifies the set of neighboring windows with physical overlap to construct a spatial dependency graph. It synchronously initializes the reference count table and sets the initial value of the atomic counter of each window to the number of downstream neighboring windows that depend on the data of that window for fusion processing. Out-of-order data acquisition and ordered batch submission: A concurrent thread pool is started to acquire remote sensing tile data from heterogeneous data sources in parallel. All acquired tile data is stored in a thread-safe ready buffer. The sequential feed controller maintains a global sequence pointer with an initial value of zero and monitors the buffer status in a high-frequency loop. When the tile data corresponding to the global sequence pointer is ready, it is taken out of the buffer and pushed into the batch aggregator. Then the pointer is incremented and the output data is arranged in row priority order. Decoupled parallel inference between GPU and PCIe: The GPU master thread, as the only thread in the system holding the GPU context, obtains ordered batches from the batch aggregator, calls the asynchronous memory copy interface to upload the data to the video memory, and performs the forward propagation of the deep learning model. After obtaining the result tensor, it does not call the synchronous data return interface, but only obtains the pointer reference of the tensor in the video memory address space and encapsulates it into a task object and pushes it into the bounded unloading queue. Each thread in the multi-threaded unloading pool obtains the task object from the unloading queue and performs a blocking Device-to-Host data copy. The GPU master thread and the unloading thread form a pipeline-like parallel operation on the time axis. Dynamic memory lifecycle management based on reference counting: the probabilistic graph data returned to system memory from the unloading thread is stored in the global cache. After the post-processing thread completes the vectorization extraction of the current window, the system queries the spatial dependency graph to obtain all upstream neighbor windows that have provided fused data for the current window, performs an atomic decrement operation on the reference count of each upstream window, and the window data whose count reaches zero is immediately physically released. Lock-free local fusion and vectorization output: The post-processing thread requests a local canvas in its private space, retrieves the probabilistic map data of the center window and its overlapping neighbor windows from the global cache in read-only mode, performs weighted fusion on the local canvas according to the Gaussian weight matrix, and after fusion, performs normalization processing and Argmax operation in sequence to generate a classification mask, calls the contour tracking algorithm to extract the vector polygon boundary, and sends the generated vector object to the main thread for storage through a thread-safe channel; Specifically, in the spatial planning and initialization phase, the system receives the target geographic range and model input dimensions, calculates the number of grid rows and columns required to cover the survey area based on the sliding step size, and generates a globally unique list of ordered window identifiers; it uses the R-Tree spatial index to pre-calculate the spatial overlap relationship between each window and surrounding windows to construct a topological dependency graph; it initializes the global reference count table, and the window sets its count value to the number of subsequent neighbors that depend on the data of that window for edge fusion.

[0036] During the out-of-order data acquisition and ordered batch submission phase, a concurrent preparation thread pool is started to concurrently acquire tiles from heterogeneous data sources, and the downloaded data is stored in a thread-safe ready buffer. The feed controller maintains a global pointer and checks the buffer in a loop. When the data corresponding to the pointer is not ready, it is forcibly blocked and waited. Only when the corresponding data is ready is it taken out, pushed into the downstream batch aggregator, and the pointer is incremented.

[0037] In the decoupled parallel inference phase between GPU and PCIe, the GPU main control thread retrieves ordered batches from the queue and performs asynchronous data upload and model inference. After obtaining the output tensor, it does not perform synchronous waiting but encapsulates the memory pointer reference as a task and pushes it into the offload queue before immediately processing the next batch. An independent multi-threaded offload pool retrieves tasks from the queue and performs blocking device-to-host data copying concurrently, achieving time overlap between computation and transmission.

[0038] During the dynamic memory lifecycle management phase based on reference counting, the probability graph returned from the unloading thread is stored in the global cache and is in an active state. After the post-processing completes the vectorization of a certain window, the dependency graph is queried to find all upstream neighbors, an atomic decrement operation is performed on them and the count is checked. If it drops to zero, the data block is immediately physically released.

[0039] In the lock-free local fusion and vectorization output stage, the post-processing thread constructs a local canvas in a private space, accesses the global cache in read-only mode to read the data of the central window and overlapping neighbors, applies a Gaussian weight matrix to weight and accumulate the overlapping areas, and after normalization, executes the mask generation and contour tracking algorithm to output vector polygons.

[0040] In some implementations, the sequential feed controller executes blocking and releasing logic: when the tile data pointed to by the global sequence pointer has not yet been stored in the ready buffer, even if there is subsequent tile data with a larger sequence number in the buffer, the controller still blocks the submission operation and enters a spin-wait state. When the tile data corresponding to the global sequence pointer is ready, it is retrieved and pushed into the batch aggregator, and then the pointer is incremented by 1. Due to the random delay in network transmission, tiles with larger spatial indices may be downloaded before tiles with smaller indices. The blocking and releasing logic reassembles the out-of-order I / O data stream in the time domain into a tensor stream arranged in row priority order in the spatial domain.

[0041] In some implementations, the GPU master thread employs a zero-wait submission mechanism. This mechanism requires that after completing the model forward propagation, the master thread does not call methods to convert GPU tensors to CPU tensors or to convert GPU tensors to array objects. Instead, it obtains a pointer reference to the result tensor in the video memory address space and encapsulates this reference as a task object, pushing it into the unloading queue. The bounded unloading queue has a capacity limit. When the queue reaches its maximum capacity, the GPU master thread's push operation is blocked until a free position appears in the queue. This backpressure control mechanism prevents video memory resources from being exhausted due to insufficient downstream processing power. When the unloading thread performs a blocking backhaul of the Nth batch of data through the PCIe bus, the GPU master thread simultaneously performs the model forward propagation calculation for the N+1th batch.

[0042] In some implementations, after the post-processing thread completes the vectorization extraction of the current window, the system queries the spatial dependency graph to obtain a list of all upstream neighbor windows, performs an atomic decrement operation on the reference count entries of each upstream window, and then checks the count value of each upstream window. The data of windows whose counts reach zero are immediately physically cleared by calling the memory release interface, and the data of the currently computed wavefront region and its related windows are retained in the system memory.

[0043] In some implementations, the Gaussian weight matrix assigns higher weights to the central region of the window and decreasing weights to the edge regions. Data from multiple windows in overlapping regions are accumulated according to their respective weights. The local canvas memory is automatically released when the processing function returns. Each post-processing thread independently completes the fusion and vectorization processing in its private space without having to perform write operations on the global data structure.

[0044] The present invention also provides a high-throughput large-scale remote sensing AI inference method based on the above system, including a spatial planning and initialization stage, a data preparation and ordered feeding stage, a decoupled inference and parallel unloading stage, a lifecycle management and memory reclamation stage, and a lock-free fusion and vectorized output stage.

[0045] During the spatial planning and initialization phase, the system receives the target geographic range and model input dimensions, calculates the number of grid rows and columns required to cover the survey area based on the sliding step size, and generates a globally unique list of ordered window identifiers. It uses the R-Tree spatial index to pre-calculate the spatial overlap relationship between each window and its surrounding windows to construct a topological dependency graph. It initializes the global reference count table, and each window sets its count value to the number of subsequent neighbors that depend on the data of that window for edge fusion.

[0046] During the data preparation and ordered feeding phase, a concurrent preparation thread pool is started to concurrently acquire tiles from heterogeneous data sources, and the downloaded data is stored in a thread-safe ready buffer. The feeding controller maintains a global pointer and checks the buffer in a loop. When the data corresponding to the pointer is not ready, it is forcibly blocked and waited. Only when the corresponding data is ready is it taken out, pushed into the downstream batch aggregator, and the pointer is incremented.

[0047] During the decoupled inference and parallel offloading phase, the GPU main control thread retrieves ordered batches from the queue and performs asynchronous data upload and model inference. After obtaining the output tensor, it does not perform synchronous waiting but encapsulates the memory pointer reference as a task and pushes it into the offloading queue before immediately processing the next batch. An independent multi-threaded offloading pool retrieves tasks from the queue and performs blocking device-to-host data copying concurrently, achieving time overlap between computation and transmission.

[0048] During the lifecycle management and memory reclamation phase, the probability graph returned from the unloading thread is stored in the global cache and is in an active state. After the post-processing completes the vectorization of a certain window, the dependency graph is queried to find all upstream neighbors, an atomic decrement operation is performed on them and the count is checked. If it drops to zero, the data block is immediately physically released.

[0049] In the lock-free fusion and vectorization output stage, the post-processing thread constructs a local canvas in a private space, accesses the global cache in read-only mode to read the data of the central window and overlapping neighbors, applies a Gaussian weight matrix to weight and accumulate the overlapping areas, and after normalization, executes the mask generation and contour tracking algorithm to output vector polygons.

[0050] Example 1: System Overall Architecture and Spatial Planning Initialization like Figure 1 As shown, the high-throughput, large-scale remote sensing AI inference system based on asynchronous decoupling architecture and spatial lifecycle management provided by this invention consists of four asynchronous subsystems, which are connected and coordinated through a back-pressure queue. This invention reconstructs the remote sensing AI inference process into four independently running asynchronous subsystems connected by the back-pressure queue using software-defined pipeline technology. The data flow path of the entire system is as follows: raw tile data enters the out-of-order preparation and ordered submission subsystem from heterogeneous data sources, and after being rearranged, forms an ordered tensor batch stream; the ordered batch enters the GPU computing and data transmission decoupling subsystem to execute deep learning model inference, and the generated probability map is returned to system memory through an offloading queue; the probability map data enters the global cache of the spatial lifecycle memory management subsystem, and its lifecycle is managed by a reference counting mechanism; finally, the lock-free local fusion subsystem reads data from the global cache, performs fusion and vectorization, and outputs standard geographic vector data.

[0051] like Figure 2As shown, this embodiment constructs the spatial cognitive foundation of the system, which is the cornerstone of memory management. The system initialization phase first completes the construction of the grid coordinate system: the system receives the boundary parameters of the target geographic area (GeoScope) and the input size required by the deep learning model (e.g., 1024 pixels). Let the geographic area of ​​the target survey area be (x... min , y min , x max , y max The model input dimensions are (H, W), and the sliding step size is set accordingly. h stride w Calculate the number of grid rows and columns required to cover the target survey area. (Number of grid rows) count and column number col count It can be calculated using the following formula: After generating window coordinates using a coordinate transformation algorithm (such as Web Mercator projection transformation), the system assigns a globally unique identifier to each sliding window, generating a list of globally unique window identifiers (0, 1, ..., N) arranged in row-major order. The window identifier in the i-th row and j-th column is i × col. count + j, the sorting rule follows the scan line order from left to right and then from top to bottom.

[0052] Topological dependency calculation is a core component of spatial planning. The system uses R-Tree spatial indexing to pre-calculate the spatial relationships between each window Wi and its surrounding windows. For each window Wi, the system identifies a set N(Wi) of all neighboring windows that physically overlap with Wi. Assume the current sliding window has a size of (H, W) and a sliding step size of (stride). h stride w If each window has a physical overlap with its neighboring windows in up to eight directions (up, down, left, right, upper left, upper right, lower left, and lower right), then these overlapping areas constitute the scope of the subsequent fusion algorithm. For windows located at the boundary of the test area, their neighborhood set may have fewer than eight elements. The spatial dependency graph simultaneously records the list of downstream windows that each window depends on as an upstream dependency, and the list of upstream windows that depend on as downstream dependencies.

[0053] The reference counting pre-setting phase initializes the reference counting table (RefTable). The reference counting table maintains an atomic counter for each window. For each window Wi, its initial count is set to the number of subsequent neighbors that rely on that window's data for edge fusion. For windows in the center of the survey area, the initial reference count is typically 8; for windows in the boundary areas, the initial count is between 3 and 5; and for windows in corner locations, the initial count is 3. The reference count is implemented using atomic integers, supporting lock-free concurrent decrementing operations.

[0054] Regarding alternative implementations of spatial indexes, in addition to using R-Tree spatial indexes, the dependency graph can also be constructed using quadtree indexes or grid-based adjacency matrices, as long as they accurately describe the neighborhood topology. The system provides unified adaptation for different index structures through an abstract spatial index interface layer.

[0055] Example 2: Out-of-order preparation and ordered submission subsystem like Figure 3 As shown, to address the high-concurrency, out-of-order characteristics of remote sensing tile network downloads or disk reads, the out-of-order preparation and ordered submission subsystem ensures that the data entering the AI ​​computation stream has strict spatial determinism. This embodiment details how to solve the uncertainty problem caused by high-concurrency I / O. This subsystem comprises three core components: a concurrent acquisition pool, a ready buffer, and a sequential feed controller.

[0056] The concurrent acquisition pool (shared tile prep executor) is implemented using a thread pool model. The system maintains a shared concurrent thread pool with a fixed number of worker threads, typically configured to be 8 to 32 threads. This thread pool reads remote sensing tile data in parallel from heterogeneous data sources. When the system starts the inference task, the task scheduler submits all window data acquisition requests to the task queue of the concurrent acquisition pool. Each worker thread competes to acquire a task from the queue and performs the actual data reading operation. Due to network fluctuations, windows with larger indices (e.g., ID=5) may download before windows with smaller indices (e.g., ID=1), meaning tile T5 may download before T1.

[0057] The Data Source Interface (DMI) provides unified encapsulation and adaptation for different types of data sources: For network WMS or TMS tile services, the DMI constructs an HTTP request conforming to the OGC specification and handles the decoding of the response data; for local GeoTIFF files, the DMI calls the GDAL library to read pixel data within a specified geographic range; for HDF5 files, the DMI accesses the specified dataset through the h5py library; for in-memory NumPy array formats (such as the PATI dataset), the DMI directly performs array slicing operations; for custom NPY format files, the DMI performs binary loading through the NumPy library. Regardless of the data source type, the DMI ultimately outputs tensor data in a unified format, containing an array of pixel values ​​and their corresponding window identifiers.

[0058] The Ready Buffer is a thread-safe hash table structure implemented using a thread-safe dictionary provided by the Python standard library. It uses window identifiers as keys and tensor data as values ​​to temporarily store downloaded tensor data that is not spatially ordered. When a worker thread finishes retrieving data, it stores the result in the Ready Buffer. Due to the randomness of network transmission delays, the order in which data from different windows arrive at the Ready Buffer is uncertain; it's possible for a window with a larger identifier to become ready before one with a smaller identifier.

[0059] The Sequential Feeder Controller is a key component ensuring the efficiency of AI batch processing and the core mechanism for achieving out-of-order preparation and ordered submission. Internally, the controller maintains a global sequence pointer, Current_Ptr (initially 0), representing the identifier of the window to be submitted. The controller runs a high-frequency monitoring loop, executing blocking and allowing logic in each iteration: During the high-frequency loop, the controller monitors the ready buffer status; if the tile data pointed to by Current_Ptr is not ready, even if subsequent tiles (e.g., ID=5) have been downloaded and stored in the buffer, the controller forcibly blocks the submission operation and enters a spin-wait state. Only when the data corresponding to Current_Ptr is ready is the controller retrieved from the buffer and pushed into the batch aggregator, then increments the pointer by 1. The batch aggregator is responsible for aggregating continuously arriving tensor data according to a set batch size (e.g., 8 or 16) to form input tensors suitable for GPU batch processing.

[0060] The technical advantage of this mechanism lies in its ability to reassemble randomly arriving out-of-order I / O data streams in the time domain into a continuous tensor stream arranged strictly in row-major order in the spatial domain. Through this mechanism, regardless of the arrival time order of the data, the final batches of data fed to the GPU are always strictly arranged in ascending order of the window identifier. This mechanism avoids memory fragmentation, ensures the correctness of subsequent memory reclamation algorithms, and shields against the volatility of network I / O, ensuring the logical determinism and stability of the AI ​​inference pipeline.

[0061] Anomaly Handling Placeholder Mechanism: To enhance system robustness, a maximum waiting timeout is set. If the tile data corresponding to a certain window identifier fails to be acquired after multiple retries, the system generates a placeholder tensor (dummy tensor) with all zeros and marks it as a failure. This placeholder tensor is then submitted normally to the downstream via the feed controller to prevent the entire inference pipeline from deadlocking due to a single point of failure in data acquisition. This placeholder tensor will be overwritten by valid data from neighboring windows in subsequent fusion stages, without affecting the overall quality of the final result.

[0062] Example 3: Decoupling Subsystem for GPU Computing and Data Transmission like Figure 4 As shown, the Decoupled Inference Engine is the core engine for improving inference speed. Its core idea is "Pipeline Masking," which enables parallel execution of GPU forward propagation computation and PCIe data backhaul, eliminating the idle waiting of GPU computing units in traditional synchronous architectures. This subsystem consists of three components: the GPU master thread, the offload queue, and the multi-threaded offloader pool.

[0063] The GPU master thread is the only thread in the system that owns the GPU context; all operations involving GPU resources are executed in this thread. The master thread's work loop (atomic loop) includes the following steps: First, it retrieves the next ordered input batch (Bin) from the batch aggregator. This batch tensor is pinned to host memory using pinned memory to accelerate transfer. Then, it calls the cudaMemcpyAsync asynchronous memory copy interface to upload the input data from host memory to GPU memory. Next, it executes the forward propagation of the deep learning model's Tensor. Out = Model(Bin), obtain the output probability graph tensor.

[0064] The zero-wait submission mechanism is the core innovation of this subsystem. This mechanism is implemented when the result tensor is obtained. Out Subsequently, the GPU master thread never performs any data return waiting operations. Specifically, the master thread does not call any synchronous data return methods, prohibits calling interfaces that convert GPU tensors to CPU tensors (such as the `.cpu()` method), prohibits calling interfaces that convert tensors to NumPy arrays (such as the `.numpy()` method), and prohibits calling any stream synchronization waiting instructions (such as `cudaStreamSynchronize`). The master thread only obtains a pointer reference (Handle) of the result tensor in the VRAM address space, encapsulates this reference along with batch metadata into a task object, and pushes it into the unloading queue. This mechanism allows the master thread to return to the loop start point within milliseconds after pushing the task object and begin processing the next batch of computation tasks.

[0065] The Offload Queue is a bounded blocking queue (bounded priority queue) connecting the GPU master thread and the multi-threaded offload pool, used for backpressure control. Its maximum capacity is an important system configuration parameter, typically ranging from 2 to 8. When the queue is not full, the GPU master thread's push operation returns immediately; when the queue is full and reaches its maximum capacity, the push operation is blocked until the offload thread retrieves the task object from the queue. This backpressure control mechanism ensures that the number of tensors accumulating in video memory awaiting return does not exceed the queue capacity, preventing video memory overflow (GPU OOM) due to insufficient downstream CPU processing power.

[0066] The offloader pool consists of a group of CPU threads dedicated to moving data; the number of threads is typically configured to be 2 to 4. Each offloader thread runs independently, retrieving task objects from the offloader queue and performing a blocking device-to-host data copy.

[0067] The above design achieves physical-level parallelism: GPU computation and PCIe data transfer overlap in a pipelined manner along the timeline. Assume the GPU computation time for a single batch is T. compute PCIe return time is T transfer In a traditional synchronous architecture, the total time to process M batches is: In the decoupled architecture of this invention, except for the first and last batches, the computation and transmission completely overlap, and the total time formula is: In the formula, T total To decouple the architecture, the total time for processing M batches is T. computeT represents the GPU computation time per batch. transfer M represents the PCIe data return time per batch, and M represents the total number of batches. When these two are nearly balanced, a performance improvement of almost 2x can be achieved. Real-world testing shows that by using a decoupled architecture, GPU load can be increased from 40% to over 95% compared to traditional solutions, and the overall throughput (FPS) of AI inference can be increased by 2-3 times.

[0068] Example 4: Space Lifecycle Memory Management Subsystem like Figure 5 As shown, the Spatial Lifecycle Manager subsystem enables precise "use-and-discard" memory management for the massive amounts of intermediate data generated by remote sensing AI inference. This subsystem utilizes three core data structures—spatial dependency graph, RefTable, and Prob Cache—to achieve refined lifecycle management of probabilistic graph data. This embodiment demonstrates how to achieve unlimited region processing capabilities.

[0069] Data loading phase: The probabilistic map data returned to system memory from the unloading thread is stored in the global cache, ProbCache. The global cache uses a thread-safe dictionary structure (Hash Map), with the window identifier as the key and the probabilistic map data as the value. At this time, the data is in an "active" state, waiting for the post-processing thread to read and merge it.

[0070] Cleanup trigger mechanism: Once the post-processing thread completes the final vectorization extraction of the current window Wcurr, it indicates that the window and its dependent data have fulfilled their purpose. At this point, the system triggers garbage collection logic and enters the reverse dependency update phase.

[0071] Reverse dependency update: The system queries the spatial dependency graph to find a list of all upstream neighbor windows that have provided fused data to the current window Wcurr. An atomic decrement operation is performed on the reference count of each window in the list, mathematically expressed as: In the formula, RefCount(W neighbor ) for neighbor window W neighbor The current reference count value is used for this operation, which is an atomic operation and ensures correctness under multi-threaded concurrency.

[0072] Conditional physical destruction: The system then immediately checks the count values ​​of each upstream window. If the decremented count value reaches zero, the system immediately calls the memory release interface (such as the del operator in Python or the free function in C++) to physically delete the data block. The system determines that the data is "dead data," removes the data of that window from the global cache, and releases the memory space it occupies. This mechanism ensures that memory usage does not accumulate as processing progresses.

[0073] The characteristics of wavefront destruction are as follows: Because the inference task proceeds in the order of spatial scanning, the release of window data exhibits a wavefront-like pattern. Window data that has completed all downstream processing is promptly reclaimed, while window data still within or downstream of the computation wavefront is retained. Only a narrow "computation wavefront" region and its associated window data are always maintained in system memory, and memory usage no longer increases linearly with map area.

[0074] The reference counting-based spatial garbage collection (GC) algorithm replaces the operating system's non-deterministic garbage collection mechanism, achieving deterministic memory destruction based on geographic topology relationships (R-Tree). This mechanism breaks the limitations of physical memory, enabling continuous and stable processing of TB-level national-level imagery on edge devices or ordinary workstations with only 16GB of memory, without the risk of memory overflow (OOM), and supporting unlimited area processing for survey areas of any size.

[0075] Example 5: Lock-free Local Fusion Subsystem like Figure 6 As shown, the lock-free local fusion subsystem utilizes thread isolation technology to accelerate post-processing and integrates core mathematical fusion formulas. This subsystem employs a unique private canvas strategy to achieve multi-threaded parallel fusion, completely eliminating the lock contention problem in traditional global fusion algorithms. This embodiment demonstrates the specific algorithm for lock-free concurrent processing.

[0076] Local Canvas Construction: When processing a central window, each post-processing thread first allocates a local canvas within its private stack space. The local canvas is a temporary array of all zeros, H × W × C, with the same size as the sliding window (where H is the height, W is the width, and C is the number of channels, i.e., the number of categories), used to store the merged probability values. Because the stack space is thread-private, the local canvases of different threads are independent of each other, requiring no synchronization mechanism.

[0077] Read-only weighted projection: The thread then retrieves the probabilistic map data of the center window and all its overlapping neighbor windows from the global cache with read-only permissions. For each neighbor window, the thread calculates the mapping relationship between its overlapping area with the center window in two coordinate systems. Let the coordinates of the top left corner of the center window be (x0, y0), the coordinates of the top left corner of a neighbor window be (x1, y1), and the window size be (H, W). Then the range of the overlapping area of ​​the two windows in the coordinate system of the center window is [max(0, x1-x0), min(W, x1-x0+W)] × [max(0, y1-y0), min(H, y1-y0+H)].

[0078] The Gaussian weight matrix G assigns higher confidence weights to the central region of the window, with the weights gradually decreasing towards the edges. The formula for generating the Gaussian weight matrix is: Among them (c x, c y ) represents the coordinates of the window center, and σ is the standard deviation parameter of the Gaussian kernel, typically ranging from 1 / 4 to 1 / 3 of the window size. This weight distribution in the overlapping region achieves a smooth transition between multi-source data.

[0079] The fusion process iterates through each pixel position of the local canvas, weighting and summing all visible window data at that position according to their Gaussian weights. The core mathematical fusion formula is: In the formula, V local V represents the fused value at coordinates (x, y) on the local canvas. center V represents the probability value of the center window at that coordinate. neighbor G represents the probability value of each neighboring window. center and G neighbor Here, (x, y) represents the Gaussian weight coefficients at the corresponding positions, (x, y) represents the position in the center window coordinate system, (x′, y′) represents the mapped positions of the neighboring window data in the local canvas coordinate system, N(center) represents the set of all neighboring windows that physically overlap with the center window, and k represents the index of the neighboring window. The summation symbol Σ indicates that all neighboring windows that overlap with the center window are summed.

[0080] Isolated Vectorization: After data accumulation, the thread sequentially performs the following post-processing operations on the local canvas: First, normalization is performed, dividing the accumulated probability value by the sum of the weights at the corresponding positions to obtain a normalized probability distribution; then, an Argmax operation is performed on each pixel position to select the class index with the highest probability and generate a classification mask (binarization); finally, a contour tracking algorithm (Vectorization, such as OpenCV's findContours function or rasterio.features.shapes function) is called to extract vector polygons along the class boundaries. Since the entire process does not involve writing to a large global array, lock contention is completely eliminated.

[0081] Output and Release: The extracted vector polygons, accompanied by geographic coordinate transformation parameters, can be converted to a standard geographic coordinate system representation. The generated vector objects are sent to the main thread for persistent storage via a thread-safe channel (such as Python's `queue.Queue`). The main thread is responsible for writing the vector data to standard geographic data formats such as GeoPackage, Shapefile, or GeoJSON. When the processing function returns, the stack memory occupied by the local canvas is automatically released upon function return, requiring no explicit memory management. The entire process involves no write lock operations, achieving true linear parallel acceleration with zero-lock submission.

[0082] This mechanism employs a "space-for-time" strategy, utilizing local memory to achieve lock-free concurrency. Each post-processing thread independently completes fusion and vectorization processing within its private space without performing write operations on global data structures, thus achieving true lock-free parallelism. This architecture allows post-processing performance to increase linearly with the number of CPU cores, and thread scalability to be nearly linear, fully exploiting the computing potential of modern multi-core servers.

[0083] Example 6: Complete Processing Flow and Stage Collaboration The complete processing flow of the method of this invention includes five collaborative stages: Task Initialization: The user submits an inference task request, specifying the geographical boundary parameters of the target survey area, the type and access path of the input data source, the deep learning model path, the sliding window size parameters, the sliding step size parameters, the output format, and other configurations. After parsing the configuration parameters, the system loads the deep learning model into the GPU memory and completes model warm-up. It creates a data source adapter instance of the corresponding type, calculates the sliding window grid size based on the geographical range and sliding parameters, constructs a spatial dependency graph and initializes the reference count table, creates thread resources such as a concurrent acquisition pool, a multi-threaded unloading pool, and a post-processing thread pool, and initializes the backpressure queues and their capacity parameters between each stage. The system then simultaneously starts the preparation thread pool, the GPU main control thread, the unloading thread pool, and the post-processing thread pool. Each thread pool is connected through a bounded backpressure queue to form a complete asynchronous pipeline, with all stages running concurrently rather than starting sequentially.

[0084] Data Acquisition and Batch Submission: The system initiates a concurrent acquisition pool, submitting all data acquisition tasks from all windows to the work queue. Worker threads execute data reading in parallel, acquiring remote sensing tiles from configured heterogeneous data sources. Due to the randomness of network latency, completed data is stored in the ready buffer in out-of-order. The sequential feed controller continuously monitors the buffer status, maintains a global sequence pointer, and detects and extracts ready data according to the sequence pointer order. When the data corresponding to the pointer is ready, it is pushed into the batch aggregator and the pointer is incremented; when the data corresponding to the pointer is not ready, it blocks and waits. After the batch aggregator accumulates a sufficient number (e.g., 8 or 16) of tensors, it forms an input batch and pushes it into the GPU input queue.

[0085] GPU Inference and Data Offloading: The GPU master thread retrieves ordered batches from the input queue, calls the asynchronous memory copy interface (cudaMemcpyAsync) to upload the data to GPU memory, performs model forward propagation to obtain the output probability map tensor, and uses a zero-wait commit mechanism to encapsulate the GPU memory pointer of the result tensor into a task object and push it into a bounded offloading queue. In an independent multi-threaded offloading pool, each thread retrieves a task from the offloading queue, performs blocking Device-to-Host data backhaul, and stores the probability map data in a global cache. The GPU master thread and the offloading thread execute in parallel on the timeline, forming a pipelined overlap.

[0086] Fusion and Vectorization: Each thread in the post-processing thread pool obtains the window identifier to be processed from the task queue. Each thread requests a local canvas in its private space and retrieves the probabilistic map data of the center window and its neighboring windows from the global cache in read-only mode, performing weighted fusion calculations based on the Gaussian weight matrix. After fusion, normalization processing, Argmax classification to generate a mask, and contour tracing to extract vector polygon boundaries are performed sequentially. The generated vector results are sent to the main thread through a thread-safe channel. After processing, wavefront destruction logic is triggered, atomically decrementing the reference count of upstream neighboring windows, immediately physically releasing data with a count of zero, and reclaiming the memory resources occupied by the relevant windows.

[0087] Output and Resource Cleanup: The main thread continuously receives vector objects submitted by each post-processing thread, appends geographic coordinate information to the vector data, and writes it to the output file according to the user-specified output format (GeoPackage, Shapefile, GeoJSON, etc.). Once all windows have finished processing, the system executes a resource cleanup process: closing all thread pools, waiting for all worker threads to exit, releasing GPU model resources and video memory usage, clearing remaining cached data, closing data source connections, and generating a task execution report containing processing statistics.

[0088] Example 7: Change Detection Task and Dual-Temporal Image Processing The system of this invention is scalable to support change detection tasks that require dual-temporal image input, which is an important scenario in remote sensing applications. In this application scenario, the system architecture remains unchanged, and only moderate expansion is needed in the data acquisition stage to support new application requirements.

[0089] Dual-phase data acquisition: In the concurrent acquisition pool, each data acquisition task is expanded into two subtasks, which read tile data from the preceding phase (T1) and the following phase (T2), respectively. The two subtasks can be executed in parallel to improve efficiency. Once the data from both phases has been acquired, the system pairs them to form a dual-channel input tensor.

[0090] The ready buffer has been expanded: it stores paired biphase tensors, with each entry containing preceding and following phase imagery data for the same geographic location. The logic of the sequential feed controller remains unchanged, still pushing paired data into the batch aggregator in spatial order.

[0091] GPU Inference Phase: The GPU inference phase uses a specially trained change detection deep learning model to process the bi-temporal input. The model receives the concatenated bi-temporal tensor and outputs a change probability map, representing the probability of a change at each pixel location. The zero-wait submission mechanism and multi-threaded unloading mechanism are exactly the same as for the single-temporal task.

[0092] Post-processing stage: The processing logic for the lock-free local fusion and vectorization stages is consistent with that of the semantic segmentation task, using the same Gaussian weight fusion algorithm to eliminate window edge effects. Finally, the system extracts the vector polygon boundaries of the changed regions, labels the change type attributes, and outputs change detection results data that conform to industry standards.

[0093] This design reuses all the core architectural components of the system (out-of-order preparation and ordered submission, GPU and transmission decoupling, spatial lifecycle management, and lock-free local fusion), and only requires moderate expansion at the data acquisition layer to seamlessly support the important remote sensing application scenario of change detection.

[0094] Example 8: System Parameter Configuration and Performance Tuning The key parameters involved in this invention can be flexibly adjusted according to actual hardware configuration and application requirements to achieve optimal performance on a specific hardware platform. The configurations for each parameter are given below: Number of concurrent acquisition pool threads: Typical configuration is 8 to 32 threads. For network data sources, the number of threads should match the network bandwidth and server concurrent connection limits; for local disk data sources, the number of threads can be appropriately reduced to avoid disk I / O contention.

[0095] Batch aggregator batch size: Typical configuration is 8 to 32. The batch size should be determined based on the GPU memory capacity and model size. Larger batches can improve GPU computing efficiency, but will increase memory usage and processing latency.

[0096] Unloading queue capacity limit: Typical configuration is 2 to 8. Too small a capacity will reduce pipeline parallelism, while too large a capacity will increase memory pressure. It is recommended to adjust according to the available memory.

[0097] Number of threads in the multi-threaded offload pool: A typical configuration is 2 to 4. Having too many threads will not improve PCIe bandwidth utilization; instead, it will increase thread switching overhead.

[0098] Post-processing thread pool thread count: It is recommended to configure it to 50% to 100% of the number of CPU cores. The number of threads should match the number of available CPU cores to fully utilize the multi-core parallel capabilities.

[0099] The standard deviation parameter σ of the Gaussian weight matrix typically takes the value of 1 / 4 to 1 / 3 of the window size. A smaller σ value makes the weights more concentrated in the center of the window, while a larger σ value makes the weights more evenly distributed.

[0100] Sliding step size to window size ratio: Typical overlap is 12.5% ​​to 25% (i.e., a step size of 75% to 87.5% of the window size). Larger overlap can improve fusion quality, but will increase computational cost.

[0101] Users can optimize these configuration parameters by comprehensively considering factors such as GPU computing power, number of CPU cores, network bandwidth, memory capacity, and target test area size, in order to achieve the best balance between throughput, latency, memory usage, and result quality.

[0102] Example 9: Hardware Environment and Deployment Requirements This invention is applicable to servers or workstations equipped with NVIDIA GPUs and CUDA computing environments. Hardware requirements include: GPU Requirements: NVIDIA graphics card supporting CUDA computing, with 8GB or more of VRAM recommended. Supported GPU models include, but are not limited to, GeForce RTX series, Quadro series, Tesla series, and data center-grade GPUs such as A100 / H100. The system will automatically detect and adaptively configure parameters based on the GPU's computing power and VRAM capacity.

[0103] CPU Requirements: Multi-core processor, 8 cores or more recommended. The number of CPU cores directly affects the parallelism of the concurrent acquisition pool and the post-processing thread pool.

[0104] Memory Requirements: 16GB or more of system memory is recommended. Thanks to the spatial lifecycle memory management mechanism, even edge devices with 16GB of memory can handle ultra-large-scale images at the terabyte level.

[0105] Software environment: CUDA Toolkit 11.0 or later, cuDNN deep learning acceleration library, Python 3.8 or later, deep learning frameworks such as PyTorch or TensorFlow, and GDAL geographic data processing library.

[0106] Key data structure definitions: RefTable uses a Hash Map structure to store reference counts, supporting O(1) time complexity for lookup and update; OffloadQueue uses a bounded priority queue structure for back pressure control, supporting thread-safe blocking enqueue and dequeue operations.

[0107] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this invention shall still fall within the scope of the technical solution of this invention. All other aspects not described in detail belong to the prior art.

Claims

1. A high-throughput, large-scale remote sensing AI inference system based on asynchronous decoupling architecture and space lifecycle management, characterized in that, It includes four asynchronous subsystems connected via a backpressure queue: out-of-order preparation and ordered submission subsystem, GPU computing and data transmission decoupling subsystem, spatial lifecycle memory management subsystem, and lock-free local fusion subsystem; The out-of-order preparation and ordered submission subsystem includes a concurrent acquisition pool, a ready buffer, and a sequential feed controller. The concurrent acquisition pool has a shared concurrent thread pool that reads remote sensing tile data in parallel from heterogeneous data sources. The ready buffer adopts a thread-safe hash table structure to temporarily store tensor data that has been downloaded but is not arranged in spatial order. The sequential feed controller has a global sequence pointer with an initial value of zero. The controller monitors the ready buffer in a high-frequency loop. When the tile data corresponding to the global sequence pointer is ready, it is retrieved and pushed into the batch aggregator. Subsequently, the pointer is incremented. The GPU computing and data transmission decoupling subsystem includes a GPU master thread, an unloading queue, and a multi-threaded unloading pool. The GPU master thread is the only thread in the system that holds the GPU context. This thread executes the forward propagation of the deep learning model in a loop. After obtaining the result tensor, it encapsulates the pointer reference of the tensor in the video memory into a task object and pushes it into the unloading queue. The unloading queue is a bounded queue with a maximum capacity limit. The multi-threaded unloading pool includes multiple CPU threads. Each thread obtains a task object from the unloading queue and performs a blocking Device-to-Host data copy. The spatial lifecycle memory management subsystem includes a spatial dependency graph, a reference counting table, and a global cache. Before the inference task starts, the spatial dependency graph uses spatial indexes to pre-calculate the spatial topological relationships between each sliding window and identifies a set of neighboring windows with physical overlap for each window. The reference counting table maintains an atomic counter for each window, and the initial value of the counter is the number of downstream neighboring windows that depend on the data of that window for fusion processing. The global cache is used to store the probability graph data returned from the unloading thread. In the lock-free local fusion subsystem, each post-processing thread applies for a local canvas in its private space. Each thread obtains the data of the center window and its overlapping neighbor windows from the global cache in read-only mode. Weighted fusion is performed on the local canvas according to the Gaussian weight matrix. After fusion, normalization processing, classification mask generation and contour tracking are performed sequentially on the local canvas to extract vector polygons.

2. The system according to claim 1, characterized in that, The sequential feed controller executes blocking and releasing logic: when the tile data pointed to by the global sequence pointer has not yet been stored in the ready buffer, even if there is subsequent tile data with a larger sequence number in the buffer, the controller still blocks the submission operation and enters a spin-waiting state; when the tile data corresponding to the global sequence pointer is ready, the controller takes it out of the buffer and pushes it into the batch aggregator, then increments the pointer by 1, and the output data is arranged in row priority order to form a spatially continuous tensor stream.

3. The system according to claim 2, characterized in that, The GPU master thread employs a zero-wait submission mechanism: after obtaining ordered batches from the batch aggregator, it calls the asynchronous memory copy interface to upload the data to the GPU memory and executes model forward propagation. After obtaining the result tensor, it does not call the synchronous data return interface, but only obtains the pointer reference of the result tensor in the GPU memory address space and encapsulates it into a task object and pushes it into the unloading queue. The threads in the multi-threaded unloading pool run in parallel with the GPU master thread. When the unloading thread performs the blocking return of the Nth batch of data through the PCIe bus, the GPU master thread simultaneously executes the model forward propagation calculation of the N+1th batch. The two form a pipeline-like parallel overlap on the time axis.

4. The system according to claim 3, characterized in that, The construction of the spatial dependency graph includes: the system receiving the target geographic range and model input size, calculating the number of grid rows and columns required to cover the target survey area according to the sliding step size, generating a list of globally unique window identifiers arranged in row priority order, using R-Tree spatial index to pre-calculate the spatial relationship between each window and surrounding windows and identify the set of neighboring windows with physical overlap, the physical overlap area being the scope of subsequent fusion processing; in the initialization of the reference count table, for each window, all its downstream dependencies are traversed and the counter of that window is incremented by the corresponding value.

5. The system according to claim 4, characterized in that, The spatial lifecycle memory management subsystem executes wavefront destruction logic: after the post-processing thread completes the vectorization extraction of the current window, the system queries the spatial dependency graph to obtain all upstream neighbor windows that have provided fusion data for the current window, performs an atomic decrement operation on the reference count of each upstream window, and then checks the count value of each upstream window. The data of windows with counts reaching zero is immediately physically released, and the data of the currently computed wavefront region and its related windows are retained in the system memory; the weighted fusion in the lock-free local fusion subsystem adopts a Gaussian weight matrix, and the fusion formula is: In the formula, Vlocal is the fusion result value at coordinate (x, y) on the local canvas, Vcenter is the probability value of the center window at that coordinate, Vneighbor is the probability value of the neighboring windows, Gcenter and Gneighbor are the Gaussian weight coefficients at the corresponding positions, (x, y) is the position in the coordinate system of the center window, (x′, y′) is the mapping position of the neighboring window data in the coordinate system of the local canvas, N(center) is the set of all neighboring windows that physically overlap with the center window, and k is the index of the neighboring window; After fusion, normalization and Argmax operations are performed sequentially to generate a classification mask. Then, the contour tracking algorithm is called to extract the vector polygon boundaries. The generated vector objects are sent to the main thread for storage through a thread-safe channel, and the local canvas memory is automatically released when the function returns.

6. The system according to claim 5, characterized in that, The heterogeneous data sources include network WMS tile services, network TMS tile services, local GeoTIFF files, HDF5 files, or NumPy array formats. The system adapts to different data formats through an abstract data source interface layer. The spatial indexes include R-Tree indexes, quadtree indexes, or adjacency matrices based on grid partitioning. The deep learning models include semantic segmentation models, object detection models, or change detection models that require dual-temporal image input. The asynchronous decoupling architecture remains transparent to the network structure inside the model.

7. A high-throughput, large-scale remote sensing AI inference method based on asynchronous decoupling architecture and space lifecycle management, characterized in that, Includes the following steps: The system receives the boundary parameters of the target geographic area and the input size required by the deep learning model. Based on the set sliding step size, it calculates the number of grid rows and columns required to cover the target survey area, generates a list of globally unique window identifiers arranged in row priority, uses spatial index to pre-calculate the spatial relationship between each sliding window and surrounding windows, and identifies the set of neighboring windows with physical overlap to construct a spatial dependency graph. It synchronously initializes the reference count table and sets the initial value of the atomic counter of each window to the number of downstream neighboring windows that depend on the data of that window for fusion processing. A concurrent thread pool is started to acquire remote sensing tile data from heterogeneous data sources in parallel. All acquired tile data is stored in a thread-safe ready buffer. The sequential feed controller maintains a global sequence pointer with an initial value of zero and monitors the buffer status in a high-frequency loop. When the tile data corresponding to the global sequence pointer is ready, it is taken out of the buffer and pushed into the batch aggregator. Then the pointer is incremented and the output data is arranged in row priority order. The GPU master thread, as the only thread in the system holding the GPU context, obtains ordered batches from the batch aggregator, calls the asynchronous memory copy interface to upload the data to the video memory, and executes the forward propagation of the deep learning model. After obtaining the result tensor, it does not call the synchronous data return interface, but only obtains the pointer reference of the tensor in the video memory address space and encapsulates it into a task object and pushes it into the bounded unloading queue. Each thread in the multi-threaded unloading pool obtains the task object from the unloading queue and performs a blocking Device-to-Host data copy. The GPU master thread and the unloading thread form a pipeline-like parallel operation on the timeline. The probabilistic graph data returned to system memory from the unloading thread is stored in the global cache. After the post-processing thread completes the vectorization extraction of the current window, the system queries the spatial dependency graph to obtain all upstream neighbor windows that have provided fused data for the current window, performs an atomic decrement operation on the reference count of each upstream window, and the data of the window whose count reaches zero is immediately physically released. The post-processing thread requests a local canvas in its private space, retrieves the probabilistic map data of the center window and its overlapping neighbor windows from the global cache in read-only mode, performs weighted fusion on the local canvas according to the Gaussian weight matrix, and after fusion, performs normalization processing and Argmax operation in sequence to generate a classification mask, calls the contour tracking algorithm to extract the vector polygon boundary, and sends the generated vector object to the main thread for storage through a thread-safe channel.

8. The method according to claim 7, characterized in that, The sequential feed controller executes blocking and releasing logic: when the tile data pointed to by the global sequence pointer has not yet been stored in the ready buffer, even if there is subsequent tile data with a larger sequence number in the buffer, the controller still blocks the submission operation and enters a spin-wait state. When the tile data corresponding to the global sequence pointer is ready, it is retrieved and pushed into the batch aggregator, and then the pointer is incremented by 1. Due to the random delay in network transmission, tiles with larger spatial indices may be downloaded before tiles with smaller indices. The blocking and releasing logic reassembles the out-of-order I / O data stream in the time domain into a tensor stream arranged in row priority order in the spatial domain.

9. The method according to claim 8, characterized in that, The GPU master thread employs a zero-wait submission mechanism. This mechanism requires that after completing the model forward propagation, the master thread does not call methods to convert GPU tensors to CPU tensors or to convert GPU tensors to array objects. Instead, it obtains a pointer reference to the result tensor in the video memory address space and encapsulates this reference into a task object, pushing it into the unloading queue. The bounded unloading queue has a capacity limit. When the queue reaches its maximum capacity, the GPU master thread's push operation is blocked until a free position appears in the queue, forming a backpressure control mechanism. This backpressure control mechanism prevents video memory resources from being exhausted due to insufficient downstream processing power. When the unloading thread performs the blocking backhaul of the Nth batch of data through the PCIe bus, the GPU master thread simultaneously performs the model forward propagation calculation of the N+1th batch.

10. The method according to claim 9, characterized in that, After the post-processing thread completes the vectorization extraction of the current window, the system queries the spatial dependency graph to obtain a list of all upstream neighbor windows, performs an atomic decrement operation on the reference count entries of each upstream window, and then checks the count value of each upstream window. The data of windows with counts of zero are immediately physically cleared by calling the memory release interface, and the data of the currently computed wavefront region and its related windows are retained in the system memory.

Images