A heterogeneous resource management method of a containerized platform of an underwater acoustic signal processor

Abstract

The application discloses a heterogeneous resource management method of a water acoustic signal processor containerization platform, and belongs to the technical field of the water acoustic signal processor containerization platform.The method comprises the following steps: constructing a unified capability representation resource pool, quantizing a heterogeneous computing unit hardware architecture and a water acoustic benchmark algorithm performance; analyzing a water acoustic task to generate an acoustic processing logic graph containing nodes and a dependency relationship; matching and mapping according to an algorithm type and a real-time constraint of each node in the graph, combining a resource pool performance and a load state, generating a task-resource mapping scheme; instantiating a processing container according to the scheme, connecting the container according to a data flow through a software-defined network and configuring storage, forming a distributed computing pipeline; and monitoring performance and triggering dynamic rescheduling and reconstruction during operation.The application realizes fine scheduling of heterogeneous resources, intelligent deployment and adaptive optimization of a task pipeline, and improves the efficiency, real-time performance guarantee and resource utilization rate of water acoustic signal processing.

Description

Technical Field

[0001] This invention belongs to the technical field of containerized platforms for underwater acoustic signal processors, and specifically relates to a heterogeneous resource management method for a containerized platform for underwater acoustic signal processors. Background Technology

[0002] Underwater acoustic signal processing is a key technology in fields such as ocean observation, underwater communication, and target detection and identification. With increasing task complexity (such as broadband signal processing, multi-target tracking, and deep learning-enhanced detection), processing algorithms are becoming increasingly complex, leading to exponential growth in computational demands. Traditional dedicated underwater acoustic signal processors are typically based on fixed hardware architectures (such as DSP arrays, dedicated ASICs, or single-model GPUs), resulting in inherent drawbacks such as system rigidity, difficulty in algorithm updates, low resource utilization, and long development and deployment cycles.

[0003] In recent years, containerization and cloud-native technologies have provided new deployment paradigms for compute-intensive applications due to their advantages in environment consistency, resource isolation, and elastic scaling. Containerizing underwater acoustic signal processing tasks and running them on heterogeneous computing platforms composed of general-purpose CPUs, GPUs, FPGAs, NPUs, etc., is considered an important direction for improving system flexibility, utilization, and iteration speed. However, existing general-purpose containerization platforms (such as systems based on Kubernetes and its ecosystem) and their resource management methods have significant shortcomings when dealing with the specific domain of underwater acoustic signal processing, mainly in the following aspects:

[0004] Insufficient awareness and quantification of the computing power characteristics of heterogeneous hardware: Existing platforms typically only manage the number of CPU cores and memory size in an abstract way. For dedicated acceleration devices such as GPUs and FPGAs, allocation is limited to a simple "number of devices," lacking a fine-grained description of the computing power characteristics of these devices. Different core algorithms in underwater acoustic processing (such as FFT, beamforming, and convolutional neural networks) exhibit significant differences in performance and power consumption across different hardware architectures. Existing methods lack a benchmarking and quantification system tailored to the characteristics of underwater acoustic computing, failing to provide precise data support for intelligent scheduling on "which algorithm runs better on which hardware," resulting in coarse resource allocation and difficulty in maximizing the energy efficiency of heterogeneous hardware.

[0005] The task scheduling and heterogeneous resource matching are not highly intelligent: Underwater acoustic processing tasks typically present as multi-stage data flow pipelines, with different algorithms at each stage and varying requirements for real-time performance and throughput. Existing container scheduling strategies are mostly based on simple resource requests (e.g., requiring two GPUs), failing to understand the internal data flow graph of the task, the dependencies between processing stages, and the real-time constraints of each stage. The scheduling process often ignores network latency caused by data exchange between computing nodes and lacks optimization mechanisms to schedule processing stages with close data dependencies to neighboring nodes to reduce communication overhead. This easily leads to increased overall task processing latency, failing to meet the stringent end-to-end latency requirements of underwater acoustic systems (especially real-time sonar processing).

[0006] The system lacks adaptive reconfiguration capabilities for dynamic underwater acoustic processing environments: Underwater acoustic signal processing environments are dynamic and time-varying; fluctuations in input data rate, target density, and channel conditions can lead to drastic changes in processing load. Existing platforms rely on static or threshold-triggered elastic scaling, which is slow to react and typically limited to increasing or decreasing the number of replicas. They cannot perform fine-grained, cross-heterogeneous computing unit dynamic rescheduling and pipeline reconfiguration targeting performance bottlenecks during task execution. When a processing stage becomes a bottleneck due to hardware performance degradation or a sudden increase in load, the entire pipeline performance suffers, and the system lacks online self-healing and performance maintenance capabilities.

[0007] In summary, existing general-purpose containerized platform resource management methods fail to deeply integrate the specialized knowledge of underwater acoustic signal processing tasks, such as algorithmic heterogeneity, data flow pipeline characteristics, strong real-time constraints, and dynamic environments. This results in significant shortcomings in heterogeneous computing resource utilization, task processing timeliness, system adaptability, and overall energy efficiency. Therefore, there is an urgent need for a heterogeneous resource management method specifically designed for containerized underwater acoustic signal processing platforms to address these issues. Summary of the Invention

[0008] In view of this, the purpose of this invention is to provide a heterogeneous resource management method for a containerized platform of underwater acoustic signal processors.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] A method for managing heterogeneous resources in a containerized platform for underwater acoustic signal processing includes the following steps:

[0011] S1: Build a unified capability representation resource pool for all heterogeneous computing units managed by the containerization platform, and label each heterogeneous computing unit with its basic hardware architecture, supported instruction set types, type and number of dedicated acceleration devices, and its performance data under the preset underwater acoustic signal processing benchmark operator set.

[0012] S2: Receive underwater acoustic signal processing tasks, parse the task description file of the underwater acoustic signal processing tasks to obtain its acoustic processing logic diagram. The acoustic processing logic diagram includes multiple processing nodes and the data flow and processing stage dependencies between each processing node. Each processing node corresponds to a core underwater acoustic signal processing algorithm.

[0013] S3: Based on the algorithm type, real-time constraints and data throughput requirements of each processing node in the acoustic processing logic diagram, match and map them with the real-time load status and performance data of each heterogeneous computing unit in the capability characterization resource pool, schedule and allocate matching heterogeneous computing units for each processing node, and generate a task-resource mapping scheme.

[0014] S4: Based on the task-resource mapping scheme, instantiate the corresponding processing containers on the allocated heterogeneous computing units, connect each processing container according to the data flow direction of the acoustic processing logic diagram through software-defined networking technology, and configure the corresponding storage access paths for them to form a physically distributed computing pipeline.

[0015] S5: During the operation of the computing pipeline, continuously monitor the resource utilization and data processing latency of each processing container. When a performance bottleneck is detected, trigger dynamic rescheduling and reconfiguration of the pipeline based on the current capacity characterization of the resource pool status.

[0016] As a further preferred embodiment of the present invention, the construction of a unified capability representation resource pool in S1 specifically includes:

[0017] S11: By deploying device probes on each computing node of the platform, basic hardware architecture data of each heterogeneous computing unit contained in each computing node is collected. The basic hardware architecture data includes CPU instruction set architecture, operating system type, memory capacity, and the model, video memory or onboard memory capacity and driver version information of dedicated acceleration devices such as GPU, FPGA or NPU.

[0018] S12: When the heterogeneous computing unit is first connected or the hardware configuration is changed, drive it to run a standardized underwater acoustic signal processing benchmark test package. The package includes beamforming, pulse compression, matched filtering and constant false alarm rate detection algorithms. Record the average processing latency, peak throughput and processing efficiency per unit energy consumption of each algorithm on the heterogeneous computing unit to generate performance data.

[0019] S13: Link and store basic hardware architecture data with performance data to form a capability profile of each heterogeneous computing unit, and aggregate all capability profiles to build a capability representation resource pool.

[0020] As a further preferred embodiment of the present invention, S2 specifically includes:

[0021] S21: Receive underwater acoustic signal processing tasks submitted in the form of a structured description file, which includes the task's processing flow, the algorithms involved, and the data interfaces.

[0022] S22: Parse the structured description file, identify and extract multiple atomic operations that constitute the core computational process of the task, and map and encapsulate each atomic operation into an independent processing node;

[0023] S23: Based on the execution order and data transfer relationship defined in the structured description file, establish directional data flow and processing stage dependencies between processing nodes to generate a preliminary topology.

[0024] S24: For each processing node, based on its encapsulated atomic operations, match and label the specific type of the corresponding underwater acoustic signal processing core algorithm from the preset underwater acoustic algorithm library.

[0025] S25: Based on the performance declaration in the structured description file or the preset algorithm type-constraint rule library, mark the real-time constraints and data throughput requirements for processing nodes and data flow directions;

[0026] S26: Based on the labeled processing nodes, the data and dependencies between them, as well as the algorithm type and constraints, generate an acoustic processing logic diagram that can be parsed by subsequent scheduling steps.

[0027] As a further preferred embodiment of the present invention, the matching mapping in step S3 specifically includes:

[0028] S31: For each processing node in the acoustic processing logic diagram, determine the required core computational operation based on the type of its acoustic processing core algorithm, and query the corresponding preferred hardware type and instruction set from the preset mapping rule base according to the computational operation.

[0029] S32: Based on the real-time constraints of the processing nodes, select heterogeneous computing units from the capability characterization resource pool that have a current load rate lower than the first threshold and meet the requirements of preferred hardware type to form the first candidate set;

[0030] S33: For each heterogeneous computing unit in the first candidate set, calculate the benchmark performance index corresponding to the algorithm of the current processing node in its performance data, and predict the expected completion time of processing the current node by combining the current queue length of the heterogeneous computing unit's pending tasks.

[0031] S34: Select the allocation combination that minimizes the completion time of the entire acoustic processing logic graph. For adjacent processing nodes with data dependencies, map them to the same computing node or different computing nodes with network latency below the second threshold, and generate a task-resource mapping scheme.

[0032] As a further preferred embodiment of the present invention, step S4, which instantiates the processing container, connects the container, and configures the storage access path according to the task-resource mapping scheme, specifically includes:

[0033] S41: Based on the task-resource mapping scheme generated by S3, obtain the heterogeneous computing unit identifier, the required container image identifier, and its startup configuration parameters allocated to each processing node;

[0034] S42: On each heterogeneous computing unit specified by the mapping scheme, pull the corresponding container image and instantiate the processing container, and inject the algorithm identifier, data input / output endpoint information and resource quota parameters of the corresponding processing node into each processing container;

[0035] S43: Based on the data flow defined in the acoustic processing logic diagram, establish virtual network links between each processing container through a software-defined network controller, configure link bandwidth, transmission protocol and quality of service policies to ensure that data transmission meets real-time constraints.

[0036] S44: Mount and configure the corresponding storage access path for each processing container, including configuring a read-only access path for the raw underwater acoustic data for the data input node container, a read-write path for the temporary data storage for the intermediate processing node container, and a persistent storage path for the result data for the final output node container.

[0037] S45: After completing the instantiation, network connection and storage configuration of all processing containers, start the containers in sequence according to the topological order of the acoustic processing logic diagram to form a physically distributed and logically coherent computing pipeline, and report the pipeline readiness status.

[0038] As a further preferred embodiment of the present invention, triggering dynamic rescheduling and reconfiguration of the pipeline in S5 specifically includes:

[0039] S51: When it is detected that the data processing latency of a processing container continuously exceeds its preset latency threshold, the processing container is determined to be a performance bottleneck container.

[0040] S52: Analyze the resource usage of the performance bottleneck container. If the CPU or dedicated acceleration device utilization of the computing unit where it is located is consistently higher than the third threshold, it is determined to be a computing resource bottleneck.

[0041] S53: If the bottleneck is computing resources, then find a backup computing unit in the capacity representation resource pool that matches the bottleneck container algorithm and whose current load rate is lower than the fourth threshold.

[0042] S54: Create a new processing container instance on the standby computing unit and migrate the necessary context state from the original bottleneck container to the new container through a data synchronization mechanism;

[0043] S55: Switch the data flow from the original bottleneck container to the new container, and gradually terminate the original bottleneck container to complete the pipeline refactoring.

[0044] As a further preferred embodiment of the invention, it also includes preemptive scheduling and resource reservation for underwater acoustic processing tasks with deadline constraints:

[0045] Before generating the task-resource mapping scheme in S3, identify the set of processing nodes marked as critical real-time paths in the acoustic processing logic diagram. The end-to-end processing latency of this set must meet the preset deadline.

[0046] For the processing nodes included in the critical real-time path, heterogeneous computing units and network bandwidth resources that meet their performance and time constraints are pre-locked in the capability representation resource pool to form a resource reservation area;

[0047] When there is a conflict between a pre-defined non-critical task request and a reserved resource, the scheduler prioritizes the allocation of the reserved resource area and performs interruptible preemption or queuing for the conflicting non-critical task.

[0048] The beneficial effects of this invention are as follows:

[0049] 1. This invention achieves refined perception and quantitative characterization of heterogeneous hardware computing power, laying a data foundation for intelligent scheduling: By constructing a unified capability characterization resource pool, it not only collects basic hardware architecture information but also obtains real performance and energy consumption data of various heterogeneous computing units (CPU, GPU, FPGA, NPU) on underwater acoustic core algorithms by running standardized underwater acoustic signal processing benchmark test packages. This solves the problem of "insufficient perception and quantification of heterogeneous hardware computing power characteristics" in the background technology, transforming the abstract "number of devices" management into "capability profile" management based on specific algorithm performance, providing a reliable quantitative basis for subsequent accurate matching and energy efficiency optimization.

[0050] 2. This invention improves the intelligence and optimization level of task scheduling and heterogeneous resource matching, ensuring processing timeliness: By parsing tasks to generate acoustic processing logic diagrams containing algorithm types, real-time constraints, and data dependencies, and performing multi-target matching mapping based on a capability representation resource pool, the scheduling process can understand the data flow and performance requirements within the task. This method not only considers the performance matching between algorithms and hardware but also optimizes the layout of processing nodes with data dependencies on physical nodes (such as mapping to the same node or low-latency nodes), effectively reducing communication overhead. This addresses the shortcomings of the background technology, namely "low intelligence level of task scheduling and heterogeneous resource matching" and "ignoring network latency," thereby significantly shortening the end-to-end processing latency of tasks and meeting the strict real-time requirements of underwater acoustic systems.

[0051] 3. This invention endows the system with online adaptive reconfiguration and self-healing capabilities for dynamic environments: by continuously monitoring performance and triggering dynamic rescheduling and reconfiguration during pipeline operation, when a performance bottleneck caused by sudden load changes or hardware performance degradation is detected, the system can automatically find backup computing units in the resource pool, migrate processing containers, and switch data streams. This overcomes the limitations of existing platforms in the background technology, which are "based on static or threshold-triggered elastic scaling, resulting in lag" and "unable to perform fine-grained dynamic rescheduling," achieving rapid response to performance bottlenecks and online pipeline optimization, significantly improving the system's robustness and performance maintenance capabilities in dynamic, time-varying underwater acoustic environments.

[0052] 4. This invention enhances the guarantee capability for critical real-time tasks: by preemptively scheduling and reserving resources for tasks with deadline constraints, resources that meet the performance and time constraints of the core processing path are pre-locked. This solves the problem that critical tasks may be delayed due to resource contention in general scheduling, ensures deterministic latency of the most critical processing links in applications such as sonar, and improves the task guarantee level of the system under complex loads.

[0053] In summary, this invention, by deeply integrating domain knowledge of underwater acoustic signal processing (algorithm features, pipeline models, real-time constraints) with platform technologies such as containerization and software-defined networking, forms a complete management closed loop from resource representation, task parsing, intelligent mapping to dynamic reconstruction. This effectively overcomes the limitations of existing general methods and significantly improves the overall performance of the containerized platform for underwater acoustic signal processing in terms of heterogeneous resource utilization, task processing timeliness, and system adaptability.

[0054] Other advantages, objectives, and features of the invention will be set forth in the following description and will be apparent to those skilled in the art in some respects, or may be learned by practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0055] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration:

[0056] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0057] Figure 2 This is a schematic diagram of the process of S2 in this invention;

[0058] Figure 3 This is a schematic diagram of the process of S3 in this invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Generally, the embodiments of the present invention described and shown in the accompanying drawings can be combined with each other in any way without conflict. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0060] like Figure 1-3 As shown, the platform described in this invention consists of multiple physical computing nodes, each of which may contain one or more heterogeneous computing units, such as general-purpose CPUs, GPUs, FPGAs, or NPUs. The core of the management platform is a centralized resource scheduler responsible for executing the methods described below.

[0061] S1: Construct a unified capability representation resource pool.

[0062] The resource scheduler first needs to create a unified resource profile model for all available heterogeneous computing units within the platform. To this end, four types of information are labeled for each computing unit:

[0063] Basic hardware architecture: such as x86_64 or ARM CPU architecture, Linux or real-time operating system type.

[0064] Instruction set type: such as AVX-512, CUDA Core, OpenCL supported versions.

[0065] Dedicated acceleration equipment: such as the model and quantity of NVIDIA A100 GPUs or the model of Xilinx Alveo FPGAs.

[0066] Performance data: obtained by running a preset set of underwater acoustic signal processing benchmark operators (including core algorithms such as beamforming, pulse compression, matched filtering, and constant false alarm rate detection), recording the average delay of each operator. Peak throughput

[0067] (MB / s) and processing efficiency per unit energy consumption (GOPS / W).

[0068] The above information from all heterogeneous computing units is aggregated to form a global, queryable capability representation resource pool, whose data structure can be represented as a set of key-value pairs.

[0069] This step transforms heterogeneous hardware from an abstract "device" into a "capability carrier" with clear algorithmic performance characteristics, providing a quantifiable data foundation for subsequent precise scheduling based on algorithmic characteristics and solving the problem of extensive resource awareness.

[0070] S2: Analyze the task and generate an acoustic processing logic diagram.

[0071] The system receives underwater acoustic signal processing tasks submitted by users in structured description files such as YAML or JSON. The scheduler parses the file and extracts the computational logic of the task. The task is decomposed into multiple atomic operations (such as "perform a 1024-point FFT" or "perform beamforming calculations"), and each atomic operation is encapsulated as a processing node. Based on the execution order and data transfer relationships in the description file, directed data flow edges and processing stage dependency edges are established between processing nodes, thereby generating an acoustic processing logic graph. Among them, vertex set Represents the processing node, each edge Indicates from node arrive Data flow and dependencies. For each node... Label the corresponding core algorithm type for underwater acoustic signal processing. Real-time constraints (maximum allowable processing delay) and data throughput requirements .

[0072] This step transforms the user task into a machine-understandable graph model containing rich domain semantics (algorithm type, real-time constraints, data dependencies), enabling the scheduler to optimize from the perspective of the overall task flow, rather than allocating resources in isolation.

[0073] S3: Graph-based intelligent matching mapping to generate task-resource mapping schemes.

[0074] The scheduler takes the acoustic processing logic graph G and the real-time refreshed resource pool status as input for optimized matching. For each processing node in the graph... According to its algorithm type The system queries a pre-defined mapping rule base to obtain the preferred set of hardware types. (e.g., "FFT algorithm preferentially maps to GPU"). Next, the current load rate is selected from the resource pool. Below the threshold (e.g., 70%), and the hardware type belongs to Heterogeneous computing units constitute the candidate set .for Each candidate unit in Combined with its performance data, the algorithm reference delay and the current length of the pending task queue Predict execution nodes Expected completion time:

[0075]

[0076] in, It is the average latency for this unit to process similar tasks.

[0077] The scheduler's optimization objective is to minimize the critical path completion time while providing support for adjacent nodes with data dependencies. Prioritize network latency Below the threshold The computing unit for Perform the mapping. The final output will be a task-resource mapping scheme. ,in It is a collection of heterogeneous computing units.

[0078] This step achieves fine-grained matching of algorithm and hardware and dependency-aware layout optimization. By predicting completion time and taking network latency into account, it effectively reduces overall task processing latency and data transmission overhead, significantly improving scheduling quality and timeliness.

[0079] Step S4: Instantiate the container and build the distributed computing pipeline.

[0080] According to the mapping scheme The scheduler performs containerized deployment on the target heterogeneous computing units. For each mapped node... and its allocated computing units Pull the pre-defined, algorithm-containing Implement a Docker image of the code. Instantiate a processing container. This involves injecting configuration parameters, such as IP ports for data input / output and resource limits (CPU cores, GPU memory, etc.). A software-defined network controller (such as Calico or Cilium) is then used to process the acoustic processing logic graph G according to its edges. Establish virtual network links between containers and configure QoS policies to guarantee bandwidth and low latency for these links. Mount appropriate storage volumes for the containers; for example, mount read-only raw acoustic data volumes for source node containers, high-speed temporary storage volumes for intermediate nodes, and persistent result storage volumes for output nodes.

[0081] All containers are started in the topological order of G, forming a physically distributed and logically coherent computing pipeline. Data starts from the input container, flows through each processing container along the virtual links established by SDN, and finally outputs the result.

[0082] This step physically implements the optimized scheduling scheme, using container technology to achieve environmental isolation and rapid deployment, and SDN technology to achieve flexible and high-performance pipeline connections, forming a truly operational distributed processing system.

[0083] S5: Runtime monitoring and dynamic rescheduling refactoring.

[0084] During pipeline operation, monitoring probes deployed on each compute node continuously collect data from each processing container. Resource utilization (CPU utilization) GPU utilization ) and actual data processing delay .

[0085] When a container is detected The delay continues to exceed its threshold When this occurs, the dynamic refactoring process is triggered:

[0086] Bottleneck diagnosis: If or (e.g., 90%) is identified as a bottleneck in computing resources.

[0087] Finding backup units: Searching for units in the capability representation resource pool that are compatible with... Algorithm matching and current load (e.g., 30%) of spare heterogeneous computing units .

[0088] State transition and switching: in Create a new container instance Through the checkpoint mechanism The necessary computational context is synchronized to Then, the SDN data stream is transferred from... Smooth transition to Finally terminated .

[0089] This step endows the system with online self-healing and adaptive optimization capabilities. It can quickly respond to load fluctuations or hardware anomalies, eliminating performance bottlenecks while maintaining service continuity through fine-grained container migration rather than simple replica scaling, greatly improving the system's robustness in dynamic underwater environments.

[0090] The detailed construction of the capability representation resource pool, and the specific implementation of S1.

[0091] S11 (Data Acquisition): Deploys lightweight device probes (DaemonSets) on each compute node. The probes communicate via system calls ( lscpu , nvidia-smiIt uses hardware driver APIs to periodically collect data such as architecture, memory, acceleration device model, and driver version.

[0092] S12 (Benchmark): When a new device is added or hardware is updated, the scheduler automatically sends out benchmark packages. These packages call highly optimized CUDA, OpenCL, or VHDL kernels to execute standard underwater acoustic algorithms. Performance data is calculated using formulas, such as energy efficiency per unit of power. ,in It is the total number of operands. This is the average power consumption. This is the total execution time.

[0093] S13 (Profile Building and Aggregation): This step associates collected static data with dynamic performance data from testing to create capability profiles. All profiles are aggregated in the scheduler and provided for querying via a REST API.

[0094] The acoustic processing logic diagram is generated in detail, and the specific implementation of S2 is explained in detail.

[0095] S21-S23 (Parsing and Graphing): Using a parser (such as PyYAML) to read the structured description file and identify... stages , dependencies Using fields such as [field name], a preliminary topology graph is constructed.

[0096] S24 (Algorithm Labeling): Maintains an underwater acoustic algorithm library containing metadata such as algorithm name and computational characteristics (e.g., computationally intensive, memory-intensive). Labels each processing node with the specific algorithm type using string matching or feature matching.

[0097] S25 (Constraint Annotation): Based on the description file deadline Field or query algorithm type - constraint rule base (e.g., "beamforming delay should be <10ms"), to label constraints for nodes and edges.

[0098] S26 (Logical Graph Generation): Outputs a graph structure object (such as a NetworkX Graph object) containing complete metadata for use by the scheduler.

[0099] The optimization strategy for intelligent matching mapping is explained in detail, along with the optimization matching process of S3.

[0100] The matching problem can be formalized as a constrained optimization problem. The optimization objective is to minimize the critical path length:

[0101]

[0102] in, This is the communication overhead, which is proportional to network latency and the amount of data transmitted. Constraints include: For real-time nodes, This problem can be solved using heuristic algorithms (such as list scheduling and genetic algorithms) to obtain a better mapping scheme. .

[0103] Detailed deployment of containerized pipelines, including detailed instructions on S4 deployment.

[0104] S41-S42 (Container Deployment): The scheduler calls the Kubernetes API to create a Pod that carries a nodeSelector (specifying the heterogeneous computing unit) and resource limits. Configuration is injected using Init Container.

[0105] S43 (Network Configuration): Through the CNI plugin interface, call the SDN controller API to create virtual network policies that meet the constraints of bandwidth Breq and maximum latency Lmax.

[0106] S44 (Storage Configuration): Mounts the appropriate type of PersistentVolumeClaim to the Pod via the CSI driver.

[0107] S45 (Pipeline Startup): The scheduler sends startup commands to Kubernetes sequentially according to the graph topology, ensuring that dependencies start after data providers.

[0108] The failover mechanism for dynamic rescheduling and reconfiguration is explained in detail, along with the reconfiguration process of S5.

[0109] For stateful processing nodes, log replication or differential checkpointing techniques are employed. Let the state of the original container `ck` be `S`, and the incremental state generated during synchronization be `ΔS`. During migration, a snapshot of `S` is first transferred to the new node, then `ΔS` is applied within a second-level time window, and finally the data stream is switched. This process ensures exactly-once semantics for data processing.

[0110] Preemptive scheduling of critical real-time tasks provides a high-priority task guarantee mechanism.

[0111] For critical real-time paths Pcrit marked with a deadline Tdeadline, the scheduler performs resource reservation before regular mapping:

[0112] 1. Calculate the sum of the worst-case execution times (WCET(Pcrit)) for all nodes on the path.

[0113] 2. In the capability characterization resource pool, find heterogeneous computing unit combinations that can satisfy WCET(Pcrit)≤Tdeadline, and mark their status as reserved.

[0114] 3. During the scheduling cycle, reserved resources are not allocated to non-critical tasks. When a non-critical task's request conflicts with a reserved resource, the scheduler either places it in a waiting queue or preempts its allocated resources and allocates them to a critical task.

[0115] This mechanism ensures the determinism and reliability of high-priority underwater acoustic processing tasks, such as active sonar detection.

[0116] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.

Claims

1. A heterogeneous resource management method for a containerized platform of underwater acoustic signal processors, characterized in that, Includes the following steps: S1: Construct a unified capability representation resource pool for all heterogeneous computing units managed by the containerization platform, labeling each heterogeneous computing unit with its basic hardware architecture, supported instruction set types, type and quantity of dedicated acceleration devices, and performance data under a preset underwater acoustic signal processing benchmark operator set; S2: Receive underwater acoustic signal processing tasks, parse the task description file of the underwater acoustic signal processing tasks to obtain their acoustic processing logic diagrams, which include multiple processing nodes and the data flow and processing stage dependencies between each processing node, where each processing node corresponds to a core underwater acoustic signal processing algorithm; S3: Based on the algorithm type, real-time constraints, and data throughput requirements of each processing node in the acoustic processing logic diagram... The task-resource mapping scheme is generated by matching and mapping the real-time load status and performance data of each heterogeneous computing unit in the capability representation resource pool to schedule and allocate matching heterogeneous computing units for each processing node; S4: Based on the task-resource mapping scheme, the corresponding processing container is instantiated on the allocated heterogeneous computing unit, and each processing container is connected according to the data flow direction of the acoustic processing logic diagram through software-defined networking technology, and the corresponding storage access path is configured for it to form a physically distributed computing pipeline; S5: During the operation of the computing pipeline, the resource utilization and data processing latency of each processing container are continuously monitored. When a performance bottleneck is detected, the pipeline is dynamically rescheduled and reconstructed according to the current capability representation resource pool status. The construction of a unified capability representation resource pool in S1 specifically includes: S11: Collecting basic hardware architecture data of each heterogeneous computing unit contained in each computing node through device probes deployed on each computing node of the platform. The basic hardware architecture data includes CPU instruction set architecture, operating system type, memory capacity, and the model, video memory or onboard memory capacity and driver version information of dedicated acceleration devices such as GPU, FPGA or NPU; S12: When a heterogeneous computing unit is first connected or its hardware configuration is changed, driving it to run a standardized underwater acoustic signal processing benchmark test package. The package includes beamforming, pulse compression, matched filtering and constant false alarm rate (CFAR) detection algorithms, recording the average processing latency, peak throughput and processing efficiency per unit of energy consumption of each algorithm on the heterogeneous computing unit, and generating performance data; S13: Associating and storing the basic hardware architecture data and performance data to form a capability profile of each heterogeneous computing unit, and aggregating all capability profiles to construct a capability representation resource pool. The matching mapping in S3 specifically includes: S31: For each processing node in the acoustic processing logic graph, determine the required core computing operations based on the type of its acoustic processing core algorithm, and query the corresponding hardware type and instruction set from the preset mapping rule base according to the computing operations; S32: Based on the real-time constraints of the processing nodes, select heterogeneous computing units with current load rates below the first threshold and meeting hardware type requirements from the capability representation resource pool to form a first candidate set; S33: For each heterogeneous computing unit in the first candidate set, calculate the benchmark performance index corresponding to the algorithm of the current processing node in its performance data, and predict its expected completion time for processing the current node by combining the current task queue length of the heterogeneous computing unit; S34: Select the allocation combination that minimizes the completion time of the entire acoustic processing logic graph. For adjacent processing nodes with data dependencies, map them to the same computing node or different computing nodes with network latency below the second threshold, and generate a task-resource mapping scheme.

2. The heterogeneous resource management method for a containerized platform of underwater acoustic signal processor according to claim 1, characterized in that: S2 specifically includes: S21: Receive underwater acoustic signal processing tasks submitted in the form of a structured description file, which includes the task's processing flow, involved algorithms, and data interfaces; S22: Parse the structured description file, identify and extract multiple atomic operations that constitute the core computational flow of the task, and map and encapsulate each atomic operation into an independent processing node; S23: Based on the execution order and data transfer relationships defined in the structured description file, establish directional data flow and processing stage dependencies between processing nodes, generating a preliminary topology; S24: For each processing node, based on its encapsulated atomic operations, match and label the specific type of its corresponding underwater acoustic signal processing core algorithm from a preset underwater acoustic algorithm library; S25: Based on the performance declaration in the structured description file or the preset algorithm type-constraint rule library, label the processing nodes and data flow with real-time constraints and data throughput requirements; S26: Based on the labeled processing nodes, the data and dependencies between them, and the algorithm type and constraints, generate an acoustic processing logic diagram that can be parsed by subsequent scheduling steps.

3. A heterogeneous resource management method for a containerized platform of underwater acoustic signal processor according to claim 1, characterized in that: S4 instantiates processing containers, connects containers, and configures storage access paths according to the task-resource mapping scheme. Specifically, this includes: S41: Based on the task-resource mapping scheme generated in S3, obtaining the heterogeneous computing unit identifier allocated to each processing node, the required container image identifier, and its startup configuration parameters; S42: On each heterogeneous computing unit specified by the mapping scheme, pulling the corresponding container image and instantiating the processing container, injecting the algorithm identifier, data input / output endpoint information, and resource quota parameters of its corresponding processing node into each processing container; S43: According to the data flow direction defined in the acoustic processing logic diagram, the software-defined network controller connects the processing containers... Establish virtual network links between them, configure link bandwidth, transmission protocol and quality of service policies to ensure data transmission meets real-time constraints; S44: Mount and configure corresponding storage access paths for each processing container, including configuring read-only access paths for raw underwater acoustic data for data input node containers, configuring read-write paths for temporary data storage for intermediate processing node containers, and configuring persistent storage paths for result data for final output node containers; S45: After completing the instantiation, network connection and storage configuration of all processing containers, start the containers sequentially according to the topological order of the acoustic processing logic diagram to form a physically distributed and logically coherent computing pipeline, and report the pipeline readiness status.

4. A heterogeneous resource management method for a containerized platform of underwater acoustic signal processor according to claim 1, characterized in that: In S5, triggering dynamic pipeline rescheduling and reconfiguration specifically includes: S51: When it is detected that the data processing latency of a processing container continuously exceeds its preset latency threshold, the processing container is determined to be a performance bottleneck container. S52: Analyze the resource usage of the performance bottleneck container. If the CPU or dedicated acceleration device utilization of the computing unit where it is located is consistently higher than the third threshold, it is determined to be a computing resource bottleneck. S53: If the bottleneck is computing resources, then find a backup computing unit in the capacity representation resource pool that matches the bottleneck container algorithm and whose current load rate is lower than the fourth threshold. S54: Create a new processing container instance on the standby computing unit and migrate the necessary context state from the original bottleneck container to the new container through a data synchronization mechanism; S55: Switch the data flow from the original bottleneck container to the new container, and gradually terminate the original bottleneck container to complete the pipeline refactoring.

5. A heterogeneous resource management method for a containerized platform of underwater acoustic signal processor according to claim 1, characterized in that: It also includes preemptive scheduling and resource reservation for underwater acoustic processing tasks with deadline constraints: before generating the task-resource mapping scheme in S3, the set of processing nodes marked as critical real-time paths in the acoustic processing logic diagram is identified, and the end-to-end processing delay of this set must meet the preset deadline. For the processing nodes included in the critical real-time path, heterogeneous computing units and network bandwidth resources that meet their performance and time constraints are pre-locked in the capability representation resource pool to form a resource reservation area; When there is a conflict between a pre-defined non-critical task request and a reserved resource, the scheduler prioritizes the allocation of the reserved resource area and performs interruptible preemption or queuing for the conflicting non-critical task.

Images