Architecture system and method for large-scale task allocation

Abstract

The invention discloses an architecture system and method for large-scale task allocation, and belongs to the field of spaceflight measurement and control, and the system comprises a large-scale hardware device which is used for deploying processing units with different performances and various heterogeneous physical communication link resources; the hardware resource dynamic modeling and visualization framework is used for realizing a dynamic modeling system based on multi-source heterogeneous hardware topology perception; the resource topology awareness database captures and synchronizes system deployment changes by maintaining the dynamic topological relation of hardware resources, presents the life cycle state of the hardware resources, and supports system-level resource situation visualization and decision making; a versioning management engine of component arrangement is applied, a multi-version executable unit is packaged based on an atomization component, and an application instance is completed through a combined modeling component; and deploying a scheduling decision system for deploying a scheduling decision. The method can adapt to task deployment requirements of a current large-scale spaceflight measurement and control system.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of spaceflight TT&C technology, and more particularly to a large-scale task allocation architecture system and method. BACKGROUND

[0002] With the development of deep space exploration, constellation networking and other multi-task parallelism, the task density of spaceflight TT&C systems is growing exponentially, exposing the adaptability crisis of multi-modal task hardware precise adaptation architecture in multi-functional spaceflight TT&C networks. The most significant problem is that it causes insufficient utilization of hardware resources, such as long-term high-value equipment idling due to the use of fixed task mapping mode, or high-speed data transmission tasks being blocked due to high-performance hardware resources being occupied by medium and low-speed data transmission tasks, etc. Therefore, it is necessary to consider the task deployment architecture research for large-scale spaceflight TT&C systems. SUMMARY

[0003] The present application aims to overcome the shortcomings of the prior art and provide a large-scale task allocation architecture system and method that can adapt to the task deployment needs of current large-scale spaceflight TT&C systems.

[0004] The purpose of the present application is achieved by the following scheme: A large-scale task allocation architecture system, comprising: a large-scale hardware device, a hardware resource dynamic modeling and visualization framework, a resource topology awareness database, a version management engine for application component orchestration, and a deployment scheduling decision system; The large-scale hardware device includes a plurality of computing nodes, each computing node constitutes an integrated subsystem, including a plurality of functional modules for performing core computing tasks, a management and control module for node state monitoring and task coordination, an exchange module for processing internal and external data interaction of the node, an exchange module for processing high-speed data interaction between the node and the outside, and peripheral devices; The hardware resource dynamic modeling and visualization framework is used to realize a dynamic modeling system based on multi-source heterogeneous hardware topology awareness; The resource topology awareness database is used to maintain the dynamic topology relationship of hardware resources, capture and synchronize system deployment changes, present the life cycle state of hardware resources, and support resource situation visualization and decision-making at the system level; The version management engine for application component orchestration is used to encapsulate multiple versions of executable units based on atomized components, and to complete the application instance through combined modeling components; The deployment scheduling decision system is used for deployment scheduling decision.

[0005] Further, each corresponding hardware module of the large-scale hardware device is deployed with processing units of different performance and a variety of heterogeneous physical communication link resources, forming a hierarchical computing and communication infrastructure with rich heterogeneous resources to realize efficient collaboration and optimized resource utilization.

[0006] Further, the deployment scheduling decision system comprises a scheduling decision unit and a task deployment unit; the scheduling decision unit is configured to obtain component metadata from a versioned management engine of application component orchestration through a standardized API based on application identification submitted by the deployment unit, obtain a resource state snapshot from a resource topology awareness database and make a resource decision; the task deployment unit is configured to extract application identification and submit it to the scheduling decision unit based on an application request issued by a user, inject executable bit streams into a hardware computing node through a collaborative interface protocol based on a set of component deployment parameters issued, establish an asynchronous deployment confirmation channel and realize atomic deployment closed loop with full-link energy observation.

[0007] Further, the dynamic modeling system based on multi-source heterogeneous hardware topology awareness is implemented by means of visual resource view acquisition through the types, main frequencies, memory sizes, communication capabilities, connection relationships and operation speed information of each hardware processing unit, including a hardware information modeling module, a node capability dynamic registration module and a dynamic resource topology visualization module. The hardware information modeling module is configured to accurately model hardware information resources by defining inter-slot interconnection relationships, integrating multi-chip instances and their interconnection topologies, including recording communication paths between processing units, processing unit and connector communication architectures, connector physical locations and communication protocol attributes; The node capability dynamic registration module is configured to periodically report processing unit type identifiers, on-chip and on-board storage resource capacities and access bandwidths, operation core numbers and configurations, working frequency ranges and peak computing power indicators, allocated input and output channel bandwidth thresholds through an in-band management channel of a board deployed in a chassis, and real-time analyze hardware resource configuration parameter changes; The dynamic resource topology visualization module is configured to integrate hardware information modeling data and node capability dynamic registration real-time data and dynamically update a global topology relationship graph, and observe heterogeneous physical communication link resource information deployed on different hardware boards in the topology relationship graph in real time; for each link, the topology graph displays a link unique identifier, a physical protocol type, a measured data transmission rate, an endpoint interconnection topology relationship, a real-time available state flag and a remaining effective transmission bandwidth.

[0008] Further, the life cycle state of the hardware resource comprises available, idle and fault.

[0009] Further, the encapsulating multiple versions of executable units based on the atomized component, through the combined modeling component, the complete application instance is included: for cross-component deployment constraints, the dynamic implantation of collaborative scheduling strategy is realized, the task deployment and resource topology adaptation are realized, and through the dynamic decoupling of version, constraint and resource, the decision unit is driven to perform multi-dimensional strategy binding deployment operation.

[0010] Further, the deployed component deployment parameter set specifically includes hardware capability fingerprint and topology constraint.

[0011] A construction method of a resource topology-aware database of a large-scale task allocation architecture system, based on any one of the above large-scale task allocation architecture systems, using a resource management module to perform the following steps: Step S1, node capability feature dynamic registration: online sensing and registering key capability parameters of each heterogeneous processing unit in the system, the key capability parameters including static and dynamic parameters; Step S2, communication link resource topology accurate mapping: scanning and mapping heterogeneous physical communication link resources deployed on different hardware boards; for each link, recording its link unique identifier, physical protocol type, and measured data transmission rate, endpoint interconnection topology relationship, real-time available state flag, and remaining effective transmission bandwidth; Step S3, normalizing and abstractly expressing the original resource data collected in steps S1 and S2 via a resource modeling engine to form a unified resource constraint matrix and a communication cost model, and synchronously updating to the persistent resource state database embedded in the task deployment and scheduling module.

[0012] Further, in step S1, the key capability parameters at least include: processing unit type identifier, on-chip and on-board storage resource capacity and access bandwidth, number and configuration of operation cores, working frequency range and peak computing power indicator, and assignable input / output channel bandwidth threshold.

[0013] Further, in step S2, the heterogeneous physical communication link resources specifically include LVDS differential pair channels, Aurora protocol links, Ethernet, RS422 / 485 serial buses, PCIe exchange channels, RapidIO interconnection architecture links, and CAN PD controllers.

[0014] The beneficial effects of the present application include: By using the scheme of the present application, a continuously evolving resource topology-aware database can be provided, which provides accurate and real-time hardware resource configuration and load view in the whole system range for deployment and scheduling algorithms, and can constitute a bottom data support foundation for realizing adaptive dynamic task resource mapping and collaborative scheduling decision according to inter-task communication dependency relationship and multi-dimensional resource demand constraint, which meets the quality of service (Qos) indicator. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a system architecture diagram of an embodiment of the present invention. Detailed Implementation

[0017] All features disclosed in all embodiments of this specification, or steps in all methods or processes implied in the disclosure, may be combined and / or extended or replaced in any way, except for mutually exclusive features and / or steps.

[0018] The specific implementation process of this invention is as follows: As a first aspect of the present invention, such as Figure 1 As shown, an architecture system for large-scale task allocation is provided, including: large-scale hardware devices, a dynamic modeling and visualization framework for hardware resources, a resource topology-aware database, a versioned management engine for application component orchestration, and a deployment scheduling decision system. The technical implementations of these parts are detailed below: Large-scale hardware devices are comprised of a vast number of computing nodes at their core. Each computing node constitutes an integrated subsystem, containing several functional modules that execute core computing tasks, a management module responsible for node status monitoring and task coordination, a switching module that handles data exchange between the node and external systems, a switching module that handles high-speed data exchange between the node and external systems, and necessary peripheral devices such as power supply and cooling. These hardware units integrate diverse processing resources, including not only processing units with varying performance but also physical communication link resources with different characteristics and connection methods (such as LVDS, Aurora, Ethernet, RS422, RapidIO, etc.). This design aims to form a hierarchical computing and communication infrastructure with abundant heterogeneous resources to achieve efficient collaboration and optimized resource utilization.

[0019] A hardware resource dynamic modeling and visualization framework is used to implement a dynamic modeling system based on multi-source heterogeneous hardware topology awareness. It obtains a visualized resource view by managing information including, but not limited to, the type, clock speed, memory size, communication capabilities, connectivity, and processing speed of each hardware processing unit. It comprises three parts: hardware information modeling, dynamic node capability registration, and dynamic resource topology visualization.

[0020] Hardware information modeling is to define the interconnection relationship between slots, integrate multi-chip instances and their interconnection topology, including recording the communication path between processing units, the communication architecture of processing units and connectors (such as distinguishing Ethernet / RapidIO exchange topology, etc.), the physical location of connectors and the communication protocol attributes (accurate to the physical layer standard, such as 1000Base-X / SFP+ or 1000Base-T / RJ45), to realize accurate modeling of hardware information resources. The functions of dynamic registration of node capabilities are: through the in-band management channel deployed in the chassis, the board card periodically reports the processing unit type identifier, on-chip and on-board storage resource capacity and access bandwidth, the number of operation cores and configuration, the working frequency range and peak computing power indicators, the assignable input / output channel bandwidth threshold, and changes in hardware resource configuration parameters in real time. Dynamic resource topology visualization integrates hardware information modeling data and real-time data of dynamic registration of node capabilities and dynamically updates the global topology relationship diagram. In the topology relationship diagram, the information of heterogeneous physical communication link resources (such as LVDS differential pair channel, Aurora protocol link, Ethernet, RS422 / 485 serial bus, PCIe exchange channel, RapidIO interconnection architecture link, CAN PD controller, etc.) deployed on different hardware board cards can be observed in real time. For each link, the topology diagram displays the unique identifier of the link, the physical protocol type, the measured data transmission rate, the endpoint interconnection topology relationship, the real-time available state flag, and the remaining effective transmission bandwidth.

[0021] The resource topology awareness database continuously maintains the dynamic topology relationship of hardware resources, actively captures and synchronizes system deployment changes, accurately presents the real-time full life cycle state of hardware resources (such as available, idle, fault), and supports resource situation visualization and decision-making at the system level.

[0022] The version management engine of application component orchestration is used to encapsulate multiple versions of executable units based on atomic components, and to complete the application instance through combined modeling components. For cross-component deployment constraints, dynamic cooperative scheduling strategies are implanted to realize automatic adaptation of task deployment and resource topology. Through the dynamic decoupling of version-constraint-resource, the engine can drive the decision unit to accurately perform multi-dimensional strategy binding deployment operations.

[0023] The deployment scheduling decision system comprises a scheduling decision unit and a task deployment unit. The scheduling decision unit in the deployment scheduling decision system is configured to obtain component metadata from a version management engine of an application component arrangement through a standardized API based on an application identifier submitted by the deployment unit, obtain a resource state snapshot from a resource topology awareness database, and perform a resource decision process. The task deployment unit in the deployment scheduling decision system is configured to extract an application identifier based on an application request issued by a user and submit the application identifier to the scheduling decision unit. Based on a submitted component deployment parameter set (including a hardware capability fingerprint and a topology constraint), executable bit streams are injected into hardware computing nodes through a collaborative interface protocol. Finally, an asynchronous deployment confirmation channel is established to realize atomic deployment of a full-link observable closed loop.

[0024] As a second aspect of the present application, a resource topology awareness database construction method of a large-scale task allocation architecture system is provided, comprising the following steps: Step S1, computing node capability feature dynamic registration: online sensing and registration of key capability parameters of various heterogeneous processing units (such as FPGA, ZYNQ, PPC) in the system, which at least include: processing unit type identifier, on-chip and on-board storage resource capacity and access bandwidth, number and configuration of operation cores, working frequency range and peak computing power indicator, and assignable input and output channel bandwidth threshold. These parameters cover static attributes and monitorable dynamic load states. Static and dynamic parameters such as memory capacity and bandwidth, peak frequency / computing power, and available I / O bandwidth.

[0025] Step S2, accurate mapping of communication link resources: scanning and mapping of heterogeneous physical communication link resources (such as LVDS differential pair channels, Aurora protocol links, Ethernet, RS422 / 485 serial buses, PCIe exchange channels, RapidIO interconnection architecture links, and CAN PD controllers) deployed on different hardware boards. For each link, record its: link unique identifier, physical protocol type, and measured data transmission rate, endpoint interconnection topology relationship, real-time available state flag, and remaining effective transmission bandwidth.

[0026] Step S3, the original resource data collected in steps S1 and S2 are normalized and abstractly expressed through a resource modeling engine to form a unified resource constraint matrix and a communication cost model, and are synchronously updated in real time to a persistent resource state database embedded in a task deployment scheduling module.

[0027] The units described in the embodiments of the present application can be implemented in a software manner, or can be implemented in a hardware manner, and the described units can also be arranged in a processor. In some cases, the names of these units do not constitute a limitation on the units themselves.

[0028] According to one aspect of the present invention, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations described above.

[0029] In another aspect, embodiments of the present invention also provide a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

Claims

1. An architecture system for large-scale task allocation, characterized in that, include: A large-scale hardware device and hardware resource dynamic modeling and visualization framework, a resource topology-aware database, a version management engine for application component orchestration, and a deployment and scheduling decision system; The large-scale hardware device includes multiple computing nodes, each computing node constituting an integrated subsystem, which includes multiple functional modules for executing core computing tasks, a management and control module for node status monitoring and task coordination, an exchange module for handling data interaction between the node and the outside world, an exchange module for handling high-speed data interaction between the node and the outside world, and peripheral devices. The hardware resource dynamic modeling and visualization framework is used to realize a dynamic modeling system based on multi-source heterogeneous hardware topology awareness. The resource topology-aware database is used to capture and synchronize system deployment changes by maintaining the dynamic topology relationship of hardware resources, present the life cycle status of hardware resources, and support system-level resource status visualization and decision-making. The version management engine for the application component orchestration is used to encapsulate multiple versions of executable units based on atomic components, and to complete application instances through composite modeling components. The deployment and scheduling decision system is used for deployment and scheduling decisions.

2. The architecture system for large-scale task allocation according to claim 1, characterized in that, Each corresponding hardware module of the large-scale hardware device is equipped with processing units of different performance and various heterogeneous physical communication link resources, forming a hierarchical computing and communication infrastructure with abundant heterogeneous resources, so as to achieve efficient collaboration and optimized resource utilization.

3. The architecture system for large-scale task allocation according to claim 1, characterized in that, The deployment scheduling decision system includes a scheduling decision unit and a task deployment unit; the scheduling decision unit is used to obtain component metadata from the version management engine of the application component orchestration through a standardized API based on the application identifier submitted by the deployment unit, obtain resource status snapshots from the resource topology awareness database, and make resource decisions. The task deployment unit is used to extract the application identifier based on the application request issued by the user and submit it to the scheduling decision unit. Based on the distributed component deployment parameter set, an executable bit stream is injected into the hardware computing node through a collaborative interface protocol to establish an asynchronous deployment confirmation channel, thereby achieving an atomic deployment closed loop that is observable across the entire link.

4. The architecture system for large-scale task allocation according to claim 1, characterized in that, The aforementioned implementation of a dynamic modeling system based on multi-source heterogeneous hardware topology awareness is specifically achieved by managing the type, clock frequency, memory size, communication capability, connection relationship, and computing speed information of each hardware processing unit and obtaining a visualized resource view. This includes a hardware information modeling module, a node capability dynamic registration module, and a dynamic resource topology visualization module. The hardware information modeling module is used to define the interconnection relationship between slots, integrate multiple chip instances and their interconnection topology, including recording the communication path between processing units, the communication architecture between processing units and connectors, the physical location of connectors and communication protocol attributes to achieve accurate modeling of hardware information resources; The node capability dynamic registration module is used to periodically report the processing unit type identifier, on-chip and onboard storage resource capacity and access bandwidth, number and configuration of computing cores, operating frequency range and peak computing power index, and allocated input / output channel bandwidth threshold of the boards deployed in the chassis through the in-band management channel, and to parse changes in hardware resource configuration parameters in real time. The dynamic resource topology visualization module is used to integrate hardware information modeling data and node capability dynamic registration real-time data and dynamically update the global topology diagram. In the topology diagram, heterogeneous physical communication link resource information deployed on different hardware boards can be observed in real time. For each link, the topology diagram displays the link's unique identifier, physical protocol type, measured data transmission rate, endpoint interconnection topology, real-time availability status flag, and remaining effective transmission bandwidth.

5. The architecture system for large-scale task allocation according to claim 1, characterized in that, The lifecycle status of the hardware resources includes available, idle, and faulty.

6. The architecture system for large-scale task allocation according to claim 1, characterized in that, The aforementioned multi-version executable unit encapsulated based on atomic components, and the complete application example through composite modeling components, includes: dynamically embedding collaborative scheduling strategies for cross-component deployment constraints, realizing task deployment and resource topology adaptation, and driving the decision unit to execute multi-dimensional strategy-bound deployment operations through dynamic decoupling of versions, constraints and resources.

7. The architecture system for large-scale task allocation according to claim 1, characterized in that, The issued component deployment parameter set specifically includes hardware capability fingerprints and topology constraints.

8. A method for constructing a resource topology-aware database for a large-scale task allocation architecture system, characterized in that, The architecture system based on any one of claims 1 to 7 for large-scale task allocation performs the following steps using the resource management module: Step S1, Dynamic Registration of Computing Node Capability Features: Online sensing and registration of key capability parameters of each heterogeneous processing unit in the system, including static and dynamic parameters; Step S2, accurate mapping of communication link resource topology: scan and map heterogeneous physical communication link resources deployed on different hardware boards; for each link, record its unique link identifier, physical protocol type, measured data transmission rate, endpoint interconnection topology, real-time availability status flag, and remaining effective transmission bandwidth; Step S3: The raw resource data collected in steps S1 and S2 is normalized and abstracted by the resource modeling engine to form a unified resource constraint matrix and communication cost model, and is synchronously updated to the persistent resource status database embedded in the task deployment and scheduling module.

9. The method for constructing a resource topology-aware database for a large-scale task allocation architecture system according to claim 8, characterized in that, In step S1, the key capability parameters include at least: processing unit type identifier, on-chip and onboard storage resource capacity and access bandwidth, number and configuration of computing cores, operating frequency range and peak computing power index, and allocable input / output channel bandwidth threshold.

10. The method for constructing a resource topology-aware database for a large-scale task allocation architecture system according to claim 8, characterized in that, In step S2, the heterogeneous physical communication link resources specifically include LVDS differential pair channel, Aurora protocol link, Ethernet, RS422 / 485 serial bus, PCIe switching channel, RapidIO interconnect architecture link, and CAN PD controller.

Images