Element scheduling method, device and equipment based on storage-calculation separation architecture of computing power network

Abstract

The invention discloses an element scheduling method, device and equipment based on a storage-calculation separation architecture of a computing power network. The method comprises the following steps: receiving a computing power resource application, computing power state perception and computing power discovery request from an internal service through a standardized API (Application Program Interface) provided by a computing power network service layer, and deploying the computing power network service layer in an optical shutter outer area and docking with a computing power network; after receiving the computing power resource application, scheduling a to-be-executed task based on the task queue, and allocating computing resources for the task according to a preset resource allocation strategy; acquiring historical computing power task execution data and extracting time features, resource features, business features and environment features; and inputting the extracted feature data into a pre-trained prediction model to obtain a prediction result of the task resource demand, and optimizing resource scheduling according to the prediction result. According to the embodiment of the invention, the resource utilization rate and the task execution efficiency are improved.

Description

Technical Field

[0001] This application belongs to the field of big data processing technology, and in particular relates to a component scheduling method, apparatus and equipment based on a computing power network storage-computing separation architecture. Background Technology

[0002] With the acceleration of data elementization and the popularization of artificial intelligence applications, data processing tasks, especially the development, processing, and trading of data elements, are exhibiting dynamic, heterogeneous, and large-scale demands for computing resources. A data element refers to a structured or unstructured dataset or data feature formed after data has undergone de-identification, modeling, and other processing.

[0003] Traditional computing architectures typically employ a compute-in-memory (CIM) model, where compute nodes simultaneously handle both computing and storage functions. This tightly coupled architecture suffers from the following inherent drawbacks: compute and storage resources are interdependent, resulting in low resource utilization. Furthermore, when business growth necessitates expansion, both compute and storage resources must be scaled up simultaneously, leading to resource waste and increased costs. Task scheduling often fails to adequately consider data locality, network topology, and cross-cluster resource conditions, resulting in high data migration overhead, high task execution latency, and weak cross-domain collaboration capabilities. Summary of the Invention

[0004] This application provides a component scheduling method, apparatus, and device based on a computing power network storage-computing separation architecture.

[0005] In a first aspect, embodiments of this application provide a component scheduling method based on a computing power network storage-computing separation architecture, including: Through the standardized API interface provided by the computing power network service layer, it receives computing power resource applications, computing power status awareness and computing power discovery requests from internal services. The computing power network service layer is deployed in the area outside the optical gate and is connected to the computing power network. Upon receiving a request for computing resources, the system schedules the tasks to be executed based on the task queue and allocates computing resources to the tasks according to a preset resource allocation configuration strategy, which includes a guarantee mechanism for resource upper and lower limits. Acquire historical computing power task execution data, and extract the time characteristics, resource characteristics, business characteristics, and environmental characteristics of the historical computing power task execution data; The time characteristics, resource characteristics, business characteristics, and environmental characteristics are input into a pre-trained prediction model to obtain the prediction results of task resource requirements, and resource scheduling is optimized based on the prediction results.

[0006] Optionally, the standardized API interfaces provided by the computing power network service layer include at least: The computing power application interface is used to apply for CPU, memory, and storage resources according to task requirements; The computing power awareness interface is used to monitor the liveness status of computing power nodes and notify events. The computing power discovery interface is used to proactively detect available computing power nodes when resources are insufficient.

[0007] Optionally, the computing power discovery interface is specifically used to actively detect available computing power nodes by deploying resource probes on computing power network nodes to collect and report node resource information in a periodic or event-driven manner.

[0008] Optionally, the resource allocation configuration strategy is defined in the form of a configuration file to ensure that each task or tenant is allocated guaranteed resources no less than the resource minimum, and to dynamically allocate additional resources when the actual needs of a task exceed the guaranteed resources, but not exceeding the resource maximum; the remaining resources can be used by other task queues.

[0009] Optionally, the method further includes: transmitting computing power tasks via a data channel based on the UDP protocol.

[0010] Optionally, the transmission of computing power tasks via a data channel based on the UDP protocol includes: The data sender divides the original data packet into fragments according to a preset fragment size, and constructs a packet header for each data fragment containing a session ID, sequence number, total number of fragments, current fragment index, data checksum, timestamp, and encryption identifier; Encryption algorithms are used to encrypt the data fragment content and key fields in the packet header; A sliding window mechanism is used for data transmission. The window size is configurable, and the data retransmission strategy is dynamically adjusted based on network conditions. The data receiver reassembles data fragments based on the session ID and sequence number, verifies data integrity, and decrypts the data.

[0011] Secondly, embodiments of this application provide a component scheduling device based on a computing power network storage-computing separation architecture, comprising: The receiving module is used to receive computing power resource applications, computing power status awareness and computing power discovery requests from internal services through the standardized API interface provided by the computing power network service layer. The computing power network service layer is deployed in the area outside the optical gate and is connected to the computing power network. The allocation module is used to schedule tasks to be executed based on the task queue after receiving a computing resource request, and allocate computing resources to the tasks according to a preset resource allocation configuration strategy, which includes a guarantee mechanism for resource upper limit and resource lower limit. The extraction module is used to acquire historical computing power task execution data and extract the time characteristics, resource characteristics, business characteristics and environmental characteristics of the historical computing power task execution data; The optimization module is used to input the time features, resource features, business features and environmental features into a pre-trained prediction model to obtain the prediction results of task resource requirements, and optimize resource scheduling based on the prediction results.

[0012] Thirdly, embodiments of this application provide an electronic device, which includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the steps of the component scheduling method based on the computing power network storage-computing separation architecture as described in any embodiment of the first aspect.

[0013] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the steps of the component scheduling method based on a computing power network storage-computing separation architecture as described in any embodiment of the first aspect.

[0014] Fifthly, embodiments of this application provide a computer program product, which is stored in a storage medium and executed by at least one processor to implement the steps of the component scheduling method based on a computing power network storage-computing separation architecture provided in the first aspect of embodiments of this application.

[0015] The component scheduling method, apparatus, and device based on the computing power network storage-computing separation architecture of this application embodiment rely on the access method provided by the computing power network to design and develop a computing power network service layer. It connects to the computing power network with a facade design pattern, shielding the complex use of the computing power network and integrating computing power network-related content through a unified interface. By combining task queues with resource upper and lower limit guarantee strategies, it achieves fair allocation and efficient utilization of resources. Furthermore, it utilizes a machine learning model based on historical data for dynamic resource prediction and scheduling, making resource allocation more in line with actual task needs and improving resource utilization and task execution efficiency. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating a component scheduling method based on a computing power network storage-compute separation architecture provided in an embodiment of this application; Figure 2 This is a diagram illustrating the connection architecture between the computing power network service layer and the computing power network provided in this application embodiment; Figure 3 This is a schematic diagram of the structure of a component scheduling device based on a computing power network storage-computing separation architecture provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0018] Figure label: The component scheduling device 300 based on a computing power network storage-computing separation architecture comprises a receiving module 301, an allocation module 302, an extraction module 303, and an optimization module 304. Electronic device 400, processor 401, memory 402, communication interface 403, bus 410. Detailed Implementation

[0019] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0020] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0021] It should be noted that the acquisition, storage, use, and processing of data in this application embodiment all comply with the relevant provisions of national laws and regulations.

[0022] With the acceleration of data elementization and the popularization of artificial intelligence applications, data processing tasks, especially the development, processing, and trading of data elements, are exhibiting dynamic, heterogeneous, and large-scale demands for computing resources. A data element refers to a structured or unstructured dataset or data feature formed after data has undergone de-identification, modeling, and other processing.

[0023] Traditional computing architectures typically employ a compute-in-memory (CIM) model, where compute nodes simultaneously handle both computing and storage functions. This tightly coupled architecture suffers from the following inherent drawbacks: compute and storage resources are interdependent, resulting in low resource utilization. Furthermore, when business growth necessitates expansion, both compute and storage resources must be scaled up simultaneously, leading to resource waste and increased costs. Task scheduling often fails to adequately consider data locality, network topology, and cross-cluster resource conditions, resulting in high data migration overhead, high task execution latency, and weak cross-domain collaboration capabilities.

[0024] Specifically, server utilization is insufficient because storage and computing are integrated. Storage mainly consumes resources such as hard drives, while computing mainly consumes resources such as CPU and memory. Their respective characteristics cannot be fully utilized. For example, GPUs consume a lot of resources for AI computing, but storage consumption is very low. Expansion is limited. For example, increasing computing resources also requires simultaneous expansion of storage resources. There is no unified management for computing unstructured data elements, making it difficult to control resource consumption when processing these elements.

[0025] Computing power network refers to a new type of computing resource organization and scheduling architecture. Its core idea is to integrate scattered computing resources (such as CPU, GPU, FPGA, etc.), storage resources and network resources into a unified resource pool through software definition, and achieve global optimization through intelligent scheduling.

[0026] The compute-in-memory separation architecture refers to a new computing architecture design that physically and logically decouples computing resources from storage resources. Its core idea breaks away from the traditional tightly coupled model of computing and storage in servers, connecting distributed computing nodes, memory resources, and storage devices into a unified resource pool through high-speed networks, enabling independent expansion and dynamic combination of resources.

[0027] As can be seen, storage-compute separation allows for separate management of storage and computing, significantly improving their respective utilization rates. Furthermore, storage and computing can be upgraded independently without interference. After connecting to a computing network, computing resources can be dynamically acquired through the network's resource scheduling, and the complex network topology is shielded from the network's constraints.

[0028] To address the problems in the related technologies, this application provides a component scheduling method, apparatus, and device based on a computing power network storage-computing separation architecture.

[0029] The component scheduling method based on the computing power network storage-computing separation architecture provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0030] Figure 1 A flowchart illustrating a component scheduling method based on a computing power network-in-memory separation architecture, according to an embodiment of this application, is shown. Figure 1As shown, the component scheduling method based on the computing power network storage-computing separation architecture may specifically include the following steps: S101. Through the standardized API interface provided by the computing power network service layer, receive computing power resource applications, computing power status awareness and computing power discovery requests from internal services. The computing power network service layer is deployed in the area outside the optical shutter and is connected to the computing power network. S102. After receiving a computing resource request, the task to be executed is scheduled based on the task queue, and computing resources are allocated to the task according to the preset resource allocation configuration strategy. The resource allocation configuration strategy includes a guarantee mechanism for the upper and lower limits of resources. S103. Obtain historical computing power task execution data, and extract the time characteristics, resource characteristics, business characteristics, and environmental characteristics of the historical computing power task execution data; S104. Input the time features, resource features, business features and environmental features into the pre-trained prediction model to obtain the prediction results of task resource requirements, and optimize resource scheduling based on the prediction results.

[0031] Therefore, relying on the access method provided by the computing power network, a computing power network service layer was designed and developed. It connects to the computing power network with a facade design pattern, shielding the complex use of the computing power network and integrating relevant content of the computing power network through a unified interface. By combining task queues with resource upper and lower limit guarantee strategies, fair allocation and efficient utilization of resources are achieved. Furthermore, machine learning models based on historical data are used for dynamic resource prediction and scheduling, making resource allocation more in line with the actual needs of tasks and improving resource utilization and task execution efficiency.

[0032] The specific implementation methods for each of the above steps are described below.

[0033] refer to Figure 2 This is a diagram illustrating the connection architecture between the computing power network service layer and the computing power network. (See diagram for example.) Figure 2 As shown, under the data element system, the component development and component trading related services are mainly deployed in the area outside the optical gate, which can be connected to the Internet computing power network or a self-built private domain computing power network; and the component development platform and the component trading platform do not directly call external computing power resources, but rather encapsulate and abstract a layer of computing power service.

[0034] In some embodiments, for a component development platform, the main use of computing resources is for component model development and component model debugging; for a component trading platform, the main use of computing resources is for transaction-related algorithm services and fusion processing calculations in the component result data delivery stage.

[0035] In some embodiments, the external service of the optical gate adds a computing power network service layer and a unified interface layer that connects to the computing power network, providing interfaces such as computing power application, computing power awareness, and computing power discovery. The internal service obtains computing power capabilities by connecting to the computing power network service layer and submits them to the computing power network for computation in the form of tasks.

[0036] Specifically, the standardized API interfaces provided by the computing power network service layer include at least: a computing power application interface, used to apply for CPU, memory, and storage resources according to task requirements; a computing power awareness interface, used to monitor the liveness status of computing power nodes and notify events; and a computing power discovery interface, used to proactively detect available computing power nodes when resources are insufficient.

[0037] As an optional implementation, the computing power service layer provides standardized API interfaces that support RESTful calls. Key interfaces include: ` / computeRes / apply`, with main input parameters including the number of CPU cores, memory size, and storage resource size; and ` / computeRes / monitor`, with main input parameters including machine model, machine name, processor architecture, operating system information, number of CPU cores, memory size, disk size, and whether a GPU is included. This interface uses a background heartbeat mechanism to ensure the availability of computing resources and a background periodic inspection method to ensure the stability of computing resources. The core technical logic involves recording the liveness status of machine services through temporary nodes in the ZooKeeper component, combined with a watch mechanism. Listeners are set up on nodes, and corresponding event notifications are generated when nodes change, thus achieving the ability to detect changes. Computing power discovery ( / computeRes / discover) is mainly used to proactively explore available computing power resources when computing power resources are insufficient. Its main implementation method is to deploy computing power resource probes (Agents) on the network nodes of the computing power network. The probes collect resource node information and report it level by level to the master node probe of the computing power node, i.e., periodic reporting, or event-driven reporting to the node probe. These two methods can be configured through the system. The default method is periodic reporting. In other words, the computing power discovery interface is specifically used to actively detect available computing power nodes by deploying resource probes on computing power network nodes to collect and report node resource information in a periodic or event-driven manner.

[0038] In some embodiments, the downstream of the computing power service layer connects to the computing power domain, which mainly comprises a three-layer architecture: a resource hardware layer, a computing power infrastructure layer, and a computing power scheduling layer. These three layers form a complete chain from physical resources to computing power services. Specifically, the resource hardware layer, as the physical foundation of the computing power system, carries the material basis of computing power capabilities. Through the pooled management of heterogeneous hardware resources, it constructs a high-density, high-reliability, and high-performance computing power foundation, primarily used to manage computing power hardware resources through hardware virtualization and resource pooling technologies. The computing power infrastructure layer, as the abstraction and encapsulation layer of hardware resources, transforms physical resources into standardized computing power service units through software-defined technologies, constructing a computing power delivery platform. Its main technologies are service abstraction, resource orchestration, and elastic architecture scaling. The computing power scheduling layer, as the brain of the computing power system, achieves unified scheduling and optimization across layers, domains, and resources through a global resource view and intelligent algorithms, maximizing computing power resource utilization and business value. Its core technologies are scheduling algorithms and scheduling visualization, strategic resource allocation, and cross-domain collaboration.

[0039] In some embodiments, in S102, the resource allocation configuration strategy is defined in the form of a configuration file to ensure that each task or tenant is allocated a guaranteed resource not lower than the resource lower limit, and to dynamically allocate additional resources when the actual needs of the task exceed the guaranteed resources, but not exceeding the resource upper limit; the remaining resources can be used by other task queues.

[0040] In practical implementation, the computing power service layer achieves load balancing by deploying multiple service instances. It uses Nacos' Feign call method to obtain relevant interfaces for the computing power service from internal services. For resource acquisition and scheduling, the design incorporates the concept of task queues and resource allocation configuration methods. The purpose of the task queue is to store tasks in a queue data structure in the background for scheduling when multiple tasks need to acquire resources. Resource allocation is based on ensuring minimum available resource capacity. The main logic involves setting a resource configuration sheet, stored in .yml format. Its main attributes include a resource upper limit and a resource lower limit, ensuring that even small tasks have sufficient resources. For example: if the total resource size is 100, the requested resource is 30, and the minimum resource usage is 10, this scheduling method will allocate 10 resources to the tenant. These 10 resources are guaranteed not to be occupied. When the task execution is insufficient with 10 resources, additional resources will be provided, but the total available resources will not exceed 30. If 10 resources are sufficient, the remaining 20 resources (30-10) can be used by other task queues.

[0041] In some embodiments, in S103, based on the data accumulation of computing power task usage, a dynamic resource scheduling system based on historical data is adopted to extract features, including: time features, namely execution time period, execution duration, and periodicity; resource features, namely CPU peak utilization, memory usage pattern, and IO intensive features; business features, namely task type, data size, and processing complexity; and environmental features, namely network latency, storage access speed, and concurrency competition.

[0042] In some embodiments, in S104, the application of the pre-trained prediction model includes: using an XGBoost regression model to predict task resource requirements; employing an LSTM neural network to learn time-series patterns of resource usage; and utilizing a clustering algorithm to identify similar task patterns; implementing an anomaly detection algorithm to identify resource usage anomalies; and recording and quantifying SLAs, including response time, success rate, throughput, etc. In this way, based on the model's output, resource usage can be dynamically recommended, gradually providing more resources.

[0043] In some optional embodiments, since the optical gate securely isolates the internal and external network environments, a data channel based on the UDF protocol is designed for the transmission of computing task data. This allows the computing task data to be transferred from inside the optical gate to outside and then forwarded to the computing network via data transfer. The specific process of computing task data passing through the optical gate is as follows: the computing task service connects to the data sending end, which is configured with a data encapsulation module, an encryption module, a sequence management module, a UDP sending module, and a data reassembly module.

[0044] In practical implementation, the data encapsulation module fragments the original data packets according to a preset fragment size, and constructs a packet header for each data fragment containing the following fields: Session ID (4 bytes), unique identifier of a data transmission session, Sequence Number (4 bytes), data fragment sequence number, total number of fragments in this transmission, current fragment index (4 bytes), current fragment index position, data checksum (8 bytes, using CRC64 checksum algorithm), timestamp (8 bytes, i.e., the timestamp of data transmission), encryption identifier (1 byte), etc.; the encryption module uses the AES algorithm to encrypt the content of the data fragments and the national cryptographic SM4 algorithm to encrypt key fields in the packet header; the sequence management module implements a sliding window mechanism with a configurable window size (default 32 fragments), an adaptive retransmission strategy, and dynamically adjusts the retransmission timeout based on network conditions; through data reassembly, the receiving end reassembles the data according to the session ID and sequence number, verifies the data integrity and timeliness, and decrypts it to restore the original data. In this way, it is possible to maximize the utilization of computing power under security constraints.

[0045] In other words, according to the embodiments of this application, the method further includes: transmitting computing power tasks via a data channel based on the UDP protocol. Specifically, this includes: the data sending end fragmenting the original data packet into fragments according to a preset fragment size, and constructing a packet header for each data fragment containing a session ID, sequence number, total number of fragments, current fragment index, data checksum, timestamp, and encryption identifier; encrypting the data fragment content and key fields of the packet header using an encryption algorithm; using a sliding window mechanism for data transmission, wherein the window size is configurable and the data retransmission strategy is dynamically adjusted based on network conditions; and the data receiving end reassembling the data fragments according to the session ID and sequence number, verifying data integrity, and decrypting the data.

[0046] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0047] Based on the same technical concept, corresponding to any of the above embodiments, this application also provides a component scheduling device 300 based on a computing power network storage-computing separation architecture.

[0048] like Figure 3 As shown, the component scheduling device 300 based on the computing power network storage-compute separation architecture may include: The receiving module 301 is used to receive computing power resource applications, computing power status awareness and computing power discovery requests from internal services through the standardized API interface provided by the computing power network service layer. The computing power network service layer is deployed in the area outside the optical gate and is connected to the computing power network. The allocation module 302 is used to schedule the tasks to be executed based on the task queue after receiving the computing power resource application, and allocate computing resources to the tasks according to the preset resource allocation configuration strategy, wherein the resource allocation configuration strategy includes a guarantee mechanism for the upper limit and lower limit of resources. The extraction module 303 is used to acquire historical computing power task execution data and extract the time characteristics, resource characteristics, business characteristics and environmental characteristics of the historical computing power task execution data; The optimization module 304 is used to input the time features, resource features, business features and environmental features into a pre-trained prediction model to obtain the prediction results of task resource requirements, and optimize resource scheduling based on the prediction results.

[0049] It should be noted that, for ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0050] The apparatus in the above embodiments is used to implement the corresponding component scheduling method based on the computing power network storage-computing separation architecture in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0051] Based on the same technical concept, corresponding to any of the above embodiments, this application also provides an electronic device.

[0052] Figure 4 A schematic diagram of a more specific electronic device hardware structure provided in this embodiment is shown.

[0053] The electronic device 400 may include a processor 401 and a memory 402 storing computer program instructions.

[0054] Specifically, the processor 401 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0055] Memory 402 may include mass storage for data or instructions. For example, and not limitingly, memory 402 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 402 may include removable or non-removable (or fixed) media. Where appropriate, memory 402 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 402 is non-volatile solid-state memory.

[0056] In certain embodiments, the memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Thus, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this application.

[0057] The processor 401 reads and executes computer program instructions stored in the memory 402 to implement any of the component scheduling methods based on the computing power network storage-computing separation architecture in the above embodiments.

[0058] In some examples, electronic device 400 may also include communication interface 403 and bus 410. For example, Figure 4 As shown, the processor 401, memory 402, and communication interface 403 are connected through bus 410 and complete communication with each other.

[0059] The communication interface 403 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0060] Bus 410 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not as a limitation, bus 410 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 410 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0061] For example, the electronic device 400 can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc.

[0062] Based on the same technical concept, corresponding to any of the methods in the above embodiments, this application also provides a non-transitory computer-readable storage medium. This computer-readable storage medium stores computer program instructions; when executed by a processor, these computer program instructions implement any of the component scheduling methods based on a computing power network storage-compute separation architecture in the above embodiments. Examples of computer-readable storage media include non-transitory computer-readable storage media, such as portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, etc.

[0063] Based on the same technical concept, corresponding to any of the above embodiments, this application also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processors to execute the component scheduling method based on a computing power network-in-memory separation architecture. Corresponding to the execution entity for each step in each embodiment of the component scheduling method based on a computing power network-in-memory separation architecture, the processor executing the corresponding step can belong to the corresponding execution entity.

[0064] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0065] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0066] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0067] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0068] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A component scheduling method based on a computing power network storage-computing separation architecture, characterized in that, include: Through the standardized API interface provided by the computing power network service layer, it receives computing power resource applications, computing power status awareness and computing power discovery requests from internal services. The computing power network service layer is deployed in the area outside the optical gate and is connected to the computing power network. Upon receiving a request for computing resources, the system schedules the tasks to be executed based on the task queue and allocates computing resources to the tasks according to a preset resource allocation configuration strategy, which includes a guarantee mechanism for resource upper and lower limits. Acquire historical computing power task execution data, and extract the time characteristics, resource characteristics, business characteristics, and environmental characteristics of the historical computing power task execution data; The time characteristics, resource characteristics, business characteristics, and environmental characteristics are input into a pre-trained prediction model to obtain the prediction results of task resource requirements, and resource scheduling is optimized based on the prediction results.

2. The method according to claim 1, characterized in that, The standardized API interfaces provided by the computing power network service layer include at least: The computing power application interface is used to apply for CPU, memory, and storage resources according to task requirements; The computing power awareness interface is used to monitor the liveness status of computing power nodes and notify events. The computing power discovery interface is used to proactively detect available computing power nodes when resources are insufficient.

3. The method according to claim 2, characterized in that, The computing power discovery interface is specifically used to actively detect available computing power nodes by deploying resource probes on computing power network nodes to collect and report node resource information in a periodic or event-driven manner.

4. The method according to claim 1, characterized in that, The resource allocation configuration strategy is defined in the form of a configuration file to ensure that each task or tenant is allocated guaranteed resources no less than the resource minimum limit, and to dynamically allocate additional resources when the actual needs of a task exceed the guaranteed resources, but not exceeding the resource upper limit; the remaining resources can be used by other task queues.

5. The method according to claim 1, characterized in that, The method further includes: Data transmission of computing power tasks is carried out through a data channel based on the UDP protocol.

6. The method according to claim 5, characterized in that, The transmission of computing power tasks via a data channel based on the UDP protocol includes: The data sender divides the original data packet into fragments according to a preset fragment size, and constructs a packet header for each data fragment containing a session ID, sequence number, total number of fragments, current fragment index, data checksum, timestamp, and encryption identifier; Encryption algorithms are used to encrypt the data fragment content and key fields in the packet header; A sliding window mechanism is used for data transmission. The window size is configurable, and the data retransmission strategy is dynamically adjusted based on network conditions. The data receiver reassembles data fragments based on the session ID and sequence number, verifies data integrity, and decrypts the data.

7. A component scheduling device based on a computing power network storage-computing separation architecture, characterized in that, include: The receiving module is used to receive computing power resource applications, computing power status awareness and computing power discovery requests from internal services through the standardized API interface provided by the computing power network service layer. The computing power network service layer is deployed in the area outside the optical gate and is connected to the computing power network. The allocation module is used to schedule tasks to be executed based on the task queue after receiving a computing resource request, and allocate computing resources to the tasks according to a preset resource allocation configuration strategy, which includes a guarantee mechanism for resource upper limit and resource lower limit. The extraction module is used to acquire historical computing power task execution data and extract the time characteristics, resource characteristics, business characteristics and environmental characteristics of the historical computing power task execution data; The optimization module is used to input the time features, resource features, business features and environmental features into a pre-trained prediction model to obtain the prediction results of task resource requirements, and optimize resource scheduling based on the prediction results.

8. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; when the processor invokes the computer program instructions, it implements the component scheduling method based on the computing power network storage-computing separation architecture as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when invoked by a processor, implement the component scheduling method based on a computing power network storage-computing separation architecture as described in any one of claims 1-6.

10. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the component scheduling method based on the computing power network storage-computing separation architecture as described in any one of claims 1-6.

Images