Task processing method and device

Abstract

The embodiment of the invention provides a task processing method and related equipment / products, and belongs to the field of distributed computing. The task processing method is used for scheduling service and comprises the steps that a target task needing heterogeneous computing resources is received, and the heterogeneous computing resources comprise a first type of computing resources and a second type of computing resources; under the condition that a first server of the first type of computing resources has available second type of computing resources, creating a computing unit for the target task in the first server; and under the condition that the first server of the first type of computing resources has no available second type of computing resources, creating a first computing unit for the target task in the first server, and creating a second computing unit in a second server of the second type of computing resources. According to the technical scheme, the creation position of the computing unit can be dynamically selected, then efficient matching of tasks and computing resources is achieved, and the computing efficiency and the resource utilization rate are improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of distributed computing, and in particular to a task processing method and device, computer equipment, computer readable storage medium, and computer program product. BACKGROUND

[0002] With the rapid development of artificial intelligence, big data, and high-performance computing technologies, distributed computing environments are widely used in various computing-intensive and data-intensive scenarios. To improve computing efficiency and resource utilization, more and more systems introduce heterogeneous computing architectures, i.e., combining multiple types of computing resources (e.g., central processing units (CPUs), graphics processing units (GPUs), etc.) to take advantage of the performance of different processors in specific tasks.

[0003] However, existing task processing methods lack efficient matching mechanisms between tasks and heterogeneous hardware resources, resulting in a mismatch between task types and computing resource types, which affects overall computing efficiency and resource utilization.

[0004] It should be noted that the above content is not necessarily prior art and is not intended to limit the patent protection scope of the present application. SUMMARY

[0005] Embodiments of the present application provide a task processing method, device, computer equipment, computer readable storage medium, and computer program product to solve or alleviate one or more technical problems presented above.

[0006] One aspect of embodiments of the present application provides a task processing method, which includes: receiving a target task requiring heterogeneous computing resources, the heterogeneous computing resources including first-type computing resources and second-type computing resources; in a case where the first-type computing resources of a first server have available second-type computing resources, creating a computing unit for the target task in the first server; in a case where the first-type computing resources of the first server have no available second-type computing resources, creating a first computing unit for the target task in the first server and a second computing unit in a second server of the second-type computing resources.

[0007] Optionally, the first-type computing resources are GPU resources. The second-type computing resources are CPU resources.

[0008] Optionally, the target task includes a video transcoding task; the method further includes: splitting the video transcoding task into an encoding task and a decoding task, and generating corresponding encoding execution commands and decoding execution commands. creating an encoding computing unit for the encoding task, and creating a decoding computing unit for the decoding task; establishing a TCP connection between the encoding computing unit and the decoding computing unit; cooperating the encoding computing unit and the decoding computing unit to perform the video transcoding operation through the TCP connection; wherein, in the case that there is available second type of computing resource in the first server of the first type of computing resource, the encoding computing unit is created in the first server for the encoding task, and the decoding computing unit is created in the first server for the decoding task; in the case that there is no available second type of computing resource in the first server of the first type of computing resource, the encoding computing unit is created in the first server, and the decoding computing unit is created in the second server.

[0009] Optionally, the establishing a TCP connection between the encoding computing unit and the decoding computing unit comprises: executing an encoding execution command by the encoding computing unit, and creating a TCP connection port to wait for the TCP connection after the execution of the encoding execution command; sending a decoding task execution request by the scheduling service, the decoding task execution request comprising node meta information of the encoding computing unit; responding to the decoding task execution request by the decoding computing unit to create the TCP connection based on the node meta information.

[0010] Optionally, the method further comprises: creating a first transcoding container and running a first daemon process for the encoding computing unit; creating a second transcoding container and running a second daemon process for the decoding computing unit; wherein, the first daemon process is used to connect the encoding computing unit and the scheduling service for information exchange, and the second daemon process is used to connect the decoding computing unit and the scheduling service for information exchange.

[0011] Optionally, the method further comprises: in the case that the video transcoding operation is completed, destroying the encoding computing unit and the decoding computing unit.

[0012] Another aspect of the embodiments of the present application provides a task processing device for a scheduling service, the device comprising: receiving a target task requiring heterogeneous computing resources, the heterogeneous computing resources comprising a first type of computing resource and a second type of computing resource; in the case that the second type of computing resource is available in the first server of the first type of computing resource, the first server creates a computing unit for the target task; in the case that the second type of computing resource is not available in the first server of the first type of computing resource, the first server creates a first computing unit for the target task, and a second server of the second type of computing resource creates a second computing unit.

[0013] Another aspect of the embodiments of the present application provides a computer device, comprising: at least one processor; and a memory in communication connection with the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0014] Another aspect of the embodiments of the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are executed by a processor to implement the method as described above.

[0015] Another aspect of the embodiments of the present application provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the method as described above.

[0016] The embodiments of the present application can include the following advantages by adopting the above technical solutions: after receiving a target task requiring heterogeneous computing resources, if the first server of the first type of computing resource contains available second type of computing resource, the first server creates a computing unit for the target task; if the first server does not contain available second type of computing resource, the first server creates a first computing unit, and a second server with second type of computing resource creates a second computing unit. In this way, by judging whether the heterogeneous resources are centrally deployed, the creation position of the computing unit is dynamically selected, and then the efficient matching of the task and the computing resource is realized, and the computing efficiency and the resource utilization rate are improved. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are included to provide a further understanding of the application and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain exemplary implementations of the application. The illustrated embodiments are provided merely for purposes of example and are not intended to limit the scope of the claims. In all the drawings, like reference numerals refer to like parts throughout the several views.

[0018] Figure 1 An operating environment diagram of a task processing method according to the embodiments of the present application is schematically shown; Figure 2A flow chart of the task processing method according to Embodiment One of the present application is schematically shown; Figure 3 A flow chart of the task processing method according to Embodiment One of the present application is schematically shown; Figure 4 A flow chart of the task processing method according to Embodiment One of the present application is schematically shown; Figure 3 A flow chart of the task processing method according to Embodiment One of the present application is schematically shown; Figure 5 A flow chart of the task processing method according to Embodiment One of the present application is schematically shown; Figure 6 A flow chart of the task processing method according to Embodiment One of the present application is schematically shown; Figure 7 A flow chart of the task processing method according to Embodiment One of the present application is schematically shown; DETAILED DESCRIPTION

[0019] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0020] It should be noted that the terms "first", "second", etc. used in the embodiments of the present application are only for the purpose of description and should not be understood as indicating or implying the relative importance of the technical features indicated or implying the number of the technical features indicated. Therefore, the features with "first", "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the fact that a person of ordinary skill in the art can realize it, and when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist and is not within the scope of protection claimed by the present application.

[0021] It should be noted that, in the present application, if the collection, storage, use, transmission and processing of data are involved, each link of the data strictly follows the legal regulations, industry standards and regulatory requirements of the data source, use location and relevant countries and regions to ensure the legality and compliance of data activities. In the collection link, the data subject is informed of the collection purpose, method and scope in a conspicuous manner, and the collection is carried out after obtaining the legal authorization of the data subject, ensuring that the collection process follows the "minimum necessary" principle and does not collect data beyond the scope. In the storage link, the storage period is limited, and the data is deleted or anonymized and encrypted in a timely manner after achieving the storage purpose. In the use link, a strict data security protection mechanism will be implemented, and the original data will be processed according to the preset desensitization rules through field-level desensitization technology. For different types of data, various desensitization strategies such as data generalization, data anonymization and data encryption are adopted to effectively avoid the risk of sensitive information leakage and ensure that the final used data are desensitized data after security processing, thereby comprehensively protecting the rights and interests of data subjects and data security. In the transmission and processing link, the confidentiality and security of data in the transmission and processing process are ensured.

[0022] In the description of the present application, it should be understood that the numerical reference before the step does not indicate the order of execution of the steps before and after the step, but is only used to facilitate the description of the present application and to distinguish each step, and therefore cannot be understood as a limitation of the present application.

[0023] Firstly, the term involved in the present application is explained: Video transcoding: converting video format and compressing encoding into different code rates through encoding, decoding, compression and other technologies to adapt to different network bandwidth and terminal processing capacity.

[0024] TCP (Transmission Control Protocol): a connection-oriented, reliable, byte-stream-based transport layer communication protocol.

[0025] The embodiment of the present application provides a task processing technical solution. In the technical solution, whether the heterogeneous resources are centrally deployed can be judged, the creation position of the computing unit is dynamically selected, and then the efficient matching of the task and the computing resource is realized, and the computing efficiency and the resource utilization rate are improved. See the following.

[0026] Finally, in order to facilitate understanding, an exemplary operating environment is provided as follows.

[0027] As shown in Figure 1 , the operating environment diagram includes: a scheduling service 2, a resource pool 4 and a requestor 6; The dispatch service 2 can be composed of multiple computing devices or can be a component of a container orchestration platform. The multiple computing devices can include virtualized computing instances. The virtualized computing instances can include virtual machines, such as emulations of computer systems, operating systems, servers, and the like. The computing devices can load the virtual machines based on virtual images and / or other data defining particular software (e.g., operating systems, specialized applications, servers) for emulation. As demand for different types of processing services changes, different virtual machines can be loaded and / or terminated on one or more computing devices. A hypervisor can be implemented to manage use of different virtual machines on the same computing device.

[0028] The resource pool 4 can be composed of multiple servers or multiple containers managed by a container orchestration platform running on multiple physical or virtual machines. The resource pool 4 is used to provide distributed computing resources. The resource pool 4 can provide distributed computing services via a network. The network includes various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and / or the like. The network can include physical links, such as coaxial cable links, twisted-pair cable links, fiber-optic links, combinations thereof, and the like, or wireless links, such as cellular links, satellite links, Wi-Fi links, and the like.

[0029] The requestor 2 is a party requesting a computing task, which can be a local application, a server, a microservice in a cloud platform, and the like.

[0030] The dispatch service 2 can provide task dispatching services for the requestor 6, such as forwarding of computing tasks for the requestor 6.

[0031] It should be noted that the number of the dispatch service 2, the resource pool 4, and the requestor 6 in the figure is only illustrative and does not limit the patent protection scope of the present application. According to actual conditions, there can be any number of dispatch services 2, resource pools 4, and requestors 6.

[0032] The following describes the technical solutions of the present application through multiple embodiments with the dispatch service 2 as the execution subject. It should be understood that these embodiments can be implemented in various forms and should not be interpreted as being limited to the embodiments described herein.

[0033] Embodiment One Figure 2 A flowchart of a task processing method according to Embodiment One of the present application is schematically shown.

[0034] As shown in Figure 2 the task processing method can include steps S200-S204, wherein: Step S200, receiving a target task requiring heterogeneous computing resources, the heterogeneous computing resources including first-type computing resources and second-type computing resources.

[0035] Step S202, in the case that there is available second-type computing resource in the first server of the first-type computing resource, creating a computing unit for the target task in the first server.

[0036] Step S204, in the case that there is no available second-type computing resource in the first server of the first-type computing resource, creating a first computing unit for the target task in the first server and a second computing unit in the second server of the second-type computing resource.

[0037] The task processing method provided in the embodiment can dynamically select the creation position of the computing unit by judging whether the heterogeneous resources are centrally deployed, thereby realizing efficient matching of the task and the computing resource and improving the computing efficiency and the resource utilization rate.

[0038] The following will be described in detail in combination with Figure 2 The steps S200-S204 and optional other steps will be described in detail.

[0039] Step S200 receiving a target task requiring heterogeneous computing resources, the heterogeneous computing resources including first-type computing resources and second-type computing resources.

[0040] The target task can include, but is not limited to, a video transcoding task, an image recognition task, a model inference task, etc. In the execution process, the target task requires the cooperative support of heterogeneous computing resources, that is, it does not depend on a single type of computing resource, but requires two or more different types of computing resources to participate in the completion together.

[0041] Regarding the heterogeneous computing resources, in an optional embodiment, the first-type computing resources are GPU resources; and the second-type computing resources are CPU resources.

[0042] The GPU resources can be used to perform graphics rendering, video processing and large-scale parallel computing tasks, and the CPU resources can be used to interpret and execute computer instructions and process general computing tasks in the operating system and various software. In some embodiments, the computing resources can also include FPGA resources, NPU resources, etc. The FPGA is a chip that can reprogram the logic circuit connection mode after leaving the factory, and can realize custom hardware acceleration logic. The NPU is a processor for artificial intelligence tasks, mainly used to accelerate the forward propagation, convolution operation and matrix multiplication of neural networks.

[0043] In the embodiment, the GPU resources and the CPU resources can be used to cooperatively execute the target task requiring heterogeneous computing resources.

[0044] Step S202 In the case that the first server of the first type of computing resource has available second type of computing resource, the first server creates a computing unit for the target task.

[0045] The first server can be a GPU server and the second server can be a CPU server.

[0046] The computing unit is the smallest execution entity that can independently complete a computing task. When the first server of the first type of computing resource has available second type of computing resource, it indicates that the first server has the ability to meet the full resource requirements of the target task, and a computing unit for executing the target task can be directly created on the first server, thereby avoiding the transmission of data between multiple servers during the task process and reducing the delay caused by cross-server communication. In some embodiments, when a GPU server has sufficient CPU resources to meet the requirements of a target task, a computing unit that meets the resource requirements of the target task can be created on the GPU server.

[0047] Step S204 In the case that the first server of the first type of computing resource has no available second type of computing resource, the first server creates a first computing unit for the target task and a second computing unit on a second server of the second type of computing resource.

[0048] When the first server of the first type of computing resource does not have available second type of computing resource, it indicates that the first server cannot meet the full resource requirements of the target task. In this case, a first computing unit containing the first type of computing resource can be created on the first server, and a second computing unit can be created on a second server of the second type of computing resource to cooperatively complete the execution of the target task. By cooperatively deploying resources between different types of servers, the waste of existing computing resources can be effectively avoided while ensuring the normal operation of the task, thereby improving the overall resource scheduling efficiency. For example, if a first server (GPU server) does not have sufficient CPU resources to meet the requirements of a target task, a computing unit for providing GPU resources can be created on the GPU server, and a computing unit for providing CPU resources can be created on a second server (CPU server).

[0049] The following will take a video transcoding task as an example to exemplarily illustrate the specific process of executing the video transcoding task.

[0050] In optional embodiments, the method further comprises: Step S300, splitting the video transcoding task into an encoding task and a decoding task, and generating corresponding encoding execution commands and decoding execution commands.

[0051] Step S302, creating an encoding computing unit for the encoding task and a decoding computing unit for the decoding task.

[0052] Step S304, establishing a TCP connection between the encoding computing unit and the decoding computing unit.

[0053] Step S306, the encoding computing unit and the decoding computing unit cooperate to perform video transcoding operation through the TCP connection.

[0054] In the case that there is available second type of computing resource in the first server of the first type of computing resource, the encoding computing unit is created in the first server for the encoding task, and the decoding computing unit is created in the first server for the decoding task; in the case that there is no available second type of computing resource in the first server of the first type of computing resource, the encoding computing unit is created in the first server, and the decoding computing unit is created in the second server.

[0055] The core operation of the video transcoding task includes decoding and encoding, and the encoding and decoding process can adopt a soft-decode and hard-encode mode, that is, decoding in a software manner and encoding in a hardware manner; or a hard-decode and soft-encode mode, that is, decoding in a hardware manner and encoding in a software manner. Among them, software decoding or software encoding can be executed on a computing unit based on CPU resource, and hardware decoding or hardware encoding can be executed on a computing unit based on GPU resource. On this basis, the video transcoding task can be split into a decoding task and an encoding task, which correspond to creating a decoding computing unit and an encoding computing unit, respectively. The decoding computing unit and the encoding computing unit can interact data through a point-to-point communication mode based on TCP protocol. In some embodiments, the computing unit can be the smallest deployable and manageable unit in the container orchestration platform. The container orchestration platform is used for automating deployment, expansion and management of containerized applications.

[0056] In this embodiment, the video transcoding task is split into an encoding task and a decoding task, so that each computing unit focuses on the processing phase it is responsible for, which is beneficial to allocate corresponding computing resources according to different computing requirements, thereby improving resource utilization and transcoding efficiency, and overcoming the limitation that only soft-decode and hard-encode or hard-decode and soft-encode tasks can be executed on a single node.

[0057] In optional embodiments, step S304 can include: Step S400, executing an encoding execution command through the encoding computing unit, and creating a TCP connection port to wait for the TCP connection after the encoding execution command is executed.

[0058] In step S402, a decoding task execution request is sent by the scheduling service, and the decoding task execution request includes node meta information of the encoding computing unit.

[0059] In step S404, the decoding computing unit responds to the decoding task execution request to create the TCP connection based on the node meta information.

[0060] Since the encoding task depends on the video data output by the decoding task, although the encoding computing unit has executed the encoding execution command, it is still in a state of waiting for the decoded video data. The encoding computing unit can create a TCP connection port after receiving the encoding execution command and listen to the port to wait for the decoding computing unit to initiate a TCP connection and obtain the decoded video data through the connection. The scheduling service can trigger the decoding computing unit to perform the video decoding operation by sending a decoding task execution request to the decoding computing unit. The decoding computing unit can send a TCP connection request to the encoding computing unit based on the node meta information carried in the task execution request, so that the encoding computing unit establishes a TCP connection with the decoding computing unit. The node meta information can include an IP address, a TCP connection port, and the like. In the case where the encoding computing unit and the decoding computing unit have established a TCP connection, the decoding computing unit can send the decoded video data to the encoding computing unit through the TCP connection for encoding operation.

[0061] In this embodiment, by establishing a point-to-point TCP connection communication between the encoding computing unit and the decoding computing unit, the decoded video data can be transmitted from the decoding computing unit to the encoding computing unit in real time, effectively avoiding intermediate data delay or loss, and improving the stability and execution efficiency of the transcoding process.

[0062] It should be noted that the embodiments of the present application are not limited to TCP connection, and other communication protocols such as QUIC (Quick UDP Internet Connections, Quick UDP Internet Connection), UDP (User Datagram Protocol, User Datagram Protocol) can also be used.

[0063] In an optional embodiment, the method further includes: creating a first transcoding container for the encoding computing unit and running a first daemon process; creating a second transcoding container for the decoding computing unit and running a second daemon process; wherein the first daemon process is used to connect the encoding computing unit and the scheduling service for information exchange, and the second daemon process is used to connect the decoding computing unit and the scheduling service for information exchange.

[0064] Each computing unit can contain multiple containers, which share resources such as network, storage, etc., and jointly constitute the computing unit. The computing unit can create a transcoding container to run a daemon process. The daemon process can be a process running in the background continuously and not dependent on a user interactive terminal, used to provide network services, log recording, time synchronization, etc. The daemon process can send the node meta information of the corresponding computing unit to the scheduling service through a callback function. The callback function is a function passed in as a parameter, which can be called when a certain condition or event is triggered, to achieve a flexible control inversion mechanism. In some embodiments, the first daemon process can send the node meta information of the encoding computing unit to the scheduling service through the callback function. The second daemon process can send the node meta information of the decoding computing unit to the scheduling service through the callback function. In this embodiment, the state of the computing unit is managed by the daemon process, and the key running information is reported to the scheduling service in real time, so that the scheduling service can flexibly schedule tasks.

[0065] In optional embodiments, the method further comprises: in the case where the video transcoding operation is completed, destroying the encoding computing unit and the decoding computing unit.

[0066] The encoding computing unit and the decoding computing unit can perform self-destruction operation after completing their respective tasks. For example, the decoding computing unit has completed its task after completing video decoding and successfully transmitting the decoded video data to the encoding computing unit, and can be destroyed by itself. Similarly, the encoding computing unit can also be destroyed by itself after obtaining the decoded video data, completing encoding and storing the result. In addition, the scheduling service can also actively notify the encoding computing unit and the decoding computing unit to be destroyed after the video transcoding product is successfully stored, so as to release resources.

[0067] In this embodiment, by destroying the encoding computing unit and the decoding computing unit, it can be effectively ensured that the computing resources are recycled and used for subsequent other tasks, so as to improve the resource utilization rate.

[0068] In order to make the present application easier to understand, the following will be combined with Figure 5 An exemplary application is provided.

[0069] Taking the video transcoding mode of soft decoding and hard encoding as an example, i.e. using CPU resources for decoding and GPU resources for encoding, the operation process is as follows: Step S501, the scheduling service receives a video transcoding task requiring heterogeneous computing resources.

[0070] Step S502, according to the decoding and encoding computing resources required in the video transcoding task, the corresponding computing units are created in the resource pool.

[0071] If the GPU server has sufficient CPU resources, the computing unit satisfying the video transcoding task is created on the GPU server in priority, and the video transcoding task is executed on the computing unit.

[0072] If the GPU server has insufficient CPU resources, the GPU computing unit is created on the GPU server, the CPU computing unit is created on the CPU server, and the following steps are performed. In step S503, the scheduling service splits the video transcoding task into encoding tasks and decoding tasks, and generates corresponding encoding execution commands and decoding execution commands.

[0073] In step S504, the GPU computing unit and the CPU computing unit create transcoding containers and run daemon processes, respectively.

[0074] In step S505, the encoding execution commands are executed on the CPU computing unit, a TCP connection port is created, and the decoded video data is waited for.

[0075] In step S506, the daemon process of the GPU computing unit sends the corresponding node meta information to the scheduling service through a callback function.

[0076] In step S507, the scheduling service sends a decoding task execution request to the CPU computing unit, wherein the decoding task execution request carries the node meta information of the GPU computing unit.

[0077] In step S508, the CPU computing unit executes the decoding task to generate decoded video data in response to the decoding task execution request, and establishes a TCP connection with the GPU computing unit through the node meta information of the GPU computing unit.

[0078] In step S509, the GPU computing unit obtains the decoded video data from the CPU computing unit through the TCP connection, and encodes to generate video transcoding products.

[0079] In step S510, the GPU computing unit stores the video transcoding products.

[0080] In step S511, the GPU computing unit and the CPU computing unit are destroyed.

[0081] In this exemplary application, by judging whether the heterogeneous resources are centrally deployed, the creation position of the computing unit is dynamically selected, and then the efficient matching of the task and the computing resource is realized, and the computing efficiency and the resource utilization rate are improved.

[0082] Embodiment Two Figure 6Fig. 2 shows a block diagram of a task processing apparatus according to Embodiment Two of the present application, which can be divided into one or more program modules stored in a storage medium and executed by one or more processors to complete the embodiments of the present application. The program modules referred to in the embodiments of the present application are a series of computer program instruction segments capable of completing a specific function, and the functions of the program modules in the embodiments will be described in detail below. As shown in Fig. 2, the apparatus 600 can include a receiving module 610, a first creating module 620, and a second creating module 630, wherein: Figure 6 The receiving module 610 is configured to receive a target task requiring heterogeneous computing resources, the heterogeneous computing resources including first-type computing resources and second-type computing resources. The first creating module 620 is configured to, in a case where there is available second-type computing resource in a first server of the first-type computing resources, create a computing unit for the target task in the first server. The second creating module 630 is configured to, in a case where there is no available second-type computing resource in a first server of the first-type computing resources, create a first computing unit for the target task in the first server and create a second computing unit in a second server of the second-type computing resources.

[0083] As an optional embodiment, the first-type computing resources are GPU resources, and the second-type computing resources are CPU resources.

[0084] As an optional embodiment, the target task includes a transcoding task, and the task processing apparatus 600 is further configured to: split the video transcoding task into an encoding task and a decoding task, and generate corresponding encoding execution commands and decoding execution commands; create an encoding computing unit for the encoding task and a decoding computing unit for the decoding task; establish a TCP connection between the encoding computing unit and the decoding computing unit; perform a video transcoding operation by the encoding computing unit and the decoding computing unit through the TCP connection; wherein, in a case where there is available second-type computing resource in a first server of the first-type computing resources, an encoding computing unit is created for the encoding task in the first server, and a decoding computing unit is created for the decoding task in the first server; in a case where there is no available second-type computing resource in a first server of the first-type computing resources, the encoding computing unit is created in the first server, and the decoding computing unit is created in the second server.

[0085] ​As an optional embodiment, establishing a TCP connection between the encoding calculation unit and the decoding calculation unit includes: The encoding calculation unit executes the encoding execution command, and after the encoding execution command is executed, a TCP connection port is created to wait for the TCP connection. The decoding task execution request is sent through the scheduling service, and the decoding task execution request includes the node metadata of the encoding calculation unit; The decoding computing unit responds to the decoding task execution request to create the TCP connection based on the node metadata.

[0086] As an optional embodiment, the task processing device 600 is further configured to: A first transcoding container is created and a first daemon process is run for the encoding calculation unit; A second transcoding container is created and a second daemon process is run for the decoding computing unit; The first daemon process is used to connect the encoding calculation unit and the scheduling service for information exchange, and the second daemon process is used to connect the decoding calculation unit and the scheduling service for information exchange.

[0087] As an optional embodiment, the task processing device 600 is further configured to: Upon completion of the video transcoding operation, the encoding calculation unit and the decoding calculation unit are destroyed.

[0088] Example 3 Figure 7 This illustration schematically depicts the hardware architecture of a computer device 10000 suitable for implementing a task processing method according to Embodiment 3 of this application. In some embodiments, the computer device 10000 may be a terminal device such as a smartphone, wearable device, tablet computer, personal computer, in-vehicle terminal, game console, virtual machine, workbench, digital assistant, set-top box, or robot. In other embodiments, the computer device 10000 may be a rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple servers), etc. Figure 7 As shown, the computer device 10000 includes, but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate and be linked with each other via a system bus. Wherein: The memory 10010 includes at least one type of computer-readable storage media, such as a flash memory, a hard disk, a multimedia card (e.g., SD or DX memory), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read-Only Memory (ROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Programmable Read-Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the memory 10010 can be an internal storage unit of the computer device 10000, such as a hard disk or a memory of the computer device 10000. In other embodiments, the memory 10010 can also be an external storage device of the computer device 10000, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, and the like. Of course, the memory 10010 can include both an internal storage unit and an external storage device of the computer device 10000. In this embodiment, the memory 10010 is generally used to store an operating system and various application programs installed in the computer device 10000, such as program codes of the task processing method, and the like. In addition, the memory 10010 can also be used to temporarily store various data that have been output or will be output.

[0089] The processor 10020 can be a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor, or other chips in some embodiments. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication of the computer device 10000, and the like. In this embodiment, the processor 10020 is used to run program codes or process data stored in the memory 10010.

[0090] The network interface 10030 can include a wireless network interface or a wired network interface, and is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 with an external terminal through a network, establish a data transmission channel and a communication link between the computer device 10000 and the external terminal, and the like. The network can be an Intranet, the Internet, a Global System of Mobile communication (GSM), a Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth, Wi-Fi, and the like wireless or wired network.

[0091] It should be noted that, Figure 7 Only the computer device with the components 10010-10030 is shown, but it should be understood that all the shown components are not required to be implemented, and more or fewer components can be alternatively implemented.

[0092] In this embodiment, the task processing method stored in the memory 10010 can also be divided into one or more program modules, and executed by one or more processors (such as the processor 10020) to complete the embodiments of the present application.

[0093] Embodiment Four The embodiments of the present application also provide a computer readable storage medium, and the computer readable storage medium has a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the task processing method in the embodiments.

[0094] In this embodiment, the computer readable storage medium includes a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read only memory (ROM), an electrically erasable programmable read only memory (EEPROM), a programmable read only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the computer readable storage medium can be an internal storage unit of the computer device, for example, a hard disk or a memory of the computer device. In other embodiments, the computer readable storage medium can also be an external storage device of the computer device, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Of course, the computer readable storage medium can also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the computer readable storage medium is usually used to store an operating system and various application software installed on the computer device, for example, program codes of the task processing method in the embodiments, etc. In addition, the computer readable storage medium can also be used to temporarily store various data that have been output or will be output.

[0095] Embodiment five The embodiments of the present application also provide a computer program product, comprising a computer program which, when executed by a processor, implements the method in the above embodiments.

[0096] Obviously, those skilled in the art should understand that each module or each step of the above-mentioned embodiments of the present application can be implemented by using a general computer device, which can be concentrated on a single computer device or distributed on a network composed of multiple computer devices, and optionally, each module or each step can be implemented by using program codes executable by a computer device, so that each module or each step can be stored in a storage device and executed by a computer device, and in some cases, the steps shown or described can be executed in an order different from that shown here, or each module or each step can be manufactured into an individual integrated circuit module, or multiple modules or steps can be manufactured into a single integrated circuit module. Therefore, the embodiments of the present application are not limited to any particular combination of hardware and software.

[0097] It should be noted that the above is only the preferred embodiment of the present application, and does not limit the patent protection scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings of the present application, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A task processing method, characterized in that, For scheduling services, the method includes: Receive a target task that requires heterogeneous computing resources, wherein the heterogeneous computing resources include a first type of computing resources and a second type of computing resources; If a second type of computing resource is available on a first server containing a first type of computing resource, a computing unit is created on the first server for the target task. If no second type of computing resources are available on the first server of the first type of computing resources, a first computing unit is created on the first server for the target task, and a second computing unit is created on the second server of the second type of computing resources.

2. The method according to claim 1, characterized in that, The first type of computing resource is GPU resources; The second type of computing resource is CPU resources.

3. The method according to claim 1, characterized in that, The target task includes a transcoding task; the method further includes: The video transcoding task is split into encoding and decoding tasks, and corresponding encoding and decoding execution commands are generated. An encoding computation unit is created for the encoding task, and a decoding computation unit is created for the decoding task; A TCP connection is established between the encoding calculation unit and the decoding calculation unit; Through the TCP connection, the encoding calculation unit and the decoding calculation unit cooperate to perform video transcoding operations; Where, if there are available second-class computing resources on the first server of the first-class computing resources, an encoding computing unit is created on the first server for the encoding task, and a decoding computing unit is created on the first server for the decoding task. If no second type of computing resources are available in the first server of the first type of computing resources, the encoding computing unit is created on the first server, and the decoding computing unit is created on the second server.

4. The method according to claim 3, characterized in that, Establishing a TCP connection between the encoding calculation unit and the decoding calculation unit includes: The encoding calculation unit executes the encoding execution command, and after the encoding execution command is executed, a TCP connection port is created to wait for the TCP connection. The decoding task execution request is sent through the scheduling service, and the decoding task execution request includes the node metadata of the encoding calculation unit; The decoding computing unit responds to the decoding task execution request to create the TCP connection based on the node metadata.

5. The method according to claim 3, characterized in that, The method further includes: A first transcoding container is created and a first daemon process is run for the encoding calculation unit; A second transcoding container is created and a second daemon process is run for the decoding computing unit; The first daemon process is used to connect the encoding calculation unit and the scheduling service for information exchange, and the second daemon process is used to connect the decoding calculation unit and the scheduling service for information exchange.

6. The method according to claim 3, characterized in that, The method further includes: Upon completion of the video transcoding operation, the encoding calculation unit and the decoding calculation unit are destroyed.

7. A task processing device, characterized in that, For scheduling services, the apparatus includes: A receiving module is used to receive target tasks that require heterogeneous computing resources, wherein the heterogeneous computing resources include a first type of computing resources and a second type of computing resources. The first creation module is used to create a computing unit for the target task on the first server when there are available second-type computing resources on the first server of the first type of computing resources. The second creation module is used to create a first computing unit for the target task on the first server and a second computing unit on the second server with second-type computing resources, provided that there are no available second-type computing resources on the first server with first-type computing resources.

8. A computer device, characterized in that, include: At least one processor; and A memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in claims 1 to 6.

Images