Image rendering method and computing system

Abstract

The invention provides an image rendering method and a computing system, relates to the technical field of computers, and is used for improving image rendering efficiency. A first computing node in the computing system receives a control instruction sent by a user terminal, an image rendering task corresponding to the control instruction is determined, and the image rendering task comprises a plurality of subtasks; the first computing node determines the computing node corresponding to each subtask based on the affinity of the plurality of subtasks and the computing nodes; wherein the plurality of subtasks correspond to at least two types of computing nodes, and different types of computing nodes comprise different types of processors; and the first computing node schedules the computing node corresponding to each sub-task and executes the corresponding sub-task. The first computing node allocates at least two types of computing nodes to the image rendering task, the corresponding sub-tasks are executed through the at least two types of computing nodes, and the image rendering efficiency can be improved based on the heterogeneous architecture of the multiple types of computing nodes.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image rendering method and computing system. Background Technology

[0002] Rendering refers to the process of generating images from models using software. A model is a strictly defined language or data structure describing a three-dimensional object, including geometry, viewpoint, texture, and lighting information. With the development of deep neural networks, optimizing image synthesis and rendering processes using neural network models is becoming increasingly common.

[0003] Currently, neural rendering technology typically uses a graphics processing unit (GPU) for direct rendering. However, as real-time rendering applications demand increasingly higher image quality and real-time performance, the computational load and speed during the rendering process become extremely high. For example, in 3D games with a refresh rate of 30 frames per second, the rendering time for each frame cannot exceed 33 milliseconds. Therefore, rendering based on a single GPU is inefficient and cannot meet the requirements of real-time rendering. Summary of the Invention

[0004] This application provides an image rendering method and computing system to improve image rendering efficiency.

[0005] Firstly, this application provides an image rendering method, which can be executed by a first computing node in a computing system, by a chip inside the first computing node, or by a processor in the first computing node (e.g., a central processing unit (CPU), GPU, neural network processing unit (NPU), data processing unit (DPU), field-programmable gate array (FPGA), etc.). The following example illustrates execution by the first computing node. The first computing node receives control commands sent by a user terminal and determines the image rendering task corresponding to the control commands. The image rendering task includes multiple subtasks. Based on the affinity between the multiple subtasks and the computing node, the first computing node determines the computing node corresponding to each subtask. The multiple subtasks correspond to at least two types of computing nodes, and the different types of computing nodes include different types of processors. The first computing node schedules the computing node corresponding to each subtask to execute the corresponding subtask.

[0006] Optionally, the first computing node schedules the computing nodes corresponding to each subtask to execute the corresponding subtask. Specifically, the first computing node may execute the corresponding subtask, and / or schedule the computing nodes corresponding to other subtasks to execute the corresponding subtasks.

[0007] Using the above method, when the first computing node executes an image rendering task according to the control commands sent by the user terminal, it can rationally allocate corresponding computing nodes to each subtask in the image rendering task based on the affinity between the subtask and the computing node. When the affinity between the computing node and the subtask is high, the task execution efficiency can be improved. Furthermore, the first computing node allocates at least two types of computing nodes to the image rendering task, and executes their respective subtasks through at least two types of computing nodes. Based on this heterogeneous architecture of multiple types of computing nodes, image rendering efficiency can be improved.

[0008] In one possible implementation, the image rendering task includes at least one subtask comprising a first function. For the first function, a first computing node can determine the corresponding computing node based on the affinity between the first function and the computing node.

[0009] Using the above method, for a first function (e.g., an operator), the first computing node can determine the corresponding computing node based on the affinity between the first function and the computing node. Based on this method, the first computing node can dynamically adjust the computing node corresponding to the first function during the execution of subtasks on various computing nodes, further improving image rendering efficiency.

[0010] In one possible implementation, the first computing node determines the computing node corresponding to each subtask based on the load information of multiple computing nodes.

[0011] In this way, the first computing node can reasonably allocate corresponding computing nodes to each subtask based on the current load information of each computing node, so that the computing power of each computing node is more balanced, thereby improving the efficiency of each computing node in performing image rendering.

[0012] In one possible implementation, the first computing node determines a type of computing node corresponding to each subtask based on the affinity between multiple subtasks and computing nodes; for each subtask, if the type of computing nodes corresponding to the subtask includes multiple computing nodes, then the computing node corresponding to the subtask is determined based on the load information of the multiple computing nodes.

[0013] In this way, the first computing node can combine the affinity between the subtask and the computing node, as well as the load signal of each computing node, to more rationally allocate the corresponding computing node to each subtask, making the subtask and the corresponding computing node more compatible and the computing power of each computing node more balanced, thereby improving the efficiency of each computing node in performing image rendering.

[0014] In one possible implementation, the first computing node determines multiple candidate computing nodes for each subtask based on the load information of multiple computing nodes, and the multiple candidate computing nodes are computing nodes of different types; for each subtask, the computing node corresponding to the subtask is determined from the multiple candidate computing nodes based on the affinity between the subtask and each candidate computing node.

[0015] In this way, the first computing node can combine the affinity between the subtask and the computing node, as well as the load signal of each computing node, to more rationally allocate the corresponding computing node to each subtask, making the subtask and the corresponding computing node more compatible and the computing power of each computing node more balanced, thereby improving the efficiency of each computing node in performing image rendering.

[0016] In one possible implementation, a pass-through link is included between two different types of computing nodes, which is used to transfer data between the two different types of computing nodes.

[0017] In this way, direct links are included between different types of computing nodes, which can improve the data transmission efficiency during the image rendering process.

[0018] In one possible implementation, the computing node includes at least one of CPU, GPU, NPU, DPU, and FPGA.

[0019] In the above manner, this application can perform image rendering tasks using various types of computing nodes.

[0020] In one possible implementation, the multiple subtasks include a first type of subtask and a second type of subtask. The first type of subtask includes at least one subtask for implementing raster rendering, and the second type of subtask includes at least one subtask for implementing neural rendering. The first type of subtask corresponds to the GPU, and at least one subtask in the second type of subtask corresponds to the NPU.

[0021] In this way, the first computing node allocates a GPU to the raster rendering subtask and an NPU to the neural rendering subtask, making the subtasks more compatible with the computing node and thus improving the efficiency of image rendering.

[0022] Secondly, this application provides a computing system that may include multiple computing nodes; a first computing node among the multiple computing nodes is used to determine an image rendering task corresponding to a control command sent by a user terminal, wherein the image rendering task includes multiple subtasks; based on the affinity between the multiple subtasks and the computing nodes, a computing node corresponding to each subtask is determined; wherein the multiple subtasks correspond to at least two types of computing nodes, and the different types of computing nodes include different types of processors; each computing node in the computing system is used to execute the corresponding subtask.

[0023] In one possible implementation, the image rendering task includes at least one subtask comprising a first function, and the first computing node is further configured to: determine the computing node corresponding to the first function based on the affinity between the first function and the computing node.

[0024] In one possible implementation, the first computing node is also used to: determine the computing node corresponding to each subtask based on the load information of multiple computing nodes.

[0025] In one possible implementation, the first computing node is further used to: determine a type of computing node corresponding to each subtask based on the affinity between multiple subtasks and computing nodes; for each subtask, if the type of computing nodes corresponding to the subtask includes multiple computing nodes, then determine the computing node corresponding to the subtask based on the load information of the multiple computing nodes.

[0026] In one possible implementation, the first computing node is further configured to: determine multiple candidate computing nodes corresponding to each subtask based on the load information of multiple computing nodes, wherein the multiple candidate computing nodes are computing nodes of different types; and for each subtask, determine the computing node corresponding to the subtask from the multiple candidate computing nodes based on the affinity between the subtask and each candidate computing node.

[0027] In one possible implementation, a pass-through link is included between two different types of computing nodes, which is used to transfer data between the two different types of computing nodes.

[0028] In one possible implementation, the computing node includes at least one of CPU, GPU, NPU, DPU, and FPGA.

[0029] In one possible implementation, the multiple subtasks include a first type of subtask and a second type of subtask. The first type of subtask includes at least one subtask for implementing raster rendering, and the second type of subtask includes at least one subtask for implementing neural rendering. The first type of subtask corresponds to the GPU, and at least one subtask in the second type of subtask corresponds to the NPU.

[0030] Thirdly, this application also provides a computing device that has the function of implementing the behavior in the method examples of the first aspect described above. The beneficial effects are described in the first aspect description and will not be repeated here. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the computing device may include an information transmission module, a computing power allocation module, and a scheduling module; wherein:

[0031] The information transmission module is used to receive control commands sent by the user terminal, determine the image rendering task corresponding to the control command, and the image rendering task includes multiple sub-tasks;

[0032] The computing power allocation module is used to determine the computing node corresponding to each subtask based on the affinity between multiple subtasks and computing nodes; wherein, multiple subtasks correspond to at least two types of computing nodes, and different types of computing nodes include different types of processors;

[0033] The scheduling module is used to schedule the computing nodes corresponding to each subtask and execute the corresponding subtask.

[0034] For details on the functions that each module can perform, please refer to the detailed description in the method examples in the first aspect; they will not be repeated here.

[0035] Fourthly, this application also provides a computing device including at least one processor and at least one memory; wherein the one or more memories store one or more computer programs, the one or more computer programs including instructions that, when executed by the one or more processors, cause the computing device to perform the methods provided by any possible implementation of the first aspect above.

[0036] Fifthly, this application also provides a communication system that includes a user terminal and a computing system provided as in any possible implementation of the second aspect above.

[0037] Sixthly, this application provides a computer-readable storage medium that, when executed by a computing device, allows the computing device to perform the method provided by any possible implementation of the first aspect described above. The storage medium stores computer program instructions. The storage medium includes, but is not limited to, volatile memory, such as random access memory, and non-volatile memory, such as flash memory, hard disk drive (HDD), and solid-state drive (SSD).

[0038] In a seventh aspect, this application provides a computer program product, which includes computer program instructions. When executed by a computing device, the computing device performs the method provided by any possible implementation of the first aspect. The computer program product can be a software installation package, which can be downloaded and executed on a computer when the method provided by any possible implementation of the first aspect is required.

[0039] Eighthly, this application also provides a computer chip connected to a memory, the chip being used to read and execute computer program instructions stored in the memory, and to execute the method provided by any possible implementation of the first aspect above. Attached Figure Description

[0040] Figure 1 A schematic diagram of a system architecture provided for an embodiment of this application;

[0041] Figure 2A This application provides a schematic diagram of the structure of a computing system according to an embodiment of the present application.

[0042] Figure 2B This application provides a schematic diagram of the structure of a computing system according to an embodiment of the present application.

[0043] Figure 2C This application provides a schematic diagram of the structure of a computing system according to an embodiment of the present application.

[0044] Figure 3A This application provides a schematic diagram of the structure of a computing system according to an embodiment of the present application.

[0045] Figure 3B This application provides a schematic diagram of the structure of a computing system according to an embodiment of the present application.

[0046] Figure 3C This application provides a schematic diagram of the structure of a computing system according to an embodiment of the present application.

[0047] Figure 4 This application provides a schematic diagram of the structure of a computing system according to an embodiment of the present application.

[0048] Figure 5 A flowchart illustrating an image rendering method provided in an embodiment of this application;

[0049] Figure 6 A schematic diagram of a load sensing principle provided in an embodiment of this application;

[0050] Figure 7 This is a schematic diagram of an image rendering process provided in an embodiment of this application;

[0051] Figure 8This is a schematic diagram illustrating image rendering performed by a cloud server, as provided in an embodiment of this application.

[0052] Figure 9 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0053] Figure 10 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0054] Figure 11 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0056] Before introducing the specific solutions provided in the embodiments of this application, some terms used in this application will be explained to facilitate understanding by those skilled in the art; it should be understood that the explanation of some terms is only used to explain the specific embodiments of this application, and is not intended to limit this application.

[0057] 1. Neural rendering:

[0058] Neural rendering is a rendering method based on deep neural networks and physical engines, capable of creating new images or video clips from existing scenes. Users can freely define scene attributes such as lighting, camera parameters, pose, geometry, shape, and model structure. Neural rendering technology can render controllable and realistic 3D scene models. It is a rendering generation technique that generates new scenes based on existing image and video data.

[0059] 2. Rasterization-based rendering:

[0060] Rasterization is the process of converting a 3D model into a 2D image; it is also called rasterization. It is a fundamental technique in computer graphics, breaking down a 3D model into a series of pixels and calculating the color value of each pixel to ultimately generate a 2D image. Rasterization is suitable for most computer graphics applications, including games, film special effects, and more. The basic steps of rasterization include:

[0061] a. Modeling: Representing the geometric structure of a 3D object using vertices, edges, and faces.

[0062] b. Transformation: Matrix operations are used to perform transformations on 3D objects, such as translation, rotation, and scaling, as well as to transform them from the world coordinate system to the camera coordinate system.

[0063] c. Cropping: Using the frustum to remove objects or parts of objects that are not within the field of view.

[0064] d. Projection: Use perspective projection or orthographic projection to transform a 3D object from the camera coordinate system to the screen coordinate system.

[0065] e. Rasterization: Use scanline algorithm or triangle filling algorithm to convert triangular patches into pixels and calculate the color value of each pixel.

[0066] f. Shading: Use shader programs to perform lighting calculations, texture mapping, fog effects, and other processing on each pixel to enhance the realism and aesthetics of the image.

[0067] 3. Graphics rendering pipeline:

[0068] The rendering pipeline, also known as the rendering pipeline, is primarily responsible for generating or rendering a 2D image given a virtual camera, 3D objects, light sources, lighting modes, and textures. The rendering pipeline is not static; due to differences in hardware, software, and platforms, there can be some subtle variations. Broadly speaking, the process comprises four stages: application, geometry processing, rasterization, and pixel processing; each stage is processed in parallel.

[0069] 4. Graphics Processor:

[0070] Graphics processors are used to process computer graphics and image data, such as image / graphics rendering and image / graphics processing. A graphics processor (GPU) is used in a graphics card. GPUs have a large number of processing units, enabling them to quickly process large amounts of graphics and image data, significantly improving computing speed and efficiency. In the field of AI, GPUs can be used to train neural networks or perform inference based on neural networks. An image processor can refer to an image processing chip or a board that integrates an image processing chip. The board is plugged into the motherboard and communicates with components such as the CPU and image processor via a bus.

[0071] 5. Neural Network Processor:

[0072] An NPU (Neural Processing Unit) is a dedicated processor designed specifically for the computational needs of neural network models, aiming to perform machine learning, especially deep learning tasks, efficiently and with low power consumption. The core concept of the NPU is to simulate the working principle of the human brain's neural network, accelerating complex calculations such as large-scale matrix operations and convolution operations in deep neural networks through massively parallel processing units (similar to neurons) and efficient interconnect structures (similar to synapses). NPUs are primarily used in artificial intelligence-related fields such as autonomous driving, image recognition, speech recognition, natural language understanding, and recommendation systems, playing a crucial role, especially in edge computing devices and AI servers in data centers. Unlike traditional general-purpose processors, NPUs integrate highly customized computing units, storage structures, and data flow scheduling mechanisms at the hardware level, enabling efficient handling of operations such as feature extraction, weight updates, and activation function calculations in deep learning models. This design gives NPUs extremely high energy efficiency when processing artificial intelligence tasks, providing computational performance far exceeding that of general-purpose processors at the same power consumption, especially in the inference stage (such as image recognition, speech recognition, and natural language processing applications), where their advantages are particularly evident.

[0073] 6. Data processor:

[0074] A data processor is a data-centric, input-output (I / O) intensive chip capable of improving computing system efficiency and reducing costs. Primarily designed for data center infrastructure applications, it flexibly offloads virtualization, networking, storage, and security services to reduce the load on the central processing unit (CPU). Data processors offload some functions from the CPU; for example, they can offload high-performance data processing functions from networking, storage, or the operating system that are unsuitable for CPU processing, freeing up CPU computing power. A data processor can be a Data Processing Unit (DPU), Infrastructure Processing Unit (IPU), System-on-Chip (SoC), iNIC, or smartNIC—any computing unit with offloading capabilities. An iNIC or smartNIC can be understood as a smart network interface card (NIC). A data processor can refer to a data processing chip or a board that integrates a data processing chip. This board is plugged into the motherboard and communicates with components such as the CPU and graphics processor via a bus.

[0075] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where each of a, b, and c can be an element itself or a set containing one or more elements.

[0076] In this application, terms such as "exemplary," "in some embodiments," and "in other embodiments" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the term "exemplary" is used to present the concept in a specific manner.

[0077] In this application, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, unless a distinction is emphasized, their intended meanings are consistent. Similarly, in the embodiments of this application, "communication" and "transmission" may sometimes be used interchangeably. It should be noted that, unless a distinction is emphasized, their intended meanings are consistent. For example, transmission can include sending and / or receiving, and can be a noun or a verb.

[0078] It should be noted that the terms "first" and "second" used in the embodiments of this application are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance or order.

[0079] The image rendering method provided in this application embodiment can be applied to, for example... Figure 1 The system architecture shown includes at least one user terminal 10 and a computing system 20; wherein Figure 1 At least one user terminal 10 is included, for example, two user terminals. At least one user terminal 10 can access the computing system 20 through a communication network.

[0080] User terminal 10 responds to user-triggered operations and sends control commands to computing system 20. For example, when running a game app on user terminal 10, the user can trigger game-related operations (such as swiping up, swiping down, rotating the screen, etc.) in the display interface of the running game application (APP). User terminal 10 responds to the user's operation in the display interface of the running game app, generates control commands related to the operation, and sends the control commands to computing system 20 through the communication network.

[0081] After receiving a control command from the user terminal 10, the computing system 20 decodes the received control command to obtain image rendering control parameters corresponding to the control command; for example, image rendering control parameters include light intensity, material properties, and viewing angle. Based on the image rendering control parameters, the computing system 20 determines the image rendering task corresponding to the control command and executes the image rendering task. The computing system 20 executes the image rendering task to obtain rendered image data; optionally, this image data can be encoded image data. The computing system 20 sends the rendered image data to the user terminal 10 via a communication network. Correspondingly, after receiving the rendered image data sent by the computing system 20, the user terminal 10 displays the rendered image on its interface; optionally, the user terminal 10 decodes the received image data and displays the rendered image on its interface based on the decoded image data.

[0082] In the system architecture of this application embodiment, the user terminal 10 can be a mobile phone, tablet computer, laptop computer, wearable device, smart screen device, etc.

[0083] The computing system provided in the embodiments of this application will now be described with reference to the accompanying drawings. The computing system 20 includes multiple computing nodes. The computing system 20 in the embodiments of this application can be a heterogeneous architecture, with the multiple computing nodes including different types of computing nodes; wherein the different types of computing nodes can include different types of processors, or the different types of computing nodes can be different types of processors. Optionally, the processors in the embodiments of this application may include CPUs, GPUs, NPUs, DPUs, FPGAs, etc.

[0084] In one possible implementation, a computing node is a node that includes a processor. For example, a computing node may include a processor and other units / components / devices, such as a computing node including a processor and memory. In this implementation, the computing system 20 may include multiple different types of computing nodes, and there may be one or more computing nodes of each type in the computing system 20. Optionally, each computing node may include one or more processors. When a computing node includes multiple processors, the multiple processors may be processors of the same type or processors of different types. Figure 2A Taking the computing system 20 shown as an example, the computing system 20 includes computing node 1, computing node 2, and computing node 3, wherein computing node 1 includes a CPU, computing node 2 includes a GPU, and computing node 3 includes an NPU. Alternatively, taking... Figure 2B Taking the computing system 20 shown as an example, the computing system 20 includes computing node 1, computing node 2, and computing node 3, wherein computing node 1 includes a DPU, computing node 2 includes a GPU, and computing node 3 includes an NPU. Alternatively, taking... Figure 2C Taking the computing system 20 shown as an example, the computing system 20 includes computing node 1, computing node 2, and computing node 3, where computing node 1 includes a CPU and a DPU, computing node 2 includes a GPU, and computing node 3 includes an NPU. It should be noted that... Figure 2A , Figure 2B , Figure 2C The example shown is one of each type of compute node, but in reality, there can be multiple compute nodes of each type.

[0085] In another possible implementation, the computing node is a processor. In this implementation, the computing system 20 includes multiple different types of processors, and each type of processor can have one or more. Figure 3A Taking the computing system 20 shown as an example, the computing system 20 includes a CPU, GPU, and NPU. Or, taking... Figure 3B Taking the computing system 20 shown as an example, the computing system 20 includes a DPU, a GPU, and an NPU. Or, taking... Figure 3C Taking the computing system 20 shown as an example, the computing system 20 includes a CPU, a DPU, a GPU, and an NPU. Figure 3A , Figure 3B , Figure 3C The example shown is of one processor of each type; in reality, there can be multiple processors of each type.

[0086] The aforementioned computing system 20 can be a single computing device or a cluster of computing devices, with the cluster including multiple computing devices. Each computing device can be understood as a computer or server, etc.; the server can be a locally deployed server or a cloud server. When the computing system is a cluster of multiple computing devices, different computing nodes can be deployed on the same computing device or on different computing devices. For example, if the computing system is a cluster of two computing devices, including computing node 1, computing node 2, and computing node 3, computing node 1 and computing node 2 can be deployed on one computing device, and computing node 3 can be deployed on another computing device. As another example, if the computing system is a cluster of three computing devices, including computing node 1, computing node 2, and computing node 3, computing node 1, computing node 2, and computing node 3 can each be deployed on a separate computing device.

[0087] When multiple computing nodes are deployed on the same computing device, they can be connected via a Peripheral Component Interconnect Express (PCIe) bus or an Extended Industry Standard Architecture (EISA) bus. For example, computing system 20 includes a CPU, GPU, and NPU. When the CPU, GPU, and NPU are deployed on a single computing device, they can be connected via a PCIe bus or an EISA bus.

[0088] Optionally, when multiple computing nodes are deployed on the same computing device, pass-through links exist between different types of computing nodes. These pass-through links, also known as peer-to-peer (P2P) links, are used to transmit data between two different types of computing nodes. For example, ... Figure 4As shown, the computing system 20 includes multiple computing nodes of various types, each type of computing node can include multiple nodes; such as multiple computing nodes including CPUs and dynamic random access memory (DRAM), multiple computing nodes including GPUs and high bandwidth memory (HBM), and multiple computing nodes including NPUs and HBM. Data transfer between the CPU and GPU can be performed via the PCIe bus, as can data transfer between the CPU and NPU. A pass-through link exists between the GPU and NPU; the GPU's data processing results can be transmitted to the NPU via the pass-through link, and correspondingly, the NPU's data processing results can also be transmitted to the GPU via the pass-through link. Based on this, the GPU's data processing results do not need to be forwarded by the CPU and can be directly transmitted to the NPU via the pass-through link; similarly, the NPU's data processing results also do not need to be forwarded by the CPU and can be directly transmitted to the GPU via the pass-through link. This reduces data transfer efficiency and latency between the GPU and NPU.

[0089] When multiple computing nodes are deployed on different computing devices, the computing nodes on different computing devices can transmit data through a communication network.

[0090] Based on the computing system described above, the computing system can be used to execute image rendering processes. For example... Figure 5 The schematic diagram shown in this application embodiment illustrates the image rendering process performed by the computing system, which may specifically include the following steps:

[0091] Step 501: The first computing node in the computing system determines the image rendering task corresponding to the control command sent by the user terminal. The image rendering task includes multiple subtasks.

[0092] In this embodiment, a user can operate a user terminal. The user terminal's display interface can run an app or display a web page. The user can trigger operations related to page display within the user terminal's display interface. The user terminal responds to the user's operations by sending control commands to the computing system. Optionally, the control commands can be used to characterize parameters related to page refresh; that is, the control commands can characterize the page that will be displayed on the user terminal's display interface based on the user's operations.

[0093] In one possible implementation, the computing system may include an information interaction unit; the user terminal responds to the user's operation and sends the control command to the information interaction unit in the computing system. For example, when the computing system is deployed in the cloud, it may include a cloud container, also known as a cloud phone container; the cloud container receives the control command sent by the user terminal and decodes it to obtain rendering control parameters. For instance, taking a game app running on a user terminal as an example, the user can trigger game-related operations (such as swiping up, swiping down, screen rotation, etc.) in the game app's display interface. The user terminal responds to the user's triggered operation in the game app's display interface, generates a control command related to that operation, and sends the control command to the cloud container. After receiving the control command, the cloud container decodes it to obtain the corresponding image rendering control parameters; for example, image rendering control parameters include light intensity, material, and viewpoint. The cloud container sends the decoded image rendering control parameters to the first computing node in the computing system.

[0094] The first computing node obtains the image rendering control parameters. Based on these parameters, it can determine the image rendering task corresponding to the control command. By executing this image rendering task, the computing system can complete the operation corresponding to the control command sent by the user terminal and obtain the rendered image corresponding to the control command.

[0095] Optionally, after determining the image rendering task, the first computing node can break down the image rendering task into multiple subtasks. These subtasks can form the rendering pipeline for the image rendering task. For example, the image rendering task may include at least two of the following subtasks: raster rendering, super-resolution, color gamut conversion, noise reduction, optical flow, frame interpolation, and encoding.

[0096] Step 502: The first computing node in the computing system determines the computing node corresponding to each subtask.

[0097] In step 502, the first computing node performs a computing power scheduling operation to allocate a corresponding computing node to each subtask in the image rendering task.

[0098] In this embodiment, multiple subtasks in the image rendering task correspond to at least two types of computing nodes, and different types of computing nodes include different types of processors. For example, multiple subtasks in the image rendering task correspond to at least two types of processors; such as multiple subtasks corresponding to CPU, GPU, and NPU, that is, the image rendering task is jointly executed by the CPU, GPU, and NPU in the computing system.

[0099] The following section details the scheme for determining the computing node corresponding to each subtask using the first computing node.

[0100] Option 1: The first computing node determines the computing node corresponding to each subtask based on the affinity between multiple subtasks and computing nodes.

[0101] Optionally, the first computing node can be a computing node in the computing system that includes a CPU or a DPU, or the first computing node can be a CPU or a DPU in the computing system.

[0102] In this embodiment, the affinity between a subtask and a computing node can be used to characterize whether the subtask and the computing node are compatible. A subtask being compatible with a computing node can be understood as the subtask being suitable to run on the computing node. Conversely, a subtask being incompatible with a computing node can be understood as the subtask being unsuitable to run on the computing node.

[0103] After determining the multiple subtasks included in the image rendering task, the first computing node assesses the affinity between each subtask and various types of computing nodes in the computing system. When determining the computing node corresponding to each subtask, the first computing node assigns the subtask to a node with affinity. For example, if the subtask is encoding, and the encoding subtask has a higher affinity with the NPU, then the first computing node can determine that the encoding subtask corresponds to the NPU.

[0104] In this embodiment of the application, the first computing node can determine the affinity between each subtask and various types of computing nodes in the computing system in a variety of different ways, which are described below:

[0105] Implementation Method 1: The first computing node detects whether each subtask is compatible with various types of computing nodes in the computing system; based on the detection results, it determines the computing nodes in the computing system that are compatible with the subtask.

[0106] During implementation, the first computing node can detect whether each subtask is compatible with the computing node in the following ways:

[0107] For each subtask, the first computing node can obtain the execution time of the subtask on the corresponding computing node; based on the execution time of the subtask, it determines whether the subtask is compatible with the computing node currently executing it. For example, if the execution time of the subtask is greater than a first set threshold, it is determined that the subtask is not compatible with the computing node currently executing it; if the execution time of the subtask is not greater than the first set threshold, it is determined that the subtask is compatible with the computing node currently executing it.

[0108] Optionally, if the subtask includes a first function, the first computing node can determine the affinity between the subtask and the computing node based on the affinity between the first function and the computing node. The first computing node can determine whether the first function is compatible with the computing node currently executing the first function based on the execution time of the first function on the corresponding computing node; for example, if the execution time of the first function exceeds a second preset threshold, then the first function is determined to be incompatible with the computing node currently executing the first function; if the execution time of the first function is not greater than the second preset threshold, then the first function is determined to be compatible with the computing node currently executing the first function. The first preset threshold and the second preset threshold can be the same or different.

[0109] It should be understood that if a subtask includes multiple first functions, the first computing node can determine the affinity between the subtask and the computing node based on the affinity between the multiple first functions and the computing node. For example, if the number of first functions in the subtask that are affinity-compatible with the computing node is greater than the number of first functions that are not affinity-compatible with the computing node, then the first computing node can determine that the subtask is affinity-compatible with the computing node. Alternatively, if a subtask includes multiple first functions, the first computing node can determine the affinity between the subtask and the computing node based on the affinity between the target first function among the multiple first functions and the computing node; the first computing node can select the target first function from the multiple first functions according to the set rules.

[0110] As one possible implementation, if the subtask does not include the first function, the first computing node can determine that the subtask is compatible with a set computing node; for example, if the subtask does not include the first function, the first computing node can determine that the subtask is compatible with the CPU or a computing node including the CPU.

[0111] Optionally, the first function can be an operator. For example, an artificial intelligence (AI) operator.

[0112] Implementation Method 2: The first computing node can obtain the pre-configured mapping relationship between subtasks and whether they are compatible with each type of computing node; based on this mapping relationship, the computing node corresponding to each subtask included in the image rendering task is determined.

[0113] Optionally, this mapping relationship can be pre-configured based on the design of the subtasks and the hardware design of the computing nodes.

[0114] In implementation, the mapping relationship can record the type of computing node that is compatible with each subtask. For example, the mapping relationship includes the affinity between the encoding subtask and the NPU, and the affinity between the raster rendering subtask and the GPU. For each subtask in the image rendering task, the first computing node selects a computing node that is compatible with the subtask from the various types of computing nodes included in the computing system, according to the mapping relationship.

[0115] It should be noted that, in the embodiments of this application, after the first computing node assigns a corresponding computing node to each subtask, during the execution of the corresponding subtask by each computing node in the computing system, the first computing node can detect the affinity between the subtask and the corresponding computing node, thereby updating the mapping relationship.

[0116] Option 2: The first computing node determines the computing node corresponding to each subtask based on the load information of multiple computing nodes in the computing system.

[0117] In this embodiment of the application, the first computing node can obtain the current load information of each computing node in the computing system, and allocate a corresponding computing node to each subtask based on the current load information of each computing node.

[0118] Optionally, the first computing node can obtain the load information of each computing node through the load awareness unit.

[0119] like Figure 6 The diagram illustrates the load awareness principle. The first computing node can call the load awareness unit to obtain the tasks currently being executed by each computing node from the task manager in the device manager, and to obtain the number of threads currently being executed by each computing node from the thread manager; for each computing node, the load awareness unit is called to estimate the computing load of the computing node and perform performance analysis on the computing node to obtain the load information of the computing node.

[0120] For example, when the first computing node assigns a corresponding computing node to each subtask, the first computing node can select a computing node with a smaller load from multiple computing nodes in the computing system to execute the corresponding subtask.

[0121] Alternatively, the first computing node can allocate a corresponding computing node to each subtask based on the current load information of each computing node in the computing system and the computing power required by the subtask.

[0122] For example, when the first computing node assigns a corresponding computing node to each subtask, for each subtask, the first computing node can select a matching computing node from the computing system for that subtask based on the computing power required by the subtask.

[0123] Option 3: The first computing node determines the computing node corresponding to each subtask based on the affinity between multiple subtasks and computing nodes, as well as the load information of multiple computing nodes.

[0124] In this scheme, the first computing node combines the affinity between the subtask and the computing node, as well as the load information of the computing node, to jointly determine the computing node corresponding to each subtask.

[0125] For Scheme 3, this application provides several different implementation methods.

[0126] Implementation method 1:

[0127] The first computing node determines a type of computing node corresponding to each subtask based on the affinity between multiple subtasks and computing nodes. For each subtask, if the type of computing nodes corresponding to the subtask includes multiple computing nodes, the computing node corresponding to the subtask is determined based on the load information of the multiple computing nodes.

[0128] Implementation method 1 can specifically include two steps:

[0129] Step 1a: The first computing node first determines a type of computing node corresponding to each subtask based on the affinity between multiple subtasks and computing nodes.

[0130] For each subtask in the image rendering task, the first computing node determines the affinity between the subtask and various types of computing nodes in the computing system, and selects a type of computing node with affinity for the subtask from the computing system. For example, a type of computing node can be a computing node including a GPU, or a computing node including an NPU, or a computing node including a CPU, or a computing node including a DPU; or a type of computing node can be at least one GPU, or at least one NPU, or at least one CPU, or at least one DPU.

[0131] It should be noted that in step 1, the method by which the first computing node determines the affinity between the subtask and various computing nodes can be found in the affinity determination method described in Scheme 1 above, and will not be repeated here.

[0132] Step 1b: For each subtask, if there are multiple computing nodes of the same type in the computing nodes corresponding to the subtask, the first computing node determines the computing node corresponding to the subtask based on the load information of the multiple computing nodes.

[0133] After the first computing node obtains a type of computing node corresponding to each subtask in the image rendering task based on step 1, the following operations are performed for each subtask:

[0134] The first computing node obtains the load information of each computing node in a certain type of computing nodes corresponding to the subtask, and selects the computing node corresponding to the subtask from the multiple computing nodes included in the certain type of computing nodes based on the load information of each computing node in the certain type of computing nodes.

[0135] For example, if the first computing node determines that there are multiple GPUs as a type of computing node for subtask A, then the first computing node selects the GPU with the lower load from the multiple GPUs as the GPU corresponding to subtask A based on the load information of each GPU, that is, the first computing node executes subtask A on the selected GPU.

[0136] It should be noted that the method by which the first computing node obtains the load information of each computing node in the type of computing nodes corresponding to each subtask can refer to the load information acquisition method introduced in Scheme 2 above, and will not be repeated here.

[0137] Implementation method 2:

[0138] The first computing node determines multiple candidate computing nodes for each subtask based on the load information of multiple computing nodes. These candidate computing nodes are of different types. For each subtask, the computing node corresponding to the subtask is determined from the multiple candidate computing nodes based on the affinity between the subtask and each candidate computing node.

[0139] Implementation method 1 can specifically include two steps:

[0140] Step 2a: The first computing node determines multiple candidate computing nodes corresponding to each subtask based on the load information of multiple computing nodes.

[0141] Among them, the multiple candidate computing nodes corresponding to each subtask are computing nodes of different types.

[0142] The first computing node obtains the current load information of each computing node in the computing system. For each type of computing node, the first computing node can select computing nodes with lower loads to form multiple candidate computing nodes. For example, the computing system includes computing node types such as CPU type computing nodes, GPU type computing nodes, and NPU type computing nodes; for example, CPU type computing nodes include CPU 0 and CPU 1, GPU type computing nodes include GPU 0, GPU 1, and GPU 2, and NPU type computing nodes include NPU 0, NPU 1, and NPU 2. Based on the load information of each computing node, the first computing node selects CPU 0 (lower load) from CPU 0 and CPU 1, GPU 1 (lower load) from GPU 0, GPU 1, and GPU 2, and NPU 1 (lower load) from NPU 0, NPU 1, and NPU 2. Thus, the multiple candidate computing nodes determined by the first computing node include CPU 0, GPU 1, and NPU 1.

[0143] It should be noted that the method by which the first computing node obtains the load information of each computing node can refer to the load information acquisition method introduced in Scheme 2 above, and will not be repeated here.

[0144] Step 2b: For each subtask, determine the corresponding computing node from multiple candidate computing nodes based on the affinity between the subtask and each candidate computing node.

[0145] For each subtask in the image rendering task, the first computing node determines the affinity between the subtask and each of the multiple candidate computing nodes; based on the affinity, the computing node corresponding to the subtask is selected from the multiple candidate computing nodes.

[0146] For example, the computing system includes computing node types such as CPU type computing nodes, GPU type computing nodes, and NPU type computing nodes; for instance, CPU type computing nodes include CPU 0 and CPU 1, GPU type computing nodes include GPU 0, GPU 1, and GPU 2, and NPU type computing nodes include NPU 0, NPU 1, and NPU 2. The first computing node, based on the load information of each computing node, determines multiple candidate computing nodes including CPU 0, GPU 1, and NPU 1. If the subtask is super-resolution, and super-resolution is compatible with NPU 1 among CPU 0, GPU 1, and NPU 1, then super-resolution corresponds to NPU 1; if the subtask is raster rendering, and raster rendering is compatible with GPU 1 among CPU 0, GPU 1, and NPU 1, then raster rendering corresponds to GPU 1.

[0147] In this application embodiment, the first computing node can be implemented in any of the above-described ways to determine the computing node corresponding to each subtask in the image rendering task.

[0148] Step 503: Each computing node in the computing system executes its corresponding subtask.

[0149] In this embodiment of the application, after determining the computing node corresponding to each subtask in the image rendering task, the first computing node schedules the computing node corresponding to each subtask and executes the subtask corresponding to the computing node.

[0150] It should be understood that when a subtask in a series of subtasks corresponds to the first computing node, the first computing node executes the corresponding subtask.

[0151] Optionally, the multiple subtasks in the image rendering task include a first type of subtask and a second type of subtask; wherein the first type of subtask includes at least one subtask for implementing raster rendering, and the second type of subtask includes at least one subtask for implementing neural rendering.

[0152] In this embodiment of the application, when the computing node is a computing node including a processor, the first type of subtask corresponds to a computing node including a GPU, and at least one subtask in the second type of subtask corresponds to a computing node including an NPU. For example, the first type of subtask includes raster rendering, and the GPU computing node is used to perform raster rendering; if the subtasks used to implement neural rendering include multiple subtasks, such as super-resolution, optical flow, frame interpolation, color gamut conversion, and noise reduction, then the NPU computing node is used to perform super-resolution, optical flow, frame interpolation, color gamut conversion, and noise reduction, or the NPU computing node is used to perform super-resolution, optical flow, and frame interpolation, and the GPU computing node is used to perform color gamut conversion and noise reduction.

[0153] When the computing node is a processor, the first type of subtask corresponds to the GPU, and at least one subtask in the second type of subtask corresponds to the NPU. For example, the first type of subtask includes raster rendering, which is performed by the GPU; if the subtasks used to implement neural rendering include multiple subtasks, such as super-resolution, optical flow, frame interpolation, color gamut conversion, and noise reduction, then the NPU is used to perform super-resolution, optical flow, frame interpolation, color gamut conversion, and noise reduction, or the NPU is used to perform super-resolution, optical flow, and frame interpolation, while the GPU is used to perform color gamut conversion and noise reduction.

[0154] In the computing system of this application embodiment, during the execution of corresponding subtasks, each computing node can also detect the affinity between the first function in each subtask and the corresponding computing node. Based on the affinity detection result, the first computing node determines whether to adjust the computing node corresponding to the first function.

[0155] Optionally, the image rendering task includes at least one subtask that includes a first function, and the first computing node determines the computing node corresponding to the first function based on the affinity between the first function and the computing node.

[0156] In implementation, for the first function in each subtask, the first computing node detects the affinity between the first function and the computing node executing the subtask. If the detection result indicates that the first function and the corresponding computing node are not compatible, the first computing node adjusts the computing node corresponding to the first function. Based on this method, the first computing node can dynamically adjust the computing node corresponding to the first function in the subtask, thereby improving the data processing efficiency of the computing system.

[0157] After each computing node in the computing system in this embodiment executes its corresponding subtask, the computing system generates rendered image data; the computing system can then send the rendered image data to the user terminal.

[0158] Optionally, after the computing system generates the rendered image data, it can encode the rendered image data and send the encoded rendered image data to the user terminal.

[0159] Accordingly, after receiving the rendered image data sent by the computing system, the user terminal decodes the rendered image data and displays it based on the decoded rendered image data, so that the user terminal displays the rendered image on the display interface.

[0160] The image rendering scheme of this application embodiment will be introduced below with an example.

[0161] like Figure 7As shown, taking a computing node as the processor as an example, the computing system includes at least one CPU, at least one GPU, at least one NPU, and a cloud container. A pass-through link exists between the GPU and NPU. The first computing node is the first CPU among the at least one CPU. The user terminal responds to a user-triggered operation by sending a control command to the cloud container. The cloud container decodes the control command to obtain the corresponding image rendering control parameters. The cloud container sends the image rendering control parameters corresponding to the control command to the first CPU, and the first CPU receives the image rendering control parameters. Based on the image rendering control parameters, the first CPU determines an image rendering task comprising multiple subtasks. Based on the affinity between the multiple subtasks and each CPU, GPU, and NPU, and / or based on the load information of each CPU, GPU, and NPU, the first CPU determines the processor corresponding to each subtask, i.e., orchestrates the various subtasks in the image rendering task and allocates a processor to each subtask. The first CPU invokes a load awareness unit to perceive the load information of each CPU, GPU, and NPU. For example, image rendering tasks include raster rendering, super-resolution, color gamut conversion, noise reduction, optical flow, frame interpolation, and encoding. Raster rendering corresponds to the GPU, while optical flow, super-resolution, frame interpolation, noise reduction, color gamut conversion, and encoding correspond to the NPU. Other computing power in image rendering tasks can be executed by the CPU.

[0162] Each CPU, GPU, and NPU in the computing system executes its corresponding subtask. During the execution of the corresponding subtask by each CPU, GPU, and NPU, the first CPU can detect the affinity between the first function in the subtask and the corresponding computing node. If the detection result indicates that the first function is not compatible with the corresponding processor, the first CPU adjusts the processor corresponding to the first function.

[0163] After each CPU, GPU, and NPU in the computing system completes its corresponding subtask, the rendered image data is obtained. The rendered image data can be sent to the user terminal via a cloud container; after receiving the rendered image data, the user terminal displays the rendered image on its display interface.

[0164] Taking a computing system deployed on a cloud server as an example, such as Figure 8As shown, the hardware structure of a cloud server can include a CPU, GPU, and NPU. After the user terminal sends control commands, the cloud server executes the image rendering process through the CPU, GPU, and NPU. A pass-through link (such as a P2P path) exists between the GPU and NPU. The base image and Mesa in the cloud container are common components, providing the basic runtime environment for the cloud container. The image capture engine in the cloud container is used to execute the raster rendering process; the AI ​​engine is used to execute the neural rendering process within the image rendering workflow; the host kernel of the cloud server includes a P2P management module, which interfaces with the P2P management module in the cloud container to enable direct data transmission between the GPU and NPU. The NPU driver in the host kernel drives the AI ​​engine to run on the NPU to execute the neural rendering process.

[0165] Based on the same technical concept as the above-described method embodiments, this application also provides a computing device for performing the above-described... Figure 5 The method shown in the embodiment executes the method on the first computing node. Figure 9 As shown, the computing device 900 includes an information transmission module 901, a computing power allocation module 902, and a scheduling module 903.

[0166] The information transmission module 901 can be used to receive control commands sent by the user terminal, determine the image rendering task corresponding to the control command, and the image rendering task includes multiple sub-tasks.

[0167] The computing power allocation module 902 can be used to determine the computing node corresponding to each subtask based on the affinity between multiple subtasks and computing nodes; wherein, multiple subtasks correspond to at least two types of computing nodes, and different types of computing nodes include different types of processors.

[0168] The scheduling module 903 can be used to schedule the computing nodes corresponding to each subtask and execute the corresponding subtask.

[0169] Optionally, the information transmission module 901 can provide the function of transmitting information between the computing device 900 and an external device. This information transmission includes receiving information input from the external device or outputting information to the external device. For example, the external device can be a user terminal, and the information transmission module 901 can be used to receive control commands input from the user terminal. Alternatively, the information transmission module 901 can include an information input module and an information output module. The information input module is used to receive information input from the external device, and the information output module is used to output information to the external device. For example, the external device can be a user terminal, and the information input module can be used to receive control commands input from the user terminal.

[0170] It should be noted that, in other embodiments, the information transmission module 901, the computing power allocation module 902, and the scheduling module 903 can all execute any step in the image rendering method. The steps implemented by the information transmission module 901, the computing power allocation module 902, and the scheduling module 903 can be specified as needed. By implementing different steps in the image rendering method through the information transmission module 901, the computing power allocation module 902, and the scheduling module 903, all functions of the computing device can be achieved.

[0171] For example, as described above, the information transmission module 901 can be used to receive control commands sent by the user terminal and determine the image rendering task corresponding to the control commands; in other embodiments, the information transmission module 901 can be used to receive control commands sent by the user terminal, and the computing power allocation module 902 can be used to determine the image rendering task corresponding to the control commands.

[0172] As one possible implementation, the image rendering task includes at least one subtask that includes a first function, and the computing power allocation module 902 is further configured to: determine the computing node corresponding to the first function based on the affinity between the first function and the computing node.

[0173] As one possible implementation, the computing power allocation module 902 is also used to: determine the computing node corresponding to each subtask based on the load information of multiple computing nodes.

[0174] As one possible implementation, the computing power allocation module 902 is also used to: determine a type of computing node corresponding to each subtask based on the affinity between multiple subtasks and computing nodes; for each subtask, if the type of computing nodes corresponding to the subtask includes multiple computing nodes, then determine the computing node corresponding to the subtask based on the load information of the multiple computing nodes.

[0175] As one possible implementation, the computing power allocation module 902 is also used to: determine multiple candidate computing nodes corresponding to each subtask based on the load information of multiple computing nodes, wherein the multiple candidate computing nodes are computing nodes of different types; and for each subtask, determine the computing node corresponding to the subtask from the multiple candidate computing nodes based on the affinity between the subtask and each candidate computing node.

[0176] As one possible implementation, the information transmission module 901 is used to: decode the received control command to obtain the rendering control parameters of the image rendering task; and determine the image rendering task including multiple subtasks based on the rendering control parameters.

[0177] Optionally, a pass-through link may be included between two different types of compute nodes, which is used to transfer data between the two different types of compute nodes.

[0178] Optionally, the computing node includes at least one of CPU, GPU, NPU, DPU, and FPGA.

[0179] Optionally, the multiple subtasks include a first type of subtask and a second type of subtask. The first type of subtask includes at least one subtask for implementing raster rendering, and the second type of subtask includes at least one subtask for implementing neural rendering. The first type of subtask corresponds to the GPU, and at least one subtask in the second type of subtask corresponds to the NPU.

[0180] As one possible implementation method, such as Figure 10 As shown, the computing device 900 in this embodiment may further include a raster rendering module 904, a neural rendering module 905, and an encoding module 906;

[0181] The raster rendering module 904 is used to synthesize images based on raster rendering; the neural rendering module 905 is used to perform neural rendering on images based on a deep neural network model, such as enhancing images based on a deep neural network model to improve image quality; and the encoding module 906 is used to encode the rendered image to obtain encoded rendered image data.

[0182] As one possible implementation, the information transmission module 901 is also used to send encoded rendered image data to the user terminal.

[0183] It should be understood that the code executing the functions of the raster rendering module 904, the neural rendering module 905, and the encoding module 906 in the embodiments of this application can run on the computing nodes corresponding to the various subtasks described above, so as to realize the functions of the raster rendering module 904, the neural rendering module 905, and the encoding module 906. For example, the code for implementing the function of the raster rendering module 904 can run on the GPU, and the code for implementing the functions of the neural rendering module 905 and the encoding module 906 can run on the NPU.

[0184] It should be noted that, in other embodiments, the raster rendering module 904, the neural rendering module 905, and the encoding module 906 can all execute any step in the image rendering method. The steps implemented by the raster rendering module 904, the neural rendering module 905, and the encoding module 906 can be specified as needed. By implementing different steps in the image rendering method through the raster rendering module 904, the neural rendering module 905, and the encoding module 906, all functions of the computing device can be achieved.

[0185] The module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0186] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a terminal device (which may be a personal computer, mobile phone, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0187] This application also provides, as in the embodiments, Figure 11 The computing device 1100 shown includes a bus 1101, at least one processor 1102, and a memory 1103. The processor 1102 and the memory 1103 communicate via the bus 1101.

[0188] When the computing system in this embodiment is a single computing device, the computing device 1100 can be a computing device; when the computing system in this embodiment is a cluster of computing devices, the computing device 1100 can be a computing device in the cluster of computing devices. For example, a computing device can be a server (including but not limited to a local server or a cloud server).

[0189] Optionally, the computing device 1100 may also include a network interface card 1104.

[0190] In this application embodiment, the processor 1102 included in the computing device 1100 can be a CPU, GPU, NPU, DPU, FPGA, etc. The processor 1102 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), discrete gate or transistor logic devices, discrete hardware components, etc.

[0191] The memory 1103 can be dynamic random access memory (DRAM). Besides DRAM, the memory 1103 can also be other types of random access memory, such as static random access memory (SRAM). Alternatively, the memory 1103 can also be read-only memory (ROM). For example, a read-only memory could be programmable read-only memory (PROM) or erasable programmable read-only memory (EPROM). The memory 1103 can also be flash memory, hard disk drive (HDD), or solid-state drive (SSD).

[0192] The memory 1103 stores computer program instructions, and the processor 1102 calls these computer program instructions to execute the aforementioned... Figure 5 The steps performed by the computing node in the described method. The memory 1103 may also include other software modules required for running processes, such as an operating system (e.g., Figure 9 or Figure 10 (Multiple modules in the computing device 900 shown).

[0193] This application also provides a communication system, which includes a user terminal and a computing system. The functions implemented by the user terminal and the computing system can be found in the description above.

[0194] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computer device, it causes the at least one computer device to perform the above-described... Figure 5The method performed by the computing node in the embodiment is described in [reference]. Figure 5 The details of each step will not be repeated here.

[0195] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing node can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing node to perform the above-described... Figure 5 The method performed by the computing node in the embodiment is described in [reference]. Figure 5 The details of each step will not be repeated here.

[0196] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0197] The various illustrative logic units and circuits described in the embodiments of this application can be implemented or operate the described functions using a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can also be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing devices, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0198] The steps of the methods or algorithms described in the embodiments of this application can be directly embedded in hardware, software units executed by a processor, or a combination of both. The software units can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium can be housed in an ASIC.

[0199] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0200] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely illustrative descriptions of the application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Thus, if such modifications and modifications fall within the scope of the claims and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. An image rendering method, characterized in that, The method includes: Receive control commands sent by user terminals, determine the image rendering task corresponding to the control commands, and the image rendering task includes multiple sub-tasks; Based on the affinity between the multiple subtasks and computing nodes, the computing node corresponding to each subtask is determined; wherein, the multiple subtasks correspond to at least two types of computing nodes, and the different types of computing nodes include different types of processors; Schedule the computing nodes corresponding to each subtask and execute the corresponding subtask.

2. The method as described in claim 1, characterized in that, The image rendering task includes at least one subtask comprising a first function, and the method further includes: The computing node corresponding to the first function is determined based on the affinity between the first function and the computing node.

3. The method as described in claim 1 or 2, characterized in that, The method further includes: Based on the load information of the multiple computing nodes, the computing node corresponding to each subtask is determined.

4. The method as described in claim 1 or 2, characterized in that, The method further includes: Based on the affinity between the multiple subtasks and computing nodes, a type of computing node is determined for each subtask. For each subtask, if a type of computing node corresponding to the subtask includes multiple computing nodes, then the computing node corresponding to the subtask is determined based on the load information of the multiple computing nodes.

5. The method as described in claim 1 or 2, characterized in that, The method further includes: Based on the load information of the multiple computing nodes, multiple candidate computing nodes are determined for each subtask, and the multiple candidate computing nodes are computing nodes of different types. For each subtask, the computing node corresponding to the subtask is determined from the plurality of candidate computing nodes based on the affinity between the subtask and each candidate computing node.

6. The method according to any one of claims 1 to 5, characterized in that, A pass-through link is included between two different types of computing nodes, the pass-through link being used to transmit data between the two different types of computing nodes.

7. The method according to any one of claims 1 to 6, characterized in that, The computing node includes at least one of CPU, GPU, NPU, DPU, and FPGA.

8. The method as described in claim 7, characterized in that, The plurality of subtasks includes a first type of subtask and a second type of subtask. The first type of subtask includes at least one subtask for implementing raster rendering, and the second type of subtask includes at least one subtask for implementing neural rendering. The first type of subtask corresponds to the GPU, and at least one subtask in the second type of subtask corresponds to the NPU.

9. A computing system, characterized in that, The system includes multiple computing nodes; The first computing node among the plurality of computing nodes is used to determine the image rendering task corresponding to the control command sent by the user terminal, wherein the image rendering task includes a plurality of subtasks; Based on the affinity between the multiple subtasks and computing nodes, the computing node corresponding to each subtask is determined; wherein, the multiple subtasks correspond to at least two types of computing nodes, and the different types of computing nodes include different types of processors; The computing node is used to execute the corresponding subtask.

10. The system as described in claim 9, characterized in that, The image rendering task includes at least one subtask comprising a first function, and the first computing node is further configured to: The computing node corresponding to the first function is determined based on the affinity between the first function and the computing node.

11. The system as described in claim 9 or 10, characterized in that, The first computing node is also used for: Based on the load information of the multiple computing nodes, the computing node corresponding to each subtask is determined.

12. The system as described in claim 9 or 10, characterized in that, The first computing node is also used for: Based on the affinity between the multiple subtasks and computing nodes, a type of computing node is determined for each subtask. For each subtask, if a type of computing node corresponding to the subtask includes multiple computing nodes, then the computing node corresponding to the subtask is determined based on the load information of the multiple computing nodes.

13. The system as described in claim 9 or 10, characterized in that, The first computing node is also used for: Based on the load information of the multiple computing nodes, multiple candidate computing nodes are determined for each subtask, and the multiple candidate computing nodes are computing nodes of different types. For each subtask, the computing node corresponding to the subtask is determined from the plurality of candidate computing nodes based on the affinity between the subtask and each candidate computing node.

14. The system according to any one of claims 11 to 13, characterized in that, A pass-through link is included between two different types of computing nodes, the pass-through link being used to transmit data between the two different types of computing nodes.

15. The system according to any one of claims 11 to 14, characterized in that, The computing node includes at least one of CPU, GPU, NPU, DPU, and FPGA.

16. The system as described in claim 15, characterized in that, The plurality of subtasks includes a first type of subtask and a second type of subtask. The first type of subtask includes at least one subtask for implementing raster rendering, and the second type of subtask includes at least one subtask for implementing neural rendering. The first type of subtask corresponds to the GPU, and at least one subtask in the second type of subtask corresponds to the NPU.

17. A computing device, characterized in that, The device includes: The information transmission module is used to receive control commands sent by the user terminal, determine the image rendering task corresponding to the control command, and the image rendering task includes multiple sub-tasks; The computing power allocation module is used to determine the computing node corresponding to each of the multiple subtasks based on the affinity between the multiple subtasks and the computing nodes; wherein, the multiple subtasks correspond to at least two types of computing nodes, and the different types of computing nodes include different types of processors; The scheduling module is used to schedule the computing nodes corresponding to each subtask and execute the corresponding subtask.

18. A computing device, characterized in that, The device includes at least one processor and at least one memory; wherein the one or more memories store one or more computer programs, the one or more computer programs including instructions that, when executed by the one or more processors, cause the computing device to perform the method as described in any one of claims 1 to 8.

19. A communication system, characterized in that, The system includes a user terminal and a computing system as described in any one of claims 9 to 16.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a computer program that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 8.

Images