Electronic device and method for scheduling an arithmetic accelerator

Abstract

The embodiment of the application discloses an electronic device and a method for scheduling an operation accelerator, and relates to the technical field of computers. The electronic device comprises a receiver configured to receive at least one first data unit, wherein the at least one first data unit comprises a control signal and a control parameter; a parser configured to parse the at least one first data unit to obtain the control signal and the control parameter, store the control parameter in a memory, and transmit the control signal to a hardware task scheduler; the hardware task scheduler is configured to schedule at least one operation accelerator based on the control signal; and the at least one operation accelerator is configured to obtain the control parameter from the memory in response to the scheduling, and perform data processing by using the control parameter. The electronic device provided by the embodiment of the application can reduce the transmission delay of the control signal.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly to an electronic device and a method for scheduling a computing accelerator. Background Technology

[0002] With the development of electronic and computer technologies, the performance of electronic devices has improved rapidly. Existing electronic devices typically incorporate a system-on-a-chip (SoC). In addition to integrating a central processing unit (CPU), the SoC also integrates an increasing number of domain-specific computing accelerators, such as domain-specific architectures (DSAs), to provide hardware acceleration for specific business scenarios, thereby achieving various services with higher energy efficiency, such as video playback and game super-resolution. Examples of such DSAs include neural network processing units (NPUs) and digital signal processors (DSPs).

[0003] Beyond the aforementioned SoC, other components are typically included, such as the display driver integrated circuit (DDIC) within the display. External chips usually have limited computing power due to area and manufacturing process limitations. In some scenarios, it's necessary to leverage the computing power of the SoC, which involves cross-chip computing power collaboration solutions. For example, to achieve artificial intelligence (AI)-based photography, the external camera chip needs to call the SoC's NPU for calculations and return the results to the external camera chip. However, existing cross-chip computing power collaboration solutions suffer from high latency. Summary of the Invention

[0004] The electronic device and method for scheduling computing accelerators provided in this application can reduce latency. To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0005] In a first aspect, embodiments of this application provide an electronic device comprising: a receiver for receiving at least one first data unit, the at least one first data unit including a control signal and control parameters; a parser for parsing the at least one first data unit to obtain the control signal and control parameters, storing the control parameters in a memory, and transmitting the control signal to a hardware task scheduler; a hardware task scheduler for scheduling at least one computing accelerator based on the control signal; and at least one computing accelerator for retrieving the control parameters from the memory in response to the scheduling, and performing data processing using the control parameters.

[0006] The electronic device provided in this application embodiment, by setting up a receiver, a parser, a hardware task scheduler, and a computing accelerator, allows the electronic device to receive data units carrying control signals and parameters from external chips or components when an external chip or component needs to access the computing power of the computing accelerator on the electronic device. The parser, after parsing the data units to obtain the control signals and parameters, can directly send the control signals to the hardware task scheduler, enabling the hardware task scheduler to schedule the computing accelerator without requiring multi-layered switching and invocation of kernel-mode driver software, user-mode service software, and interface software. After responding to the scheduling by the hardware task scheduler, the computing accelerator can perform data processing based on the control parameters. Therefore, compared with existing cross-chip computing power collaboration schemes, this application embodiment can significantly shorten the control signal transmission latency. Furthermore, in this application embodiment, by setting up a parser to parse the data units, there is no need to modify the existing bus transmission protocol and bus hardware communication path. This allows for efficient and fast inter-chip task scheduling with a simple design and low cost.

[0007] Based on the first aspect, in one possible implementation, the parser is either a software-driven processor or a hardware parser. A software-driven processor may include, but is not limited to, a central processing unit or other dedicated processor.

[0008] Based on the first aspect, in one possible implementation, at least one first data unit includes a third data unit and a fourth data unit, wherein the third data unit includes a control signal and the fourth data unit includes control parameters.

[0009] Based on the first aspect, in one possible implementation, at least one first data unit includes a fifth data unit, which includes control signals and control parameters. By setting the control signals and control parameters in the fifth data unit, the control signals and control parameters can be transmitted using the same data unit, reducing the number of data unit transmissions.

[0010] Based on the first aspect, in one possible implementation, the receiver is further configured to: receive at least one second data unit, the at least one second data unit including first service data; the parser is further configured to: parse the at least one second data unit to obtain the first service data, and store the first service data in a memory; and at least one computing accelerator is specifically configured to: perform data processing on the first service data using control parameters.

[0011] Based on the first aspect, in one possible implementation, at least one first data unit further includes second service data; the parser is also configured to: obtain the second service data while parsing at least one first data unit, and store the second service data in a memory; at least one computing accelerator is specifically configured to: perform data processing on the second service data using control parameters.

[0012] Based on the first aspect, in one possible implementation, the receiver is specifically configured to: receive at least one first data unit from an external chip, wherein the external chip includes at least one of the following: a communication chip, a power amplifier chip, a display driver integrated circuit chip, or a camera chip.

[0013] Based on the first aspect, in one possible implementation, at least one computing accelerator includes at least one of the following: an image signal processor (ISP), a graphics processing unit (GPU), a neural network processor (NPU), or a digital signal processor (DSP).

[0014] Based on the first aspect, in one possible implementation, the electronic devices are integrated into the same chip.

[0015] Secondly, embodiments of this application provide a method for scheduling a computing accelerator. The method includes: receiving at least one first data unit from a receiver, the first data unit including a control signal and control parameters; parsing the at least one first data unit from a parser to obtain the control signal and control parameters; storing the control parameters in a memory by the parser and transmitting the control signal to a hardware task scheduler; scheduling at least one computing accelerator based on the control signal by the hardware task scheduler; and having at least one computing accelerator, in response to scheduling, retrieve the control parameters from the memory and perform data processing using the control parameters.

[0016] Based on the second aspect, in one possible implementation, at least one first data unit includes a third data unit and a fourth data unit, wherein the third data unit includes a control signal and the fourth data unit includes control parameters.

[0017] Based on the second aspect, in one possible implementation, at least one first data unit includes a fifth data unit, which includes control signals and control parameters.

[0018] Based on the second aspect, in one possible implementation, the method further includes: receiving at least one second data unit by a receiver, the at least one second data unit including first service data; parsing the at least one second data unit by a parser to obtain the first service data, and storing the first service data in a memory; and performing data processing using control parameters, including: performing data processing on the first service data using control parameters.

[0019] Based on the second aspect, in one possible implementation, at least one first data unit further includes second service data; the method further includes: obtaining the second service data while parsing at least one first data unit, and storing the second service data in a memory; and performing data processing using control parameters, including: performing data processing on the second service data using control parameters.

[0020] Based on the second aspect, in one possible implementation, receiving at least one first data unit by the receiver includes: receiving at least one first data unit by the receiver from an external chip, wherein the external chip includes at least one of the following: a communication chip, a power amplifier chip, a display driver integrated circuit chip, or a camera chip.

[0021] Thirdly, embodiments of this application also provide an electronic device, which includes the electronic device and storage device as described in the first aspect.

[0022] Fourthly, embodiments of this application also provide a computer-readable storage medium for storing a computer program, which, when executed by an electronic device, is used to implement the method described in the second aspect or any possible implementation thereof.

[0023] Fifthly, embodiments of this application also provide a computer program product, which, when executed by an electronic device, is used to implement the method described in the second aspect or any possible implementation thereof.

[0024] It should be understood that the second to fifth aspects of this application are consistent with the technical solutions of the first aspect of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, so they will not be described again. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of a chip architecture installed within an electronic device;

[0026] Figure 2 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0027] Figure 3The embodiments provided in this application are as follows Figure 2 A schematic diagram illustrating the interaction flow between the components in the electronic device shown.

[0028] Figure 4A This is a schematic diagram of data unit A1 provided in an embodiment of this application;

[0029] Figure 4B This is a schematic diagram of data unit A2 and data unit A3 provided in an embodiment of this application;

[0030] Figure 4C This is a schematic diagram of data unit B provided in an embodiment of this application;

[0031] Figure 4D This is a schematic diagram of data unit A4 provided in an embodiment of this application;

[0032] Figure 5 The embodiments provided in this application are as follows Figure 2 This is another schematic diagram illustrating the interaction process between the components in the electronic device shown.

[0033] Figure 6 The embodiments provided in this application are as follows Figure 2 This is another schematic diagram illustrating the interaction process between the components in the electronic device shown.

[0034] Figure 7 This is another hardware structure diagram of the electronic device provided in the embodiments of this application;

[0035] Figure 8 This is a flowchart of a method for scheduling a computing accelerator provided in an embodiment of this application. Detailed Implementation

[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the embodiments of this application.

[0037] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0038] The terms "first" and "second," etc., in the specification and drawings of the embodiments of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0039] Furthermore, the term "comprising" and any variations thereof mentioned in the description of the embodiments of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0040] It should be noted that in the description of the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0041] In the description of the embodiments of this application, unless otherwise stated, "a plurality of" means two or more.

[0042] Existing electronic devices typically include a System-on-a-Chip (SoC) and peripheral chips located outside the SoC, such as... Figure 1 As shown, Figure 1 This is a schematic diagram of a chip architecture for an electronic device. A System-on-a-Chip (SoC) integrates a CPU or processor, and in addition, an NPU is also integrated within the SoC. Figure 1 The peripheral chips, excluding the SoC, schematically illustrate the Wi-Fi chip. Furthermore, Figure 1 The chip architecture shown also includes memory. Typically, peripheral chips, such as Wi-Fi chips, have limited computing power due to limitations in area and manufacturing process. Therefore, to achieve certain functions, peripheral chips usually need to utilize the computing power of the SoC (System-on-a-Chip), which involves cross-chip computing power collaboration solutions.

[0043] Taking AI-enabled Wi-Fi functionality as an example, the Wi-Fi chip needs to call upon the NPU (Neural Processing Unit) on the SoC (System-on-a-Chip) side for computation. When the Wi-Fi chip invokes the NPU's computing power, it needs to send control signals and parameters to the SoC via the bus. Since the transmission paths of the control signals and parameters on the SoC are different, the Wi-Fi chip typically transmits these signals and parameters using different data frames. The Wi-Fi chip usually first stores the parameter signals in memory via the bus; then, it sends the control signals to the processor via the bus to notify the kernel-mode Wi-Fi driver software. The Wi-Fi driver software switches back to user mode to notify the Wi-Fi service software, which then calls the NPU interface software via inter-process communication. The NPU interface software triggers the NPU to execute hardware calculations through the NPU driver software. The NPU reads the parameter signals from memory and performs hardware calculations based on these signals. Furthermore, after the NPU completes its calculations, it sends a reverse notification to the Wi-Fi chip via the NPU interface software, the Wi-Fi service software, the Wi-Fi driver software, and the bus, enabling the Wi-Fi chip to obtain the calculation results. As can be seen from the above, in existing cross-chip computing power collaboration solutions, if peripheral chips call upon the SoC-side accelerator, the CPU needs to switch between kernel-mode software and user-mode software multiple times, resulting in high latency, for example, on the order of several milliseconds (ms). For some latency-sensitive services, this excessive latency cannot meet the requirements.

[0044] The electronic device provided in this application embodiment, by setting up a receiver, a parser, a hardware task scheduler, and a computing accelerator, allows the electronic device to receive data units carrying control signals and parameters from external chips or components when an external chip or component needs to access the computing power of the computing accelerator on the electronic device. The parser, after parsing the data units to obtain the control signals and parameters, can directly send the control signals to the hardware task scheduler, enabling the hardware task scheduler to schedule the computing accelerator without requiring multi-layered switching and invocation of kernel-mode driver software, user-mode service software, and interface software. After responding to the scheduling by the hardware task scheduler, the computing accelerator can perform data processing based on the control parameters. Therefore, compared with existing cross-chip computing power collaboration schemes, this application embodiment can significantly shorten the control signal transmission latency. Furthermore, in this application embodiment, by setting up a parser to parse the data units, there is no need to modify the existing bus transmission protocol and bus hardware communication path. This allows for efficient and fast inter-chip task scheduling with a simple design and low cost. The following is a combination of... Figures 2 to 7 The embodiments of this application will be described in detail below.

[0045] Please refer to Figure 2 , Figure 2This is a schematic diagram of the hardware architecture of an electronic device 100 provided in an embodiment of this application. The electronic device 100 may be located within an electronic device. The electronic devices include, but are not limited to: mobile phones, tablets, personal computers, handheld computers, mobile internet devices (MIDs), cameras, wearable devices (such as smartwatches, smart bracelets, pedometers, etc.), audio equipment, audio and video players, set-top boxes, game consoles, printers, mice, keyboards, in-vehicle equipment (such as equipment on vehicles like cars, airplanes, ships, trains, and high-speed trains), virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, smart home devices (such as refrigerators, televisions, air conditioners, electricity meters, etc.), intelligent robots, workshop equipment, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, or wireless terminals in smart homes, and flying equipment (such as intelligent robots, hot air balloons, drones, airplanes), etc. Figure 2 This is merely one example of electronic device 100. Electronic device 100 can be any type of electronic component located within the electronic device or the electronic device itself. For example, electronic device 100 can be a chip or chipset, or a circuit board equipped with a chipset, etc. This embodiment is not limited in this respect. The aforementioned chipset, or circuit board equipped with a chipset, etc., can operate under suitable software drivers.

[0046] Electronic device 100 may include multiple components, including a receiver 11, a parser 12, a hardware task scheduler 13, and at least one computing accelerator 14. The hardware task scheduler 13 may also be referred to as a heterogeneous graph task scheduler (HTS), and will be described as such hereinafter. The electronic device 100 provided in this application embodiment may also include more or fewer components, such as a central processing unit (CPU). Optionally, the electronic device 100 shown in this application embodiment may be integrated into one or more chips, which may be placed in a chipset. For example, the receiver 11, parser 12, and hardware task scheduler 13 are integrated into chip 1, and at least one computing accelerator is integrated into chip 2; or, for another example, all components of the electronic device 100 are integrated into the same chip, which may also be referred to as a system on a chip (SOC), such as... Figure 2 As shown. In this embodiment, the electronic device 100 is integrated into the same chip (e.g., Figure 2 The description uses electronic device 100 as an example of SOC (System-on-Chip), but it is not intended to limit the solution.

[0047] Receiver 11, also known as an interface, communicates with external chips (hereinafter referred to as external chips) or components via bus 20. Bus 20 may include, but is not limited to, at least one of the following: Peripheral Component Interconnect Express (PCIE) bus, Universal Serial Bus (USB), Inter-Integrated Circuit (I2C) bus, or Mobile Industry Processor Interface (MIPI) bus. Therefore, the SOC may include one or more receivers 11, and each receiver 11 may be of the following types, including but not limited to: USB interface, PCIE interface, I2C interface, or MIPI interface. External chips may include, but are not limited to: communication chips, power amplifier (PA) chips, display driver integrated circuit chips, or camera chips. Communication chips may include, for example, WIFI chips, Bluetooth chips, or 5G chips; external components may include, for example, audio components, display components, or storage components. It is understood that different types of external chips or components may communicate with the SOC via different buses. Figure 2The diagram schematically illustrates an external chip including a Wi-Fi chip and a DDIC chip. Therefore, the SOC has two receivers 11, one with a PCIe interface and the other with a MIPI interface. The Wi-Fi chip communicates with the SOC via the PCIe bus, and the DDIC chip communicates with the SOC via the MIPI bus. The PCIe interface receives data units from the Wi-Fi chip via the PCIe bus; the MIPI interface receives data units from the DDIC chip via the MIPI bus. When the bus is a PCIe bus or a USB bus, the data unit can also be called a data packet; when the bus is I2C, the data unit can also be called a byte. For the specific structure and content of the data unit using the PCIe bus as an example, please refer to [reference needed]. Figures 4A to 4D Related descriptions.

[0048] The parser 12 can act as a proxy for the external chip 30, decapsulating the data units sent by the external chip 30 to recover control signals and control parameters from the data units, and sending the control signals to the HTS 13 and storing the control parameters in the aforementioned memory 40. The control signals are used to instruct the arithmetic accelerator 14 to start executing data processing tasks; the control parameters may include, but are not limited to, wireless signal strength sequences or histogram statistics. The parser 12 can include various implementations: in one possible implementation, the parser 12 can be a hardware parser, such as, but not limited to, an integrated circuit (ASIC), discrete gates, transistor logic devices, or discrete hardware components; in another possible implementation, the parser 12 can also be a software-driven processor, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), or other types of programmable logic devices. In one possible implementation, the bus used to connect the external chip 30 to the SoC includes various types, and multiple parsers 12 can be included. Each parser 12 corresponds to a specific bus type to decode data units based on the bus protocol. For example, a PCIe interface is connected to parser 12A, which parses data units received by the PCIe interface based on the PCIe bus protocol. An I2C interface is connected to parser 12B, which decodes data units sent by the I2C bus based on the I2C bus protocol. In another possible implementation, the parser 12 configured in the SoC can decode data units sent by various types of buses. For example, both the PCIe interface and the I2C interface are connected to the same parser 12. Parser 12 can decode data units received from the PCIe interface using the PCIe bus protocol, and it can parse data units received from the I2C interface using the I2C bus protocol. The embodiments in this application are described below using the example of a parser 12 that can parse data units sent by multiple types of buses, but this is not intended to limit the solution.

[0049] HTS13 can schedule at least one computing accelerator 14 based on the control signals sent by parser 12. Specifically, HTS13 can generate dependency information, which indicates the dependencies between multiple tasks and the computing accelerators 14 to which the multiple tasks belong; thus, HTS13 can also schedule multiple computing accelerators 14 based on the dependency information. HTS13 can forward control signals to the corresponding computing accelerators 14 based on the multiple computing accelerators 14 indicated by the dependency information. For example, if the ISP and NPU are called by external chip 30, and the output of NPU depends on the input of ISP, HTS13 can forward control signals to ISP. After ISP sends a signal to HTS13 indicating that data processing is complete, HTS13 can forward control signals to NPU. The scheduling of HTS13 includes scheduling a task, i.e., the business data to be processed, to the corresponding computing accelerator. This scheduling includes, but is not limited to: assigning execution tasks to any one or more computing accelerators 14, controlling the task execution time of one or more computing accelerators, controlling the execution order of multiple computing accelerators, or stopping the execution of any one or more computing accelerators. For details on the specific scheduling process of HTS13 for computing accelerator 14, please refer to the patent with publication number "CN118377587A".

[0050] The computing accelerator 14 may include, but is not limited to, an NPU, a graphics processing unit (GPU), an image signal processor (ISP), and a DSP. It is understood that the computing accelerator 14 may also include more or fewer domain-specific processors or domain-specific processing modules; this embodiment does not impose specific limitations. The computing accelerator 14 responds to the scheduling of HTS 13 (i.e., responds to control signals), reads control parameters from memory 40, and performs data processing using the control parameters.

[0051] In this embodiment of the application, the electronic device equipped with electronic device 100 may further include one or more other components, such as memory 40. Memory 40 may exemplary include volatile memory, such as dynamic random access memory (DRAM), synchronous dynamic random-access memory (SDRAM), or double data rate (DDR) SDRAM, etc. Memory 40 may be selectively disposed on or outside the aforementioned SoC. Figure 2The diagram schematically illustrates a scenario where the memory 40 is located outside the aforementioned SoC. The memory 40 can be used to store data units, and it can also be used to store control parameters after the parser 12 has parsed the data units. Furthermore, the memory 40 can also be used to store service data transmitted between the SoC and the external chip 30, which belongs to the tasks of the corresponding computing accelerators according to scheduling. In one possible implementation, a portion of the storage space in the memory 40 can be shared by the receiver 11, the parser 12, and the computing accelerator 14, and the data stored in this portion of the storage space can be accessed by the receiver 11, the parser 12, and the computing accelerator 14. In addition to the aforementioned portion of the storage space, the memory 40 can also store various instructions and programs of the computing accelerator 14; when the parser 14 is a software-driven processor, the memory 40 can also store the instructions and programs of the parser 14. In addition, the SoC may integrate one or more other components, such as a cache and a system bus for connecting the various components inside the SoC. The receiver 11, parser 12, computing accelerator 14 and cache are all coupled to the system bus to enable communication between the receiver 11, parser 12, computing accelerator 14 and cache.

[0052] based on Figure 2 The hardware architecture shown below will be explained in the following sections. Figure 3 The interactive flow 300 shown further describes the electronic device provided in the embodiments of this application. For example... Figure 3 As shown, Figure 3 This describes the interaction process 300 between the external chip 30 and the SoC. This interaction process 300 includes the following steps S301 to S308.

[0053] S301, the external chip 30 encapsulates control signals and control parameters based on a bus protocol to generate data unit A. S302, the external chip 30 sends data unit A to the receiver 11 on the SOC side via bus 20. This data unit is capable of carrying service data and is not interface control information for implementing chip interface protocol control.

[0054] When the external chip 30 needs to utilize the computing power of the integrated computing accelerator 14 within the SoC, it needs to provide control signals and control parameters to the computing accelerator 14. The control parameters provided by the external chip 30 to the computing accelerator 14 are the input parameters required for the computing accelerator 14 to perform calculations. The type of these control parameters is typically related to the type of algorithm running on the computing accelerator 14 and the computing characteristics of the computing accelerator 14. In a specific example, when the external chip 30 is a Wi-Fi chip, the aforementioned bus 20 can be a PCIe bus. If the Wi-Fi chip needs to enable AI functions, it can utilize the computing power of the integrated NPU within the SoC. The control parameters provided by the Wi-Fi chip to the NPU can be a wireless signal strength sequence. When the external chip 30 is a camera chip, the aforementioned bus 20 can be a MIPI bus. If the camera chip needs to activate image processing functions, it can utilize the computing power of the integrated ISP within the SoC. The control parameters provided by the camera chip to the ISP can be histogram statistical information.

[0055] In one possible implementation, the chips on both sides of bus 20 can communicate according to the standard protocol architecture of the bus. That is, the data unit A described in this embodiment can be a data unit obtained by external chip 30 using the standard protocol of bus 20 to encapsulate control signals and control parameters, rather than the interface standard protocol control information of bus 20. A more detailed description is given using the PCIe bus as an example. The standard protocol of the PCIe bus includes the physical layer, the data link layer, and the transport layer. When encapsulating control signals and control parameters, the standard cooperative header of the PCIe bus (including the physical layer header, the data link layer header, and the transport layer header) can be kept unchanged. The aforementioned control signals and control parameters are added to the data field in the transport layer to generate data unit A. For example... Figure 4A As shown, Figure 4AThe diagram schematically illustrates the structure of a data unit A1 based on the PCIe bus protocol. Building upon the PCIe bus protocol, the data field can further include a type field, an event field, and a parameter field. The type field indicates whether the data unit contains control signals. For example, the type field can be two bits: "00" represents that there are no control signals in data unit A1, and "01" represents that data unit A1 contains control signals. Control signals can be written to the event field, which is only valid when the type field indicates the presence of control signals in data unit A1. The event field can include multiple bits, which can include multiple combinations, each combination indicating the type of a computational accelerator 14. Assuming the SoC includes four computing accelerators 14: NPU, ISP, GPU, and DSP, the event field can include two bits: "00" for NPU, "01" for ISP, "10" for GPU, and "11" for DSP. Control parameters provided by the external chip 30 to the computing accelerator 14 are written to the control parameter field. For example, assuming the Wi-Fi chip utilizes the computing power of the NPU, the Wi-Fi chip can... Figure 4A In the data unit A1 shown, the type field is set to "01", the event field is set to "00", and the wireless signal strength sequence is written to the parameter field.

[0056] In this embodiment, the external chip 30 can use one or more data units A to transmit the aforementioned control signals and control parameters to the receiver 11. In one possible implementation, a data unit A may include control signals and control parameters, for example... Figure 4A As shown in data unit A1, external chip 30 can transmit control signals and control parameters to receiver 11 through the same data unit A1. In another possible implementation, the control signals and control parameters can be set in multiple data units A. When control signals and control parameters are set in multiple data units A, each data unit in A conforms to the PCIe bus protocol to encapsulate the control signals and control parameters. Taking data units A2 and A3 as an example, data unit A2 includes control signals; data unit A3 includes control parameters, such as... Figure 4B As shown. Data unit A2's data field includes a type field and an event field; data unit A3's data field includes a type field and a parameter field. The information indicated by the event field and the parameter field can be compared with... Figure 4AThe corresponding fields shown indicate the same information and will not be described again. When control signals and control parameters are set in multiple data units A, the type field can be used to indicate the type of signal included in the data field; for example, the type field in data unit A2 is used to indicate that data unit A2 includes only control signals; the type field in data unit A3 is used to indicate that data unit A3 includes only control parameters.

[0057] In step S303, receiver 11 stores data unit A in memory 40 and sends interrupt signal 1 to parser 12. In one possible implementation, parser 12 can be directly connected to receiver 11 located on the SoC via a physical line. After receiver 11 receives data unit A from external chip 30 and data unit A is transmitted to the SoC side, bus 20 can send interrupt signal 1 to parser 12 via a physical line.

[0058] S304, Parser 12 reads data unit A from memory 40 based on interrupt signal 1. S305, Parser 12 parses data unit A to obtain control signals and control parameters. S306, Parser 12 sends the control signals to HTS 13 and stores the control parameters in memory 40. In one possible implementation, parser 12 can be directly connected to HTS 13 via hardware lines. After receiving interrupt signal 1 from bus 20, parser 12 can read data unit A from memory 40; then, parser 12 can decode data unit A based on the protocol type of bus 20 and using the corresponding bus 20 protocol to recover the control signals and control parameters; then, it stores the control parameters in memory 40 and sends the control signals to HTS 13.

[0059] S307, HTS13 sends a control signal to the computation accelerator 14 to trigger the computation accelerator 14 to perform the corresponding calculation. HTS13 can schedule the control signal based on the type of computation accelerator 14 indicated by the control signal. Specifically, HTS13 can forward the control signal to the corresponding computation accelerator 14 to trigger the computation accelerator 14 to perform the corresponding calculation. For example, assuming the computation accelerator 14 indicated by the control signal is an NPU, then HTS13 forwards the control signal to the NPU.

[0060] S308, the computation accelerator 14 reads control parameters from the memory 40 based on the control signal. S309, the computation accelerator 14 performs calculations using the control parameters to generate data processing results. The computation accelerator 14 reads control parameters from the memory 40 based on the triggering of the control signal; then it performs calculations on the business data using the control parameters. In a specific example, if the computation accelerator 14 is an NPU, the calculations performed by the computation accelerator 14 can be, for example, neural network calculations; if the computation accelerator 14 is an ISP, the calculations performed by the computation accelerator 14 can be, for example, image processing calculations; if the computation accelerator 14 is a DSP, the calculations performed by the computation accelerator 14 can be, for example, data processing calculations. Thus, the computation accelerator 14 generates the final data processing results.

[0061] The above has been approved. Figure 3 The interactive flow shown illustrates how the external chip 30 sends control signals and control parameters to the computing accelerator 14 in the SoC, thereby utilizing the computing power of the computing accelerator 14 to perform calculations based on the control parameters. Figure 3 As can be seen from the interactive flow shown, the electronic device provided in this application embodiment, by setting up a receiver 11, a parser 12, an HTS 13, and a computing accelerator 14, allows the receiver 11 on the SOC to receive data units carrying control signals and control parameters from the external chip 30 when the external chip 30 needs to utilize the computing power of the computing accelerator 14 on the SOC. The parser 12, after parsing the data units and obtaining the control signals and parameters, can directly send the control signals to the HTS 13, enabling the HTS 13 to schedule the computing accelerator 14 without requiring multi-layered switching and invocation through kernel-mode driver software, user-mode service software, and interface software. After responding to the scheduling by the HTS 13, the computing accelerator 14 can perform data processing based on the control parameters. Therefore, compared with the cross-chip computing power collaboration scheme in the prior art, the embodiments of this application can greatly shorten the control signal transmission delay and improve the running speed. In addition, in the embodiments of this application, by setting up a parser 12, the data unit is parsed by the parser 12. There is no need to modify the existing bus transmission protocol and bus hardware communication path. It can be compatible with the processor architecture in the prior art. Based on the existing processor architecture, efficient and fast inter-chip task scheduling can be achieved with simple design and low cost.

[0062] In such Figure 3 Above the interaction flow between the various components shown, the business data required for computation by the computing accelerator 14 can be obtained from the external chip 30. The SOC can obtain business data from the external chip 30 in various ways. Among these, business data may include, but is not limited to, RGB images or YUV images.

[0063] In the first possible implementation, the external chip 30 can, based on the standard protocol using bus 20, set the service data required by the computing accelerator 14 in data unit B, and transmit data unit B to the receiver 11 on the SOC side via bus 20. Data unit B can be a different data unit from data unit A as described above, but encapsulated using the same interface standard protocol. Taking the PCIe bus as an example, the structure of data unit B can be similar to... Figure 4A The structure shown is similar; business data can be set in the data field of the transport layer, such as... Figure 4C As shown. Data unit B may include a type field, which indicates that the data type transmitted by data unit B is business data; for example, the type field can be "10" to indicate that the data field carries business data. Based on this first possible implementation, refer to... Figure 5 ,like Figure 5 As shown, the process also includes the following steps: S401, receiver 11 receives data unit B from external chip 30. S402, receiver 11 stores data unit B in memory 40 and sends interrupt signal 2 to parser 12. S403, parser 12 retrieves data unit B from memory 40. S404, parser 12 parses data unit B to obtain service data. S405, parser 12 saves the service data to memory 40. S406, accelerator 14 retrieves service data from memory 40. In step S309 above, accelerator 14 uses control parameters to perform data processing on the service data and generate data processing results.

[0064] In the second possible implementation, the business data required for data processing by the computing accelerator 14 can be placed in data unit A and transmitted to receiver 11 along with control signals and control parameters. Based on this second possible implementation, Figure 4A Based on the data unit A1 shown, the data field in the transport layer can also include a service data field, which is used to carry service data, such as... Figure 4D As shown. Figure 4D In the data unit A4 shown, the `type` field can be used to indicate that data unit A4 simultaneously includes control signals, control parameters, and service data. For example, the `type` field can be "11" to indicate that the `data` field carries control signals, control parameters, and service data. Therefore, based on this second possible implementation, Figure 3 The S301 shown is replaced by: an external chip 30 that can encapsulate control signals, control parameters and business data to generate data unit A; Figure 3The S305 shown is replaced by: parser 12 parses data unit A to obtain control signals, control parameters and service data; Figure 3 The S306 shown is replaced by: the parser 12 sends the control signal to the computing accelerator 14 and stores the control parameters and business data in the memory 40; Figure 3 The S308 shown is replaced by: the computing accelerator 14 uses control parameters to perform data processing on the business data and generate data processing results. That is to say, the business data can be placed in any one or more data units, including but not limited to data units used to transmit control signals and control parameters.

[0065] It is understandable that the external chip 30 can be based on a bus protocol and adopt... Figures 4A to 4D One or more data units in the system transmit control signals, control parameters, and service data to the SOC side; thus, the parser 12 can, based on the bus protocol, [process / process]... Figures 4A to 4D One or more data units are parsed to obtain control signals, control parameters, and business data.

[0066] In one possible implementation of this application embodiment, after the computation accelerator 14 in the SoC completes its computation, it can return the result to the external chip 30 via the parser 12 and the bus 20. It should be noted that the receiver 11 (also called a receiving unit) described in the above embodiments can be part of an interface, receiving data units from the external chip 30 via the bus 20. In addition to the receiving unit, the interface may also include a transmitting unit, which can communicate with the external chip or component via the bus 20 to send data units to the external chip. (Continue to refer to...) Figure 6 The interactive flow shown is based on steps S301 to S309 above. Figure 6 It also includes steps S310 to S315.

[0067] S310, the computation accelerator 14 stores the data processing result in the memory 40 and sends a notification signal to the HTS 13. The computation accelerator 14 can send the notification signal to the HTS 13 via the hardware line between them.

[0068] S311, HTS13 forwards the notification signal to parser 12.

[0069] S312, the parser 12 reads the data processing result from the memory 40 based on the notification signal. S313, the parser 12 encapsulates the data processing result based on the bus protocol of the bus 20. At this point, the parser performs the encapsulation function, i.e., it is a unified parsing and encapsulation device, generating data unit C. S314, the parser 12 stores the data unit C in the memory 40 and configures the configuration information to the transmitting unit. In one possible implementation, when the parser 12 decapsulates the data unit in step S305, it can record the mapping relationship between the bus 20 and the external chip 30. The notification signal sent from the computing accelerator 14 to the parser 12 can carry the identifier of the external chip 30. The parser 12 can compare the identifier of the external chip 30 with the recorded identifier of the external chip 30 to determine the external chip 30 to which the data processing result is to be sent, and the bus 20 that interacts with the external chip 30. Then, the parser 12 can encapsulate the data processing result based on the bus protocol of the bus 20 corresponding to the external chip 30, in a manner similar to that described in step S301 where the external chip 30 encapsulates the control signals and control parameters. Assuming that the data processing result needs to be transmitted to the WIFI chip through the PCIE bus 20, the parser 12 can... Figure 4A The data frame format shown writes the data processing result into the data field of the data frame, and then adds the standard cooperative packet header of the PCIe bus 20 to generate data unit C. The above configuration information is used to configure the transmitting unit, and the configuration information may include the device identifier of the target device and the address of the target chip to which the data is to be sent.

[0070] S315, the transmitting unit reads the data unit C from the memory 40 based on the configuration information, and sends the data unit C to the external chip 30.

[0071] S316, the external chip 30 decapsulates the data unit C and reads the calculation result. In this step, the external chip 30 can decapsulate the data unit C based on the bus 20 protocol that communicates with the SoC.

[0072] pass Figure 6As can be seen from the interaction flow shown, on the SoC side, after the computing accelerator 14 completes data processing, it can also notify the HTS 13 through hardware lines. The HTS 13 forwards the notification signal to the parser 12. Thus, the transmission path of the notification signal on the SoC does not need to go through the layers of calls of various software in the processor. In addition, the parser 12 can encapsulate the data processing result into the data field of the data unit based on the bus protocol, and send the data unit containing the data processing result to the external chip 30 through the transmission unit and the bus. Therefore, the electronic device shown in this embodiment can be compatible with the existing processor architecture without changing the signal transmission path and bus 20 protocol in the original processor architecture. It can improve the operating speed of the electronic device at a lower cost based on the existing processor architecture.

[0073] above Figure 2 , Figure 3 , Figure 5 as well as Figure 6 In the illustrated embodiment, the parser 12 is schematically shown as a separate hardware module. In one possible implementation, the parser's execution logic can be implemented via software. When the parser's execution logic is implemented via software, the software can run on a general-purpose processor such as a CPU, or it can run on other dedicated processors. In this implementation, the SoC may not have a dedicated hardware parser 12, but it may have an HTS 13. The logic for parsing data units sent by the external chip 30 can be executed by the CPU, such as... Figure 7 As shown, Figure 7 This schematically illustrates yet another hardware structure diagram of the electronic device 100. For example... Figure 7 As shown, the CPU is connected to the bus 20, memory 40, and HTS 13 via hardware lines. The CPU runs the software program for the parser 12. That is to say, the workflow executed by the parser 12 and its interaction with other components, as shown in any of the above embodiments, are all executed by the CPU. For example, when the electronic device 100 is... Figure 7 When the structure shown is used, Figure 3 In step S303, the receiver 11 sends an interrupt signal 1 to the parser 12, which is replaced by the receiver 11 sending an interrupt signal 1 to the CPU. Figure 3 The parser 12 in step S305 shown parses data unit A, which is replaced by the CPU parsing data unit A based on the bus protocol of bus 20.

[0074] Based on the same inventive concept, embodiments of this application also provide a method for scheduling a computing accelerator, which is applied to, for example... Figure 2 In the electronic device shown. Please continue reading. Figure 8 This illustrates a flow 800 of a method for scheduling a computing accelerator provided in an embodiment of this application. This flow 800 of the method for scheduling a computing accelerator can be... Figure 2 The electronic device 100 shown executes the following steps: Step 801, a receiver receives at least one first data unit, the at least one first data unit including a control signal and control parameters; Step 802, a parser parses the at least one first data unit to obtain the control signal and control parameters; Step 803, the parser stores the control parameters in a memory and transmits the control signal to a hardware task scheduler; Step 804, the hardware task scheduler schedules at least one computing accelerator based on the control signal; Step 805, in response to the scheduling, the at least one computing accelerator retrieves the control parameters from the memory and performs data processing using the control parameters.

[0075] In one possible implementation, at least one first data unit includes a third data unit and a fourth data unit, wherein the third data unit includes a control signal and the fourth data unit includes control parameters.

[0076] In one possible implementation, at least one first data unit includes a fifth data unit, which includes control signals and control parameters.

[0077] In one possible implementation, the method further includes: receiving at least one second data unit by a receiver, the at least one second data unit including first service data; parsing the at least one second data unit by a parser to obtain the first service data, and storing the first service data in a memory; and performing data processing using control parameters, including: performing data processing on the first service data using control parameters.

[0078] In one possible implementation, at least one first data unit further includes second service data; the method further includes: obtaining the second service data while parsing at least one first data unit, and storing the second service data in a memory; and performing data processing using control parameters, including: performing data processing on the second service data using control parameters.

[0079] In one possible implementation, receiving at least one first data unit by a receiver includes: receiving at least one first data unit by a receiver from an external chip, wherein the external chip includes at least one of the following: a communication chip, a power amplifier chip, a display driver integrated circuit chip, or a camera chip.

[0080] It is understood that, in order to achieve the above-mentioned functions, the electronic device 100 includes hardware and / or software modules corresponding to the execution of each function. Based on the steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application. For example, Figure 2 The receiver 11, parser 12, and computing accelerator 14 shown can be software-driven hardware components, that is, each component can include a corresponding processor and corresponding driver software, that is, implemented in software or hardware combination.

[0081] In the electronic device 100 described above, at least one of the receiver 11, the parser 12, and the computing accelerator 14 can call a computer program stored in the memory to control and manage the operation of the corresponding component, thereby supporting the steps executed by the electronic device 100. The memory can be used to support the electronic device 100 in storing program code and data, and includes, but is not limited to, cache, registers, or at least a portion of the storage space of the aforementioned memory 40. The electronic device 100 can be a combination of one or more microprocessors that implement computing functions. Furthermore, the electronic device 100 may also include other programmable logic devices, transistor logic devices, or discrete hardware components.

[0082] The multi-core processor described in this application can be implemented on integrated circuits (ICs), analog ICs, radio frequency integrated circuits, mixed-signal ICs, application-specific integrated circuits (ASICs), printed circuit boards (PCBs), electronic devices, etc. This multi-core processor can also be manufactured using various IC process technologies, such as complementary metal-oxide semiconductors (CMOS), n-type metal-oxide-semiconductor (NMOS), p-type metal-oxide semiconductors (PMOS), bipolar junction transistors (BJTs), bipolar CMOS (BiCMOS), silicon-germanium (SiGe), etc.

[0083] This application also provides a computer storage medium storing computer instructions. When the computer instructions are executed on the electronic device 100, the electronic device 100 performs the aforementioned related method steps to implement the method for scheduling a computing accelerator in the above embodiments.

[0084] This application also provides a computer program product containing instructions; when the instructions are executed on an electronic device 100, the electronic device 100 performs the aforementioned related steps to implement the method for scheduling a computing accelerator in the above embodiments.

[0085] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0086] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0087] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0088] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An electronic device, characterized in that, include: A receiver is configured to receive at least one first data unit, wherein the at least one first data unit includes a control signal and control parameters; A parser is configured to parse the at least one first data unit to obtain the control signal and the control parameters, store the control parameters in a memory, and transmit the control signal to a hardware task scheduler. The hardware task scheduler schedules at least one computing accelerator based on the control signal; The at least one computing accelerator is configured to, in response to the scheduling, retrieve the control parameters from the memory and perform data processing using the control parameters.

2. The electronic device according to claim 1, characterized in that, The parser is either a software-driven processor or a hardware parser.

3. The electronic device according to claim 1 or 2, characterized in that, The at least one first data unit includes a third data unit and a fourth data unit, wherein the third data unit includes the control signal and the fourth data unit includes the control parameters.

4. The electronic device according to claim 1 or 2, characterized in that, The at least one first data unit includes a fifth data unit, which includes the control signal and the control parameters.

5. The electronic device according to any one of claims 1 to 4, characterized in that, The receiver is further configured to: receive at least one second data unit, wherein the at least one second data unit includes first service data; The parser is further configured to: parse the at least one second data unit to obtain the first service data, and store the first service data in the memory; The at least one computing accelerator is specifically used to: perform data processing on the first business data using the control parameters.

6. The electronic device according to any one of claims 1 to 5, characterized in that, The at least one first data unit also includes second business data; The parser is further configured to: obtain the second service data when parsing the at least one first data unit, and store the second service data in the memory; The at least one computing accelerator is specifically used to: perform data processing on the second business data using the control parameters.

7. The electronic device according to any one of claims 1 to 6, characterized in that, The receiver is specifically configured to: receive the at least one first data unit from an external chip, wherein the external chip includes at least one of the following: a communication chip, a power amplifier chip, a display driver integrated circuit chip, or a camera chip.

8. The electronic device according to any one of claims 1 to 7, characterized in that, The at least one computing accelerator includes at least one of the following: an image signal processor (ISP), a graphics processing unit (GPU), a neural network processor (NPU), or a digital signal processor (DSP).

9. The electronic device according to any one of claims 1 to 8, characterized in that, The electronic devices are integrated into the same chip.

10. An electronic device, characterized in that, Includes the electronic device and the memory as described in any one of claims 1 to 9.

11. A method for scheduling a computing accelerator, characterized in that, include: At least one first data unit is received by a receiver, wherein the at least one first data unit includes a control signal and control parameters; The at least one first data unit is parsed by the parser to obtain the control signal and the control parameters; The parser stores the control parameters in the memory and transmits the control signals to the hardware task scheduler. The hardware task scheduler schedules at least one computing accelerator based on the control signal. The at least one computing accelerator, in response to the scheduling, retrieves the control parameters from the memory and performs data processing using the control parameters.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by an electronic device, is used to implement the method as described in claim 11.

13. A computer program product, characterized in that, When the computer program product is executed by an electronic device, it is used to implement the method as described in claim 11.

Images