Data processing method and related device

Abstract

The invention discloses a data processing method and a related device, and relates to the field of intelligent driving, a current to-be-executed target algorithm reasoning instruction in a neural processing unit is acquired according to a pre-configured task scheduling rule, and on the basis, a kernel identifier and a coroutine identifier are extracted from the target algorithm reasoning instruction; and then a coroutine switching process is executed on a target kernel of the neural processing unit, a current coroutine operated by the target kernel is switched to a target coroutine to execute a model reasoning task represented by the target algorithm reasoning instruction, the target kernel is a kernel represented by the kernel identifier, and the target coroutine is a coroutine in the target kernel represented by the coroutine identifier. By utilizing the characteristic that the coroutines in the kernel can be switched to execute the corresponding model reasoning tasks according to the context, when the current model reasoning task needs to wait for resources, the current model reasoning task can be switched to other coroutines to execute other model reasoning tasks, so that the utilization rate of computing power resources in the kernel is improved, and the cost is reduced.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to a data processing method and related apparatus. Background Technology

[0002] An NPU (Neural Processing Unit) is a processor specifically designed to optimize and accelerate machine learning and artificial intelligence workloads. Compared to traditional processors, NPUs offer higher efficiency and performance for deep learning algorithms, especially neural networks, and are therefore widely used in fields such as autonomous driving.

[0003] Currently used NPU management and control systems are typically bare-metal environments without an operating system. In this bare-metal environment, only one main execution context runs. This means that if resources are needed, execution must wait until resources become available before continuing, resulting in wasted computing resources due to idleness. To meet the computing power requirements of intelligent driving algorithms, the only way to increase computing power is to increase the number of NPUs, which in turn increases costs. Summary of the Invention

[0004] In view of the above problems, this application provides a data processing method and related apparatus to increase the utilization rate of computing resources and reduce costs. The specific solution is as follows:

[0005] The first aspect of this application provides a data processing method, including:

[0006] According to the pre-configured task scheduling rules, obtain the target algorithm inference instructions that are currently to be executed in the neural processing unit;

[0007] Extract the kernel identifier and coroutine identifier from the target algorithm inference instructions;

[0008] A coroutine switching process is performed on the target kernel of the neural processing unit to switch the current coroutine running on the target kernel to the target coroutine. The model inference task represented by the target algorithm inference instruction is executed through the target coroutine. The target kernel is the kernel represented by the kernel identifier, and the target coroutine is the coroutine represented by the coroutine identifier in the target kernel.

[0009] In one possible implementation, the process of determining the task scheduling rules includes:

[0010] The neural processing unit compiler is invoked to compile the algorithm model, resulting in a set of algorithm inference instructions;

[0011] The execution order, associated kernel, and coroutine of each algorithm inference instruction in the algorithm inference instruction set are determined based on the resource orchestration strategy, thus obtaining the task scheduling rule.

[0012] In one possible implementation, the coroutine switching process includes:

[0013] Determine whether the coroutine context of the current coroutine has been saved to a preset register;

[0014] If so, determine whether the target coroutine has been scheduled and whether the coroutine context of the target coroutine has been restored from the preset register;

[0015] If the target coroutine is scheduled and its coroutine context is restored from the preset register, then the target coroutine is run.

[0016] In one possible implementation, before determining whether the coroutine context of the current coroutine is saved to a preset register, the method further includes:

[0017] Entering the critical section sets the current coroutine's state to a resource-waiting blocked state.

[0018] After saving the coroutine context of the current coroutine to the preset register, the method further includes: exiting the critical section.

[0019] In one possible implementation, determining whether the coroutine context of the current coroutine is saved to a preset register includes:

[0020] Based on the data types contained in the coroutine context of the current coroutine, determine whether the context data for each data type is stored in the corresponding data type's preset register.

[0021] In one possible implementation, determining whether the target coroutine has been scheduled and whether the coroutine context of the target coroutine has been restored from the preset register includes:

[0022] Based on the data types contained in the obtained coroutine context of the target coroutine, determine whether the context data for each data type has been recovered from the preset register of the corresponding data type.

[0023] In one possible implementation, prior to executing the coroutine switching process, the following is also included:

[0024] The coroutines in the neural processing unit are initialized, and the initialized coroutines are converted into non-running coroutines and put into a sleep state.

[0025] Upon receiving an algorithm inference instruction, the sleep state ends, and the coroutine running state is entered to execute the coroutine switching process.

[0026] A second aspect of this application provides a data processing apparatus, comprising:

[0027] The algorithm inference instruction acquisition module is used to acquire the target algorithm inference instruction to be executed in the neural processing unit according to the pre-configured task scheduling rules.

[0028] The identifier extraction module is used to extract the kernel identifier and coroutine identifier from the target algorithm inference instructions; and,

[0029] The coroutine switching module is used to perform a coroutine switching process on the target kernel of the neural processing unit, switch the current coroutine running on the target kernel to the target coroutine, and execute the model inference task represented by the target algorithm inference instruction through the target coroutine. The target kernel is the kernel represented by the kernel identifier, and the target coroutine is the coroutine represented by the coroutine identifier in the target kernel.

[0030] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the data processing method described in the first aspect or any implementation thereof.

[0031] A fourth aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0032] The memory is used to store computer programs;

[0033] The processor is used to execute the computer program so that the electronic device can implement the data processing method of the first aspect or any implementation thereof.

[0034] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the data processing method described in the first aspect or any implementation thereof.

[0035] By employing the above technical solution, the data processing method provided in this application can obtain the target algorithm inference instructions currently to be executed in the neural processing unit according to pre-configured task scheduling rules, and extract the kernel identifier and coroutine identifier from the target algorithm inference instructions. Then, a coroutine switching process is executed on the target kernel of the neural processing unit, switching the current coroutine running in the target kernel to the model inference task represented by the target algorithm inference instructions executed by the target coroutine. Here, the target kernel is the kernel represented by the kernel identifier, and the target coroutine is the coroutine represented by the coroutine identifier in the target kernel. By utilizing the characteristic that coroutines in the kernel can switch to execute corresponding model inference tasks according to the context, when the current model inference task needs to wait for resources, it is possible to switch to other coroutines to execute other model inference tasks, achieving the effect of concurrent execution of model inference tasks, thereby improving the utilization rate of computing resources in the kernel, meeting the higher computing power requirements of intelligent driving algorithms, and reducing costs. Attached Figure Description

[0036] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0037] Figure 1 A structural diagram of a remote data processing system provided in this application;

[0038] Figure 2 A structural diagram of a terminal provided in this application;

[0039] Figure 3 A structural diagram of a server provided in this application;

[0040] Figure 4 A flowchart of a data processing method provided in this application;

[0041] Figure 5 A scheduling diagram of the management system in the neural processing unit provided in this application;

[0042] Figure 6 The coroutine state transition diagram provided for this application;

[0043] Figure 7 The flowchart for coroutine switching provided in this application;

[0044] Figure 8 The flowchart for coroutine initialization provided in this application;

[0045] Figure 9 A structural diagram of a data processing apparatus provided in this application;

[0046] Figure 10 This is a structural diagram of an electronic device provided in this application. Detailed Implementation

[0047] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0048] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0049] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0050] See Figure 1 , Figure 1 A schematic diagram of the architecture of a remote data processing system is shown. The system may include a terminal 100 and a server 200. The server 200 may include one or more servers (…). Figure 1 (The example includes a server), and the server 200 can provide the method provided in the embodiments of this application to one or more terminals.

[0051] The terminal 100 may be equipped with an application that generates and sends algorithm inference instructions. The application and webpage can provide an interface. The terminal 100 can receive relevant parameters input by the user on the interface and send the parameters to the server 200. The server 200 can obtain the processing result based on the received parameters and return the processing result to the terminal 100.

[0052] It should be understood that in some optional implementations, the terminal 100 can also complete the action of obtaining the processing result based on the received parameters on its own, without the need for the server to cooperate. This application embodiment is not limited to this.

[0053] The following description Figure 1 The product form of the mid-terminal 100;

[0054] The terminal 100 in this application embodiment can be a mobile phone, tablet computer, wearable device, vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.

[0055] Figure 2 A schematic diagram of an optional hardware structure for terminal 100 is shown.

[0056] refer to Figure 2 As shown, the terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), a headphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190, and other components. Those skilled in the art will understand that... Figure 2 These are merely examples of terminals or multi-functional devices and do not constitute a limitation on terminals or multi-functional devices. They may include more or fewer components than shown, or combine certain components, or different components.

[0057] The input unit 130 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the portable multi-functional device. Specifically, the input unit 130 may include a touchscreen 131 (optional) and / or other input devices 132. The touchscreen 131 can collect touch operations performed by the user on or near it (such as operations performed by the user using fingers, knuckles, styluses, or any suitable object on or near the touchscreen), and drive the corresponding connection devices according to a pre-set program. The touchscreen can detect the user's touch actions, convert the touch actions into touch signals and send them to the processor 170, and can receive and execute commands sent by the processor 170; the touch signal includes at least touch point coordinate information. The touchscreen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, various types of touchscreens, such as resistive, capacitive, infrared, and surface acoustic wave, can be used to implement the touchscreen. Besides the touchscreen 131, the input unit 130 may also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0058] Among them, the input device 132 can receive input data, etc.

[0059] The display unit 140 can be used to display information input by the user or information provided to the user, various menus of the terminal 100, interactive interfaces, file display, and / or playback of any multimedia file. In this embodiment, the display unit 140 can be used to display an interface for model inference results, processing results, etc.

[0060] The memory 120 can be used to store instructions and data. The memory 120 may primarily include an instruction storage area and a data storage area. The data storage area can store various types of data, such as multimedia files and text. The instruction storage area can store software units such as operating systems, applications, and instructions required for at least one function, or subsets or extended sets thereof. It may also include non-volatile random access memory. It provides the processor 170 with hardware, software, and data resources for managing the computing device, supporting control software and applications. It is also used for storing multimedia files, as well as storing running programs and applications.

[0061] The radio frequency unit 110 (optional) can be used to receive and transmit signals during information transmission or calls. In this embodiment, the radio frequency unit 110 can send algorithm inference instruction data to the server 200 and receive model inference processing results sent by the server 200.

[0062] It should be understood that the radio frequency unit 110 is optional and can be replaced with other communication interfaces, such as a network port.

[0063] The terminal 100 also includes a power supply 190 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0064] Terminal 100 also includes an external interface 180, which can be a standard Micro USB interface or a multi-pin connector, which can be used to connect terminal 100 to other devices for communication or to connect a charger to charge terminal 100.

[0065] Although not shown, terminal 100 may also include a flash, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with various functions, etc., which will not be described in detail here. Some or all of the methods described below can be applied to, for example... Figure 2 In the terminal 100 shown.

[0066] The following description Figure 1The product form of the mid-range server 200;

[0067] Figure 3 A structural diagram of a server 200 is provided, as follows: Figure 3 As shown, server 200 includes bus 201, processor 202, communication interface 203, and memory 204. Processor 202, memory 204, and communication interface 203 communicate with each other via bus 201.

[0068] Bus 201 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0069] The processor 202 can be any one or more of the following processors: central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP), or digital signal processor (DSP).

[0070] Memory 204 may include volatile memory, such as random access memory (RAM). Memory 204 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0071] The memory 204 can be used to store software code related to the data processing method, and the processor 202 can execute the steps of the chip's data processing method, and can also schedule other units to achieve corresponding functions.

[0072] It should be understood that the aforementioned terminal 100 and server 200 can be centralized or distributed devices. The processors (e.g., processor 170 and processor 202) in the aforementioned terminal 100 and server 200 can be hardware circuits (such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), general-purpose processors, digital signal processors (DSPs), microprocessors or microcontrollers, etc.) or combinations of these hardware circuits. For example, the processor can be a hardware system with instruction execution capabilities, such as a CPU or DSP, or a hardware system without instruction execution capabilities, such as an ASIC or FPGA, or a combination of the aforementioned hardware systems without instruction execution capabilities and hardware systems with instruction execution capabilities.

[0073] To improve the utilization of computing resources in neural processing units (NNUnits), current management methods typically incorporate an operating system and utilize its multi-threading mechanism for resource allocation. However, this approach increases the memory overhead and thread switching performance costs of the NNUnit, ultimately limiting its computing power.

[0074] To address the aforementioned problems, embodiments of this application provide a data processing method. The data processing method of this application embodiment will be described in detail below with reference to the accompanying drawings.

[0075] Reference Figure 4 , Figure 4 This is a flowchart illustrating a data processing method provided in an embodiment of this application, such as... Figure 4 As shown in the embodiment of this application, a data processing method may include steps 401 to 403, which are described in detail below.

[0076] 401. According to the pre-configured task scheduling rules, obtain the target algorithm inference instructions to be executed in the neural processing unit.

[0077] Specifically, refer to Figure 5As shown, the NPU compiler compiles the algorithm model into a set of algorithm inference instructions that the NPU can understand. These instructions are organized in different layers, such as Layer 0, Layer 1 to Layer N. Each layer contains specific types of instructions and corresponding parameters. Each algorithm inference instruction contains the instruction type (e.g., addition, multiplication, activation function, etc.) and instruction parameters (e.g., weights, biases, etc.), as well as the corresponding coroutine identifier (i.e., coroutine ID) and the kernel identifier (i.e., CPU ID) of the kernel where the coroutine resides. The coroutine identifier and kernel identifier corresponding to the algorithm inference instruction can be determined by the model inference task according to its needs (e.g., its internal resource orchestration).

[0078] A coroutine is a lightweight thread, a programming concept that is not directly managed by the operating system. Although only one coroutine runs in the kernel at any given time, switching between coroutines allows for concurrent execution.

[0079] The core data structures of coroutines mainly include: a coroutine scheduler and a coroutine control block. The coroutine scheduler is primarily responsible for scheduling and switching coroutines within each kernel. Since coroutines only run in one kernel, kernel migration is not performed. In multi-core systems, the coroutine scheduler exists as an array, and its data structures are shown in the table below:

[0080]

[0081] The coroutine control block is primarily responsible for managing the execution context of a coroutine. When a coroutine is scheduled for execution, the coroutine scheduling system prepares the execution environment for the coroutine based on this data structure. The coroutine context data is shown in the table below:

[0082]

[0083] The data on methods for controlling the state of coroutines are shown in the table below:

[0084]

[0085] When performing specific coroutine state transitions, one can follow... Figure 6 The state machine transition process is shown. After being created by the CTaskCreate method, the coroutine enters the initial state. Then, it is scheduled to switch to the execution state by the CTaskSched method. After being processed by the CTaskYield method, the coroutine is abandoned and enters the blocked state. The coroutine in the blocked state is activated by the CTaskActivate method and then returned to the initial state. Then the process repeats.

[0086] 402. Extract the kernel identifier and coroutine identifier from the target algorithm inference instructions.

[0087] Specifically, refer to Figure 5 As shown, after the management system in the neural processing unit receives the algorithm inference instruction, it parses the kernel identifier and coroutine identifier required by the target algorithm inference instruction from the algorithm inference instruction according to the pre-agreed parsing method.

[0088] It is understood that those skilled in the art can choose the method of sending and parsing instructions as needed, and will not elaborate further here.

[0089] 403. Perform a coroutine switching process on the target kernel of the neural processing unit, switch the current coroutine running on the target kernel to the target coroutine, and execute the model inference task represented by the target algorithm inference instructions through the target coroutine. The target kernel is the kernel represented by the kernel identifier, and the target coroutine is the coroutine represented by the coroutine identifier in the target kernel.

[0090] Specifically, based on the target coroutine required by the target kernel, a coroutine switching process is executed to switch the current coroutine in the target kernel to the target coroutine, so as to execute the corresponding model inference task according to the instruction parameters and instruction type in the algorithm inference instructions. The switched-out current coroutine enters a blocked state, waiting to be called again to resume execution.

[0091] As can be seen from the above, this data processing method, based on the switchable execution of coroutines, has an effect comparable to concurrent execution, enabling the algorithm inference instructions of the neural processing unit to be executed concurrently. This maximizes the computing power of the neural processing unit, improves the utilization rate of computing power, reduces the number of neural processing units required for model inference tasks, and effectively reduces costs.

[0092] In one possible implementation, to facilitate reasonable and efficient scheduling of algorithm inference instructions, the process of determining the task scheduling rules in step 401 includes:

[0093] Step 11: Call the neural processing unit compiler to compile the algorithm model and obtain the algorithm inference instruction set.

[0094] Specifically, refer to Figure 5 The set of algorithm inference instructions obtained after compilation by the neural processing unit compiler is shown (as described above, it will not be repeated here).

[0095] Step 12: Determine the execution order of each algorithm inference instruction in the algorithm inference instruction set, the associated kernel and coroutine according to the resource orchestration strategy, and obtain the task scheduling rules.

[0096] Specifically, the resource orchestration strategy here can be determined based on the execution order of the algorithm inference instructions at each level in the algorithm model, or it can be determined based on user-defined rules. On this basis, the kernel where the coroutines used by each algorithm inference instruction are located and the coroutine identifiers are obtained, ultimately forming the corresponding task scheduling strategy.

[0097] It should be noted that those skilled in the art can adjust the above task scheduling strategy as needed, and no restrictions are imposed here.

[0098] In one possible implementation, the coroutine switching process in the above embodiments may specifically include the following process:

[0099] First, determine whether the current coroutine's coroutine context has been saved to the preset register.

[0100] If so, determine whether the target coroutine has been scheduled and whether the coroutine context of the target coroutine has been restored from the preset register.

[0101] If the target coroutine is scheduled and its coroutine context is restored from the preset register, then the target coroutine is run.

[0102] Specifically, the default registers used here can be built based on the fifth-generation RISC (Reduced Instruction Set Computer, an open-source instruction set architecture). Compared to the method of fully saving all registers (31 integer registers) when switching operating system threads, the number of registers used can be reduced to about half, requiring only 14 registers.

[0103] When switching coroutines, the current coroutine's context is first saved to a preset register. After saving, the target coroutine's scheduling process is executed. Based on the target coroutine's identifier, the target coroutine's context is restored from the preset register. After restoration, the coroutine switch is completed.

[0104] When saving the context, it is necessary to determine whether the context data for each data type is stored in the corresponding preset register, based on the data type contained in the current coroutine's context. Similarly, when restoring the context, it is necessary to determine whether the context data for each data type is restored from the corresponding preset register, based on the data type contained in the obtained target coroutine's context.

[0105] Furthermore, to ensure smooth coroutine switching, before determining whether the current coroutine's coroutine context has been saved to the preset register, the following steps are also included:

[0106] Entering the critical section sets the current coroutine's state to a waiting-for-resources-blocked state. After saving the current coroutine's coroutine context to a preset register, the critical section is exited.

[0107] The critical section here can be understood as a protected area. Entering this area means that subsequent actions must be executed in sequence, that is, the current coroutine context is stored first, and then the target coroutine context is restored.

[0108] As a specific application of the above coroutine switching process, refer to Figure 7 As shown, after starting the coroutine switch, the system first enters the critical section and sets the current coroutine's state to Pend (blocked state).

[0109] The `setjmp` function is used to save the current coroutine's context into the `jmp_buf` (an array of structures) of the TCB (Thread Control Block). The specific storage process is shown on the left: first, integer registers `s0-s11`, `sp` (stack pointer), and `ra` (return address) are used to store integer data from the context. If floating-point data is also present, then floating-point registers `fs0-fs11` are used to store it; otherwise, no further storage is required. After storage, the `setjmp` function returns the corresponding parameters.

[0110] When the setjmp jump function receives the first return parameter, it indicates that the context of the current coroutine has been successfully saved. Next, it exits the critical section and selects the target coroutine from the ready (initial state) coroutines, preparing to execute the restoration of the target coroutine.

[0111] The `longjmp` function is called to restore the context of the target coroutine. The restoration process on the left is similar to that of the stored procedure described above, first restoring integer data, and then restoring floating-point data if applicable. This will not be elaborated further. After restoration, the `longjmp` function returns a second parameter (here, 1) to the `setjmp` function. Upon receiving this non-zero value, the `setjmp` function indicates that a jump has occurred, and upon receiving the second return value, the target coroutine resumes execution.

[0112] In one possible implementation, before executing the coroutine switching process, the coroutines in the neural processing unit can be initialized, and the initialized coroutines can be converted into non-running coroutines and enter a sleep state. When an algorithm inference instruction is received, the sleep state ends, and the coroutine enters the running state to execute the coroutine switching process.

[0113] Specifically, refer to Figure 8The coroutine management system shown is composed of RISC-V (fifth generation open source instruction set architecture). The first step is to initialize the scheduler (load the scheduling policy) and initialize the idle coroutines.

[0114] Then, according to business needs, the creation of coroutines includes: initializing the coroutine's TCB (Coroutine Control Block), setting the stack, coroutine entry function, parameters, state, and the coroutine's initial context. The initial context includes ra (Return Address, i.e., the next executing coroutine) and sp (Stack Pointer), where sp points to the top of the current thread's or coroutine's stack. Within the coroutine's context, sp manages the coroutine's local variables and temporary data. When a coroutine is created, it is allocated a certain amount of memory as a stack, and sp is initialized to the top address of that stack. During the coroutine's execution, sp is dynamically adjusted based on function calls and returns to maintain stack consistency.

[0115] When the algorithm inference instructions are executed, the system executes an Idle coroutine, which internally executes WFI (Wait For Interrupt) to enter a sleep state, thereby reducing power consumption. When a service needs to run, it will be triggered by an external interrupt, activating the required service coroutine and simultaneously activating the sleep Idle coroutine, which will then execute Yield (coroutine switching process) to run.

[0116] The above describes a data processing method provided by an embodiment of this application. The following describes an apparatus for performing the above data processing method.

[0117] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Figure 9 As shown, the data processing apparatus includes:

[0118] The algorithm inference instruction acquisition module 901 is used to acquire the target algorithm inference instruction to be executed in the neural processing unit according to the pre-configured task scheduling rules.

[0119] The identifier extraction module 902 is used to extract the kernel identifier and coroutine identifier from the target algorithm inference instructions; and,

[0120] The coroutine switching module 903 is used to perform a coroutine switching process on the target kernel of the neural processing unit, switch the current coroutine running on the target kernel to the target coroutine, and execute the model inference task represented by the target algorithm inference instruction through the target coroutine. The target kernel is the kernel represented by the kernel identifier, and the target coroutine is the coroutine represented by the coroutine identifier in the target kernel.

[0121] In one possible implementation, the process of determining the task scheduling rules in the algorithm inference instruction acquisition module 901 includes:

[0122] The neural processing unit compiler is invoked to compile the algorithm model, resulting in a set of algorithm inference instructions;

[0123] The execution order, associated kernel, and coroutine of each algorithm inference instruction in the algorithm inference instruction set are determined based on the resource orchestration strategy, thus obtaining the task scheduling rule.

[0124] In one possible implementation, the coroutine switching process in coroutine switching module 903 includes:

[0125] Determine whether the coroutine context of the current coroutine has been saved to a preset register;

[0126] If so, determine whether the target coroutine has been scheduled and whether the coroutine context of the target coroutine has been restored from the preset register;

[0127] If the target coroutine is scheduled and its coroutine context is restored from the preset register, then the target coroutine is run.

[0128] In one possible implementation, the coroutine protection module is used to determine whether the coroutine context of the current coroutine has been saved to a preset register:

[0129] Entering the critical section sets the current coroutine's state to a resource-waiting blocked state.

[0130] After saving the coroutine context of the current coroutine to the preset register, the method further includes: exiting the critical section.

[0131] In one possible implementation, the process by which the coroutine switching module 903 determines whether the coroutine context of the current coroutine has been saved to a preset register includes:

[0132] Based on the data types contained in the coroutine context of the current coroutine, determine whether the context data for each data type is stored in the corresponding data type's preset register.

[0133] In one possible implementation, the process by which the coroutine switching module 903 determines whether the target coroutine has been scheduled and whether the coroutine context of the target coroutine has been restored from the preset register includes:

[0134] Based on the data types contained in the obtained coroutine context of the target coroutine, determine whether the context data for each data type has been recovered from the preset register of the corresponding data type.

[0135] In one possible implementation, it also includes: a coroutine initialization module, used before executing the coroutine switching process,

[0136] The coroutines in the neural processing unit are initialized, and the initialized coroutines are converted into non-running coroutines and put into a sleep state.

[0137] Upon receiving an algorithm inference instruction, the sleep state ends, and the coroutine running state is entered to execute the coroutine switching process.

[0138] This application also provides an electronic device in its embodiments. (See reference...) Figure 10 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 10 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0139] like Figure 10 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 into a random access memory (RAM) 1003. When the electronic device is powered on, the RAM 1003 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0140] Typically, the following devices can be connected to the I / O interface 1005: input devices 1006 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1007 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1008 including, for example, memory card, hard disk, etc.; and communication devices 1009. Communication device 1009 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 10 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0141] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the data processing methods provided in this application.

[0142] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the data processing methods provided in this application.

[0143] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0144] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0145] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0146] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A data processing method, characterized in that, include: According to the pre-configured task scheduling rules, obtain the target algorithm inference instructions that are currently to be executed in the neural processing unit; Extract the kernel identifier and coroutine identifier from the target algorithm inference instructions; A coroutine switching process is performed on the target kernel of the neural processing unit to switch the current coroutine running on the target kernel to the target coroutine. The model inference task represented by the target algorithm inference instruction is executed through the target coroutine. The target kernel is the kernel represented by the kernel identifier, and the target coroutine is the coroutine represented by the coroutine identifier in the target kernel.

2. The data processing method according to claim 1, characterized in that, The process of determining the task scheduling rules includes: The neural processing unit compiler is invoked to compile the algorithm model, resulting in a set of algorithm inference instructions; The execution order, associated kernel, and coroutine of each algorithm inference instruction in the algorithm inference instruction set are determined based on the resource orchestration strategy, thus obtaining the task scheduling rule.

3. The data processing method according to claim 1, characterized in that, The coroutine switching process includes: Determine whether the coroutine context of the current coroutine has been saved to a preset register; If so, determine whether the target coroutine has been scheduled and whether the coroutine context of the target coroutine has been restored from the preset register; If the target coroutine is scheduled and its coroutine context is restored from the preset register, then the target coroutine is run.

4. The data processing method according to claim 3, characterized in that, Before determining whether the coroutine context of the current coroutine has been saved to a preset register, the process also includes: Entering the critical section sets the current coroutine's state to a resource-waiting blocked state. After saving the coroutine context of the current coroutine to the preset register, the method further includes: exiting the critical section.

5. The data processing method according to claim 3, characterized in that, The step of determining whether the coroutine context of the current coroutine is saved to a preset register includes: Based on the data types contained in the coroutine context of the current coroutine, determine whether the context data for each data type is stored in the corresponding data type's preset register.

6. The data processing method according to claim 3, characterized in that, The step of determining whether the target coroutine has been scheduled and whether the coroutine context of the target coroutine has been restored from the preset register includes: Based on the data types contained in the obtained coroutine context of the target coroutine, determine whether the context data for each data type has been recovered from the preset register of the corresponding data type.

7. The data processing method according to any one of claims 1 to 6, characterized in that, Before executing the coroutine switching process, the following is also included: The coroutines in the neural processing unit are initialized, and the initialized coroutines are converted into non-running coroutines and put into a sleep state. Upon receiving an algorithm inference instruction, the sleep state ends, and the coroutine running state is entered to execute the coroutine switching process.

8. A data processing apparatus, characterized in that, include: The algorithm inference instruction acquisition module is used to acquire the target algorithm inference instruction to be executed in the neural processing unit according to the pre-configured task scheduling rules. The identifier extraction module is used to extract the kernel identifier and coroutine identifier from the target algorithm inference instructions; as well as, The coroutine switching module is used to perform a coroutine switching process on the target kernel of the neural processing unit, switch the current coroutine running on the target kernel to the target coroutine, and execute the model inference task represented by the target algorithm inference instruction through the target coroutine. The target kernel is the kernel represented by the kernel identifier, and the target coroutine is the coroutine represented by the coroutine identifier in the target kernel.

9. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to perform the data processing method as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the data processing method as described in any one of claims 1 to 7.

11. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the data processing method as described in any one of claims 1 to 7.

Images