Multi-core synchronization system and system component in neural network processor

Abstract

The invention provides a multi-core synchronization system and system component in a neural network processor, and belongs to the technical field of computers. The system comprises a neural network processor and a system component, the neural network processor comprises a task scheduling processor and a plurality of cores; the system component is respectively connected with the task scheduling processor and each core through a system bus; and the system component is used for communicating among the cores under the control of the task scheduling processor so as to enable the cores to synchronously execute the distributed model tasks. The system is used for efficient inter-core communication and synchronization.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more particularly to a multi-core synchronization system and system components in a neural network processor. Background Art

[0002] NPU (Neural-network Processing Unit) is a chip specially designed to accelerate the calculation of artificial neural networks and has been widely used in various AI applications. Compared with traditional computing platforms such as CPU (central processing unit) and GPU (Graphics Processing Unit), NPU adopts the architecture of "data-driven parallel computing" and takes the MAC (Multiply Accumulate) array dedicated to convolution operations as the core, so it is better at processing massive multimedia data such as videos and images. Since artificial neural network models usually require a lot of computing resources for training and reasoning, a single-core NPU may not meet actual needs, while a multi-core NPU can split multiple models or a single large model and distribute them to different cores for parallel computing, which can significantly improve computing efficiency and achieve more flexible task allocation and load balancing.

[0003] As the number of cores and computing power continue to increase, NPUs require an efficient multi-core synchronization mechanism to ensure data consistency and correctness of calculation results between multiple cores during parallel computing, thereby improving the reliability and stability of applications. To date, most NPU vendors use lock-based solutions to achieve communication and synchronization between multiple cores. When the number of cores increases significantly, defects such as lock contention and increased lock overhead are prone to occur, reducing the efficiency of inter-core synchronization. Summary of the invention

[0004] The present disclosure provides a multi-core synchronization system and system components in a neural network processor.

[0005] A first aspect of the present disclosure provides a multi-core synchronization system in a neural network processor, comprising: a neural network processor and a system component; the neural network processor comprises a task scheduling processor and a plurality of cores;

[0006] The system components are respectively connected to the task scheduling processor and each of the cores via a system bus;

[0007] The system component is used to communicate between the cores under the control of the task scheduling processor so that the cores can synchronously execute the assigned model tasks.

[0008] A second aspect of the present disclosure provides a system component, which is used to communicate between the cores of a neural network processor under the control of a task scheduling processor of the neural network processor so that the cores synchronously execute the assigned model tasks.

[0009] The present disclosure has the following advantages:

[0010] By setting up a system component for multi-core synchronization in a neural network processor, the system component communicates between multiple cores under the control of the task scheduling processor of the neural network processor, so that each core executes the assigned model task synchronously, thereby providing efficient inter-core communication and synchronization for the multi-core neural network processor and realizing unified management of inter-core synchronization; the synchronization between multiple cores is completed by the system component, and there is no need to occupy too much resources of the host's central processing unit due to inter-core synchronization, thereby reducing the operating load of the host's central processing unit; and, by providing a direct connection channel between multiple cores through the system component, the interaction path and cost are shortened, thereby enabling the neural network processor to support more complex artificial intelligence modules and larger-scale data input. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic diagram of the architecture of a multi-core synchronization system in a neural network processor provided in an embodiment of the present disclosure;

[0012] Figure 2 A schematic diagram of interrupt distribution of a system component provided in an embodiment of the present disclosure;

[0013] Figure 3 A schematic diagram of event distribution of a system component provided in an embodiment of the present disclosure;

[0014] Figure 4 A schematic diagram of a barrier implementation process provided in an embodiment of the present disclosure;

[0015] Figure 5 A schematic diagram of a multi-core NPU synchronization process provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0016] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0017] As used in this disclosure, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0018] The terms used in the present disclosure are only used to describe specific embodiments and are not intended to limit the present disclosure.As used in the present disclosure, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0019] When the terms “comprising” and / or “made of…” are used in the present disclosure, it specifies the existence of the stated features, integers, steps, operations, elements and / or components, but does not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0020] Unless otherwise defined, all terms (including technical and scientific terms) used in this disclosure have the same meaning as those commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined in this disclosure.

[0021] The embodiment of the present disclosure provides a multi-core synchronization system in a neural network processor. Figure 1 The figure shows the architecture diagram of a multi-core synchronization system in a neural network processor.

[0022] The multi-core synchronization system in the neural network processor mainly includes: a neural network processor 10 and a system component (also called SaneSync) 11; the neural network processor 10 includes a task scheduling processor (also called Vector / ScalarProcessor) 101 and multiple cores (Core) 102; the system component 11 is respectively connected to the task scheduling processor 101 and each core 102 through a system bus (also called NOC) 12; the system component 11 is used to communicate between each core 102 under the control of the task scheduling processor 101 so that each core 102 synchronously executes the assigned model task.

[0023] Among them, the system component 11 provides efficient communication and synchronization functions between the cores of the multi-core NPU, thereby facilitating the parallel or pipeline running of application programs in multiple cores.

[0024] The task scheduling processor 101 is a task scheduling core shared by multiple cores, and is mainly responsible for controlling the synchronization and task allocation between multiple cores through the system component 11 .

[0025] The system bus 12 is used to connect various devices and is responsible for transmitting data, addresses and control information.

[0026] Figure 1 As shown in the figure, Core 0 to n represent multiple cores, n represents the identification (ID) of the core, and the ID of each core is fixed and unique.

[0027] In some embodiments, the neural network processor 10 also includes a shared cache 103 for storing intermediate calculation results of the task scheduling processor 101 and each of the cores 102.

[0028] In some embodiments, the system further includes a physical memory 13 connected to the shared cache 103 via a system bus 12;

[0029] The physical memory 13 is used to store the model tasks and task descriptors input by the neural network processor 10, and the task descriptors are used to describe at least one of the following information: the number of cores required for the model task operation, the model task input and / or output storage address, the intermediate data write-back address, and the event and / or status receiving address. The physical memory 13 can be any form of physical memory, for example, it can be a physical memory in the form of DDR.

[0030] The shared cache 103 and the physical memory 13 are also connected to a central processing unit 14 (also called Host CPU) of the host through a system bus 12 .

[0031] The host's central processor 14 is responsible for resource management and scheduling of upper-layer application tasks, and controls multiple NPU-like device operations. The host's central processor 14 configures system components through the system bus 12, and allocates model tasks to the NPU through the system bus, and transmits information about the model tasks allocated to the NPU to the NPU's shared cache 103. The NPU's task scheduling processor 101 reads the model tasks from the shared cache 103 and schedules them to each core for execution.

[0032] The system component 11 mainly communicates between the cores through interrupts, events, semaphores, barriers, etc. to coordinate the task execution progress of each core, so as to achieve the purpose of controlling the synchronization of each core.

[0033] In some embodiments, interrupts are mainly used for real-time communication and processing when multiple cores of the NPU are running in parallel, and can coordinate situations where some cores execute too fast or too slowly. The main applicable scenarios are: a core suspends the task being executed, notifies the target core to perform a certain operation, and waits for the target core's response to resume the task. For example, in the image detection stage of advanced driver assistance, when a core detects a target, it needs to send an interrupt to other cores to inform them to cooperate in processing the target information.

[0034] Figure 2 The figure shows a schematic diagram of interrupt distribution of a system component, wherein the system component 11 includes an interrupt manager 21, and the interrupt manager 21 includes an initiating interrupt register and a confirming interrupt register corresponding to each of the cores 102;

[0035] The interrupt manager 21 is used to record the information that the source core 23 initiates an interrupt to the target core 22 in the interrupt initiation register corresponding to the target core 22; and to record the information that the interrupt ends returned by the target core 22 in the interrupt confirmation register corresponding to the target core 22.

[0036] Specifically, the source core 23 is used to write the identifier of the source core 23 into the interrupt initiation register corresponding to the target core 22 in the interrupt manager when initiating an interrupt to the target core 22;

[0037] The interrupt manager 21 is used to stop initiating new interrupts to the target core 22 when it is determined that the target core 22 is in an interrupt pending state; when it is determined that the target core 22 is not in an interrupt pending state, initiate the interrupt initiated by the source core 23 to the target core 22, and mark the target core 22 as an interrupt pending state until an interrupt confirmation returned by the target core 22 is obtained, clear the interrupt pending state of the target core 22 and update the confirmation interrupt register corresponding to the target core 22;

[0038] The target core 22 is used to return an interrupt confirmation to the interrupt manager 21 when the interrupt is completed.

[0039] The source core may be any core among multiple cores of the NPU that initiates an interrupt, and the target core may be any core among multiple cores of the NPU that receives an interrupt initiated by the source core.

[0040] In some embodiments, the system component 11 is further used to merge the interrupt completion information of each core and report it to the host's central processor 14. Merging the information of each interrupt completion and reporting it to the host's central processor avoids the problem that the separate reporting method requires too much interrupt resources of the host's central processor, thereby reducing the operating load of the host's central processor.

[0041] Combination Figure 2 The interrupt distribution process shown in the figure mainly includes the following steps:

[0042] Step 1: Generate an interrupt

[0043] Each core can write its own ID into the corresponding core's interrupt initiation register in the system component to generate an interrupt. If core 0 needs to send an interrupt to core 1, core 0 writes its own ID "0" into core 1's interrupt initiation register RAISE_IRQ_1, and the interrupt signal will be connected to core 1.

[0044] Step 2: Check the interrupt source

[0045] As an interrupt receiver, each core can check the interrupt source by reading the interrupt initiation register RAISE_IRQ_n, that is, querying the core ID that initiated the interrupt. If core 1 needs to check its own interrupt source, it can read its corresponding interrupt initiation register RAISE_IRQ_1.

[0046] Step 3: Check the interrupt confirmation

[0047] When sending an interrupt to a target core, if the core has an interrupt that has not been serviced or confirmed, an interrupt may be overwritten. Therefore, before starting the interrupt, you need to read the interrupt confirmation register ACK_IRQ_n of the target core to check whether the core has received other interrupts and has not completed them. If the interrupt pending status is read back, no new interrupt will be sent to the target core.

[0048] Step 4: Confirm that the interrupt has been serviced

[0049] The core that is the interrupt receiver generates an interrupt confirmation to indicate that the interrupt has been serviced. After the system component receives the confirmation, the corresponding interrupt line will be canceled and the interrupt pending state will be cleared, indicating that a new interrupt can be sent to the core.

[0050] In some embodiments, events can meet lightweight synchronization scenarios. For example, in an image classification task, different cores process data of similar size. At this time, events can be used to achieve synchronization to improve processing speed and efficiency. Compared with interrupts, events are triggered by hardware, without the intervention of the control core, and the response speed is faster.

[0051] Figure 3 The diagram is a schematic diagram of event distribution of a system component, wherein the system component includes an event manager 31, and the event manager 31 includes event registers corresponding to each of the cores 102;

[0052] The event manager 31 is used to record the identifier of the source core 23 in the event register corresponding to the target core 22, generate a pulse event for the target core 22, and check the status of the target core 22 from the event register corresponding to the target core 22 to obtain the execution status of the target core 22 on the pulse event.

[0053] Specifically, the source core 23 is used to write the identifier of the source core 23 into the event register corresponding to the target core 22;

[0054] The event manager 31 is used to trigger the generation of a pulse event and send it to the target core 22 when the identifier of the source core 23 is written in the event register corresponding to the target core 22; and to check the state of the target core 22 recorded in the event register corresponding to the target core 22 to determine the execution of the pulse event by the target core 22, and resend the pulse event after a certain period of time when it is determined that the pulse event has not been executed;

[0055] The target core 22 is used to update the state of the target core 22 in the event register corresponding to the target core 22 when the state changes, and return the certain time length for which the current state needs to be maintained to the event manager 31; and receive the pulse event and execute it.

[0056] The source core may be any core among multiple cores of the NPU that initiates an event, and the target core may be any core among multiple cores of the NPU that receives an event initiated by the source core.

[0057] Exemplarily, the pulse event includes a sleep event for instructing the target core to enter a sleep mode.

[0058] Combination Figure 3 The event distribution process shown in the figure mainly includes the following steps:

[0059] Step 1: Generate events

[0060] Any core can write its own ID to register SEND_EVENT_n to generate a pulse event. If core0 needs to send a sleep event to core 1, core 0 writes its own ID "0" to core 1's event register SEND_EVENT_1, and the event line will be connected to core 1.

[0061] Step 2: Check the event status

[0062] Any core can check the state of the target core after the event is sent to the target core. When reading SEND_EVENT_n, it can be determined whether core n is sleeping and waiting for events. If core 1 receives a sleep event but is blocked from entering sleep mode, core 0 can check the state and send a new event to the target core 1 later. The interval between resending events is longer than the time to enter sleep mode.

[0063] In some embodiments, shared resources are protected by setting semaphores, and the shared resources include memory resources, register resources, etc., and data consistency is ensured. For example, when multiple cores access a task queue, mutual exclusion is achieved through semaphores to ensure data consistency.

[0064] The system component 11 includes a synchronization manager, and the synchronization manager includes an atomic counter;

[0065] The synchronization manager is used to decrement the semaphore stored in the atomic counter when any of the cores 102 occupies resources; and, when any of the cores 102 needs to release resources, increment the semaphore stored in the atomic counter; and, when the semaphore stored in the atomic counter is zero, deny resource occupation by any of the cores 102 and put the core requesting resources into a sleep state until the amount of resources stored in the atomic counter is not zero, then waking up the core requesting resources; wherein the resources include resources shared by each of the cores 102 during the execution of model tasks.

[0066] Exemplarily, the initial value of the semaphore can be the total number of cores that need to be synchronized. For example, when the number of cores that need to be synchronized is 5, the initial value of the semaphore is set to 5. The initial value of the semaphore can also be the number of resources that can be shared by each core. The operations on the semaphore include decrement operation (also called P operation) and increment operation (also called V operation); when a core needs to occupy resources, the P operation is performed, that is, the atomic counter is decremented, and the value of the semaphore is reduced by 1; when a core releases resources, the V operation is performed, the atomic counter is incremented, and the value of the semaphore is increased by 1, waking up other cores waiting for the resource.

[0067] In some embodiments, the barrier is used to wait for multiple cores to reach a certain point before continuing execution. For example, a matrix multiplication is split into N tasks running on different cores. The barrier can be used to wait for the N cores to complete execution. When all cores reach the barrier point, the next step of calculation can be awakened.

[0068] The system components include a synchronization manager, the synchronization manager including an atomic counter;

[0069] The synchronization manager is used to save the number of cores that need to reach the barrier point corresponding to the model task in the atomic counter, and when any of the cores executes the model task and reaches the barrier point, reduce the number of cores that need to reach the barrier point by one; wherein, when the number of cores that need to reach the barrier point is not zero, the core requesting the next model task is placed in a waiting state, and the core in the waiting state is unlocked until the number of cores that need to reach the barrier point is reduced to zero.

[0070] For example, Figure 4 The following is a schematic diagram of the barrier implementation process, which mainly includes the following steps:

[0071] Step 1: Create a barrier, determine the number of cores that need to reach the barrier point, and initialize the atomic counter inside the synchronization manager to the number of cores;

[0072] Step 2: When any core calls the barrier function, the atomic counter counts down by one, indicating that the core has reached the barrier point. Any core can call the barrier function once during the execution of a task to trigger the change of the atomic counter.

[0073] Step 3: When the atomic counter is 0, it means that all cores have reached the barrier point. At this time, the cores waiting at the barrier point are unlocked so that the unlocked cores can perform the next calculation.

[0074] For example, Figure 5 The figure shows a schematic diagram of the multi-core NPU synchronization process. Any core in the NPU sends interrupts to other cores in the NPU or the host's CPU through the font component, and triggers pulse events to other cores in the NPU. The system component includes three modules, namely the event manager, the interrupt manager, and the synchronization manager. The event manager includes MMIO registers corresponding to each core, namely SEND_EVENT_0~n, which are used to send event pulses; the interrupt manager includes two interrupt-related MMIO registers - RAISE_IRQ_n and ACK_IRQ_n, which are used for each core to initiate and confirm interrupts respectively; the synchronization manager includes an atomic counter, which is used to implement semaphores or barriers in the system component. The system component is also used to merge completed interrupts and report them to the host's CPU.

[0075] The overall process of multi-core NPU synchronization mainly includes:

[0076] Step 1: Task Assignment

[0077] The host's CPU distributes neural network computing tasks to multiple cores according to certain rules and determines the input data and cache size of each core. For example, a matrix multiplication calculation is split into N tasks and distributed to N cores for parallel computing, with each core processing part of the data.

[0078] Step 2: Data transmission

[0079] Technologies such as DMA are used to accelerate the transmission of NPU external data (such as matrix multiplication input data, etc.) to the internal cache, thereby reducing the load on the host CPU.

[0080] Step 3: Inter-core synchronization

[0081] System components select interrupts, events, semaphores or barriers according to the scenario to achieve synchronization and interaction between multiple cores. For example, when a core completes the calculation, the call reaches the barrier function and the atomic counter is reduced by 1.

[0082] Step 4: Calculation completed

[0083] After all cores complete the calculation, the system component can summarize the final calculation results through interrupts / events. If N cores all call the arrival barrier function, the value of the relevant atomic counter is reduced to 0, and the system component can send a completion interrupt to the host CPU.

[0084] In the disclosed embodiments, a system component for multi-core synchronization in a neural network processor is provided, and the system component communicates between multiple cores under the control of a task scheduling processor of the neural network processor, so that each core executes the assigned model task synchronously, thereby providing efficient inter-core communication and synchronization for the multi-core neural network processor and realizing unified management of inter-core synchronization; the synchronization between multiple cores is completed by the system component, and there is no need to occupy too much resources of the host's central processing unit due to inter-core synchronization, thereby reducing the operating load of the host's central processing unit; and, by providing a direct connection channel between multiple cores through the system component, the interaction path and cost are shortened, thereby enabling the neural network processor to support more complex artificial intelligence modules and larger-scale data input.

[0085] In the prior art, a lock or semaphore-based solution is used to achieve synchronization of multi-core NPUs to provide critical areas or mutually exclusive areas to ensure mutually exclusive access to data and interaction between cores. However, when the number of cores increases, lock competition and synchronization overhead will inevitably increase, and the efficiency of communication between cores is low, which makes it difficult to meet the needs of scenarios with high real-time requirements such as autonomous driving. At the same time, the load of the host's CPU increases significantly. The disclosed embodiment realizes communication and synchronization between multiple cores in the NPU through a system component independent of the host's CPU, which can solve the defects in the prior art, sink the host's CPU and device interaction and management tasks to the system component, reduce the running load of the host's CPU, provide a direct connection channel between multiple cores, shorten the interaction path and cost between multiple cores, and expand multiple types of cores to achieve unified management of interrupts, events, states and synchronization between cores, and by merging the interrupt signals of multiple cores and interacting with the host's CPU, the interrupt hardware connection with the host's CPU is saved, which improves efficiency and reduces costs.

[0086] Based on the same concept, an embodiment of the present disclosure also provides a system component, which is applied to the multi-core synchronization system in the neural network processor described above, and is used to communicate between the cores of the neural network processor under the control of the task scheduling processor in the neural network processor so that each core executes the assigned model task synchronously.

[0087] In some embodiments, the system component includes an event manager, and the event manager includes an event register corresponding to each of the cores;

[0088] The event manager is used to record the identifier of the source core in the event register corresponding to the target core, generate a pulse event for the target core, and check the status of the target core from the event register corresponding to the target core to obtain the execution status of the target core on the pulse event.

[0089] In some embodiments, the event manager is used to trigger the generation of a pulse event and send it to the target core when the identifier of the source core is written in the event register corresponding to the target core; and is used to check the status of the target core recorded in the event register corresponding to the target core to determine the execution of the pulse event by the target core, and when it is determined that the pulse event has not been executed, resend the pulse event after a certain interval.

[0090] In some embodiments, the pulse event comprises a sleep event for instructing the target core to enter a sleep mode.

[0091] In some embodiments, the system components include a synchronization manager, wherein the synchronization manager includes an atomic counter;

[0092] The synchronization manager is used to decrement the semaphore stored in the atomic counter when any of the cores occupies resources; and increment the semaphore stored in the atomic counter when any of the cores needs to release resources; and, when the semaphore stored in the atomic counter is zero, refuse resource occupancy by any of the cores and put the core requesting resources into a dormant state until the amount of resources stored in the atomic counter is not zero, waking up the core requesting resources;

[0093] The resources include resources shared by the cores during the execution of model tasks.

[0094] In some embodiments, the system components include a synchronization manager, the synchronization manager including an atomic counter;

[0095] The synchronization manager is used to save the number of cores that need to reach the barrier point corresponding to the model task in the atomic counter, and when any of the cores executes the model task and reaches the barrier point, reduce the number of cores that need to reach the barrier point by one; wherein, when the number of cores that need to reach the barrier point is not zero, the core requesting the next model task is placed in a waiting state, and the core in the waiting state is unlocked until the number of cores that need to reach the barrier point is reduced to zero.

[0096] The system components provided in the embodiments of the present disclosure have two major functions:

[0097] First, it supports interrupts / events. It can trigger interrupts on any core in the NPU, and can also send event pulses to any core in the NPU. When multiple cores are running in parallel, interrupts / events can realize communication and collaborative processing between cores and summarize the final processing results.

[0098] Second, it supports semaphores and barriers. Semaphores can be used to manage shared resources between cores, while barriers allow synchronization at a certain point in the scheduling queue when multiple models run in parallel.

[0099] The system components provided by the embodiments of the present disclosure can reduce the load on the host's CPU, provide efficient inter-core communication and synchronization for a multi-core NPU system, achieve unified management of inter-core synchronization, and save interrupt resources.

[0100] The system components and multi-core synchronization system provided by the embodiments of the present disclosure can be applied to a variety of scenarios, including but not limited to smart car scenarios, smart phone scenarios, smart home scenarios, and other scenarios involving the deployment of a large number of artificial intelligence (AI) applications and requiring multi-core NPU synchronization support, involving application scenarios in the fields of smart cockpits, autonomous driving, the Internet of Things, etc. that require multi-core NPU synchronization support.

[0101] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods applied for above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0102] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0103] Those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present embodiment and to form different embodiments.

[0104] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present disclosure, but the present disclosure is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and substance of the present disclosure, and these modifications and improvements are also considered to be within the scope of protection of the present disclosure.

Claims

1. A multi-core synchronization system in a neural network processor, characterized in that: include: Neural network processor and system components; the neural network processor includes a task scheduling processor and multiple cores; The system components are respectively connected to the task scheduling processor and each of the cores via a system bus; The system component is used to communicate between the cores under the control of the task scheduling processor so that the cores can synchronously execute the assigned model tasks.

2. The multi-core synchronization system in a neural network processor according to claim 1, characterized in that: The neural network processor also includes a shared cache for storing intermediate calculation results of the task scheduling processor and each of the cores.

3. The multi-core synchronization system in a neural network processor according to claim 2, characterized in that: The system also includes a physical memory connected to the shared cache via a system bus; The physical memory is used to store the model tasks and task descriptors input by the neural network processor, and the task descriptors are used to describe at least one of the following information: the number of cores required for the model task operation, the model task input and / or output storage address, the intermediate data write-back address, and the event and / or status receiving address.

4. The multi-core synchronization system in a neural network processor according to claim 1, characterized in that: The system component includes an interrupt manager, and the interrupt manager includes an initiating interrupt register and a confirming interrupt register corresponding to each of the cores; The interrupt manager is used to record information about the source core initiating an interrupt to the target core in an initiating interrupt register corresponding to the target core; And the interrupt end information returned by the target core is recorded in the confirmation interrupt register corresponding to the target core.

5. The multi-core synchronization system in a neural network processor according to claim 4, characterized in that: The source core is used to write the identifier of the source core into the interrupt initiation register corresponding to the target core in the interrupt manager when initiating an interrupt to the target core; The interrupt manager is used to stop initiating new interrupts to the target core when determining that the target core is in an interrupt pending state; When it is determined that the target core is not in an interrupt pending state, the interrupt initiated by the source core is initiated to the target core, and the target core is marked as in an interrupt pending state until an interrupt confirmation returned by the target core is obtained, the interrupt pending state of the target core is cleared, and a confirmation interrupt register corresponding to the target core is updated; The target core is used to return an interrupt confirmation to the interrupt manager when the interrupt is completed.

6. The multi-core synchronization system in a neural network processor according to claim 5, characterized in that: The system component is also used to merge the interrupt completion information of each core and report it to the central processing unit of the host.

7. The multi-core synchronization system in a neural network processor according to claim 1, characterized in that: The system component includes an event manager, and the event manager includes event registers corresponding to each of the cores; The event manager is used to record the identifier of the source core in the event register corresponding to the target core, generate a pulse event for the target core, and check the status of the target core from the event register corresponding to the target core to obtain the execution status of the target core on the pulse event.

8. The multi-core synchronization system in a neural network processor according to claim 7, characterized in that: The source core is used to write the identifier of the source core into the event register corresponding to the target core; The event manager is used to trigger the generation of a pulse event and send it to the target core when the identifier of the source core is written into the event register corresponding to the target core; and for checking the state of the target core recorded in the event register corresponding to the target core to determine the execution of the pulse event by the target core, and resending the pulse event after a certain time interval when it is determined that the pulse event is not executed; The target core is used to update the state of the target core in the event register corresponding to the target core when the state changes, and return the certain time length for which the current state needs to be maintained to the event manager; and receive the pulse event and execute it.

9. The multi-core synchronization system in a neural network processor according to claim 8, characterized in that: The pulse event includes a sleep event for instructing the target core to enter a sleep mode.

10. The multi-core synchronization system in a neural network processor according to claim 1, characterized in that: The system components include a synchronization manager, wherein the synchronization manager includes an atomic counter; The synchronization manager is used to decrement the semaphore stored in the atomic counter when any of the cores occupies resources; and, when any of the cores needs to release resources, incrementing the semaphore stored in the atomic counter; and, when the semaphore stored in the atomic counter is zero, refusing resource occupation by any of the cores and causing the core requesting resources to enter a dormant state, until the amount of resources stored in the atomic counter is not zero, waking up the core requesting resources; The resources include resources shared by the cores during the execution of model tasks.

11. The multi-core synchronization system in a neural network processor according to claim 1, characterized in that: The system components include a synchronization manager, the synchronization manager including an atomic counter; The synchronization manager is used to save the number of cores that need to reach the barrier point corresponding to the model task in the atomic counter, and when any of the cores executes the model task and reaches the barrier point, reduce the number of cores that need to reach the barrier point by one; wherein, when the number of cores that need to reach the barrier point is not zero, the core requesting the next model task is placed in a waiting state, and the core in the waiting state is unlocked until the number of cores that need to reach the barrier point is reduced to zero.

12. A system component, characterized in that: The system component is used to communicate between the cores of the neural network processor under the control of the task scheduling processor of the neural network processor so that each core executes the assigned model task synchronously.

Images