Multi-core synchronization system and system component

Abstract

The present disclosure provides a multi-core synchronization system and a system component. The system comprises a neural network processor and a system component. The neural network processor comprises a task scheduling processor and a plurality of cores. The system component is connected to the task scheduling processor and the plurality of cores respectively by means of a system bus. The system component is used for performing communication among the plurality of cores under the control of the task scheduling processor, so that the plurality of cores synchronously execute allocated model tasks.

Description

Multi-core synchronization system and system components

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application CN 202310789897.X, filed on June 29, 2023, entitled “Multi-core synchronization system and system components in a neural network processor,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to the field of computer technology, and in particular to a multi-core synchronization system and system components. Background Art

[0004] Neural-network Processing Unit (NPU), a chip specifically designed to accelerate artificial neural network calculations, has been widely used in various artificial intelligence (AI) applications. Compared to traditional computing platforms such as the Central Processing Unit (CPU) and Graphics Processing Unit (GPU), the NPU adopts a "data-driven parallel computing" architecture with a multiply-accumulate (MAC) array dedicated to convolution operations as its core. Therefore, it is better at processing massive multimedia data such as videos and images. Since artificial neural network models usually require a lot of computing resources for training and inference, a single-core NPU may not be able to meet actual needs. Multi-core NPUs can split multiple models or a single large model and distribute them to different cores for parallel computing, which can significantly improve computing efficiency and achieve more flexible task allocation and load balancing.

[0005] As the number of cores and computing power continue to increase, NPUs require an efficient multi-core synchronization mechanism to ensure data consistency and correct calculation results across multiple cores during parallel computing, thereby improving application reliability and stability. Currently, most NPU vendors use lock-based solutions to achieve multi-core communication and synchronization. When the number of cores increases significantly, lock contention and increased lock overhead are prone to occur, reducing the efficiency of inter-core synchronization.

[0006] Summary of the Invention

[0007] The present disclosure provides a multi-core synchronization system and system components.

[0008] The present disclosure provides a multi-core synchronization system, comprising: a neural network processor and a system component, wherein the neural network processor includes a task scheduling processor and multiple cores; the system component is respectively connected to the task scheduling processor and each of the multiple cores through a system bus, wherein the system component is used to communicate between each of the multiple cores under the control of the task scheduling processor so that each of the multiple cores synchronously executes the assigned model tasks.

[0009] The present disclosure also provides a system component, wherein the system component is used to communicate between multiple cores of a neural network processor under the control of a task scheduling processor of the neural network processor, so that each of the multiple cores synchronously executes the assigned model tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] FIG1 is a schematic diagram of the architecture of a multi-core synchronization system provided in an embodiment of the present disclosure;

[0011] FIG2 is a schematic diagram of interrupt distribution of system components provided in an embodiment of the present disclosure;

[0012] FIG3 is a schematic diagram of event distribution of system components provided in an embodiment of the present disclosure;

[0013] FIG4 is a schematic diagram of a barrier implementation process provided in an embodiment of the present disclosure;

[0014] FIG5 is a schematic diagram of a multi-core NPU synchronization process provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0015] The following describes the specific embodiments of the present disclosure in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure and are not intended to limit the present disclosure.

[0016] As used in this disclosure, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0017] The terms used in the present disclosure are only used to describe specific embodiments and are not intended to limit the present disclosure.As used in the present disclosure, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0018] When the terms “comprising” and / or “made of…” are used in the present disclosure, it specifies the existence of the stated features, integers, steps, operations, elements and / or components, but does not preclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0019] Unless otherwise defined, all terms (including technical and scientific terms) used in this disclosure have the same meanings as those commonly understood by those skilled in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning unless expressly defined in this disclosure.

[0020] An embodiment of the present disclosure provides a multi-core synchronization system. FIG1 is a schematic diagram of the architecture of the multi-core synchronization system according to an embodiment of the present disclosure.

[0021] As shown in Figure 1, the multi-core synchronization system mainly includes a neural network processor 10 and a system component 11. The neural network processor (NPU) 10 includes a task scheduling processor (also known as a vector / scalar processor) 101 and multiple cores 102. The system component 11 is connected to the task scheduling processor 101 and each core 102 via a system bus 12. Under the control of the task scheduling processor 101, the system component 11 is used to communicate between the cores 102 so that each core 102 executes the assigned model task synchronously.

[0022] The system component 11 provides efficient communication and synchronization functions between the cores 102 of the multi-core NPU, thereby facilitating the application program to run in parallel or in a pipeline manner in multiple cores 102.

[0023] The task scheduling processor 101 is a task scheduling core shared by multiple cores 102 and is mainly responsible for controlling synchronization and task allocation between the multiple cores 102 through the system component 11 .

[0024] The system bus 12 is used to connect various devices and is responsible for transmitting data, addresses and control information.

[0025] As shown in FIG. 1 , core 0 to core n represent multiple cores 102 , where n represents an identification (ID) of the core 102 . The ID of each core 102 is fixed and unique.

[0026] In some embodiments, the neural network processor 10 further includes a shared cache 103 for storing intermediate calculation results of the task scheduling processor 101 and each core 102 .

[0027] In some embodiments, the system further includes a physical memory 13 connected to a shared cache 103 via a system bus 12 .

[0028] The physical memory 13 is used to store model tasks and task descriptors input by the neural network processor 10. The task descriptor is used to describe at least one of the following information: the number of cores required for the model task operation, the storage addresses of the model task input and / or output, the intermediate data write-back address, and the event and / or status reception address. The physical memory 13 can be any form of physical memory, for example, a double data rate (DDR) synchronous dynamic random access memory (SDRAM) or other physical memory.

[0029] The shared cache 103 and the physical memory 13 are also connected to the host's central processing unit 14 (also called Host CPU) via the system bus 12 .

[0030] The host's central processing unit (CPU) 14 is responsible for resource management and scheduling of upper-layer application tasks, controlling the operations of multiple devices similar to the NPU. The CPU 14 configures the system components 11 via the system bus 12 and assigns model tasks to the NPUs via the system bus 12. The CPU 14 then transfers information about the assigned model tasks to the NPU 10 to the NPU's shared cache 103. The NPU's task scheduling processor 101 reads the model tasks from the shared cache 103 and schedules them for execution by the cores 102.

[0031] The system component 11 can communicate between the cores 102 through interrupts, events, semaphores, barriers, etc. to coordinate the task execution progress of each core 102 and achieve the purpose of controlling the synchronization of each core 102.

[0032] In some embodiments, interrupts are used for real-time communication and processing when multiple cores of the NPU are running in parallel. They can coordinate situations where some cores execute too fast or too slowly. Applicable scenarios mainly include: a core suspends the task it is executing, notifies the target core to perform a certain operation, and waits for the target core's response to resume the task. For example, in the image detection stage of advanced driver assistance, when a core detects a target, it needs to send an interrupt to other cores to inform them to cooperate in processing the target information.

[0033] 2 is a schematic diagram of interrupt distribution of the system component 11. The system component 11 includes an interrupt manager 21. The interrupt manager 21 includes an initiating interrupt register and a confirming interrupt register corresponding to each core 102.

[0034] The interrupt manager 21 is used to record information that the source core 23 initiates an interrupt to the target core 22 in the interrupt initiation register corresponding to the target core 22 , and to record information that the target core 22 returns an interrupt completion in the interrupt confirmation register corresponding to the target core 22 .

[0035] Specifically, when the source core 23 initiates an interrupt to the target core 22 , the source core 23 writes an identifier of the source core 23 into an initiated interrupt register corresponding to the target core 22 in the interrupt manager 21 .

[0036] The interrupt manager 21 is used to: when it is determined that the target core 22 is in an interrupt pending state, stop initiating new interrupts to the target core 22; when it is determined that the target core 22 is not in an interrupt pending state, initiate the interrupt initiated by the source core 23 to the target core 22, mark the target core 22 as an interrupt pending state, and after obtaining the interrupt confirmation returned by the target core 22, clear the interrupt pending state of the target core 22 and update the confirmation interrupt register corresponding to the target core 22.

[0037] When the interrupt is completed, the target core 22 returns an interrupt confirmation to the interrupt manager 21 .

[0038] The source core 23 may be any core among the multiple cores 102 of the NPU 10 that initiates an interrupt, and the target core 22 may be any core among the multiple cores 102 of the NPU 10 that receives the interrupt initiated by the source core 23 .

[0039] In some embodiments, the system component 11 is further configured to merge the interrupt completion information of each core 102 and report it to the host's central processing unit 14. Compared to individual reporting, merging the interrupt completion information of each core 102 and reporting it to the host's central processing unit 14 can avoid excessive use of the host's central processing unit 14's interrupt resources and reduce the operating load of the host's central processing unit 14.

[0040] As shown in FIG2 , interrupt distribution mainly includes the following steps 1 to 4.

[0041] Step 1: Generate an interrupt

[0042] Each core can generate an interrupt by writing its own ID to the corresponding core's interrupt initiator register in the system component. For example, if core 0 needs to send an interrupt to core 1, core 0 writes its own ID "0" to core 1's interrupt initiator register RAISE_IRQ_1, and the interrupt signal is connected to core 1.

[0043] Step 2: Check the interrupt source

[0044] As an interrupt receiver, each core can check the interrupt source by reading the interrupt initiator register RAISE_IRQ_n (n represents the core ID). That is, query the ID of the core that initiated the interrupt. For example, when core 1 needs to check its own interrupt source, it can read its corresponding interrupt initiator register RAISE_IRQ_1.

[0045] Step 3: Check the interrupt confirmation

[0046] When sending an interrupt to a target core, if that core already has an interrupt that has not been serviced (i.e., not yet acknowledged), an interrupt overwrite may occur. Therefore, before initiating an interrupt, it is necessary to read the target core's interrupt acknowledgement register, ACK_IRQ_n (where n represents the core ID), to check whether the target core has already received and completed other interrupts. If the readback indicates an interrupt pending, no new interrupt is sent to the target core.

[0047] Step 4: Confirm that the interrupt has been serviced

[0048] The core that serves as the interrupt receiver generates an interrupt confirmation to indicate that the interrupt has been serviced. After the system component receives the interrupt confirmation, the corresponding interrupt line will be canceled and the interrupt pending state will be cleared, indicating that a new interrupt can be sent to the core that sent the interrupt confirmation.

[0049] In some cases, events can be used for lightweight synchronization. For example, in an image classification task, different cores may process similar amounts of data. In this case, events can be used to synchronize data, improving processing speed and efficiency. Compared to interrupts, events are triggered by hardware, eliminating the need for control core intervention and resulting in faster response times.

[0050] 3 is a schematic diagram of event distribution of the system component 11. The system component 11 includes an event manager 31. The event manager 31 includes event registers corresponding to each core 102.

[0051] The event manager 31 is used to record the identifier of the source core 23 in the event register corresponding to the target core 22, generate a pulse event for the target core 22, and check the status of the target core 22 from the event register corresponding to the target core 22 to obtain the execution status of the target core 22 on the pulse event.

[0052] Specifically, the source core 23 writes the identifier of the source core 23 into the event register corresponding to the target core 22 .

[0053] The event manager 31 is used to: generate a pulse event and send the pulse event to the target core 22 when the identifier of the source core 23 is written in the event register corresponding to the target core 22, and check the status of the target core 22 recorded in the event register corresponding to the target core 22 to determine the execution of the pulse event by the target core 22, and re-send the pulse event to the target core 22 after a first time interval when it is determined that the pulse event has not been executed.

[0054] When the state of the target core 22 changes, the target core 22 updates the state of the target core 22 in the event register corresponding to the target core 22, and returns the first duration for which the current state needs to be maintained to the event manager 31. The target core 22 is also used to receive pulse events and execute pulse events.

[0055] The source core 23 may be any core among the multiple cores 102 of the NPU 10 that initiates an event, and the target core 22 may be any core among the multiple cores 102 of the NPU 10 that receives the event initiated by the source core 23 .

[0056] Exemplarily, the pulse event includes a sleep event for instructing the target core 22 to enter a sleep mode.

[0057] As shown in FIG3 , event distribution mainly includes the following steps 1 and 2.

[0058] Step 1: Generate events

[0059] Any core can generate a pulse event by writing its own ID to the corresponding core's event register SEND_EVENT_n (n represents the core ID) in the system component. For example, if core 0 needs to send a sleep event to core 1, core 0 writes its own ID "0" to core 1's event register SEND_EVENT_1, and the event line is connected to core 1.

[0060] Step 2: Check the event status

[0061] After sending an event to a target core, any core can check the target core's status. By reading the event register SEND_EVENT_n (n represents the core ID) corresponding to the target core, it can determine whether the target core is in sleep mode and waiting for events. If core 1 receives a sleep event but is prevented from entering sleep mode, core 0 can check the status and resend the event to core 1 later. The time interval between resending events (i.e., the first duration) is longer than the time it takes to enter sleep mode.

[0062] In some embodiments, semaphores are used to protect shared resources and ensure data consistency. Shared resources include memory resources, register resources, etc. For example, when multiple cores access a task queue, semaphores are used to achieve mutual exclusion to ensure data consistency.

[0063] The system component 11 includes a synchronization manager. The synchronization manager includes an atomic counter.

[0064] The synchronization manager is configured to: decrement the semaphore stored in the atomic counter when any core 102 occupies a resource; increment the semaphore stored in the atomic counter when any core 102 releases a resource; and, if the semaphore stored in the atomic counter is zero, reject the resource occupation request from any core 102 and put the resource-requesting core 102 into a dormant state until the resource amount stored in the atomic counter is non-zero, at which time the resource-requesting core 102 is awakened. Resources include resources shared by each core 102 during the execution of a model task.

[0065] For example, the initial value of the semaphore can be the number of cores that need to be synchronized. For example, when 5 cores need to be synchronized, the initial value of the semaphore is set to 5. The initial value of the semaphore can also be the number of resources that can be shared by each core. Operations on semaphores include decrement operations (also called P operations) and increment operations (also called V operations). When a core occupies a resource, a P operation is performed, that is, the atomic counter is decremented and the value of the semaphore is reduced by 1; when a core releases a resource, a V operation is performed, the atomic counter is incremented, the value of the semaphore is increased by 1, and other cores waiting for the resource are woken up.

[0066] In some embodiments, barriers are used in scenarios where it is necessary to wait for multiple cores to reach a certain point before continuing execution. For example, a matrix multiplication is split into N tasks running on different cores. Barrier blocking can be used to wait for these N cores to complete execution. When all cores reach the barrier point, the next step of calculation can be awakened.

[0067] The system component 11 includes a synchronization manager. The synchronization manager includes an atomic counter.

[0068] The synchronization manager is used to: store the number of cores that need to reach the barrier point corresponding to the model task in the atomic counter; when any core executes the model task and reaches the barrier point, reduce the number of cores that need to reach the barrier point by one; when the number of cores that need to reach the barrier point is not zero, put the core requesting the next model task in a waiting state until the number of cores that need to reach the barrier point is reduced to zero, and then unlock the core in the waiting state.

[0069] For example, FIG4 is a schematic diagram of the barrier implementation process, which mainly includes the following steps 1 to 3.

[0070] Step 1: Create a barrier, that is, determine the number of cores that need to reach the barrier point, and initialize the atomic counter inside the synchronization manager to the number of cores.

[0071] Step 2: When any core calls the barrier function, it means that the core reaches the barrier point, and the atomic counter count is reduced by one. Any core can call the barrier function once during the execution of a task to trigger the change of the atomic counter.

[0072] Step 3: When the atomic counter is 0, it means that all cores have reached the barrier point. At this time, the cores waiting at the barrier point are unlocked so that the unlocked cores can perform the next calculation.

[0073] For example, Figure 5 is a schematic diagram of the multi-core NPU synchronization process. Any core in the NPU sends an interrupt to other cores in the NPU or the host's CPU through the system component 11, and triggers a pulse event to other cores in the NPU. The system component 11 includes: an event manager (Event Manager), an interrupt manager (Interrupt Request Manager) and a synchronization manager (Sync Manager). The event manager includes MMIO (Memory-mapped I / O) registers corresponding to each core, namely SEND_EVENT_0 to SEND_EVENT_n, for sending event pulses. The interrupt manager includes two interrupt-related MMIO registers RAISE_IRQ and ACK_IRQ, which are used for each core to initiate and confirm interrupts, respectively. The synchronization manager includes an atomic counter (Atomic Counter) for implementing semaphores or barriers in the system component 11. The system component 11 is also used to merge completed interrupts and report them to the host's central processor 14.

[0074] The overall process of multi-core NPU synchronization mainly includes steps 1 to 4.

[0075] Step 1: Task Assignment

[0076] The host's central processing unit 14 distributes neural network computing tasks across multiple cores according to specific rules, determining the input data and cache size for each core. For example, a matrix multiplication calculation can be split into N equal parts and distributed to the N cores for parallel computing, with each core processing a portion of the data.

[0077] Step 2: Data transmission

[0078] Direct Memory Access (DMA) and other technologies are used to accelerate the transmission of NPU external data (eg, matrix multiplication input data, etc.) to the internal cache, thereby reducing the load of the host's central processing unit 14.

[0079] Step 3: Inter-core synchronization

[0080] System component 11 selects interrupts, events, semaphores, or barriers according to the scenario to achieve synchronization and interaction between multiple cores. For example, when a core completes calculation, it calls the barrier function and the atomic counter is decremented by 1.

[0081] Step 4: Calculation completed

[0082] After all cores complete the computation, system component 11 can aggregate the final computation results via interrupts / events. For example, if all N cores call the reach barrier function, the value of the associated atomic counter decreases to 0. At this point, system component 11 can send a unified completion interrupt to the host's CPU 14.

[0083] In the embodiment of the present disclosure, a system component for synchronization between multiple cores in the NPU is set up, and the system component communicates between the multiple cores under the control of the task scheduling processor of the NPU, so that each core executes the assigned model task synchronously, thereby providing efficient inter-core communication and synchronization for the multi-core NPU and realizing unified management of inter-core synchronization. The synchronization between multiple cores is completed by the system component, and there is no need to occupy too much resources of the host's central processing unit due to inter-core synchronization, thereby reducing the operating load of the host's central processing unit. In addition, by providing a direct connection channel between multiple cores through the system component, the interaction path and cost are shortened, thereby enabling the NPU to support more complex artificial intelligence modules and larger-scale data input.

[0084] In the related art, solutions based on locks or semaphores are used to synchronize multi-core NPUs, providing critical sections or mutually exclusive sections to ensure mutually exclusive access to data and interaction between cores. However, as the number of cores increases, lock contention and synchronization overhead will inevitably increase, and the efficiency of inter-core communication is low, making it difficult to meet the needs of scenarios with high real-time requirements such as autonomous driving. At the same time, the load on the host's central processing unit (CPU) is significantly increased. According to the embodiments of the present disclosure, communication and synchronization between multiple cores in the NPU are achieved through system components independent of the host's CPU, which can solve the defects existing in the related art, transfer the interaction between the host's CPU and the device and management tasks to the system components, reduce the operating load of the host's CPU, provide direct connection channels between multiple cores, shorten the interaction paths and costs between multiple cores, and expand multiple types of cores to achieve unified management of interrupts, events, states, and synchronization between cores. By merging the interrupt signals of multiple cores and interacting with the host's CPU, the interrupt hardware connection with the host's CPU is saved, improving efficiency and reducing costs.

[0085] Based on the same concept, an embodiment of the present disclosure also provides a system component, which is applied to the multi-core synchronization system described above, and is used to communicate between the cores of the NPU under the control of the task scheduling processor in the NPU so that each core executes the assigned model task synchronously.

[0086] In some embodiments, the system component includes an event manager, wherein the event manager includes event registers corresponding to each core.

[0087] The event manager is used to record the identification of the source core in the event register corresponding to the target core, generate a pulse event for the target core, and check the status of the target core from the event register corresponding to the target core to obtain the execution status of the target core on the pulse event.

[0088] In some embodiments, the event manager is used to: generate a pulse event and send the pulse event to the target core when the identifier of the source core is written into the event register corresponding to the target core, and check the status of the target core recorded in the event register corresponding to the target core to determine the execution of the pulse event by the target core, and re-send the pulse event to the target core after a first time interval when it is determined that the event has not been executed.

[0089] In some embodiments, the pulse event comprises a sleep event for instructing the target core to enter a sleep mode.

[0090] In some embodiments, the system component includes a synchronization manager. The synchronization manager includes an atomic counter.

[0091] The synchronization manager is used to decrement the semaphore stored in the atomic counter when any core occupies resources, and increment the semaphore stored in the atomic counter when any core releases resources. If the semaphore stored in the atomic counter is zero, the synchronization manager rejects any core's resource occupation request and puts the core requesting resources into a dormant state until the resource amount stored in the atomic counter is non-zero, at which point the core requesting resources is awakened. Resources include those shared by all cores during the execution of model tasks.

[0092] In some embodiments, the system component includes a synchronization manager. The synchronization manager includes an atomic counter.

[0093] The synchronization manager is used to: store the number of cores that need to reach the barrier point corresponding to the model task in the atomic counter; when any core executes the model task and reaches the barrier point, reduce the number of cores that need to reach the barrier point by one; when the number of cores that need to reach the barrier point is not zero, put the core requesting the next model task in a waiting state until the number of cores that need to reach the barrier point is reduced to zero, and then unlock the core in the waiting state.

[0094] The system components provided in the embodiments of the present disclosure have two major functions:

[0095] First, it supports interrupts and events. It can trigger interrupts on any core within the NPU and send event pulses to any core within the NPU. When multiple cores are running in parallel, interrupts and events can realize inter-core communication and coordinated processing, and summarize the final processing results.

[0096] Second, it supports semaphores and barriers. Semaphores can be used to manage shared resources between cores, and barriers allow synchronization at a certain point in the scheduling queue (i.e., the barrier point) when multiple models run in parallel.

[0097] The system components provided by the embodiments of the present disclosure can reduce the load on the host's central processing unit, provide efficient inter-core communication and synchronization for a multi-core NPU system, implement unified management of inter-core synchronization, and save interrupt resources.

[0098] The system components and multi-core synchronization system provided by the embodiments of the present disclosure can be applied to a variety of scenarios, including but not limited to smart car scenarios, smart phone scenarios, smart home scenarios and other scenarios involving the deployment of a large number of artificial intelligence (AI) applications and requiring multi-core NPU synchronization support, as well as application scenarios in the fields of smart cockpits, autonomous driving, the Internet of Things, etc. that require multi-core NPU synchronization support.

[0099] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods applied for above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable, and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0100] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0101] Those skilled in the art will understand that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present embodiment and to form different embodiments.

[0102] It is understood that the above embodiments are merely exemplary embodiments for illustrating the principles of the present disclosure, and the present disclosure is not limited thereto. Those skilled in the art may make various modifications and improvements without departing from the spirit and substance of the present disclosure, and such modifications and improvements are also considered to be within the scope of protection of the present disclosure.

Claims

1. A multi-core synchronization system, comprising: Neural network processors and system components, The neural network processor includes a task scheduling processor and multiple cores; The system components are respectively connected to the task scheduling processor and each of the multiple cores via a system bus. The system component is used to communicate between the multiple cores under the control of the task scheduling processor, so that the multiple cores can synchronously execute the assigned model tasks.

2. The multi-core synchronization system according to claim 1, wherein: The neural network processor also includes a shared cache for storing intermediate calculation results of the task scheduling processor and each of the multiple cores.

3. The multi-core synchronization system according to claim 2, wherein: The system further comprises a physical memory, wherein the physical memory is connected to the shared cache via the system bus. The physical memory is used to store the model task and the task descriptor input by the neural network processor, and the task descriptor is used to describe at least one of the following information: The number of cores required for the model task operation; The model task input and / or output storage address; Intermediate data write-back address; The address to which events and / or status are sent.

4. The multi-core synchronization system according to claim 1, wherein: The system component includes an interrupt manager, wherein the interrupt manager includes an initiation interrupt register and a confirmation interrupt register corresponding to each of the plurality of cores. The interrupt manager is used to: record information about the source core initiating an interrupt to the target core in an initiating interrupt register corresponding to the target core, and The corresponding confirmation interrupt register records the information indicating the end of the interrupt returned by the target core.

5. The multi-core synchronization system according to claim 4, wherein: In the case where the source core initiates an interrupt to the target core, the source core writes an identifier of the source core into an interrupt initiation register corresponding to the target core in the interrupt manager, The interrupt manager is used to: When determining that the target core is in an interrupt pending state, stopping initiating new interrupts to the target core; In the case of determining that the target core is not in an interrupt pending state, initiating an interrupt initiated by the source core to the target core, marking the target core as being in an interrupt pending state, and after obtaining an interrupt confirmation returned by the target core, clearing the interrupt pending state of the target core and updating a confirmation interrupt register corresponding to the target core; The target core returns the interrupt confirmation to the interrupt manager if the interrupt is completed.

6. The multi-core synchronization system according to claim 5, wherein: The system component is also used to merge the interrupt completion information of each of the multiple cores and report it to the central processing unit of the host.

7. The multi-core synchronization system according to claim 1, wherein: The system component includes an event manager, the event manager includes event registers corresponding to each of the plurality of cores, The event manager is used to: record the identifier of the source core in the event register corresponding to the target core, generate a pulse event for the target core, and check the state of the target core from the event register corresponding to the target core to obtain the execution status of the target core on the pulse event.

8. The multi-core synchronization system according to claim 7, wherein: The source core writes the identifier of the source core into the event register corresponding to the target core, The event manager is used to: generate the pulse event and send the pulse event to the target core when the identifier of the source core is written into the event register corresponding to the target core, and check the state of the target core recorded in the event register corresponding to the target core to determine the execution of the pulse event by the target core, and resend the pulse event to the target core after a first time interval when it is determined that the pulse event is not executed. When the state of the target core changes, the target core updates the state of the target core in the event register corresponding to the target core, and returns the first duration for which the current state needs to be maintained to the event manager, and the target core is also used to receive the pulse event and execute the pulse event.

9. The multi-core synchronization system according to claim 8, wherein: The pulse event includes a sleep event for instructing the target core to enter a sleep mode.

10. The multi-core synchronization system according to claim 1, wherein: The system components include a synchronization manager, the synchronization manager including an atomic counter, The synchronization manager is used to: In the case where any of the plurality of cores occupies resources, a semaphore stored in the atomic counter is decremented, When any of the plurality of cores releases resources, the semaphore stored in the atomic counter is incremented. When the semaphore stored in the atomic counter is zero, reject the resource occupation request of any of the multiple cores, and make the core requesting resources enter a dormant state, until the amount of resources stored in the atomic counter is not zero, then wake up the core requesting resources, The resources include resources shared by the multiple cores during the execution of the model tasks.

11. The multi-core synchronization system according to claim 1, wherein: The system components include a synchronization manager, the synchronization manager including an atomic counter, The synchronization manager is used to: The number of cores that need to reach the barrier point corresponding to the model task is stored in the atomic counter, When any of the plurality of cores executes the model task and reaches the barrier point, the number of cores that need to reach the barrier point is reduced by one. When the number of cores that need to reach the barrier point is not zero, the core requesting the next model task is placed in a waiting state until the number of cores that need to reach the barrier point is reduced to zero, and then the core in the waiting state is unlocked.

12. A system component, wherein: The system component is used to communicate between multiple cores of the neural network processor under the control of the task scheduling processor of the neural network processor, so that each of the multiple cores synchronously executes the assigned model tasks.

13. The system component according to claim 12, comprising an interrupt manager, the interrupt manager comprising an initiating interrupt register and a confirming interrupt register corresponding to each of the plurality of cores, The interrupt manager is used to record information that a source core initiates an interrupt to the target core in an interrupt initiation register corresponding to the target core, and to record information that an interrupt is completed returned by the target core in an interrupt confirmation register corresponding to the target core.

14. The system assembly of claim 13, wherein: In the case where the source core initiates an interrupt to the target core, the source core writes an identifier of the source core into an interrupt initiation register corresponding to the target core in the interrupt manager, The interrupt manager is used to: When determining that the target core is in an interrupt pending state, stopping initiating new interrupts to the target core; When it is determined that the target core is not in an interrupt pending state, The target core starts the interrupt initiated by the source core, marks the target core as an interrupt pending state, and after obtaining the interrupt confirmation returned by the target core, clears the interrupt pending state of the target core and updates the confirmation interrupt register corresponding to the target core; The target core returns the interrupt confirmation to the interrupt manager if the interrupt is completed.

15. The system component according to claim 14, further configured to merge the interrupt completion information of each of the plurality of cores and report the information to a central processing unit of a host.

16. The system component of claim 12, comprising an event manager, the event manager comprising event registers corresponding to each of the plurality of cores, The event manager is used to: record the identifier of the source core in the event register corresponding to the target core, generate a pulse event for the target core, and check the state of the target core from the event register corresponding to the target core to obtain the execution status of the target core on the pulse event.

17. The system assembly of claim 16, wherein: The source core writes the identifier of the source core into the event register corresponding to the target core, The event manager is used to: generate the pulse event and send the pulse event to the target core when the identifier of the source core is written into the event register corresponding to the target core, and check the state of the target core recorded in the event register corresponding to the target core to determine the execution of the pulse event by the target core, and resend the pulse event to the target core after a first time interval when it is determined that the pulse event is not executed. When the state of the target core changes, the target core updates the state of the target core in the event register corresponding to the target core, and returns the first duration for which the current state needs to be maintained to the event manager, and the target core is also used to receive the pulse event and execute the pulse event.

18. The system assembly of claim 17, wherein: The pulse event includes a sleep event for instructing the target core to enter a sleep mode.

19. The system component of claim 12, comprising a synchronization manager, the synchronization manager comprising an atomic counter, The synchronization manager is used to: In the case where any of the plurality of cores occupies resources, a semaphore stored in the atomic counter is decremented, When any of the plurality of cores releases resources, the semaphore stored in the atomic counter is incremented. When the semaphore stored in the atomic counter is zero, reject the resource occupation request of any of the multiple cores, and make the core requesting resources enter a dormant state, until the amount of resources stored in the atomic counter is not zero, then wake up the core requesting resources, in, The resources include resources shared by the multiple cores during the execution of the model task.

20. The system component of claim 12, comprising a synchronization manager, the synchronization manager comprising an atomic counter, The synchronization manager is used to: The number of cores that need to reach the barrier point corresponding to the model task is stored in the atomic counter, When any of the plurality of cores executes the model task and reaches the barrier point, the number of cores that need to reach the barrier point is reduced by one. When the number of cores that need to reach the barrier point is not zero, the core requesting the next model task is placed in a waiting state until the number of cores that need to reach the barrier point is reduced to zero, and then the core in the waiting state is unlocked.

Images