Chip design method of NPU and DSP combined acceleration AI model

Abstract

The invention relates to the technical field of chip architecture design, and discloses a chip design method of an NPU and DSP combined acceleration AI model, comprising the following steps: executing chip internal authority initialization; executing authority verification and access scheduling based on a synchronous perception type cross-core access control mechanism; constructing a synchronous cooperation relationship between the NPU and the DSP; sub-graph tasks are reasonably dispatched to heterogeneous computing units; and optimal scheduling is carried out through heterogeneous core load similarity mapping and a self-adaptive migration decision-making mechanism. Compared with the prior art that heterogeneous core task division is performed only based on a static scheduling table or a preset calculation load, and particularly in a complex AI model reasoning scene with operator distribution dynamically changing or frequent execution congestion, task granularity cross-core migration scheduling is difficult to realize, the method has the advantages that the sub-graph division mechanism is constructed, so that the task granularity cross-core migration scheduling is realized, and the task granularity cross-core migration scheduling efficiency is improved. And in combination with self-adaptive migration control logic, scheduling optimization of dynamic load balancing and resource collaborative utilization in a task execution process is realized.

Description

Technical Field

[0001] This invention relates to the field of chip architecture design technology, and in particular to a chip design method for jointly accelerating AI models using an NPU and a DSP. Background Technology

[0002] Currently, with the urgent need for complex AI inference capabilities in edge computing and smart terminals, more and more AI chip architectures are adopting integrated heterogeneous acceleration solutions. By collaboratively deploying neural network processing units (NPUs) and digital signal processors (DSPs), a better balance is achieved between power consumption, computing power, and on-chip resources. Typical solutions, such as the ARM and NPU collaborative architecture and the Qualcomm Hexagon architecture, have been widely used in mobile, automotive, and industrial devices.

[0003] However, in the current technical architecture, the inference process of AI models generally faces the following key problems and technical bottlenecks: First, in the process of deploying model operators across cores, existing systems mostly rely on static rules or offline strategies to assign operator subgraphs, lacking a sense of linkage between operator structure topology, data communication coupling degree, and real-time load status within the core. Especially when the model structure is complex (such as multi-branch, deep residual networks) or the task changes frequently, it is difficult to achieve efficient inter-core task allocation and scheduling, resulting in some subgraphs being scheduled to computing units with mismatched resources, causing problems such as load bottlenecks, cache jitter, and increased inference latency. Second, in terms of cross-core communication control, the traditional synchronization mechanism between NPU and DSP mostly uses simple event flags or interrupt triggers, lacking the ability to dynamically model event context and dependency paths. In addition, existing chips mostly use fixed address mapping and static permission configuration in inter-core access control, making it difficult to accurately constrain access behavior under dynamic scheduling structures, especially in scenarios with high security requirements, posing a risk of unauthorized instruction access or illegal rewriting.

[0004] Therefore, there is an urgent need to propose a chip design method that uses NPU and DSP to jointly accelerate AI models, integrating ONNX model structure understanding, dynamic solidification of inter-core permissions, synchronous event path modeling and load-aware scheduling mechanism, so as to achieve efficient, secure and stable operation of complex AI models under heterogeneous chip architecture. Summary of the Invention

[0005] To address the aforementioned technical shortcomings, the present invention aims to propose a chip design method for jointly accelerating AI models using an NPU and a DSP. This method addresses the technical problem of difficulty in achieving cross-core migration scheduling at the task granularity in complex AI model inference scenarios where the heterogeneous core task partitioning is based solely on static scheduling tables or preset computing loads, especially in scenarios where operator distribution changes dynamically or execution congestion is frequent.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: The present invention provides a chip design method for jointly accelerating AI models using an NPU and a DSP. The chip design method for jointly accelerating AI models using NPU and DSP includes: Step S10: Perform internal initialization tasks on the chip using a segmented permission binding and dynamic boundary fixing mechanism through the on-chip control register, and output the permission configuration table. ; Step S20: Based on the permission configuration table A synchronous-aware cross-core access control mechanism is used to perform collaborative instruction parsing and permission verification tasks, and output a cross-core synchronous instruction set. Step S30: Based on the cross-core synchronization instruction set, a two-path handshake mechanism based on event association is used to perform the synchronization handshake task between the NPU and DSP, and output the synchronization result information; Step S40: Obtain the ONNX graph structure of the AI ​​model, and use a task scheduling-aware selection mechanism based on the ONNX graph structure and synchronization result information to execute the sub-task assignment of the AI ​​model, and output the mapping task structure queue. Step S50: Based on the mapping task structure queue, the mapping task scheduling task is executed using a heterogeneous core load similarity mapping and adaptive migration decision mechanism, and the final scheduling result set is output. .

[0007] Preferably, in step S10, an internal initialization task is performed on the chip using a segmented permission binding and dynamic boundary fixing mechanism through the on-chip control register, and a permission configuration table is output. The steps specifically include: Step S101: For the shared static random access memory, a three-dimensional binding relationship is generated through the on-chip control register. The three-dimensional binding relationship includes NPU dedicated partition, DSP dedicated partition, bidirectional interactive partition and configuration status partition. Step S102: For the bidirectional interactive partitions in the 3D binding relationship, perform dynamic boundary validity checks and output dynamic boundary validity indicators. ; Step S103: Based on dynamic boundary validity index The legal interval boundaries in the bidirectional interactive partition are written to a one-time solidification register to form a solidification result set; and a permission configuration table is generated based on the solidification result set. Permission configuration table It includes the access subject identifier ID, address boundary range, and operation permission vector; wherein, the access subject identifier ID is used to distinguish whether the access originates from the NPU or DSP; the address boundary range is used to represent the range of shared storage physical addresses that the access subject can access; the operation permission vector is used to define the set of access types for the access subject to the target address range, and the set of access types includes "whether read type is allowed" bit0, "write type" bit1, and "execute type" bit2.

[0008] Preferably, in step S20, based on the permission configuration table The steps for executing collaborative instruction parsing and permission verification tasks using a synchronization-aware cross-core access control mechanism, and outputting a cross-core synchronized instruction set, specifically include: Step S201: First, construct an access request (Access). The access request (Access) includes the subject ID, target address, operation type, and synchronization dependency tag. ; Step S202: Based on the access request and permission configuration table Perform a three-dimensional joint validity check, which includes address boundary validity check, operation permission validity check, and synchronization status validity check; output the validity check code; Step S203: When any item in the three-dimensional joint validity check is invalid, the following actions are performed: write the validity check code into the error status register, set the error type in the error status register to 1, and block the access request Access. When all items in the three-dimensional joint validity check are valid, the cross-core synchronization instruction set is output. The cross-core synchronization instruction set includes the master-slave mode trigger instruction set, the layer data sharing instruction set, and the synchronization mode instruction set.

[0009] Preferably, step S30, which involves executing a synchronization handshake task between the NPU and DSP using an event-related dual-path handshake mechanism based on a cross-core synchronization instruction set and outputting synchronization result information, specifically includes: Step S301: Obtain the synchronization mode instruction set from the cross-core synchronization instruction set; Step S302: If the synchronous mode instruction set shows that the hard-wired handshake method is currently used, the NPU sends a synchronization request to the DSP by setting the handshake request signal hsk[0]; after receiving the handshake request signal hsk[0], the DSP sends a handshake response signal hsk[1] to the NPU; If the synchronous mode instruction set indicates that the interrupt handshake method is currently in use, the NPU sends an interrupt trigger instruction to the DSP, triggering the DSP to enter the interrupt service procedure, and the DSP returns an interrupt response status signal. Step S303: Determine whether the synchronous handshake task was successfully executed based on the handshake response signal hsk[1] and the interrupt response status signal, and output the synchronization result information.

[0010] Preferably, in step S30, the success condition for the synchronous handshake task is: In the current case of using hard-wired handshake, the following conditions must be met: the NPU receives the handshake response signal hsk[1] and hsk[1]=1, and at the same time, the handshake timeout count does not exceed the preset timeout count threshold. In the current interrupt handshake mode, the following conditions must be met: the NPU detects the interrupt response status signal, and the interrupt response status signal is zero.

[0011] Preferably, step S40, which involves obtaining the ONNX graph structure of the AI ​​model, assigning subtasks to the AI ​​model using a task scheduling-aware selection mechanism based on the ONNX graph structure and synchronization result information, and outputting a queue of mapped task structures, specifically includes: Step S401: Obtain the ONNX graph structure of the AI ​​model and obtain the set of available computational cores based on the synchronization result information; construct a candidate subgraph set using the GraphCuts algorithm on the ONNX graph structure, and for the i-th candidate subgraph in the candidate subgraph set... The operator set is extracted using a structure vector extraction method based on topological attributes. operator set This includes load parameters and communication topology parameters; Step S402: Based on operator set The structural similarity metric is used to calculate the operator structural coupling score. When the operator structural coupling score is greater than a threshold, the i-th candidate subgraph is prioritized. The task is assigned to the NPU for processing; otherwise, the i-th candidate subgraph is given priority by default. Dispatch to DSP for processing; output a set of scheduling structures; Step S403: Based on the set of scheduling structures, perform sequential mapping and arrangement in the set of computing cores, and finally output the queue of mapped task structures.

[0012] Preferably, in step S50, the mapping task scheduling task is executed based on the mapping task structure queue using a heterogeneous core load similarity mapping and adaptive migration decision mechanism, and the final scheduling result set is output. The steps specifically include: Step S501: Periodic Intra-core Load Awareness Vector Construction Stage: Obtain the number of currently active operators and the total number of queued operators of the NPU at a period of 10ns, and define the NPU core load factor based on the number of currently active operators and the total number of queued operators of the NPU. Simultaneously, the number of currently active operators and the total number of queued operators in the DSP are obtained with a period of 10ns, and the DSP core load factor is defined based on the number of currently active operators and the total number of queued operators in the DSP. According to NPU core load factor With DSP core load factor Generate periodic in-core load sensing vectors ; Step S502: Rescheduling Determination Phase: Based on Periodic In-Kernel Load Awareness Vector The first load similarity score is calculated using a similarity analysis principle based on normalized cosine distance with the preset NPU operator structure template. ; and based on the periodic in-core load sensing vector The second load similarity score is calculated using a similarity analysis principle based on Manhattan distance with the preset DSP operator structure template. Based on the first load similarity score Second load similarity score Calculate the difference in load similarity scores When the load similarity score difference When the difference in load similarity score exceeds the preset threshold, the task will be migrated between the NPU and DSP in the direction with the smaller similarity score. Step S503: After completing the task migration, reorder the original mapped task structure queue and output the final scheduling result set. .

[0013] Preferably, in step S102, the dynamic boundary legality check includes interval overlap determination, subject consistency determination, and operation closure determination.

[0014] Preferably, in step S201, when When the value is 1, it indicates that access to construct an Access request can only be granted after synchronization is complete; when... A value of 0 indicates support for directly constructing an Access request.

[0015] Preferably, in step S40, the final output mapping task structure queue includes subgraph number, target core ID, estimated start timestamp, and dependency list.

[0016] The present invention also provides a chip design device for jointly accelerating an AI model using an NPU and a DSP, comprising: a memory, a processor, and a chip design program for jointly accelerating an AI model using an NPU and a DSP stored in the memory and executable on the processor. When the chip design program for jointly accelerating an AI model using an NPU and a DSP is executed by the processor, a chip design method for jointly accelerating an AI model using an NPU and a DSP is implemented.

[0017] The present invention also provides a computer program product, including a chip design program for an AI model jointly accelerated by an NPU and a DSP, wherein the chip design program for the AI ​​model jointly accelerated by an NPU and a DSP, when executed by a processor, implements the chip design method for the AI ​​model jointly accelerated by an NPU and a DSP.

[0018] The beneficial effects of this invention are as follows: By introducing a subgraph partitioning algorithm and a heterogeneous core scheduling strategy driven by structural similarity, this invention enables the computational tasks between the NPU and DSP to be assigned and migrated in real time based on the operator coupling degree and the adaptability of the computational core, which significantly improves the inference efficiency and inter-core collaboration capability of complex neural network models within the chip.

[0019] This invention effectively solves the risks of resource contention and unauthorized access in the traditional heterogeneous core access process by integrating segmented permission binding, dynamic boundary solidification, and dual-path handshake mechanism, ensuring that the chip has higher security, synchronization accuracy, and stability in the multi-core concurrent access state. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the first embodiment of a chip design method for jointly accelerating AI models using an NPU and a DSP according to the present invention.

[0022] Figure 2 This is a schematic diagram of a chip design method for jointly accelerating AI models using an NPU and a DSP, according to the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Example 1: As Figure 1 The diagram shown is a flowchart of the first embodiment of the chip design method for jointly accelerating AI models using NPU and DSP according to the present invention. The first embodiment of the chip design method for jointly accelerating AI models using NPU and DSP according to the present invention is presented.

[0025] In the first embodiment, the chip design method for jointly accelerating AI models using NPU and DSP includes: Step S10: Perform internal initialization tasks on the chip using a segmented permission binding and dynamic boundary fixing mechanism through the on-chip control register, and output the permission configuration table. ; It should be noted that the "segmented permission binding" in this step refers to: logically segmenting multiple accessible areas inside the chip (such as on-chip SRAM area, NPU dedicated cache area, DSP code segment, interrupt register area, etc.) and binding a preset permission tag set (such as read / write / execute / synchronize / debug, etc.) to each logical segment; while the "dynamic boundary hardening mechanism" is to write a specific permission mask into the control register during the startup phase, so that each permission segment enters a read-only locked state after initial configuration, preventing it from being tampered with by illegal instructions during operation.

[0026] Understandably, this mechanism not only completes the basic permission registration for each functional area on the chip, but also strengthens the boundary control of sensitive access behaviors, effectively preventing abnormal behaviors such as the DSP debugging module's unauthorized access to the NPU parameter area.

[0027] It should be understood that, compared to the design approach of traditional chips that use static address access mapping (such as simple MMU mapping), this embodiment adopts a combination mechanism of segmentation, masking, and solidification. It can build a security permission system with fine-grained access control capabilities and immutability during the execution phase within the chip without relying on external configuration tools. It is particularly suitable for cross-core instruction scheduling and dynamic memory interaction scenarios that may occur during multi-core collaborative execution.

[0028] For example, during the startup process of a chip, after the NPU loads its model weights, its dedicated SRAM area (such as address range 0x1000_0000~0x1000_FFFF) will be set to "readable and writable only by this core, read-only by other cores," and a write lock state will be configured through control registers to prevent any core from modifying the permission mapping of this area. Simultaneously, the segment containing the DSP debug interface (such as 0x2000_0000~0x2000_1FFF) will be set to "read-only, non-executable" after startup diagnostics are completed, forming an on-chip permission solidification barrier. The resulting permission configuration table is cached in the SoC configuration area in key-value pair format for reference during subsequent synchronization and scheduling phases.

[0029] Step S20: Based on the permission configuration table A synchronous-aware cross-core access control mechanism is used to perform collaborative instruction parsing and permission verification tasks, and output a cross-core synchronous instruction set. It should be noted that the "synchronization-aware cross-core access control mechanism" in this step refers to dynamically identifying and controlling collaborative access requests from different cores in a multi-core collaborative environment, based on the access permissions and status flags of each computing core for specific resource segments in the permission configuration table, combined with the current instruction execution context information. This mechanism relies on the on-chip access control unit and permission status register to receive, parse, verify permissions, and schedule cross-core instructions, ensuring that cross-core access requests can only enter the execution phase when permission and synchronization conditions are met.

[0030] Understandably, AI models often interact across cores with the NPU and DSP, involving intermediate feature map results, interrupt signals, and shared parameters. Using traditional static access strategies (such as pure address mapping or whitelisting rules) can easily lead to unauthorized writes, data races, or access conflicts. This step introduces a synchronization-aware access mechanism that dynamically determines instruction execution capability while ensuring multi-core data consistency, thus guaranteeing the controllability and security of collaborative access behavior.

[0031] For example, in a typical multi-core inference process, when the DSP core is about to read the intermediate tensor feature map T1 calculated by the NPU core, it first checks the permission configuration table to confirm that the DSP has read-only permissions to the memory segment where T1 is located. Then, it confirms through the synchronization status register that the NPU has completed the calculation of T1 and released the write lock. Then, it parses the DSP's read intent instruction (such as READ_T1→ADDR:0x1800_0000) through the cross-core access control interface, encapsulates it into a cross-core synchronization instruction structure (containing fields such as source core ID, destination resource, access type, and instruction signature) by the ACU scheduling unit, and adds it to the cross-core synchronization instruction set for subsequent handshake mechanism scheduling.

[0032] Step S30: Based on the cross-core synchronization instruction set, a two-path handshake mechanism based on event association is used to perform the synchronization handshake task between the NPU and DSP, and output the synchronization result information; It should be noted that the "event-based dual-path handshake mechanism" in this step refers to establishing two synchronization logical paths—an active event path and a passive event path—to address the synchronization needs when the NPU and DSP collaboratively execute intermediate tasks of an AI model. In the active event path, the initiating core proactively initiates a handshake request based on the task progress; in the passive event path, the receiving core listens for trigger flags in the synchronization event buffer and initiates a response process. This mechanism, driven by hardware-level event triggers and on-chip event registers, enables cross-core collaborative tasks to efficiently complete synchronization under different execution rhythms.

[0033] Understandably, during the acceleration of multi-core AI models, different cores may have inconsistent task execution rates due to architectural differences (e.g., NPU uses tensor parallelism, DSP uses scalar pipelining). Direct synchronization often results in blocking or waiting issues. This step decouples the process through a dual-path handshake, allowing one core to send a signal first and the other core to respond asynchronously. This avoids deadlocks or idle cycles caused by traditional locking mechanisms and enables efficient synchronous control that integrates asynchronous scheduling and event-driven processes.

[0034] It should be understood that compared with the traditional single-path synchronization mechanism (such as one-sided signaling blocking and waiting or interruption wake-up), this step has stronger adaptability and scalability: (1) it supports the decoupling and docking of different execution logics between heterogeneous cores; (2) it improves the determinism and stability of the intermediate task delivery process; (3) it is suitable for unstable operating conditions such as runtime task switching and scheduling misalignment; (4) it is based on event matching rather than fixed timing, and has stronger fault tolerance and parallelism guarantee.

[0035] For example, during AI model inference, after the NPU completes the convolution calculation, it needs to pass the intermediate feature tensor T2 to the DSP for activation function processing. At this time, the NPU will issue a handshake event EVENT_NPU_FINISH(T2) through the active event path and update the status flag corresponding to T2 in the event register to "Ready". The DSP periodically listens for this event flag and, upon detecting the T2 status update, initiates an acknowledgment response EVENT_DSP_ACK(T2) through the passive event path. Only after bidirectional event acknowledgment is T2 allowed to be scheduled and shared in the internal bus, thereby ensuring resource synchronization, secure transmission, and data consistency. The synchronization result information generated in this process will include fields such as task ID, resource identifier, trigger timestamp, and synchronization status, which the scheduler will use for further task dispatch.

[0036] Step S40: Obtain the ONNX graph structure of the AI ​​model, and use a task scheduling-aware selection mechanism based on the ONNX graph structure and synchronization result information to execute the sub-task assignment of the AI ​​model, and output the mapping task structure queue. It should be noted that the "task scheduling-aware selection mechanism" in this step refers to: after obtaining the complete ONNX graph structure, combining the synchronization result information output in step S30 (such as the current NPU and DSP status flags, load trends, core idle windows, data transmission rates, etc.), using a combination of structural segmentation and operator structure coupling analysis, segmenting the AI ​​model graph and binding it to the target processing kernel to generate the optimal heterogeneous mapping strategy. Specifically, this includes: ① Graph structure segmentation: using the GraphCuts algorithm to identify structural breakpoints in the ONNX graph and generate a set of candidate subgraphs; ② Feature extraction: calculating the operator structure vector (including load factor, parameter scale, topological dependencies, etc.) for each subgraph; ③ Scheduling selection: based on the subgraph structure vector and the current core state, using a structural coupling scoring function, evaluating its adaptability on the NPU / DSP and making binding decisions.

[0037] Understandably, this mechanism not only considers the operator type and data characteristics of the subgraph but also incorporates feedback on the current internal operating status of the chip, possessing "dynamic load awareness" capabilities. This avoids the problems of high-load repeated binding, inter-core waiting and blocking, or transmission redundancy that arise from only considering the static graph structure and ignoring the execution state in the traditional ONNX graph partitioning process. By comprehensively weighing the processing advantages and load states between heterogeneous cores before binding subgraph tasks, it achieves more efficient heterogeneous scheduling.

[0038] For example, taking a typical ResNet model, its ONNX graph structure contains stacked modules of multiple convolutions, batch normalization, and activation functions. During step S40, a subgraph grouped into Conv1-Conv3 is first sliced ​​using GraphCuts, and its operator structure vector V1 (containing information such as total MAC counts, parameter size, and successor node dependencies) is extracted. Synchronization results show that the NPU is currently under medium load, while the DSP is idle. Based on the structural coupling scoring function (such as cosine similarity comparison between V1 and the NPU operator template vector TNPU), it is determined that the NPU is more suitable for processing this subgraph, and thus it is assigned to the NPU. Another subgraph (such as one containing a ReLU+Add+BN combination) is assigned to the DSP due to its lower data transmission cost and moderate computational density. Finally, a mapping task structure queue is formed, where each structure includes fields such as subgraph ID, target processing kernel type, estimated execution time, resource usage window, and scheduling priority.

[0039] Step S50: Based on the mapping task structure queue, the mapping task scheduling task is executed using a heterogeneous core load similarity mapping and adaptive migration decision mechanism, and the final scheduling result set is output. .

[0040] It should be noted that the "heterogeneous core load similarity mapping and adaptive migration decision mechanism" in this step refers to the periodic monitoring of the running status of the NPU and DSP cores after receiving the mapping task structure queue generated in step S40, constructing a load-aware vector across time slices, and performing structural similarity calculations with the preset NPU / DSP operator structure templates respectively, thereby obtaining the matching degree score between the current mapping subgraph and the target computing core, and further determining whether to execute the cross-core migration of the subtask based on the load matching degree difference analysis and dynamic migration cost prediction.

[0041] Understandably, traditional heterogeneous scheduling methods often solidify the subtask-kernel binding relationship during the initial mapping stage, ignoring runtime kernel state changes and operator load behavior evolution. This can easily lead to uneven core resource utilization, task backlog, or excessive migration costs. This mechanism introduces an in-kernel load-aware vector (such as operator density, nearest execution window MAC value, cache conflict frequency, etc.) and a structural similarity scoring mechanism (such as normalized cosine distance analysis) to dynamically compare the rationality of mapping and determine whether migration is necessary based on the score difference. This mechanism has the advantages of strong adaptability and high decision-making accuracy, ensuring that scheduling rationality is maintained even under multi-task concurrency or model structure changes.

[0042] It should be understood that, compared with the traditional mechanism that makes migration decisions based solely on the current kernel load size, this scheme considers the structural coupling between the subgraph operator and the target kernel template after migration, ensuring that it still has execution efficiency advantages after migration.

[0043] For example, assuming the current model is YOLOv5, during execution, the "Conv+BN+SiLU" module subgraph of layer 3... Initially assigned to the NPU for processing. However, at 200ms of scheduling, it was detected that the NPU's core idle rate had dropped to 20% due to backlogs in other highly complex subgraphs, while the DSP's current idle rate was 70%. Extracting the module subgraph. operator structure vector The similarity scores were calculated with the operator template vectors TNPU and TDSP of NPU and TDSP, respectively, which were 0.52 and 0.68, resulting in a score difference ΔS = 0.16. Since ΔS is greater than the migration threshold of 0.12, and the estimated migration cost (data copying + context loading) is 1.8ms, which is less than the preset maximum tolerable migration cost of 2ms, a migration command is triggered, and the module subgraph is moved. The task was migrated to the DSP for execution, which effectively avoided NPU congestion.

[0044] Example 2: Furthermore, the chip design system for jointly accelerating AI models using NPU and DSP provided by this invention employs a chip design method for jointly accelerating AI models using NPU and DSP as described in the above embodiments, and can solve the technical problem of chip design for jointly accelerating AI models using NPU and DSP. Compared with the prior art, the beneficial effects of the chip design system for jointly accelerating AI models using NPU and DSP provided by this invention are the same as the beneficial effects of the chip design method for jointly accelerating AI models using NPU and DSP as described in the above embodiments, and other technical features of the chip design system for jointly accelerating AI models using NPU and DSP are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0045] Example 3: This invention provides a chip design device for jointly accelerating AI models using an NPU and a DSP. Please refer to... Figure 2A chip design device for jointly accelerating an AI model using an NPU and DSP includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to execute the chip design method for jointly accelerating an AI model using an NPU and DSP as described in Embodiment 1 above. The chip design device for jointly accelerating an AI model using an NPU and DSP in this embodiment of the invention may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. This chip design device for jointly accelerating an AI model using an NPU and DSP is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the invention. A chip design device for jointly accelerating AI models using an NPU and DSP may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes based on programs stored in a read-only memory 1002 or programs loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the chip design device for jointly accelerating AI models using an NPU and DSP. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An I / O interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows a chip design device for an NPU and DSP jointly accelerating AI model to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows a chip design device for an NPU and DSP jointly accelerating AI model with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0046] Example 4: This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the chip design method for jointly accelerating an AI model using an NPU and DSP as described above. The computer program product provided by this invention can solve the technical problem of chip design for jointly accelerating an AI model using an NPU and DSP. Compared with the prior art, the beneficial effects of the computer program product provided by this invention are the same as those of the chip design method for jointly accelerating an AI model using an NPU and DSP provided in the above embodiments, and will not be repeated here.

[0047] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this invention.

[0048] It should be understood that the various parts disclosed in this invention can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0049] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A chip design method for jointly accelerating AI models using NPU and DSP, characterized in that, The methods include: Step S10: Perform internal initialization tasks on the chip using a segmented permission binding and dynamic boundary fixing mechanism through the on-chip control register, and output the permission configuration table. ; Step S20: Based on the permission configuration table A synchronous-aware cross-core access control mechanism is used to perform collaborative instruction parsing and permission verification tasks, and output a cross-core synchronous instruction set. Step S30: Based on the cross-core synchronization instruction set, a two-path handshake mechanism based on event association is used to perform the synchronization handshake task between the NPU and DSP, and output the synchronization result information; Step S40: Obtain the ONNX graph structure of the AI ​​model, and use a task scheduling-aware selection mechanism based on the ONNX graph structure and synchronization result information to execute the sub-task assignment of the AI ​​model, and output the mapping task structure queue. Step S50: Based on the mapping task structure queue, the mapping task scheduling task is executed using a heterogeneous core load similarity mapping and adaptive migration decision mechanism, and the final scheduling result set is output. .

2. The chip design method for jointly accelerating AI models using NPU and DSP as described in claim 1, characterized in that, In step S10, the chip performs internal initialization tasks using a segmented permission binding and dynamic boundary fixing mechanism through the on-chip control register, and outputs the permission configuration table. The steps specifically include: Step S101: For the shared static random access memory, a three-dimensional binding relationship is generated through the on-chip control register. The three-dimensional binding relationship includes NPU dedicated partition, DSP dedicated partition, bidirectional interactive partition and configuration status partition. Step S102: For the bidirectional interactive partitions in the 3D binding relationship, perform dynamic boundary validity checks and output dynamic boundary validity indicators. ; Step S103: Based on dynamic boundary validity index The legal interval boundaries in the bidirectional interactive partition are written to a one-time solidification register to form a solidification result set; and a permission configuration table is generated based on the solidification result set. Permission configuration table It includes the access subject identifier ID, address boundary range, and operation permission vector; wherein, the access subject identifier ID is used to distinguish whether the access originates from the NPU or DSP; the address boundary range is used to represent the range of shared storage physical addresses that the access subject can access; the operation permission vector is used to define the set of access types for the access subject to the target address range, and the set of access types includes "whether read type is allowed" bit0, "write type" bit1, and "execute type" bit2.

3. The chip design method for jointly accelerating AI models using NPU and DSP as described in claim 1, characterized in that, In step S20, based on the permission configuration table The steps for executing collaborative instruction parsing and permission verification tasks using a synchronization-aware cross-core access control mechanism, and outputting a cross-core synchronized instruction set, specifically include: Step S201: First, construct an access request (Access). The access request (Access) includes the subject ID, target address, operation type, and synchronization dependency tag. ; Step S202: Based on the access request and permission configuration table Perform a three-dimensional joint validity check, which includes address boundary validity check, operation permission validity check, and synchronization status validity check; output the validity check code; Step S203: When any item in the three-dimensional joint validity check is invalid, the following actions are performed: write the validity check code into the error status register, set the error type in the error status register to 1, and block the access request Access. When all items in the three-dimensional joint validity check are valid, the cross-core synchronization instruction set is output. The cross-core synchronization instruction set includes the master-slave mode trigger instruction set, the layer data sharing instruction set, and the synchronization mode instruction set.

4. The chip design method for jointly accelerating AI models using NPU and DSP as described in claim 1, characterized in that, Step S30, which involves executing a synchronization handshake between the NPU and DSP using an event-related dual-path handshake mechanism based on a cross-core synchronization instruction set and outputting the synchronization result information, specifically includes: Step S301: Obtain the synchronization mode instruction set from the cross-core synchronization instruction set; Step S302: If the synchronous mode instruction set shows that the hard-wired handshake method is currently used, the NPU sends a synchronization request to the DSP by setting the handshake request signal hsk[0]; after receiving the handshake request signal hsk[0], the DSP sends a handshake response signal hsk[1] to the NPU; If the synchronous mode instruction set indicates that the interrupt handshake method is currently in use, the NPU sends an interrupt trigger instruction to the DSP, triggering the DSP to enter the interrupt service procedure, and the DSP returns an interrupt response status signal. Step S303: Determine whether the synchronous handshake task was successfully executed based on the handshake response signal hsk[1] and the interrupt response status signal, and output the synchronization result information.

5. The chip design method for jointly accelerating AI models using NPU and DSP as described in claim 4, characterized in that, In step S30, the success condition for the synchronous handshake task is: In the current case of using hard-wired handshake, the following conditions must be met: the NPU receives the handshake response signal hsk[1] and hsk[1]=1, and at the same time the handshake timeout count does not exceed the preset timeout count threshold. In the current interrupt handshake mode, the following conditions must be met: the NPU detects the interrupt response status signal, and the interrupt response status signal is zero.

6. The chip design method for jointly accelerating AI models using NPU and DSP as described in claim 1, characterized in that, Step S40 involves obtaining the ONNX graph structure of the AI ​​model, using a task scheduling-aware selection mechanism based on the ONNX graph structure and synchronization result information to assign subtasks to the AI ​​model, and outputting a queue of mapped task structures. This step specifically includes: Step S401: Obtain the ONNX graph structure of the AI ​​model and obtain the set of available computational cores based on the synchronization result information; construct a candidate subgraph set using the GraphCuts algorithm on the ONNX graph structure, and for the i-th candidate subgraph in the candidate subgraph set... The operator set is extracted using a structure vector extraction method based on topological attributes. operator set This includes load parameters and communication topology parameters; Step S402: Based on operator set The structural similarity metric is used to calculate the operator structural coupling score. When the operator structural coupling score is greater than a threshold, the i-th candidate subgraph is prioritized. The task is assigned to the NPU for processing; otherwise, the i-th candidate subgraph is given priority by default. Dispatch to DSP for processing; output a set of scheduling structures; Step S403: Based on the set of scheduling structures, perform sequential mapping and arrangement in the set of computing cores, and finally output the queue of mapped task structures.

7. The chip design method for jointly accelerating AI models using NPU and DSP as described in claim 1, characterized in that, In step S50, the mapping task scheduling task is executed based on the mapping task structure queue using a heterogeneous core load similarity mapping and adaptive migration decision mechanism, and the final scheduling result set is output. The steps specifically include: Step S501: Periodic Intra-core Load Awareness Vector Construction Stage: Obtain the number of currently active operators and the total number of queued operators of the NPU at a period of 10ns, and define the NPU core load factor based on the number of currently active operators and the total number of queued operators of the NPU. Simultaneously, the number of currently active operators and the total number of queued operators in the DSP are obtained with a period of 10ns, and the DSP core load factor is defined based on the number of currently active operators and the total number of queued operators in the DSP. According to NPU core load factor With DSP core load factor Generate periodic in-core load sensing vectors ; Step S502: Rescheduling Determination Phase: Based on Periodic In-Kernel Load Awareness Vector The first load similarity score is calculated using a similarity analysis principle based on normalized cosine distance with the preset NPU operator structure template. ; and based on the periodic in-core load sensing vector The second load similarity score is calculated using a similarity analysis principle based on Manhattan distance with the preset DSP operator structure template. Based on the first load similarity score Second load similarity score Calculate the difference in load similarity scores When the load similarity score difference When the difference in load similarity score exceeds the preset threshold, the task will be migrated between the NPU and DSP in the direction with the smaller similarity score. Step S503: After completing the task migration, reorder the original mapped task structure queue and output the final scheduling result set. .

8. The chip design method for jointly accelerating AI models using NPU and DSP as described in claim 2, characterized in that, In step S102, the dynamic boundary legality check includes interval overlap determination, subject consistency determination, and operation closure determination.

9. The chip design method for jointly accelerating AI models using NPU and DSP as described in claim 3, characterized in that, In step S201, when When the value is 1, it indicates that access to construct an Access request can only be granted after synchronization is complete; when... A value of 0 indicates support for directly constructing an Access request.

10. The chip design method for jointly accelerating AI models using NPU and DSP as described in claim 6, characterized in that, In step S40, the final output mapping task structure queue includes the subgraph number, target core ID, estimated start timestamp, and dependency list.

Images