Model reasoning acceleration method, device, storage medium and program product

Abstract

The present application discloses a model reasoning acceleration method, device, storage medium and program product, which relate to the field of artificial intelligence technology. The method includes: determining whether the current operator is an operator to be offloaded. If so, all computing operations are performed on the computing task of the current operator on the main core, and the computing execution result of the current operator is output; if not, a first computing operation is performed on the computing task of the current operator on the main core, and the first execution result of the current operator is output. At the same time, a second computing operation is performed on the computing task of the current operator on the cooperative core, and the second execution result of the current operator is output. This solves the problem of slow reasoning speed due to disabling an external hardware acceleration card and limited performance of the hardware acceleration card itself, and achieves the technical effect of improving the model reasoning speed; based on the interrupt response mechanism, the computing execution result of the current operator is generated on the main core according to the first execution result and the second execution result, and achieves the technical effect of improving the real-time performance of the model reasoning.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model reasoning acceleration method, device, storage medium, and program product. Background Art

[0002] Traditional baseboard management controllers rely on low-power central processing units (CPUs) to run model inference, but due to their limited computing power, they struggle to meet real-time requirements. Hardware accelerator cards are one of the current mainstream technologies for accelerating AI inference. Their core idea is to significantly improve performance by offloading computationally intensive tasks through dedicated hardware, such as neural network processing units, field-programmable gate arrays (FPGAs), and graphics processing units (GPUs). This type of solution has been widely used in high-performance scenarios such as servers and edge computing. However, hardware accelerator cards have the following fatal flaws in baseboard management controller scenarios: (I) Cost and hardware compatibility issues: 1. Excessive hardware cost; 2. Interface compatibility limitations; 3. Physical space conflicts. (II) Power consumption and heat dissipation limitations: 1. Excessive power consumption; 2. Complex heat dissipation design. (III) Software ecosystem and maintenance costs: 1. Fragmented tool chain; 2. Poor model compatibility; 3. Firmware upgrade risks. Summary of the Invention

[0003] The present application provides a model reasoning acceleration method, device, storage medium and program product to at least solve the problem in the related art that the baseboard management controller cannot use a hardware acceleration card due to limitations such as cost, power consumption and space, making it difficult to achieve model reasoning acceleration.

[0004] This application provides a model reasoning acceleration method, which is applied to a baseboard management controller. The baseboard management controller includes a main core and a co-core. The model reasoning acceleration method includes:

[0005] In response to receiving a model inference request, loading a preset model on the main core according to the model inference request, and determining whether the current operator is an operator to be offloaded;

[0006] If not, all computations are performed on the current operator on the main core based on the input tensor of the current operator, and the computation results of the current operator are output;

[0007] If so, then based on the input tensor of the current operator, perform a first computing operation on the current operator on the main core, output the first execution result of the current operator, and simultaneously perform a second computing operation on the current operator on the co-core, output the second execution result of the current operator, wherein all computing operations include at least the first computing operation and the second computing operation;

[0008] Based on the interrupt response mechanism, the calculation execution result of the current operator is generated on the main core according to the first execution result and the second execution result of the current operator.

[0009] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing, when executing the computer program, a model reasoning acceleration method comprising at least the following steps:

[0010] In response to receiving a model inference request, loading a preset model on the main core according to the model inference request, and determining whether the current operator is an operator to be offloaded;

[0011] If not, all computations are performed on the current operator on the main core based on the input tensor of the current operator, and the computation results of the current operator are output;

[0012] If so, then based on the input tensor of the current operator, perform a first computing operation on the current operator on the main core, output the first execution result of the current operator, and simultaneously perform a second computing operation on the current operator on the co-core, output the second execution result of the current operator, wherein all computing operations include at least the first computing operation and the second computing operation;

[0013] Based on the interrupt response mechanism, the calculation execution result of the current operator is generated on the main core according to the first execution result and the second execution result of the current operator.

[0014] The present application also provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, wherein when the computer program is executed by a processor, at least a model reasoning acceleration method including the following steps is implemented:

[0015] In response to receiving a model inference request, loading a preset model on the main core according to the model inference request, and determining whether the current operator is an operator to be offloaded;

[0016] If not, all computations are performed on the current operator on the main core based on the input tensor of the current operator, and the computation results of the current operator are output;

[0017] If so, then based on the input tensor of the current operator, perform a first computing operation on the current operator on the main core, output the first execution result of the current operator, and simultaneously perform a second computing operation on the current operator on the co-core, output the second execution result of the current operator, wherein all computing operations include at least the first computing operation and the second computing operation;

[0018] Based on the interrupt response mechanism, the calculation execution result of the current operator is generated on the main core according to the first execution result and the second execution result of the current operator.

[0019] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements at least a model reasoning acceleration method comprising the following steps:

[0020] In response to receiving a model inference request, loading a preset model on the main core according to the model inference request, and determining whether the current operator is an operator to be offloaded;

[0021] If not, all computations are performed on the current operator on the main core based on the input tensor of the current operator, and the computation results of the current operator are output;

[0022] If so, then based on the input tensor of the current operator, perform a first computing operation on the current operator on the main core, output the first execution result of the current operator, and simultaneously perform a second computing operation on the current operator on the co-core, output the second execution result of the current operator, wherein all computing operations include at least the first computing operation and the second computing operation;

[0023] Based on the interrupt response mechanism, the calculation execution result of the current operator is generated on the main core according to the first execution result and the second execution result of the current operator.

[0024] Through this application, depending on whether the current operator is an operator to be unloaded, if so, all computing operations are performed on the computing task of the current operator on the main core, and the computing execution result of the current operator is output; if not, the first computing operation is performed on the computing task of the current operator on the main core, and the first execution result of the current operator is output. At the same time, the second computing operation is performed on the computing task of the current operator on the cooperative core, and the second execution result of the current operator is output. This solves the problem of slow inference speed due to the disabling of an external hardware acceleration card and the limited performance of the hardware acceleration card itself, and achieves the technical effect of improving the model inference speed; based on the interrupt response mechanism, the computing execution result of the current operator is generated on the main core according to the first execution result and the second execution result, and achieves the technical effect of improving the real-time performance of the model inference. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0026] Figure 1 1. A schematic diagram of a flow chart of a model reasoning acceleration method according to an embodiment;

[0027] Figure 2 This is a flowchart of initializing a model reasoning acceleration system in one embodiment;

[0028] Figure 3 1. A schematic diagram of a flow chart of a method for accelerating model reasoning in a main core according to an embodiment;

[0029] Figure 41. A schematic diagram of a flow chart of a method for accelerating model reasoning in a co-core according to an embodiment;

[0030] Figure 5 A flowchart of aggregating a first execution result and a second execution result in a main core in one embodiment;

[0031] Figure 6 A schematic diagram of the architecture of a model reasoning acceleration system in one embodiment;

[0032] Figure 7 FIG. 1 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0033] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0034] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0035] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0036] Hardware accelerator cards have the following critical flaws in baseboard management controller scenarios:

[0037] (1) Cost and hardware compatibility issues:

[0038] 1. Excessive hardware costs: The price of a low-power neural network processing unit accelerator card far exceeds the overall hardware cost of the baseboard management controller. The accelerator card cost accounts for more than 300% of the total hardware cost, which is completely inconsistent with the cost-sensitive positioning of the baseboard management controller.

[0039] 2. Interface compatibility limitations: Baseboard management controller chips typically only provide low-speed interfaces, such as low-pin-count bus interfaces and enhanced serial peripheral interfaces, and cannot support high-speed buses such as PCIe 3.0 / 4.0. This limits the bandwidth of the accelerator card. Specifically, the measured PCIe 2.0 x1 bandwidth is only 500MB / s, which cannot meet the data throughput requirements of the neural network processing unit.

[0040] 3. Physical space conflict: Baseboard management controller boards are typically compact at 5cm x 5cm and cannot accommodate standard-sized accelerator cards such as the M.22280 specification. Forced integration requires redesigning the printed circuit board layout, increasing development time and risk.

[0041] (2) Power consumption and heat dissipation limitations:

[0042] 1. Excessive power consumption: A typical neural network processing unit accelerator card consumes 2-4W of power, while the baseboard management controller (BMC) typically needs to consume less than 10W. The accelerator card consumes 30%-40% of the power budget, resulting in insufficient power for other key functions such as fan control and sensor acquisition.

[0043] 2. Complex heat dissipation design: Baseboard management controllers usually rely on passive cooling or heat sink-free design. When the accelerator card is running, the board temperature will increase significantly. The measured operating temperature of the neural network processing unit can reach 70°C or above, which may trigger the over-temperature protection mechanism of the baseboard management controller and cause system downtime.

[0044] (3) Software ecology and maintenance costs:

[0045] 1. Toolchain fragmentation: Accelerator cards from different manufacturers require independent software stack adaptations. For example, neural network processing units require dedicated compilers, and field-programmable gate arrays require custom drivers. This exponentially increases the complexity of baseboard management controller firmware development. For example, supporting two different accelerator cards requires maintaining two code bases, increasing testing and maintenance costs.

[0046] 2. Poor model compatibility: Accelerator cards typically only support specific operator types. For example, neural network processing units excel at convolution but do not support dynamic control flow. This results in inefficient operation of long-tail models such as custom long short-term memory neural networks, requiring extensive manual optimization or even model reconstruction.

[0047] 3. Firmware upgrade risk: The driver and firmware of the hardware accelerator card must be upgraded synchronously with the baseboard management controller firmware. If a version mismatch occurs, such as an outdated version of the neural network processing unit driver, it may cause the inference task to crash, affecting the reliability of the server management function.

[0048] In one embodiment, Figure 1As shown, a model reasoning acceleration method is provided, which is applied to a baseboard management controller. The baseboard management controller includes a main core and a co-core. The model reasoning acceleration method includes:

[0049] In response to receiving a model inference request, loading a preset model on the main core according to the model inference request, and determining whether the current operator is an operator to be offloaded;

[0050] If not, all computations are performed on the current operator on the main core based on the input tensor of the current operator, and the computation results of the current operator are output;

[0051] If so, then based on the input tensor of the current operator, perform a first computing operation on the current operator on the main core, output the first execution result of the current operator, and simultaneously perform a second computing operation on the current operator on the co-core, output the second execution result of the current operator, wherein all computing operations include at least the first computing operation and the second computing operation;

[0052] Based on the interrupt response mechanism, the calculation execution result of the current operator is generated on the main core according to the first execution result and the second execution result of the current operator.

[0053] In specific implementation, most current baseboard management controllers have built-in coprocessors, namely the co-cores in this embodiment, but these co-cores are usually only used for specific functions, such as running real-time operating systems and processing low-speed tasks, and therefore do not participate in model reasoning, resulting in the main core and the co-core being independent of each other and unable to share computing power. For example, when the main core is running the model at full load, the co-core is idle, resulting in low overall energy efficiency. Therefore, this embodiment, based on whether the current operator is an operator to be unloaded, transfers the computing tasks of the operators not to be unloaded to the main core of the baseboard management controller for calculation; unloads the computing tasks of the operators to be unloaded from the main core of the baseboard management controller to the co-core of the baseboard management controller, and transfers them to the main core and the co-core for parallel calculation; and determines whether to integrate the first execution result and the second execution result on the main core of the baseboard management controller according to the interrupt response mechanism. That is, the main core of the baseboard management controller in this embodiment is responsible for processing control flow and complex operators, and the coprocessor of the baseboard management controller focuses on rule calculation, making full use of the computing resources of the main core and coprocessor in the baseboard management controller to achieve the optimal balance between computing power and energy efficiency, thereby accelerating the model reasoning speed of the baseboard management controller.

[0054] Specifically, depending on whether the current operator is an operator to be unloaded, if so, all computing operations are performed on the computing task of the current operator on the main core, and the computing execution result of the current operator is output; if not, the first computing operation is performed on the computing task of the current operator on the main core, and the first execution result of the current operator is output. At the same time, the second computing operation is performed on the computing task of the current operator on the cooperative core, and the second execution result of the current operator is output. This solves the problem of slow inference speed due to the disabling of the external hardware acceleration card and the limited performance of the hardware acceleration card itself, and achieves the technical effect of improving the model inference speed; based on the interrupt response mechanism, the computing execution result of the current operator is generated on the main core according to the first execution result and the second execution result, and achieves the technical effect of making full use of the main core and cooperative core resources and improving the real-time performance of the model inference.

[0055] like Figure 2 As shown, before receiving the model inference request, the model inference acceleration method further includes:

[0056] (1) System startup: After power is applied, the main core and the auxiliary core will start the firmware system pre-burned into the storage device. After the main core system is started, it will initialize the system resources and start the model inference process. After the model inference process starts, a thread pool will be created to wait for the arrival of model inference tasks;

[0057] (2) Hardware resource mapping: 1. Shared memory area division. When the main core starts, the memory management unit maps the fixed-size space of the configured physical address to shared memory. The cache strategy is configured for the main core and the auxiliary core to ensure data consistency. 2. Interrupt controller configuration. Register the interrupt number of the auxiliary core. For example, bind the auxiliary core's interrupt request to the interrupt line numbered 32 to let the main core know which interrupt comes from the auxiliary core. The auxiliary core's interrupt priority is set higher than IPMI (Intelligent Platform Management Interface, a standard interface for managing hardware devices in servers and other computer systems) communication but lower than the temperature alarm. Enable edge trigger mode.

[0058] (3) Thread pool construction: Start the model inference process, which creates all working threads according to the configuration, including the main core thread and the co-core thread.

[0059] In addition, before determining whether the current operator is an operator to be offloaded, the model inference acceleration method also includes:

[0060] Identify the computational graph of the preset model on the main core and obtain multiple offloadable operators;

[0061] If the currently unloadable operator is a memory-intensive operator, the current unloadable operator is determined on the master core to be a non-unloadable operator.

[0062] If the currently offloadable operator is a computationally intensive operator, the currently offloadable operator is determined as an operator to be offloaded on the master core.

[0063] In practice, this embodiment uses the main core to traverse the computation graph of a preset model and identify offloadable operators. Based on the operator type of the current offloadable operator, it determines whether the current offloadable operator is an operator to be offloaded and marks the input / output tensor addresses of the operator to be offloaded. Compute-intensive operators, such as matrix-matrix and matrix-vector multiplication operators, can be split into independent subtasks, allowing the main core to handle control flow and complex operators while the co-core focuses on rule-based computations, achieving an optimal balance between computing power and energy efficiency.

[0064] In addition, based on the input tensor of the current operator, all computation operations are performed on the current operator on the main core, and the computation execution results of the current operator are output, including:

[0065] Generates the computation task of the current operator on the main core based on the current operator and its input tensor, and writes the computation task of the current operator to the main core thread of the thread pool.

[0066] The calculation task of the current operator in the main core thread is processed on the main core, the calculation execution result of the current operator is output, and the calculation execution result of the current operator is written into the double-buffered task queue in the shared memory.

[0067] Furthermore, if Figure 3 As shown, before performing a first computing operation on the current operator on the main core according to the input tensor of the current operator and outputting a first execution result of the current operator, the model inference acceleration method further includes:

[0068] On the main core, the computation task of the current operator is divided into corresponding multiple computation subtasks, and the multiple computation subtasks of the current operator are written into the double-buffered task queue, wherein the computation amount of the multiple computation subtasks is the same;

[0069] Based on the task allocation mechanism, multiple computing subtasks of the current operator are divided into the main core computing subtasks and co-core computing subtasks of the current operator on the main core;

[0070] If the current computing subtask of the current operator is a main core computing subtask, the current computing subtask of the current operator is written to the main core thread on the main core;

[0071] If the current computation subtask of the current operator is a co-core computation subtask, the current computation subtask of the current operator is written to the co-core thread of the thread pool on the main core;

[0072] Based on the input tensor of the current operator, the task descriptor of the current operator is generated on the main core, and the task descriptor of the current operator is written to the task descriptor queue in the shared memory.

[0073] In practice, this embodiment first updates the head pointer of the task descriptor queue and then writes the task descriptor of the current operator into the task descriptor queue's ring buffer. If the double-buffered task queue is full, the main core further reduces the task granularity to quickly free up queue space. For example, if each computation subtask of the current operator consists of 1024 elements, each computation subtask is further split into new computation subtasks consisting of 256 elements to enable faster queue entry and reduce queue backlogs.

[0074] Specifically, the current operator's computational task is split into multiple corresponding subtasks, which are then written to a double-buffered task queue. Each subtask in the double-buffered task queue is processed by either a main core thread or a co-core thread, fully utilizing the co-core computing resources in the baseboard management controller and improving model inference speed.

[0075] Furthermore, performing a first computing operation on the current operator on the main core and outputting a first execution result of the current operator includes:

[0076] Performing a first computing operation on a main core computing subtask of a current operator in a main core thread on a main core, outputting a first execution result of the current operator, and writing the first execution result of the current operator into a double-buffered task queue;

[0077] Perform a second computation operation on the current operator on the co-core, and output a second execution result of the current operator, including:

[0078] Get the task descriptor of the current operator in the task descriptor queue on the co-core;

[0079] According to the task descriptor of the current operator, a second computing operation is performed on the co-core computing subtask of the current operator in the co-core thread on the co-core, a second execution result of the current operator is output, and the second execution result of the current operator is written into the double-buffered task queue.

[0080] In practice, the dual-buffered task queue achieves parallelization of data production and consumption by alternating between two memory areas. The core process is that while the main core writes to the current buffer, the auxiliary core processes the backup buffer. When the auxiliary core completes processing the backup buffer, it switches to processing the current buffer while the main core fills the backup buffer. The main core thread fills the current buffer with tasks and then switches to the backup buffer to continue writing the next batch of tasks. The auxiliary core polls the buffer switch flag and reads the corresponding buffer data based on the buffer switch flag to avoid read-write conflicts. The auxiliary core writes the execution results to the backup buffer, updates the buffer switch flag, and simultaneously updates the auxiliary core load rate in the status register to the auxiliary core's current load rate.

[0081] Furthermore, based on the task allocation mechanism, multiple computing subtasks of the current operator are divided into co-core computing subtasks of the current operator on the main core, including:

[0082] On the main core, the load rate of the co-core thread is obtained from the status register of the shared memory, and based on the load rate of the co-core thread, the co-core calculation ratio of the current operator is obtained;

[0083] Based on the computing task of the current operator, the total computing amount of the current operator is obtained on the main core. Based on the total computing amount of the current operator and the computing ratio of the co-cores, the computing amount of the co-cores of the current operator is obtained.

[0084] Based on the multiple computing subtasks of the current operator, the average computing amount of the current operator is obtained on the main core. The computing amount of the co-core of the current operator and the average computing amount are rounded down to obtain the number of co-core subtasks of the current operator.

[0085] According to the number of co-core subtasks of the current operator, the corresponding number of computing subtasks of the current operator are divided into the co-core computing subtasks of the current operator on the main core.

[0086] like Figure 3 As shown, this embodiment first initializes the dynamic scheduler and load monitoring module, and reads the load rate of the co-core in the shared memory status register every 10ms to dynamically adjust the task slicing strategy. If the load rate of the co-core is less than 30%, 70% of the total computation of the current operator is offloaded to the co-core; if the load rate of the co-core is greater than 80%, 20% of the total computation of the current operator is offloaded to the co-core to reduce latency. By slicing the computational tasks of the current operator, the computing resources of the main core and the co-core are fully utilized, thereby improving the model inference speed.

[0087] Furthermore, based on the task allocation mechanism, multiple computing subtasks of the current operator are divided into the main core computing subtasks of the current operator on the main core, including:

[0088] Based on the multiple computing subtasks of the current operator, obtain the total number of subtasks of the current operator on the master core;

[0089] On the main core, the number of main core subtasks of the current operator is obtained based on the total number of subtasks of the current operator and the number of subtasks of the co-core;

[0090] Based on the number of main core subtasks of the current operator, the corresponding number of computation subtasks of the current operator are divided into the main core computation subtasks of the current operator on the main core. Furthermore, based on the input tensor of the current operator, a task descriptor of the current operator is generated on the main core, including:

[0091] Obtain the input tensor address, output tensor address, and operator type of the current operator on the main core, and generate a cyclic check code for the current operator based on the input tensor address, output tensor address, and operator type of the current operator;

[0092] Generate the task descriptor of the current operator on the main core based on the input tensor address, output tensor address, operator type, and cyclic check code of the current operator.

[0093] Specifically, the main core generates the task descriptor of the current operator based on the input tensor address, output tensor address, operator type and cyclic check code of the current operator, so that the co-core can process the co-core computing subtask of the current operator in the double-buffered task queue according to the task descriptor, thereby improving the efficiency of model inference.

[0094] Furthermore, the task descriptor of the current operator in the task descriptor queue is obtained on the co-core, including:

[0095] Based on the cyclic check code of the current operator, the task descriptor of the current operator is verified on the co-core;

[0096] If the verification is successful, the task descriptor of the current operator in the task descriptor queue is obtained on the co-core;

[0097] If the check fails, an error code and an interrupt request are generated on the co-core;

[0098] On the co-core, the error code is written to the error field in the status register and an interrupt request is sent to the main core.

[0099] like Figure 4 As shown, the co-core reads the task descriptor from the tail of the circular buffer in the task descriptor queue. After successfully verifying the circular checksum of the task descriptor, it parses and obtains the operator type, ensuring the security of the current operator's task descriptor. If the operator type is a matrix operation, it uses single instruction multiple data to call the co-core optimized assembly kernel to calculate the co-core calculation subtask. If the verification fails, the error code of the verification failure is written to the error_code error field, and the main core will report an error.

[0100] In addition, if Figure 5 As shown, based on the interrupt response mechanism, according to the first execution result and the second execution result of the current operator, the calculation execution result of the current operator is generated on the main core, including:

[0101] In response to the main core receiving the interrupt request, the main core compares the interrupt number of the interrupt request with the interrupt number of the cooperative core;

[0102] If the comparison is successful, the error field in the status register is identified on the main core;

[0103] If the recognition is successful, the main core generates an error prompt for the current operator based on the error code in the error field;

[0104] If the recognition fails, the second execution result of the current operator in the double-buffered task queue is obtained on the main core, and the second execution result of the current operator is moved to the main core thread;

[0105] Generate a calculation execution result of the current operator on the main core according to the first execution result and the second execution result of the current operator.

[0106] In specific implementation, in this embodiment, when the result of the cooperative core thread is not ready, that is, before the main core receives the interrupt request from the cooperative core, the main core thread enters the sleep state to reduce the idling overhead of the main core; after the result is ready, the main core thread will be awakened to aggregate the results; the main core thread checks whether the calculation subtasks of the current operator in the main core thread and the cooperative core thread are all completed; if completed, the calculation execution result of the current operator is passed to the next operator; if not completed, the dynamic scheduler is triggered to reallocate tasks.

[0107] Specifically, based on the interrupt response mechanism, the main core can obtain the status of the cooperative core in a timely manner, such as verification errors, calculation timeouts, etc. If the cooperative core generates an error interrupt, the main core can take over the cooperative core to perform calculations and other operations to ensure the continuity of the model inference business.

[0108] Furthermore, the interrupt number of the interrupt request is compared with the interrupt number of the cooperative core on the main core, including:

[0109] If the interrupt number of the interrupt request is the same as the interrupt number of the cooperative core, the comparison is determined to be successful on the main core;

[0110] If the interrupt number of the interrupt request is different from the interrupt number of the cooperative core, the comparison is determined to have failed on the main core. In addition, the error field in the status register is identified on the main core, including:

[0111] Determining, on the primary core, whether the error field includes an error code;

[0112] If the error field includes an error code, the recognition is determined to be successful on the primary core;

[0113] If the error field does not include an error code, the master core determines that the recognition has failed.

[0114] In a specific implementation, if the error field does not include an error code, the second execution result of the current operator is moved to the main core thread on the main core, and the buffer lock is released; the first execution result and the second execution result of the current operator in the double-buffered task queue are integrated into the calculation execution result of the current operator on the main core.

[0115] In addition, performing a second computing operation on the current operator on the co-core and outputting a second execution result of the current operator further includes:

[0116] If the current cooperative core subtask is calculated, the number of currently completed cooperative core subtasks is obtained on the cooperative core, and the number of currently completed cooperative core subtasks is compared with the subtask number threshold;

[0117] If the number of currently completed co-core subtasks is less than the subtask number threshold, the calculation operation is performed on the next co-core subtask on the co-core;

[0118] Record the computation time of the next co-core subtask on the co-core, and compare the computation time of the next co-core subtask with the subtask computation time threshold;

[0119] If the calculation duration of the next co-core subtask is less than the subtask calculation duration threshold, the co-core continues to perform calculation operations on the next co-core subtask until the first case or the second case is met, where the first case indicates that the calculation of the next co-core subtask is completed, and the calculation duration of the next co-core subtask is less than or equal to the subtask calculation duration threshold; the second case indicates that the calculation of the next co-core subtask is not completed, and the calculation duration of the next co-core subtask is greater than the subtask calculation duration threshold;

[0120] If the calculation of the next co-core subtask is completed, and the calculation time of the next co-core subtask is less than or equal to the subtask calculation time threshold, then the number of currently completed co-core subtasks is compared with the subtask number threshold;

[0121] If the number of currently completed cooperative core subtasks is equal to the subtask number threshold, an interrupt request is generated on the cooperative core, and the execution results of the currently completed cooperative core subtasks are written into the double-buffered task queue;

[0122] If the next cooperative core subtask is not calculated and the calculation time of the next cooperative core subtask is greater than the subtask calculation time threshold, an error code and an interrupt request are generated on the cooperative core, and the error code is written into the error field, and the interrupt request is sent to the main core.

[0123] Furthermore, the master core generates an error prompt for the current operator based on the error code in the error field, including:

[0124] If the error code is the first code, the error type is determined to be a verification error on the master core, and an error prompt indicating a verification error of the current operator is generated;

[0125] If the error code is the second code, the error type is determined to be calculation timeout on the master core, and an error prompt indicating that the current operator calculation has timed out is generated.

[0126] In a specific implementation, if the subtask number threshold is 5 and the subtask calculation duration threshold is 100ms, the calculation of the current third co-core subtask is completed, and the calculation operation continues to be performed on the fourth co-core subtask. The current calculation duration of the fourth co-core subtask is 60ms. If the fourth co-core subtask is completed at 900ms, the calculation operation continues to be performed on the fifth co-core subtask. If the fourth co-core subtask is still not completed at 100ms, a calculation timeout error code is generated and an interrupt request is sent to the main core. By setting the subtask number threshold and the subtask calculation duration threshold, the main core can obtain the calculation results of the co-core in a timely manner and handle the failure of the co-core in a timely manner.

[0127] In specific implementation, examples of model inference acceleration methods are as follows:

[0128] 1. The user triggers an inference request: a query command is sent to the main core through the interface, such as "How to create a logical disk?"

[0129] 2. Model loading and computational graph parsing: The main core loads a pre-trained AI model, inputs user questions, and parses the computational graph to identify offloadable operators, such as matrix multiplication and vector operations.

[0130] 3. Task slicing and dynamic offloading: The main core thread pool executes computationally intensive operators, such as activation function operators and layer normalization operators. The co-core thread detects parallel computing requirements, such as weight matrix multiplication, and splits the matrix into dynamic slicing. The slicing size is 512, which can be adjusted in real time based on the co-core load. The task descriptor, including the data address and operator type, is written to the shared memory lock-free queue, triggering the co-core interrupt with interrupt request number 32, notifying the co-core to obtain the task.

[0131] 4. Co-core calculation: After receiving an interrupt, the co-core enters the interrupt service process and reads the task descriptor from shared memory. The co-core uses single instruction stream multiple data stream instructions such as the dot product function to perform matrix multiplication and addition, and writes the result back to the output buffer in the double-buffered task queue module.

[0132] 5. Interrupt merging strategy: After processing two slices, a low-priority interrupt is triggered to notify the main core that the result is ready;

[0133] 6. Main core synchronization and result aggregation: interrupt processing, the main core responds to the interrupt, reads the sharding results from the shared memory, verifies the cyclic redundancy check; data aggregation, integrates all sharding results, and calculates the final result;

[0134] 7. Graph advancement and looping: Load-aware scheduling: If the current load rate of the co-core is 78%, the shard size of the next node is dynamically adjusted to 256. Double buffer switching: the main core fills the next batch of tasks into the backup buffer while the co-core processes the tasks in the current buffer.

[0135] 8. End and continue: The graph node where the current operator is located ends, and the calculation continues with the next graph node, and so on.

[0136] It should be understood that although Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0137] In one embodiment, Figure 6 As shown, a model reasoning acceleration system is provided. The model reasoning acceleration system is located in the baseboard management controller and includes a main core, a co-core and a shared memory.

[0138] The main core includes the model inference process, which is used to load the preset model, parse the calculation graph of the preset model, and initialize the thread pool;

[0139] The co-core includes a lightweight real-time operating system, which is used to run the task processing engine and supports priority preemption;

[0140] The shared memory includes a task descriptor queue, a status register, and a double-buffered task queue. The task descriptor queue is used to store metadata corresponding to model inference requests; the status register is used to provide real-time feedback on the load rate, temperature, and error code of the co-core; and the double-buffered task queue is used for alternating reading and writing between the main core and the co-core.

[0141] In a specific implementation, the metadata corresponding to the model inference request in this embodiment includes at least the operator type and data address. Specifically, the shared memory module is used to overlap computing and communication, eliminating data copy overhead and reducing communication latency.

[0142] Furthermore, the model inference process includes a thread pool and a dynamic scheduler, and the thread pool includes at least a first thread, a second thread, a third thread, and a fourth thread.

[0143] The first thread and the second thread are main core threads, used to execute computationally intensive operators;

[0144] The third and fourth threads are co-core threads, used for task offloading, co-core communication, and data synchronization between the main core and the co-core;

[0145] A dynamic scheduler is used to adjust the size and offloading strategy of computing subtasks in real time according to the load of the co-core.

[0146] In a specific implementation, computationally intensive operators include activation function operators and layer normalization operators.

[0147] Furthermore, the lightweight real-time operating system includes a task processing engine and an interrupt controller.

[0148] The task processing engine is used to parse the task descriptor task queue and perform matrix operations on the co-core subtasks in the double-buffered task queue;

[0149] Interrupt controller, used to manage task completion notification and exception reporting.

[0150] In a specific implementation, matrix operations include matrix multiplication by matrix operations and matrix multiplication by vector operations, etc.

[0151] Furthermore, the double-buffered task queue includes input and output buffers and hardware semaphores. The input and output buffers are used to provide independent production areas and consumption areas to achieve physical separation of computing and communication; the hardware semaphores are used to indicate the data readiness status to achieve buffer role switching.

[0152] In specific implementations, input and output buffers are used for ① data isolation, providing independent production and consumption areas to achieve physical separation of computing and communication, typically implemented as two buffers: the current buffer and the standby buffer; ② parallel processing support, allowing producers to fill one buffer while consumers process another buffer, achieving true computing-communication overlap; ③ pipeline optimization, forming a continuous pipeline of production → consumption → production → consumption, hiding data transmission delays; ④ data consistency guarantee, each buffer is only accessed by one role (producer or consumer) at a specific time, avoiding read-write conflicts.

[0153] Hardware semaphores are used to ① synchronize and control the access timing of producer (computing) and consumer (communication) threads / processes to ensure data integrity and prevent simultaneous reading and writing of the same buffer; ② status notification: the producer notifies the consumer "data is ready" through the semaphore, and the consumer notifies the producer "buffer is free" through the semaphore; ③ atomic operation guarantee: providing hardware-level atomic operations to avoid the overhead of software locks and ensure the atomicity of buffer switching operations.

[0154] Therefore, the normal working process of the double-buffered task queue is: the producer obtains an empty buffer through a semaphore; the producer fills data into the input buffer; the producer notifies the consumer through a semaphore; the consumer obtains a full buffer through a semaphore; the consumer processes the output buffer data; the consumer notifies the producer through a semaphore. In addition, the semaphore timeout mechanism can detect deadlocks, and the buffer status flag can detect data corruption. For the specific limitations of the model reasoning acceleration system, please refer to the limitations of the model reasoning acceleration method above, which will not be repeated here. The various modules in the above-mentioned model reasoning acceleration system can be implemented in whole or in part through software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the electronic device in the form of hardware, or can be stored in the memory of the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0155] In one embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0156] In response to receiving a model inference request, loading a preset model on the main core according to the model inference request, and determining whether the current operator is an operator to be offloaded;

[0157] If not, all computations are performed on the current operator on the main core based on the input tensor of the current operator, and the computation results of the current operator are output;

[0158] If so, then based on the input tensor of the current operator, perform a first computing operation on the current operator on the main core, output the first execution result of the current operator, and simultaneously perform a second computing operation on the current operator on the co-core, output the second execution result of the current operator, wherein all computing operations include at least the first computing operation and the second computing operation;

[0159] Based on the interrupt response mechanism, the calculation execution result of the current operator is generated on the main core according to the first execution result and the second execution result of the current operator. When the program instructions are read and executed by one or more processors, they can also perform operations corresponding to the various steps in the above method embodiment. Please refer to the description above and will not be repeated here. Figure 7, which exemplarily shows the architecture of an electronic device, which may specifically include a processor 710, a video display adapter 711, a disk drive 712, an input / output interface 713, a network interface 714, and a memory 720. The processor 710, video display adapter 711, disk drive 712, input / output interface 713, network interface 714, and memory 720 may be communicatively connected via a communication bus 730.

[0160] Among them, the processor 710 can be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in this application.

[0161] The memory 720 can be implemented in the form of a read-only memory (ROM), a random access memory (RAM), a static storage device, a dynamic storage device, etc. The memory 720 can store an operating system 721 for controlling the operation of the electronic device 700, and a basic input and output system (BIOS) 722 for controlling the low-level operations of the electronic device 700. In addition, a web browser 723, a data storage management 724, and an icon font processing system 725, etc. can also be stored. The above-mentioned icon font processing system 725 can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided by the present application is implemented by software or firmware, the relevant program code is stored in the memory 720 and is called and executed by the processor 710.

[0162] The input / output interface 713 is used to connect to input / output modules to enable information input and output. The input / output modules can be configured as components within the device (not shown) or externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, and various sensors, while output devices may include a display, speaker, vibrator, indicator light, and the like.

[0163] The network interface 714 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, Wi-Fi, Bluetooth, etc.).

[0164] The bus 730 comprises a pathway for transmitting information between the various components of the device (eg, the processor 710 , the video display adapter 711 , the disk drive 712 , the input / output interface 713 , the network interface 714 , and the memory 720 ).

[0165] In addition, the electronic device 700 can also obtain information on specific collection conditions from a virtual resource object collection condition information database (not shown in the figure) for use in condition judgment, etc.

[0166] It should be noted that although the electronic device 700 shown above only includes a processor 710, a video display adapter 711, a disk drive 712, an input / output interface 713, a network interface 714, a memory 720, a bus 730, etc., in a specific implementation, the electronic device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the solution of the present application, and does not necessarily include all the components shown in the figure.

[0167] Through the description of the above implementation methods, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling an electronic device (which can be a personal computer, a cloud server, or a network device, etc.) to execute the methods of each embodiment of the present application or certain parts of the embodiments.

[0168] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0169] In response to receiving a model inference request, loading a preset model on the main core according to the model inference request, and determining whether the current operator is an operator to be offloaded;

[0170] If not, all computations are performed on the current operator on the main core based on the input tensor of the current operator, and the computation results of the current operator are output;

[0171] If so, then based on the input tensor of the current operator, perform a first computing operation on the current operator on the main core, output the first execution result of the current operator, and simultaneously perform a second computing operation on the current operator on the co-core, output the second execution result of the current operator, wherein all computing operations include at least the first computing operation and the second computing operation;

[0172] Based on the interrupt response mechanism, the calculation execution result of the current operator is generated on the main core according to the first execution result and the second execution result of the current operator. Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0173] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0174] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make numerous variations and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application.

[0175] In one embodiment, a computer program product is provided, wherein a computer program is stored on the product, and when the computer program is executed by a processor, the following steps are performed:

[0176] In response to receiving a model inference request, loading a preset model on the main core according to the model inference request, and determining whether the current operator is an operator to be offloaded;

[0177] If not, all computations are performed on the current operator on the main core based on the input tensor of the current operator, and the computation results of the current operator are output;

[0178] If so, then based on the input tensor of the current operator, perform a first computing operation on the current operator on the main core, output the first execution result of the current operator, and simultaneously perform a second computing operation on the current operator on the co-core, output the second execution result of the current operator, wherein all computing operations include at least the first computing operation and the second computing operation;

[0179] Based on the interrupt response mechanism, a calculation execution result of the current operator is generated on the main core according to the first execution result and the second execution result of the current operator. In one embodiment, a computer program product is provided, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0180] In response to receiving a model inference request, loading a preset model on the main core according to the model inference request, and determining whether the current operator is an operator to be offloaded;

[0181] If not, all computations are performed on the current operator on the main core based on the input tensor of the current operator, and the computation results of the current operator are output;

[0182] If so, then based on the input tensor of the current operator, perform a first computing operation on the current operator on the main core, output the first execution result of the current operator, and simultaneously perform a second computing operation on the current operator on the co-core, output the second execution result of the current operator, wherein all computing operations include at least the first computing operation and the second computing operation;

[0183] Based on the interrupt response mechanism, the calculation execution result of the current operator is generated on the main core according to the first execution result and the second execution result of the current operator. Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a computer program product. When executed, the computer program can include the processes of the embodiments of the above-mentioned methods.

[0184] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0185] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make numerous variations and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application.

Claims

1. A model reasoning acceleration method is applied to a baseboard management controller, wherein the baseboard management controller includes a main core and a co-core, characterized in that: The method comprises: In response to receiving a model inference request, loading a preset model on the main core according to the model inference request, and determining whether a current operator is an operator to be offloaded; If not, perform all computation operations on the current operator on the main core according to the input tensor of the current operator, and output the computation execution result of the current operator; If so, performing a first computing operation on the current operator on the main core according to the input tensor of the current operator, outputting a first execution result of the current operator, and performing a second computing operation on the current operator on the co-core, outputting a second execution result of the current operator, wherein all computing operations include at least the first computing operation and the second computing operation; Based on an interrupt response mechanism, generating a calculation execution result of the current operator on the main core according to a first execution result and a second execution result of the current operator; Wherein, before performing a first computing operation on the current operator on the main core according to the input tensor of the current operator and outputting a first execution result of the current operator, the method further includes: Splitting the computing task of the current operator into a plurality of corresponding computing subtasks on the main core, and writing the plurality of computing subtasks of the current operator into a double-buffered task queue in a shared memory, wherein the computing amount of the plurality of computing subtasks is the same; Based on the task allocation mechanism, the multiple computing subtasks of the current operator are divided into a main core computing subtask and a co-core computing subtask of the current operator on the main core; If the current computing subtask of the current operator is a main core computing subtask, writing the current computing subtask of the current operator into the main core thread on the main core; If the current computing subtask of the current operator is a co-core computing subtask, writing the current computing subtask of the current operator into the co-core thread of the thread pool on the main core; Generate a task descriptor of the current operator on the master core according to the input tensor of the current operator, and write the task descriptor of the current operator into a task descriptor queue in the shared memory.

2. The method according to claim 1, characterized in that Before determining whether the current operator is an operator to be uninstalled, the method further includes: Identifying a computational graph of the preset model on the main core to obtain a plurality of offloadable operators; If the currently unloadable operator is a memory-intensive operator, determining on the primary core that the currently unloadable operator is an operator not to be unloaded; If the currently unloadable operator is a computationally intensive operator, the currently unloadable operator is determined on the primary core as the operator to be unloaded.

3. The method according to claim 1, characterized in that The performing all computation operations on the current operator on the main core according to the input tensor of the current operator and outputting the computation execution result of the current operator includes: Generate a computing task of the current operator on the main core according to the current operator and the input tensor of the current operator, and write the computing task of the current operator into the main core thread of the thread pool; The computing task of the current operator in the main core thread is processed on the main core, the computing execution result of the current operator is output, and the computing execution result of the current operator is written into the double-buffered task queue in the shared memory.

4. The method according to claim 1, characterized in that The performing a first computing operation on the current operator on the main core and outputting a first execution result of the current operator includes: Performing a first computing operation on a main core computing subtask of a current operator in the main core thread on the main core, outputting a first execution result of the current operator, and writing the first execution result of the current operator into the double-buffered task queue; The performing a second computing operation on the current operator on the co-core and outputting a second execution result of the current operator includes: Obtaining, on the co-core, a task descriptor of a current operator in the task descriptor queue; According to the task descriptor of the current operator, a second computing operation is performed on the co-core computing subtask of the current operator in the co-core thread on the co-core, a second execution result of the current operator is output, and the second execution result of the current operator is written into the double-buffered task queue.

5. The method according to claim 1, characterized in that: The step of dividing the multiple computing subtasks of the current operator into co-core computing subtasks of the current operator on the main core based on the task allocation mechanism includes: Obtaining, on the main core, a load rate of the co-core thread from a status register of the shared memory, and obtaining a co-core calculation ratio of the current operator according to the load rate of the co-core thread; According to the computing task of the current operator, the total computing amount of the current operator is obtained on the main core, and according to the total computing amount of the current operator and the computing ratio of the co-cores, the computing amount of the co-cores of the current operator is obtained; According to the multiple computing subtasks of the current operator, the average computing amount of the current operator is obtained on the main core, and the number of the co-core subtasks of the current operator is obtained by rounding down the computing amount of the co-core and the average computing amount of the current operator; According to the number of co-core subtasks of the current operator, a corresponding number of computing subtasks of the current operator are divided into co-core computing subtasks of the current operator on the main core.

6. The method according to claim 5, characterized in that The dividing, on the main core, of the multiple computing subtasks of the current operator into main core computing subtasks of the current operator based on the task allocation mechanism includes: Obtaining, on the master core, a total number of subtasks of the current operator according to the multiple computing subtasks of the current operator; Obtaining, on the main core, the number of main core subtasks of the current operator based on the total number of subtasks of the current operator and the number of subtasks of the co-core; According to the number of main core subtasks of the current operator, a corresponding number of computing subtasks of the current operator are divided on the main core into main core computing subtasks of the current operator.

7. The method according to claim 4, characterized in that: Generating a task descriptor of the current operator on the main core according to the input tensor of the current operator includes: Obtaining, on the main core, an input tensor address, an output tensor address, and an operator type of the current operator, and generating a cyclic check code of the current operator according to the input tensor address, the output tensor address, and the operator type of the current operator; A task descriptor of the current operator is generated on the main core according to the input tensor address, output tensor address, operator type, and cyclic check code of the current operator.

8. The method according to claim 7, characterized in that: The obtaining, on the co-core, the task descriptor of the current operator in the task descriptor queue includes: Verifying the task descriptor of the current operator on the co-core according to the cyclic check code of the current operator; If the verification is successful, the task descriptor of the current operator in the task descriptor queue is obtained on the cooperative core; If the check fails, an error code and an interrupt request are generated on the cooperative core; The error code is written into an error field in a status register on the auxiliary core, and the interrupt request is sent to the main core.

9. The method according to claim 8, characterized in that The generating, based on the interrupt response mechanism and according to the first execution result and the second execution result of the current operator, the calculation execution result of the current operator on the main core includes: In response to the main core receiving an interrupt request, the main core compares an interrupt number of the interrupt request with an interrupt number of the cooperative core; If the comparison is successful, identifying an error field in the status register on the master core; If the recognition is successful, an error prompt of the current operator is generated on the master core according to the error code in the error field; If the recognition fails, obtaining the second execution result of the current operator in the double-buffered task queue on the main core, and moving the second execution result of the current operator into the main core thread; Generate a calculation execution result of the current operator on the main core according to the first execution result and the second execution result of the current operator.

10. The method according to claim 9, characterized in that: The comparing, on the main core, the interrupt number of the interrupt request with the interrupt number of the cooperative core includes: If the interrupt number of the interrupt request is the same as the interrupt number of the cooperative core, determining that the comparison is successful on the main core; If the interrupt number of the interrupt request is different from the interrupt number of the cooperative core, it is determined on the main core that the comparison fails.

11. The method according to claim 9, characterized in that: The identifying, on the master core, an error field in the status register includes: determining, on the primary core, whether the error field includes the error code; If the error field includes the error code, determining on the primary core that the recognition is successful; If the error field does not include the error code, it is determined on the primary core that recognition has failed.

12. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the model reasoning acceleration method as claimed in any one of claims 1 to 11 when executing the computer program.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the model inference acceleration method according to any one of claims 1 to 11 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the model reasoning acceleration method according to any one of claims 1 to 11 are implemented.

Images