Hybrid engine intelligent computing method and device for artificial intelligence chip

Abstract

The application relates to an intelligent computing method and device for a hybrid engine of an artificial intelligence chip. The data processing method is applied to a processing chip, and the processing chip comprises a scalar processing unit and a computing unit. The method comprises the following steps: in the case that a processing instruction is received, a first target processing unit for processing the processing instruction is determined from the scalar processing unit and the computing unit; the processing instruction is used for instructing to run a target neural network model; the processing instruction is processed by using the first target processing unit, and processing information corresponding to the processing instruction is obtained; the processing information comprises control information and data information; the target neural network model is run by using a second target processing unit and the processing information, and a running result of the target neural network model is obtained; and the second target processing unit comprises the first target processing unit. The application can improve the problem that the artificial intelligence chip is prone to overheat in the process of running the neural network model.

Description

Technical Field

[0001] This application relates to the field of chip technology, and in particular to a hybrid engine intelligent computing method and apparatus for artificial intelligence chips. Background Technology

[0002] Currently, artificial intelligence chips are being used in the computational processing of neural network models. These chips enable the rapid execution of neural network models and the acquisition of their results.

[0003] However, the current architecture of artificial intelligence chips follows the von Neumann architecture. During the operation of neural network models, the data processing units in the artificial intelligence chip need to communicate with each other, which makes the artificial intelligence chip prone to overheating and other problems. Summary of the Invention

[0004] Therefore, it is necessary to provide a hybrid engine intelligent computing method and device for artificial intelligence chips that can improve the problems such as overheating that are prone to occur when artificial intelligence chips run neural network models, in order to address the above-mentioned technical issues.

[0005] In a first aspect, this application provides a data processing method. This data processing method is applied to a processing chip, which includes a scalar processing unit and a computation unit. The method includes: upon receiving a processing instruction, determining a first target processing unit from the scalar processing unit and the computation unit to process the processing instruction; the processing instruction instructing the execution of a target neural network model; processing the processing instruction using the first target processing unit to obtain processing information corresponding to the processing instruction; the processing information includes control information and data information; and executing the target neural network model using a second target processing unit and the processing information to obtain the execution result of the target neural network model; the second target processing unit includes the first target processing unit.

[0006] In one embodiment, upon receiving a processing instruction, determining a first target processing unit for processing the processing instruction from the scalar processing unit and the computing unit includes: upon receiving the processing instruction, obtaining the remaining memory amount of the scalar processing unit and the remaining memory amount of the computing unit; and determining the unit with the largest remaining memory amount among the remaining memory amount of the scalar processing unit and the remaining memory amount of the computing unit as the first target processing unit.

[0007] In one embodiment, the processing chip further includes a vector processing unit; before processing the processing instruction using the first target processing unit to obtain processing information corresponding to the processing instruction, the method further includes: calling the vector processing unit to perform vectorization processing on the processing instruction to obtain first vector data corresponding to the processing instruction; processing the processing instruction using the first target processing unit to obtain processing information corresponding to the processing instruction, including: processing the first vector data using the first target processing unit to obtain processing information.

[0008] In one embodiment, the second target processing unit further includes a vector processing unit; running the target neural network model using the second target processing unit and processing information to obtain the running result of the target neural network model includes: using the first target processing unit to decompose the target neural network model to obtain multiple sub-models corresponding to the target neural network model, and sending each sub-model to the vector processing unit; running each sub-model using the vector processing unit and processing information to obtain the running result.

[0009] In one embodiment, each sub-model is run using a vector processing unit and processing information to obtain running results, including: using the vector processing unit to vectorize the processing information to obtain second vector data corresponding to the processing information; and using the vector processing unit and the second vector data to run each sub-model to obtain running results.

[0010] In one embodiment, the processing chip further includes a judgment unit. When the first target processing unit is a computing unit, the method further includes: obtaining the judgment result of the judgment unit on the processing information; the judgment result is used to characterize whether the processing information is correct; if the judgment result characterizes the processing information as incorrect, a third target processing unit for processing the processing instruction is re-determined from the scalar processing unit and the computing unit; and the processing instruction is processed using the third target processing unit.

[0011] Secondly, this application also provides a data processing apparatus. The data processing apparatus is applied to a processing chip, which includes a scalar processing unit and a computing unit. The apparatus includes: a first determining module, configured to, upon receiving a processing instruction, determine a first target processing unit from the scalar processing unit and the computing unit to process the instruction; the processing instruction is used to instruct the execution of a target neural network model; a second determining module, configured to process the processing instruction using the first target processing unit to obtain processing information corresponding to the processing instruction; the processing information includes control information and data information; and a third determining module, configured to run the target neural network model using the second target processing unit and the processing information to obtain the execution result of the target neural network model; the second target processing unit includes the first target processing unit.

[0012] In one embodiment, the first determining module is further configured to, upon receiving a processing instruction, obtain the remaining memory of the scalar processing unit and the remaining memory of the computing unit; and determine the unit with the largest remaining memory among the remaining memory of the scalar processing unit and the remaining memory of the computing unit as the first target processing unit.

[0013] In one embodiment, the processing chip further includes a vector processing unit; the device further includes a fourth determining module, used to call the vector processing unit to perform vectorization processing on the processing instruction to obtain first vector data corresponding to the processing instruction; the second determining module is further used to process the first vector data using the first target processing unit to obtain processing information.

[0014] In one embodiment, the second target processing unit further includes a vector processing unit; the third determining module is further configured to use the first target processing unit to split the target neural network model to obtain multiple sub-models corresponding to the target neural network model, and send each sub-model to the vector processing unit; and run each sub-model using the vector processing unit and the processing information to obtain the running result.

[0015] In one embodiment, the third determining module is further configured to use the vector processing unit to perform vectorization processing on the processed information to obtain the second vector data corresponding to the processed information; and use the vector processing unit and the second vector data to run each sub-model to obtain the running results.

[0016] In one embodiment, the processing chip further includes a judgment unit, and the device further includes a fifth determination module for obtaining the judgment result of the judgment unit on the processing information; the judgment result is used to characterize whether the processing information is correct; if the judgment result characterizes the processing information as incorrect, a third target processing unit for processing the processing instruction is re-determined from the scalar processing unit and the calculation unit; and the processing instruction is processed using the third target processing unit.

[0017] Thirdly, this application also provides a processing chip, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any of the first aspects above.

[0018] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects above.

[0019] Fifthly, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the method described in any one of the first aspects above.

[0020] The above-mentioned data processing method, apparatus, processing chip, storage medium, and program product are described above. The data processing method is applied to the processing chip, which includes a scalar processing unit and a computing unit. Upon receiving a processing instruction to run a target neural network model, a first target processing unit for processing the instruction is determined from the scalar processing unit and the computing unit. The first target processing unit then processes the instruction to obtain processing information corresponding to the instruction, including control information and data information. Finally, a second target processing unit including the first target processing unit and the processing information are used to run the target neural network model to obtain the running result of the target neural network model. In this process, the first target processing unit is used to process processing instructions, and the second target processing unit is used to run the target neural network model. The second target processing unit includes the first target processing unit. Therefore, the data processing unit that processes the processing instructions and runs the target neural network model may be the same, that is, either a scalar processing unit or a computing unit. Therefore, in the processing chip of this application, that is, the artificial intelligence chip, during the running of the neural network model, there may be situations where the scalar processing unit or the computing unit independently completes the processing instructions and runs the target neural network model. This eliminates the need for data communication between the scalar processing unit and the computing unit, thereby reducing the data communication between the various data processing units and improving the problem of overheating that is prone to occur when the artificial intelligence chip runs the neural network model. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a data processing method in one embodiment;

[0022] Figure 2 This is a flowchart illustrating another data processing method in one embodiment;

[0023] Figure 3 This is one embodiment of a processing chip architecture;

[0024] Figure 4 This is a structural block diagram of a data processing device in one embodiment;

[0025] Figure 5 This is an internal structure diagram of a processing chip in one embodiment. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0027] Artificial intelligence (AI) chips typically refer to chips equipped with deep learning algorithms, which are already being used in the computational processing of neural network models. Currently, optimization of AI chips mainly focuses on innovation in computing architecture and hardware optimization of deep learning algorithms to achieve improvements in computing power, power consumption, and cost.

[0028] Specifically, to improve the performance of artificial intelligence chips, teams both domestically and internationally have adopted various design schemes, some of which are as follows:

[0029] (1) The deep learning processor Eyeriss emphasizes the rule of prioritizing energy efficiency and is a processor based on a dataflow architecture. It has designed an independent RS (Row stationary) computation rule for the computing units. The second-generation version, Eyeriss V2, has a sparsified and more flexible network structure.

[0030] (2) ENVISION employs multiple data reuse methods, including convolution reuse, image reuse, and convolution kernel reuse, to improve chip energy efficiency.

[0031] (3) DNPU mainly relies on heterogeneous architecture, hybrid load partitioning method, dynamic adaptive fixed-point computation, and multiplier based on quantization table to achieve chip configurability and low power consumption.

[0032] (4) UNPU is a fully variable weight bit precision DNN (Deep Neural Networks) accelerator that can change the optimal bit precision according to different precision / performance requirements.

[0033] (5) LNPU, which uses direct feedback alignment for fast online learning and has a built-in direct error propagation mechanism based on a pseudo-random number generator.

[0034] (6) GANPU proposes an adaptive spatiotemporal workload reuse method for the processing unit of generative adversarial network to maintain high utilization when accelerating multiple DNNs in a single GAN (Generative Adversarial Networks) model. At the same time, it uses a dual sparsity architecture to skip redundant computation caused by zeros of input and output features, and proposes an exponential-only ReLU prediction algorithm and its lightweight processing element architecture.

[0035] (7) APU (Acceleration Processing Unit) is a heterogeneous computing architecture that integrates a traditional CPU (Central Processing Unit) and GPU (Graphics Processing Unit) on a single chip. It flexibly distributes tasks between the CPU and GPU according to the nature of the computation, and assigns artificial intelligence operations to the GPU for processing, thereby improving the efficiency of data parallel computing.

[0036] (8) DPU (Deep-Learning Processing Unit) is a configurable computing engine based on FPGA (Field-Programmable Gate Array) to accelerate deep learning algorithms such as convolutional neural networks.

[0037] (9) TPU (Tensor Processing Units) is also a dedicated chip for accelerating neural network computing capabilities. Four generations have been launched. Compared with CPUs and GPUs of the same period, it can achieve 15-40 times performance improvement and 40-80 times energy efficiency improvement.

[0038] (10) The multimodal artificial intelligence chip Thinker can balance the resource conflicts between CNN (Convolutional Neural Networks) and RNN (Recurrent Neural Network) in terms of computation and bandwidth.

[0039] (11) Evolver, an evolvable artificial intelligence chip that supports on-chip training and reinforcement learning.

[0040] (12) ReDCIM can be used for general cloud artificial intelligence scenarios. It is based on TranCIM, which is accelerated by the self-attention neural network Transformer that integrates in-memory computing, and another Transformer accelerated chip based on approximate computing and asymptotic sparsity.

[0041] (13) STICKER-T proposed a block loop algorithm and a unified frequency domain acceleration implementation path.

[0042] (14) The DianNao series of neural network accelerators supports the accelerated processing of large-scale deep neural networks such as CNNs. It is the world's earliest neural network accelerator for dedicated computing in the field of artificial intelligence.

[0043] (15) The “Aristotle” architecture for CNN data operations and the “Descartes” architecture for DNN and RNN data operations are both DPU architectures and are semi-customized processors based on FPGA.

[0044] (16) BPU (Brain Processing Unit), the earliest two are the Journey 1.0 chip and the Sunrise 1.0 chip, and the application areas include data-intensive computing scenarios such as intelligent driving, smart life and smart city.

[0045] The functions of some traditional artificial intelligence chips used in computer devices, also known as processors, are described below:

[0046] (1) Central Processing Unit (CPU)

[0047] The CPU, as the core of a computer system's computation and control, is the final execution unit for information processing and program execution. All operations in the software layer of a computer system are ultimately mapped to CPU operations through the instruction set. Its main functions are interpreting computer instructions and processing data within the computer software. The von Neumann architecture is the foundation of modern computers. Its three core components are the CPU, internal memory, and input / output devices. The CPU mainly consists of two parts: the control unit and the arithmetic logic unit (ALU), including high-speed cache memory and the data and control buses that connect them. In this architecture, programs and data are stored uniformly, and instructions and data must be accessed from the same memory space and transmitted via the same bus, preventing overlapping execution. According to the von Neumann architecture, the CPU's operation is divided into five stages: instruction fetch, instruction decoding, instruction execution, memory access, and result write-back. The CPU's main functions are processing instructions, executing operations, controlling time, and processing data.

[0048] (2) Graphics Processing Unit (GPU)

[0049] A GPU, also known as a graphics processing unit, is a microprocessor specifically designed for performing image and graphics-related computations in personal computers, workstations, game consoles, and some mobile devices (such as tablets and smartphones).

[0050] GPUs reduce the reliance of graphics cards on CPUs and perform some of the tasks that were originally handled by the CPU. Especially in 4D graphics processing, the core technologies used by GPUs include hardware T&L (geometry transformation and lighting processing), cubic environment material mapping and vertex blending, texture compression and bump mapping, dual-texture four-pixel 256-bit rendering engines, etc. Hardware T&L technology can be said to be the hallmark of GPUs.

[0051] (3) Neural network processor

[0052] Neural network processors, also known as neural network accelerators or computing cards, or deep learning processors, are modules specifically designed to handle massive computational tasks in intelligent applications (other non-computational tasks are still handled by the CPU). Much data processing in neural networks involves matrix multiplication and addition. GPUs, which perform massively parallel operations, offer a cost-effective approach, but at the cost of higher power consumption. FPGAs, with their built-in DSP (Digital Signal Processor) modules and local memory, are more energy-efficient, but they are typically more expensive. Deep learning refers to multi-layered neural networks and the methods used to train them. In simple terms, a neural network processor uses deep neural networks to simulate the mechanisms of the human brain for learning, judgment, and decision-making.

[0053] (4) Multi-core processor

[0054] A multi-core processor integrates two or more complete computing engines (cores) into a single processor. This allows the processor to support multiple processors on the system bus, with all bus control and command signals provided by the bus controller. The development of multi-core technology stemmed from engineers' realization that simply increasing the speed of a single-core chip would generate excessive heat without providing a corresponding performance improvement, as was the case with previous processors. They recognized that at those speeds, processors generated excessive heat. Even without the heat issue, the cost-effectiveness was unacceptable; slightly faster processors were significantly more expensive. The advantages of multi-core technology in applications are twofold: providing users with more powerful computing performance; and more importantly, meeting the requirements of users performing simultaneous multitasking and multi-tasking computing environments.

[0055] (5) Heterogeneous processors

[0056] Heterogeneous computing refers to the "cooperative computing and mutual acceleration" between computing units using different instruction sets and architectures, thereby breaking through the development bottlenecks of single processor architectures and effectively solving problems such as energy consumption and scalability. Commonly used processor chips include CPUs, DSPs, GPUs, FPGAs, and ASICs (Application Specific Integrated Circuits). CPUs and GPUs require software support, while FPGAs and ASICs are integrated hardware and software architectures, where the software is the hardware. In terms of energy efficiency: ASIC > FPGA > GPU > CPU. The fundamental reason for this result is that for computationally intensive algorithms, the higher the efficiency of data movement and computation, the higher the energy efficiency. ASICs and FPGAs are closer to the underlying I / O, so their computational efficiency and data movement are high. However, FPGAs have redundant transistors and interconnects and operate at lower frequencies, so their energy efficiency is not as high as ASICs. GPUs and CPUs are general-purpose processors that require instruction fetching, instruction decoding, and instruction execution. This process shields the underlying I / O processing, decoupling the hardware and software, but it also limits the efficiency of data movement and computation, resulting in lower energy efficiency compared to ASICs and FPGAs. The difference in energy efficiency between GPUs and CPUs is mainly due to the fact that a large portion of the transistors in a CPU are used for cache and control logic units. Therefore, for computationally intensive algorithms with low computational complexity, redundant transistors cannot be utilized, resulting in a lower energy efficiency for CPUs compared to GPUs.

[0057] Over their long development, processor chips have developed distinct characteristics in use and the market. The CPU and GPU fields have a wealth of open-source software and applications. Any new technology is first implemented using CPU algorithms, making CPU programming resources abundant and readily available, with low development costs and short development cycles. FPGA implementations use low-level hardware description languages ​​such as Verilog / VHDL, requiring developers to have a deep understanding of FPGA chip characteristics. However, their high parallelism often leads to orders-of-magnitude performance improvements. Furthermore, FPGAs are dynamically reconfigurable. Once deployed in a data center, different logic can be configured to implement different hardware acceleration functions based on business needs. For example, the FPGA board currently deployed on a server might be used for image compression logic serving instant messaging. If real-time advertising prediction requires expansion to obtain more FPGA computing resources, a simple FPGA reconfiguration process can transform the FPGA board into "new" hardware to serve real-time advertising prediction, making it ideal for batch deployment. ASIC chips can achieve optimal performance, namely high area utilization, high speed, and low power consumption; however, neural network SC development is extremely risky, requires a sufficiently large market to guarantee cost, and has a long time cycle from R&D to market, making it unsuitable for fields where algorithms such as deep learning CNN are rapidly iterating.

[0058] Traditional AI chips follow the von Neumann architecture. During the execution of neural network models, the data processing units within the AI ​​chip need to communicate with each other. The computation heavily relies on massive amounts of data and requires the system to operate at very high clock frequencies, leading to a series of so-called "memory walls," "power walls," and "performance walls." These limitations prevent the AI ​​chip from achieving its full performance potential. For example, the "power wall" is designed to prevent the AI ​​chip from overheating. Furthermore, traditional AI chips suffer from low parallelism, poor versatility of mapping networks, difficulty in dynamically adjusting parameters during computation, and challenges in rapidly transferring data. Therefore, it is necessary to propose effective technical solutions to these problems.

[0059] In one embodiment, such as Figure 1 The diagram illustrates a data processing method. This embodiment describes the application of this data processing method to a processing chip, which can be a System-on-Chip (SoC) including a scalar processing unit and a computing unit. In this embodiment, the data processing method includes the following steps:

[0060] Step S101: Upon receiving a processing instruction, determine the first target processing unit from the scalar processing unit and the computation unit to process the processing instruction; the processing instruction is used to instruct the execution of the target neural network model.

[0061] The processing instructions are a string of binary 0s and 1s that instruct the processing chip to run any of the following neural network models: LSTM (Long Short-Term Memory), CNN, or RNN.

[0062] The scalar processing unit includes RISC-V (Reduced Instruction Set Computing V) and a scalar cache, while the computation unit includes a PE (Processing Element) array and a computation cache, both of which are capable of processing instructions.

[0063] Optionally, when the remaining memory in the computing unit's cache reaches a preset remaining memory threshold, the computing unit sends a notification to the state machine in the processing chip, causing the state machine to transition to a new state. When the state machine is in a pre-state, the controller in the processing chip is triggered. Then, upon receiving a processing instruction, the controller determines the first target processing unit based on the processing efficiency of the scalar processing unit and the computing unit. The processing instruction is then marked, carrying identification information. The marked processing instruction is then sent to the first target processing unit. The identification information instructs the first target processing unit on the processing action of the instruction, such as logical judgment or compilation.

[0064] For example, when the controller finds that the processing efficiency of the computing unit is greater than that of the scalar processing unit, it determines the computing unit as the first target processing unit, then packages the identification information indicating that the processing instruction should be logically judged with the processing instruction and sends it to the computing unit, which then processes the processing instruction.

[0065] Step S102: The processing instruction is processed by the first target processing unit to obtain the processing information corresponding to the processing instruction; the processing information includes control information and data information.

[0066] The control information includes address information and a neural network identifier. The address information indicates how to retrieve the data to be processed from the registers in the processing chip, while the neural network identifier indicates how to run the target neural network model. The data to be processed includes the target neural network weight parameters required to run the target neural network model, as well as the image to be recognized. This data is carried in the processing instructions; it is externally input data that is required to run the target neural network model but does not need to be retrieved from registers.

[0067] Optionally, if the first target processing unit is a scalar processing unit, the scalar processing unit performs logical judgments on the processing instructions to obtain control information and data information. If the first target processing unit is a computation unit, the ALU (Arithmetic and Logic Unit) in the computation unit performs logical judgments on the processing instructions to obtain control information and data information. The computation unit occupies some PE array overhead and computation cache overhead when performing logical judgments.

[0068] Step S103: Run the target neural network model using the second target processing unit and the processing information to obtain the running result of the target neural network model; the second target processing unit includes the first target processing unit.

[0069] Optionally, if the first target processing unit is a scalar processing unit, it can serve as the second target processing unit. It retrieves the data to be processed from the register based on the address information in the control information. During this retrieval, it performs vectorization processing on the data, including data reading, data decomposition, and position matching. Then, based on the neural network identifier in the control information, it determines the target neural network model to be run. The data and the data to be processed are then input into the target neural network model, and convolution, pooling, and activation operations are performed to obtain the result of the target neural network model. Alternatively, after inputting the data and the data to be processed into the target neural network model, the scalar processing unit performs partial convolution, pooling, and activation operations. After these operations, it packages the partial results, control information, data, and the data to be processed together and sends them to the computation unit. The computation unit then performs the computation to obtain the result of the target neural network model.

[0070] Optionally, if the first target processing unit is a computing unit, the computing unit can be used as the second target processing unit. The computing unit obtains the data information to be processed from the register according to the address information in the control information, then determines the target neural network model to be run according to the neural network identifier in the control information, and then inputs the data information and the data information to be processed into the target neural network model to start performing operations such as convolution, pooling, and activation to obtain the running result of the target neural network model.

[0071] Additionally, it should be noted that during the convolution, pooling, and activation operations, the weight parameter adjustment mechanism in the second target processing unit will flexibly adjust the weight parameters according to the operation situation to improve the accuracy of the target neural network model's results.

[0072] In summary, upon receiving a processing instruction, a first target processing unit is determined from the scalar processing unit and the computation unit to process the instruction. This instruction directs the execution of a target neural network model. The first target processing unit then processes the instruction to obtain corresponding processing information, including control and data information. Finally, a second target processing unit and the processing information are used to run the target neural network model, yielding the result. The second target processing unit includes the first target processing unit. In this application, the first target processing unit processes the instruction, and the second target processing unit runs the target neural network model. Since the second target processing unit includes the first target processing unit, the data processing unit for processing the instruction and running the target neural network model can be the same entity—either a scalar processing unit or a computation unit. Therefore, in this application, the processing chip, i.e., the artificial intelligence chip, allows either the scalar processing unit or the computation unit to independently process the instruction and run the target neural network model during neural network model execution. This eliminates the need for data communication between the scalar processing unit and the computation unit, reducing data communication between data processing units and mitigating overheating issues common in artificial intelligence chips during neural network model execution.

[0073] In one embodiment, upon receiving a processing instruction, determining a first target processing unit for processing the processing instruction from the scalar processing unit and the computing unit includes: upon receiving the processing instruction, obtaining the remaining memory amount of the scalar processing unit and the remaining memory amount of the computing unit; and determining the unit with the largest remaining memory amount among the remaining memory amount of the scalar processing unit and the remaining memory amount of the computing unit as the first target processing unit.

[0074] Optionally, when the controller in the processing chip receives a processing instruction, it obtains the remaining memory amount of the scalar cache in the scalar processing unit, then obtains the remaining memory amount of the computation cache in the computation unit, and then compares the remaining memory amount of the scalar cache with the remaining memory amount of the computation cache. If the remaining memory amount of the scalar cache is larger, the scalar processing unit is determined to be the first target processing unit; if the remaining memory amount of the computation cache is larger, the computation unit is determined to be the first target processing unit.

[0075] In this embodiment, by comparing the remaining memory of the scalar processing unit and the remaining memory of the computing unit to determine the first target processing unit, the data processing capabilities of the scalar processing unit and the computing unit can be accurately estimated, thereby ensuring that the first target processing unit avoids data overflow and processing failure during the processing of instructions.

[0076] In one embodiment, such as Figure 2 The diagram illustrates another data processing method. The processing chip also includes a judgment unit. If the first target processing unit is a computing unit, the method further includes:

[0077] Step S201: Obtain the judgment result of the judgment unit on the processed information; the judgment result is used to characterize whether the processed information is correct.

[0078] Since the main function of the computing unit is to run neural network models, when the computing unit is used to process processing instructions, such as logical judgments, the accuracy of the logical judgment results of the computing unit cannot be fully guaranteed. Therefore, a judgment unit can also be set in the processing chip.

[0079] Optionally, the judgment unit periodically acquires the processing information obtained by the computing unit from processing the processing instructions, and then compares it with the processing information obtained by the judgment unit from processing the processing instructions to obtain a judgment result. Specifically, if the processing information obtained by the computing unit from processing the processing instructions is consistent with the processing information obtained by the judgment unit from processing the processing instructions, it indicates that the processing information obtained by the computing unit from processing the processing instructions is correct, and the judgment result is 1; if they are inconsistent, it indicates that the processing information obtained by the computing unit from processing the processing instructions is incorrect, and the judgment result is 0. The controller in the processing chip acquires the judgment result of the judgment unit from processing information in real time to realize real-time monitoring of the computing unit's processing of processing instructions.

[0080] Step S202: If the judgment result indicates that the processing information is incorrect, then the third target processing unit for processing instructions is re-determined from the scalar processing unit and the calculation unit.

[0081] Step S203: The processing instructions are processed using the third target processing unit.

[0082] Optionally, if the judgment result obtained by the controller from the judgment unit is 0, the controller controls the calculation unit to stop processing the processing instruction and instructs the calculation unit to clear the data stored in the calculation cache when processing the processing instruction. Then, the controller obtains the remaining memory of the scalar processing unit and the remaining memory of the calculation unit, and determines the unit with the largest remaining memory between the remaining memory of the scalar processing unit and the remaining memory of the calculation unit as the third target processing unit, and uses the third target processing unit to process the processing instruction.

[0083] If the processing information obtained by the calculation unit from processing the processing instruction twice in a row is judged as incorrect by the judgment unit, then the controller determines the scalar processing unit as the third target processing unit and processes the processing instruction.

[0084] In addition, since the computing unit not only processes the current processing instructions, but also runs the neural network model indicated by the processing instructions of the previous moment, it is necessary to monitor the remaining memory of the computing unit in real time. The processing chip is also equipped with a memory monitoring unit to obtain the remaining memory of the computing cache in the computing unit in real time. If the remaining memory is less than the memory threshold, it indicates that the data of the computing unit is overflowing. The memory monitoring unit then feeds back to the controller. The controller re-determines the third target processing unit for processing the processing instructions from the scalar processing unit and the computing unit, and uses the third target processing unit to process the processing instructions to ensure the normal processing of the processing instructions.

[0085] In this embodiment, by obtaining the judgment result of the judgment unit on the processing information, the third target processing unit for the processing instruction is re-determined, thus ensuring the correctness of the processing instruction.

[0086] In one embodiment, the processing chip further includes a vector processing unit; before processing the processing instruction using the first target processing unit to obtain the processing information corresponding to the processing instruction, the above method further includes: calling the vector processing unit to perform vectorization processing on the processing instruction to obtain the first vector data corresponding to the processing instruction; the above step S101 includes: processing the first vector data using the first target processing unit to obtain processing information.

[0087] The vector processing unit includes SIMD (Single Instruction Multiple Data). Vectorization refers to the process of arranging messy data neatly.

[0088] Optionally, when the controller in the processing chip receives a processing instruction, it determines the first target processing unit for processing the instruction from the scalar processing unit and the calculation unit, and then calls the vector processing unit to perform vectorization processing on the instruction. That is, after sequentially performing operations such as reading the instruction, decomposing it, and matching its position, the controller obtains the first vector data. The controller then sends the first vector data to the first target processing unit, which processes the first vector data to obtain the processing information.

[0089] In this embodiment, by calling the vector processing unit to vectorize the processing instructions, the data transmission speed can be improved, thereby enhancing the parallel computing capability of the processing chip in running the target neural network model.

[0090] In one embodiment, the second target processing unit further includes a vector processing unit; running the target neural network model using the second target processing unit and processing information to obtain the running result of the target neural network model includes: using the first target processing unit to decompose the target neural network model to obtain multiple sub-models corresponding to the target neural network model, and sending each sub-model to the vector processing unit; running each sub-model using the vector processing unit and processing information to obtain the running result.

[0091] In one embodiment, each sub-model is run using a vector processing unit and processing information to obtain running results, including: using the vector processing unit to vectorize the processing information to obtain second vector data corresponding to the processing information; and using the vector processing unit and the second vector data to run each sub-model to obtain running results.

[0092] Optionally, the first target processing unit can be a computing unit. The computing unit decomposes the target neural network model to obtain multiple sub-models corresponding to the target neural network model, and sends each sub-model to the vector processing unit. The vector processing unit and the processing information are used to run each sub-model to obtain the running results.

[0093] Optionally, under normal circumstances, the vector processing unit can only perform simple multiplication and addition operations. Therefore, the first target processing unit needs to decompose the target neural network model into multiple simple multiplication and / or addition sub-models, and then send each sub-model to the vector processing unit. The first target processing unit also sends the processing information obtained from processing the processing instructions to the vector processing unit.

[0094] After receiving the sub-models and processing information, the vector processing unit changes its state from normal to computation state. The vector buffer in the vector processing unit is cleared. Then, according to the control information in the processing information, the unit obtains the data to be processed from the registers in the processing chip. During the process of obtaining the data to be processed, the unit performs vectorization processing on the data to be processed to obtain the second vector data. Then, based on the data information in the processing information and the second vectorized data, the unit runs each sub-model to obtain the running results. Finally, the state of the vector processing unit changes back from computation state to normal state.

[0095] In addition, due to the large amount of data involved in running each sub-model, the vector processing unit may experience data overflow, calculation errors, and other problems. Therefore, a verification unit is also set up in the processing chip. This verification unit is used to periodically obtain the remaining memory in the vector buffer of the vector processing unit and to determine whether the running results of each sub-model are correct. If the remaining memory in the vector buffer is insufficient or the running results of each sub-model are incorrect, the verification unit notifies the vector processing unit to stop running each sub-model and clears all the data stored in the vector buffer generated during the running of each sub-model. The verification unit also notifies the controller, and the controller re-determines the fourth target processing unit to run the target neural network model from the scalar processing unit and the calculation unit.

[0096] In this embodiment, after the vector processing unit performs vectorization processing on the processed information to obtain the second vector processing data, it does not need to return the second vector processing data to the first target processing unit. The vector processing unit can complete the calculation of the target neural network model, reducing data transmission and thus improving the problem of overheating that is prone to occur when the artificial intelligence chip runs the neural network model.

[0097] In addition, such as Figure 3 As shown, an architecture for a processing chip is provided. The architecture of the processing chip includes a scalar processing unit, a computing unit, a vector processing unit, and a specific domain integration interface. These components are interconnected through an on-chip network, and the scalar processing unit, computing unit, vector processing unit, and specific domain integration interface are all integrated on a System-on-a-Chip (SoC).

[0098] The scalar processing unit includes RISCV, which is used to implement support for various neural network mappings, instruction compilation, and multi-level operations.

[0099] The computing unit includes a PE array, and the functions of each PE can be customized to reduce the interaction between off-chip and on-chip units.

[0100] Vector processing units include SIMD, which are used to enhance the parallel computing capabilities of neural network models.

[0101] The Network On Chip (NOC) can be one or more of the following: Advanced Microcontroller Bus Architecture (AMBA), namely Advanced High-performance Bus (AHB), Advanced System Bus (ASB), Advanced Peripheral Bus (APB), and Advanced eXtensible Interface (AXI). Furthermore, when the RISC core in this application is increased to multi-core or even many-core, an on-chip network protocol can be adopted. Each type of bus implies that the corresponding functional module needs to be integrated into the SOC and adapted for integration.

[0102] Specific domain interfaces can be Peripheral Component Interconnect Express (PCI-Express) interfaces, Double Data Rate Synchronous Dynamic Random Access Memory (DDR) interfaces, etc. These protocol interfaces include multiple performance versions, and each external interface also means that the corresponding functional modules need to be integrated into the SOC and adapted and debugged.

[0103] In summary, the most detailed embodiment of this application is as follows:

[0104] Step S1: Upon receiving a processing instruction, obtain the remaining memory of the scalar processing unit and the remaining memory of the computation unit, and determine the unit with the largest remaining memory among the remaining memory of the scalar processing unit and the remaining memory of the computation unit as the first target processing unit for processing the processing instruction, wherein the processing instruction is used to instruct the execution of the target neural network model.

[0105] In step S2, the processing chip further includes a judgment unit. When the first target processing unit is a calculation unit, the judgment unit obtains the judgment result of the processing information. The judgment result is used to indicate whether the processing information is correct. If the judgment result indicates that the processing information is incorrect, a third target processing unit for processing the processing instruction is re-determined from the scalar processing unit and the calculation unit. The processing instruction is processed using the third target processing unit.

[0106] Step S3: Call the vector processing unit to vectorize the processing instruction to obtain the first vector data corresponding to the processing instruction.

[0107] Step S4: The first target processing unit processes the first vector data to obtain processing information, which includes control information and data information.

[0108] Step S5: Run the target neural network model using the second target processing unit and the processing information to obtain the running result of the target neural network model. The second target processing unit includes the first target processing unit.

[0109] Step S6: The second target processing unit further includes a vector processing unit, which uses the first target processing unit to split the target neural network model to obtain multiple sub-models corresponding to the target neural network model, and sends each sub-model to the vector processing unit.

[0110] Step S7: Use the vector processing unit to vectorize the processed information to obtain the second vector data corresponding to the processed information; use the vector processing unit and the second vector data to run each sub-model and obtain the running results.

[0111] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0112] Based on the same inventive concept, this application also provides a data processing apparatus for implementing the data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data processing apparatus embodiments provided below can be found in the limitations of the data processing method described above, and will not be repeated here.

[0113] In one embodiment, such as Figure 4 As shown, a structural block diagram of a data processing device is provided. The data processing device is applied to a processing chip, which includes a scalar processing unit and a computing unit. The data processing device 400 includes: a first determining module 401, a second determining module 402, and a third determining module 404, wherein:

[0114] The first determining module 401 is used to determine, upon receiving a processing instruction, a first target processing unit from the scalar processing unit and the computation unit to process the processing instruction; the processing instruction is used to instruct the execution of the target neural network model.

[0115] The second determining module 402 is used to process the processing instruction using the first target processing unit to obtain the processing information corresponding to the processing instruction; the processing information includes control information and data information.

[0116] The third determining module 404 is used to run the target neural network model using the second target processing unit and the processing information to obtain the running result of the target neural network model; the second target processing unit includes the first target processing unit.

[0117] In one embodiment, the first determining module 401 is further configured to, upon receiving a processing instruction, obtain the remaining memory of the scalar processing unit and the remaining memory of the computing unit; and determine the unit with the largest remaining memory among the remaining memory of the scalar processing unit and the remaining memory of the computing unit as the first target processing unit.

[0118] In one embodiment, the processing chip further includes a vector processing unit; the device further includes a fourth determining module, used to call the vector processing unit to perform vectorization processing on the processing instruction to obtain first vector data corresponding to the processing instruction; the second determining module 402 is further used to process the first vector data using the first target processing unit to obtain processing information.

[0119] In one embodiment, the second target processing unit further includes a vector processing unit; the third determining module 404 is further configured to use the first target processing unit to split the target neural network model to obtain multiple sub-models corresponding to the target neural network model, and send each sub-model to the vector processing unit; and run each sub-model using the vector processing unit and the processing information to obtain the running result.

[0120] In one embodiment, the third determining module 404 is further configured to use the vector processing unit to perform vectorization processing on the processed information to obtain the second vector data corresponding to the processed information; and use the vector processing unit and the second vector data to run each sub-model to obtain the running results.

[0121] In one embodiment, the processing chip further includes a judgment unit, and the device further includes a fifth determination module for obtaining the judgment result of the judgment unit on the processing information; the judgment result is used to characterize whether the processing information is correct; if the judgment result characterizes the processing information as incorrect, a third target processing unit for processing the processing instruction is re-determined from the scalar processing unit and the calculation unit; and the processing instruction is processed using the third target processing unit.

[0122] Each module in the aforementioned data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0123] In one embodiment, a processing chip is provided, the internal structure of which can be shown as follows: Figure 5 As shown, the processing chip includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data such as processing information from the above method embodiments. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a processing chip method.

[0124] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0125] In one embodiment, a processing chip is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0126] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0127] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0128] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0129] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data processing method, characterized in that, The data processing method is applied to a processing chip, which includes a scalar processing unit, a computation unit, and a vector processing unit. The scalar processing unit includes a RISC-V 5th Edition Reduced Instruction Set Computing (RISC-V) and a scalar cache. The computation unit includes a PE computation unit array and a computation cache. The vector processing unit includes a SIMD Single Instruction Multiple Data (SIMD) system. The method includes: Upon receiving a processing instruction, the remaining memory of the scalar processing unit and the remaining memory of the computing unit are obtained; the unit with the largest remaining memory between the remaining memory of the scalar processing unit and the remaining memory of the computing unit is determined as the first target processing unit; the processing instruction is used to instruct the execution of the target neural network model; The vector processing unit is invoked to perform vectorization processing on the processing instruction to obtain the first vector data corresponding to the processing instruction; The first target processing unit processes the first vector data to obtain processing information corresponding to the processing instruction; the processing information includes control information and data information; the control information includes address information of the data to be processed and neural network identifier of the target neural network model; the address information is used to indicate that the data to be processed is obtained from the register in the processing chip, and the data to be processed includes the target neural network weight parameters required to run the target neural network model and the image to be recognized; The target neural network model is run using the second target processing unit and the processing information to obtain the running result of the target neural network model; the second target processing unit includes the first target processing unit and the vector processing unit; The step of running the target neural network model using the second target processing unit and the processing information to obtain the running result of the target neural network model includes: using the first target processing unit to split the target neural network model to obtain multiple sub-models corresponding to the target neural network model, and sending each sub-model to the vector processing unit; using the vector processing unit to vectorize the processing information to obtain second vector data corresponding to the processing information; and using the vector processing unit and the second vector data to run each sub-model to obtain the running result.

2. The data processing method according to claim 1, characterized in that, The processing chip further includes a determination unit; if the first target processing unit is the computing unit, the method further includes: The judgment result of the judgment unit on the processed information is obtained; the judgment result is used to characterize whether the processed information is correct. If the judgment result indicates that the processing information is incorrect, then a third target processing unit for processing the processing instruction is re-determined from the scalar processing unit and the calculation unit. The processing instructions are processed using the third target processing unit.

3. The method according to claim 1, characterized in that, The method further includes: The processing instructions are marked to obtain processing instructions carrying marking information; The processing instruction carrying the identification information is sent to the first target processing unit; wherein the identification information is used to instruct the first target processing unit to perform the processing action on the processing instruction.

4. The method according to claim 2, characterized in that, The step of obtaining the judgment result of the judgment unit on the processed information includes: If the processing information obtained by the calculation unit from processing the processing instruction is consistent with the processing information obtained by the judgment unit from processing the processing instruction, then the judgment result is determined to be that the processing information obtained by the calculation unit from processing the processing instruction is correct.

5. The method according to claim 4, characterized in that, The method further includes: If the processing information obtained by the calculation unit from processing the processing instruction is inconsistent with the processing information obtained by the judgment unit from processing the processing instruction, then the judgment result is determined to be that the processing information obtained by the calculation unit from processing the processing instruction is incorrect.

6. The method according to claim 2, characterized in that, The step of re-determining the third target processing unit for processing the processing instruction from the scalar processing unit and the computing unit includes: The system controls the computing unit to stop processing the processing instructions and instructs the computing unit to clear the data stored in the computing cache when processing the processing instructions. After clearing the data stored in the computation cache when processing the processing instructions, the remaining memory of the scalar processing unit and the remaining memory of the computation unit are obtained, and the unit with the largest remaining memory between the remaining memory of the scalar processing unit and the remaining memory of the computation unit is determined as the third target processing unit.

7. A data processing apparatus, characterized in that, The data processing device is applied to a processing chip, which includes a scalar processing unit and a computation unit. The scalar processing unit includes a RISC-V 5th Edition Reduced Instruction Set Computing (RISC-V) and a scalar cache. The computation unit includes a PE computation unit array and a computation cache. The vector processing unit includes a SIMD Single Instruction Multiple Data (SIMD) system. The device includes: The first determining module is configured to, upon receiving a processing instruction, obtain the remaining memory of the scalar processing unit and the remaining memory of the computing unit; determine the unit with the largest remaining memory between the remaining memory of the scalar processing unit and the remaining memory of the computing unit as the first target processing unit; the processing instruction is used to instruct the execution of the target neural network model; The vector processing unit is used to call the vector processing unit to perform vectorization processing on the processing instruction to obtain the first vector data corresponding to the processing instruction; The second determining module is used to process the first vector data using the first target processing unit to obtain processing information corresponding to the processing instruction; the processing information includes control information and data information; the control information includes address information of the data information to be processed and the neural network identifier of the target neural network model; the address information is used to indicate that the data information to be processed is obtained from the register in the processing chip, and the data information to be processed includes the target neural network weight parameters required to run the target neural network model and the image to be identified; The third determining module is used to run the target neural network model using the second target processing unit and the processing information to obtain the running result of the target neural network model; the second target processing unit includes the first target processing unit and the vector processing unit; The third determining module is specifically used to use the first target processing unit to split the target neural network model to obtain multiple sub-models corresponding to the target neural network model, and send each sub-model to the vector processing unit; use the vector processing unit to vectorize the processing information to obtain second vector data corresponding to the processing information; use the vector processing unit and the second vector data to run each sub-model to obtain the running result.

8. A processing chip, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images