Neural network hardware circuit, data selection method, chip and electronic equipment

Abstract

The invention relates to a neural network hardware circuit, a data selection method, a chip and electronic equipment, and belongs to the field of neural networks. The neural network hardware circuit comprises an input data selector, an actuator and an output data selector, wherein the actuator is positioned between the input data selector and the output data selector; the input data selector is used for selecting external input data or internal data of the neural network hardware circuit as an input data source according to a first selection control signal; the actuator comprises a customized hardware circuit corresponding to at least one layer in the neural network; and the output data selector is used for selecting to output the output data of the actuator as the external data or the internal data according to a second selection control signal. According to the invention, the circuit area and power consumption during hardware conversion of the neural network can be reduced.

Description

Technical Field

[0001] The present application relates to the field of neural networks, and more specifically, to a neural network hardware circuit and data selection method, chip, and electronic equipment. Background Art

[0002] Neural networks typically consist of multiple layers, such as convolutional layers, fully connected layers, and activation layers. Each layer can be considered an independent functional module, and the hardware modules for each layer can be highly customized and optimized to meet the specific computing requirements of that layer. For example, independent optimization of convolutional and fully connected layers can achieve optimal performance. Because the hardware for each layer is independent, the control logic is relatively simple, eliminating the need for complex data scheduling and regulation. However, since each layer is an independent hardware module, expanding the number of network layers or processing larger models requires additional hardware resources, which significantly increases the area and power consumption of the entire hardware module. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a neural network hardware circuit and data selection method, chip, and electronic device to improve the problems of large area and power consumption of neural network hardware circuits in the prior art.

[0004] The embodiment of the present application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a neural network hardware circuit, comprising: an input data selector, an executor, and an output data selector, wherein the executor is located between the input data selector and the output data selector; the input data selector is used to select external input data or internal data of the neural network hardware circuit as an input data source according to a first selection control signal; the executor includes a customized hardware circuit corresponding to at least one layer in the neural network; the output data selector is used to select to output the output data of the executor as external data or as the internal data according to a second selection control signal.

[0006] In the above embodiment, the neural network hardware circuit of the above structure can be used to control the input data source of the input data selector and the output direction of the output data selector, so that when the neural network is hardwareized, for the repeated structures in the neural network, only a single hardware resource needs to be instantiated (or customized), and then the repeated hardware resources (executors) need to be called multiple times, without the need to separately instantiate hardware resources for the repeated structures in the neural network. Since a large number of hardware modules are reduced, the purpose of reducing the circuit area and power consumption when the neural network is hardwareized can be achieved. At the same time, by controlling the input data source of the input data selector and the output direction of the output data selector, the processing effect of separately instantiating hardware resources for the repeated structures in the neural network can also be achieved.

[0007] In combination with a possible implementation manner of the embodiment of the first aspect, the neural network hardware circuit also includes: a state machine, which is connected to the input data selector and the output data selector respectively; the state machine is used to send a first selection control signal to the input data selector, and send a second selection control signal to the output data selector.

[0008] In the above embodiment, by adding a state machine to output a control signal to control the input data selector and the output data selector, precise control can be performed, thereby achieving the processing effect of individually instantiating hardware resources for repeated structures in the neural network.

[0009] In combination with a possible implementation manner of the embodiment of the first aspect, the neural network hardware circuit further includes: a cache; the states of the state machine include: a pre-start phase, a cache write phase, a cache balancing phase, a cache clear phase and an end phase; wherein, in the pre-start phase, the input data selector selects external input data as the input data source, and the output data selector has no output; in the cache write phase, the input data selector selects external input data as the input data source, and the output data selector selects to write the output data of the actuator as internal data into the cache; in the cache balancing phase, the input data selector selects internal data as the input data source, and the output data selector selects to write the output data of the actuator as internal data into the cache; in the cache clear phase, the input data selector selects internal data as the input data source, and the output data selector selects to output the output data of the actuator as external data; in the end phase, the input data selector has no input, and the output data selector selects to output the output data of the actuator as external data.

[0010] In the above embodiment, the above five states of the state machine cover the workflow of the entire circuit. By controlling the state machine to be in different states, the input data source of the input data selector and the output direction of the output data selector can be accurately controlled, thereby achieving precise control of the data flow and efficient processing of the data flow.

[0011] In combination with a possible implementation manner of the embodiment of the first aspect, the state machine is also connected to the cache and the executor; the initial state of the state machine is the pre-start stage; when the state machine receives a first signal from the executor, it switches to the cache write stage; when the state machine receives an indication signal from the cache, it switches to the cache balancing stage; when the state machine receives a task completion signal from the executor, it switches to the cache clearing stage; when the state machine receives an empty signal from the cache, it switches to the end stage; when the state machine receives a second signal from the executor, it switches to the pre-start stage.

[0012] In the above embodiment, the state machine can switch states by receiving signals from the cache and the executor without introducing other control logic. This can reduce dependence on external control logic while accurately controlling the state of the state machine, thereby simplifying the circuit area and control logic. At the same time, since the state machine switches states by receiving signals from the cache and the executor, it is beneficial to improve the timeliness of switching and switch to different states in time to accurately process data.

[0013] In combination with a possible implementation manner of the embodiment of the first aspect, the neural network hardware circuit further includes: a cache, which is connected to the input data selector and the output data selector respectively; and the cache is used to cache the internal data.

[0014] In the above embodiment, by adding a cache to store internal data, the input data selector has enough time to select the input data source, which can reduce the timing requirements of the circuit and thus improve the applicability of the circuit. At the same time, by adding a cache to cache data, the data can be transmitted in multiple times, which can reduce the requirements for single data transmission.

[0015] In combination with a possible implementation of the embodiment of the first aspect, the cache includes a FIFO memory.

[0016] In the above embodiment, a FIFO (First Input First Output) memory is used to ensure that the data that enters the cache first leaves the cache first, without the need for additional control logic to ensure that the data that enters the cache first leaves the cache first, thereby reducing the control logic of the circuit.

[0017] In combination with a possible implementation manner of the embodiment of the first aspect, the number of the executors is multiple, and the neural network hardware circuit further includes a first switch and a second switch; the output end of the input data selector is connected to the input end of each of the executors through the first switch, and the input end of the output data selector is connected to the output end of each of the executors through the second switch.

[0018] In the above embodiment, by adding the first switch and the second switch, the input or output of each actuator can be selected by controlling the state (closed or open) of the first switch and / or the second switch to adapt to various complex scenarios. Different actuators can be selected by switching the switches, thereby covering neural networks with different structures and improving the flexibility of data processing.

[0019] In combination with a possible implementation manner of the embodiment of the first aspect, the first switch and the second switch are both transistor switches, and the transistor switch includes: one of: a BJT switch, an IGBT switch, a MOS tube switch, and a Darlington transistor switch.

[0020] In the above embodiment, the above transistor switch is selected as the switching element, which can achieve its purpose while reducing circuit area and saving costs, and simplifying the difficulty of circuit design.

[0021] In combination with a possible implementation manner of the embodiment of the first aspect, the neural network corresponding to the neural network hardware circuit includes: a feedforward neural network, a convolutional neural network, a recurrent neural network, a long short-term memory network, a generative adversarial network, a Transformer network, an autoencoder, and a deep residual network.

[0022] In the above embodiment, the above mainstream neural network can be processed into circuit hardware to obtain a corresponding reusable neural network hardware circuit. When instantiating, for repeated structures, only a single hardware resource needs to be instantiated (or customized) and then called multiple times, thereby achieving the purpose of reducing the circuit area and power consumption when the neural network is hardwareized, and being able to adapt to various complex scenarios to improve the flexibility of data processing.

[0023] In a second aspect, an embodiment of the present application further provides a chip comprising a neural network hardware circuit provided by any possible implementation of the embodiment of the first aspect described above.

[0024] In a third aspect, an embodiment of the present application further provides an electronic device comprising the chip as described above.

[0025] In a fourth aspect, an embodiment of the present application further provides a data selection method, which is applied to a neural network hardware circuit provided by any possible implementation method of the embodiment of the first aspect above, the method comprising: in a pre-startup phase, controlling the input data selector to select external input data as an input data source; in a cache write phase, controlling the input data selector to select external input data as an input data source, and controlling the output data selector to select to write the output data of the executor as internal data into the cache; in a cache balancing phase, controlling the input data selector to select internal data as an input data source, and controlling the output data selector to select to write the output data of the executor as internal data into the cache; in a cache clearing phase, controlling the input data selector to select internal data as an input data source, and controlling the output data selector to select to output the output data of the executor as external data; in an ending phase, controlling the output data selector to select to output the output data of the executor as external data.

[0026] In combination with a possible implementation manner of the fourth aspect, if the neural network hardware circuit includes a state machine, wherein the initial state of the state machine is the pre-startup stage; the method further includes: the state machine switches to the cache write stage when receiving a first signal from the executor; the state machine switches to the cache balancing stage when receiving an indication signal from the cache; the state machine switches to the cache clearing stage when receiving a task completion signal from the executor; the state machine switches to the end stage when receiving an empty signal from the cache; and the state machine switches to the pre-startup stage when receiving a second signal from the executor.

[0027] Among them, the technical effects that can be achieved in the second to fourth aspects refer to the corresponding technical effects in the first aspect and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 A schematic diagram showing the structure of a first neural network hardware circuit provided in an embodiment of the present application is shown;

[0030] Figure 2 A schematic diagram showing the structure of a second neural network hardware circuit provided in an embodiment of the present application is shown;

[0031] Figure 3 A schematic diagram showing the structure of a third neural network hardware circuit provided in an embodiment of the present application is shown;

[0032] Figure 4 A schematic structural diagram of a fourth neural network hardware circuit provided in an embodiment of the present application is shown;

[0033] Figure 5 A schematic diagram of state switching of a state machine provided in an embodiment of the present application is shown;

[0034] Figure 6 A schematic diagram of the structure of a neural network model provided in an embodiment of the present application is shown;

[0035] Figure 7 A working principle diagram of a neural network hardware circuit provided by an embodiment of the present application is shown;

[0036] Figure 8 A schematic diagram showing the structure of a fifth neural network hardware circuit provided in an embodiment of the present application is shown;

[0037] Figure 9 A schematic diagram of a data selection process according to an embodiment of the present invention is shown;

[0038] Figure 10 A structural block diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The following embodiments can be used as examples to more clearly illustrate the technical solutions of the present application, and are not intended to limit the scope of protection of the present application. It will be understood by those skilled in the art that, in the absence of conflict, the features in the following embodiments and embodiments can be combined with each other.

[0040] It should be noted that similar numbers and letters represent similar items in the following figures, so once an item is defined in one figure, it does not need to be further defined or explained in subsequent figures. At the same time, in the description of this application, relational terms such as "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0041] Furthermore, the term "and / or" in this application is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0042] In the description of the embodiments of the present application, unless otherwise clearly specified or limited, the technical term "connection" may refer to a direct connection or an indirect connection through an intermediate medium.

[0043] In a neural network, the same layer structure will appear. In order to reduce the circuit area and power consumption when the neural network is hardwareized, the embodiment of the present application provides a reusable neural network hardware circuit. For this repeated structure, it is only necessary to instantiate (or customize) a single hardware resource and then call it multiple times, thereby achieving the purpose of reducing the circuit area and power consumption when the neural network is hardwareized. The neural network shown in this application can be a neural network of various deep learning, that is, the neural network corresponding to the neural network hardware circuit in this application includes but is not limited to feedforward neural networks (FNN), convolutional neural networks (CNN), recurrent neural networks (RNN), long short-term memory networks (LSTM), generative adversarial networks (GAN), transformer networks, autoencoders, deep residual networks (ResNet), etc.

[0044] The following combination Figure 1The neural network hardware circuit provided in the embodiment of the present application is described. The neural network hardware circuit provided in the embodiment of the present application includes: an input data selector, an executor, and an output data selector. The executor is located between the input data selector and the output data selector.

[0045] In one possible implementation, the neural network hardware circuit shown in this application may be a part of the entire hardware circuit of the neural network. Figure 1 The input data selector shown may include other actuators before it, and / or, after the output data selector, may include other actuators. In some embodiments, multiple Figure 1 The circuit structure shown.

[0046] The input data selector is used to select external input data or internal data of the neural network hardware circuit as the input data source according to a first selection control signal (such as represented by C1). In some possible implementations, the input data selector can be a two-or-one selector, and the input data source can be external input data or internal data of the neural network hardware circuit. The first selection control signal is used to control the selection logic of the input data selector. For example, the first selection control signal can be a level signal. When the first selection control signal is a high level 1, the input data source can be selected as the input data source by external input data; when the first selection control signal is a low level 0, the input data source can be selected as the input data source by internal data of the neural network hardware circuit. Of course, the reverse can also be true, that is, when the first selection control signal is a low level 0, the input data source can be selected as the input data source by external input data; when the first selection control signal is a high level 1, the input data source can be selected as the input data source by internal data of the neural network hardware circuit.

[0047] The external input data may be external data input to the entire neural network. In some embodiments, the external input data may also be output data from other actuators.

[0048] An output data selector is configured to select whether to output the actuator's output data as external data or as internal data based on a second selection control signal (e.g., represented by C2). In some possible implementations, the output data selector can be a one-to-two selector (i.e., a selector with one input and two outputs) or a demultiplexer, allowing the actuator's output data to be output as external data or internal data. The second selection control signal is used to control the selection logic of the output data selector. For example, the second selection control signal can be a level signal. When the second selection control signal is a high level 1, the actuator's output data can be selected as external data; when the second selection control signal is a low level 0, the actuator's output data can be selected as internal data. Of course, the reverse can also be true, i.e., when the second selection control signal is a low level 0, the actuator's output data can be selected as external data; when the second selection control signal is a high level 1, the actuator's output data can be selected as internal data.

[0049] The external data may be external data output by the entire neural network. In some embodiments, the external data may be input data input to the next executor.

[0050] The executor includes a custom hardware circuit corresponding to at least one layer of the neural network. Specifically, the executor includes a custom circuit for hardware implementation of at least one layer of the neural network. When hardware implementation of a neural network, a single layer of the neural network can be hardwareized as an independent functional module, or several layers of the neural network can be hardwareized as independent functional modules. The function of the executor can be determined by the structure of the neural network. For example, for a convolution operation, the executor's function includes convolution operations, which involve multiplication, addition, and other operations.

[0051] The number of actuators located between the input data selector and the output data selector can be multiple, and these multiple actuators can be connected in the order of their layer structure in the neural network. The actuators located between the input data selector and the output data selector can be actuators that need to be reused. In this application, data can be input in the form of an input stream, that is, the input data is continuously input in the form of small data packets in a certain order and processed immediately. An input data selector is used at the input to select whether the data source is an input stream from the inside or outside of the neural network hardware circuit. After passing through the actuator, the output data obtained is output through the output data selector, and the data is selected to be input into the cache or output in the form of an output stream. The input of the input data selector and the output of the output data selector are controlled by selecting a control signal.

[0052] The neural network hardware circuit using the above structure can realize multiple calls to repeated hardware resources (executors) by controlling the input data source of the input data selector and the output direction of the output data selector, thereby achieving the purpose of reducing the circuit area and power consumption when the neural network is hardwareized.

[0053] In some embodiments, as Figure 2 As shown, the neural network hardware circuit also includes a cache, which is connected to the input data selector and the output data selector. The cache is used to cache internal data. The output data selector can select to write internal data to the cache, and the input data selector can select to read internal data from the cache.

[0054] The cache can be any device with a cache function. In some possible implementations, the cache can include a FIFO memory. A FIFO memory is a first-in, first-out dual-port buffer, where the first data entered is the first data removed. One port is the memory input port, and the other port is the memory output port.

[0055] In order to facilitate the control of the above-mentioned input data selector and output data selector, a possible implementation method is as follows: Figure 3 As shown, the neural network hardware circuit further includes: a state machine, the state machine is connected to the input data selector and the output data selector respectively. The state machine is used to send a first selection control signal to the input data selector and a second selection control signal to the output data selector.

[0056] In some possible implementations, when the neural network hardware circuit includes: an input data selector, an executor, an output data selector, a cache, and a state machine, the states of the state machine may include: a pre-start phase, a cache write phase, a cache balance phase, a cache clear phase, and an end phase.

[0057] In the pre-start phase, the input data selector selects external input data as the input data source, and the output data selector has no output.

[0058] Cache writing phase: The input data selector selects external input data as the input data source, and the output data selector selects the output data of the executor as internal data to be written into the cache.

[0059] Cache balancing phase: The input data selector selects internal data as the input data source, and the output data selector selects the output data of the executor as internal data to be written into the cache.

[0060] Cache clearing phase: The input data selector selects internal data as the input data source, and the output data selector selects the output data of the executor as the external data output.

[0061] End stage: The input data selector has no input, and the output data selector selects to output the output data of the actuator as external data.

[0062] In some possible implementations, the state machine is also connected to the cache and the executor. In this case, the schematic diagram of the neural network hardware circuit can be as follows: Figure 4 The switching of the above five stages can be controlled by the state machine, as shown in the schematic diagram. Figure 5 As shown. The initial state of the state machine is the pre-start phase. If the state machine receives a first signal from the executor (such as out_valid=1), it switches to the cache write phase, where the first signal indicates that the data in the output path of the executor is valid. Next, if the state machine receives an indication signal from the cache (such as fifo_full=1), it switches to the cache balancing phase, where when the number of bytes of data written to the cache reaches a preset number of bytes (configurable), the cache sends the indication signal to the state machine. Subsequently, if the state machine receives a task completion signal from the executor (such as loop_finish=1), it switches to the cache clear phase. Subsequently, if the state machine receives an empty signal from the cache (such as fifo_empty=1), it switches to the end phase. Finally, if the state machine receives a second signal from the executor (such as out_valid=0), it switches to the pre-start phase, where the second signal indicates that there is no data in the output path of the executor. When the executor needs to transmit data, it may send a first signal to the state machine. When the executor does not need to transmit data, it may send a second signal to the state machine.

[0063] In order to better illustrate the above situation of multiplexing executors, the following Figure 6 The example shown is used for illustration. Figure 6 For a six-layer neural network model, every two layers form a repeating structure, referred to here as a basic multiplexing unit. For this basic multiplexing unit, two executors are instantiated, one for each model layer. Then, using time-division multiplexing, these two executors are repeatedly called by controlling the input and output data selectors. After three calls, the complete six-layer network model is processed.

[0064] like Figure 7 As shown in FIG, the workflow of the neural network hardware circuit in this example is divided into five stages, namely, the pre-start stage, the cache write stage, the cache balance stage, the cache clearing stage, and the end stage. Figure 7In the table, each row represents the working status of each executor. For example, in the first row, executor 0 completes the processing of model layer 0, model layer 2, and model layer 4 in chronological order; in the second row, executor 2 completes the processing of model layer 1, model layer 3, and model layer 5 in chronological order. The specific process of the five stages is as follows:

[0065] 1. The pre-start phase, from the time the external input stream reaches the input data selector until the last executor outputs its first data packet but before it is sent to the output data selector, is called the pre-start phase. During this phase, the input data selector permanently selects channel 0, which represents the external input stream; the output data selector has no output.

[0066] 2. The cache write phase occurs from the end of the pre-start phase until the first executor completes its first model layer processing. During this phase, the last executor has already begun outputting data, while the first executor still needs to receive data from the input data stream and cannot reuse it. Therefore, the output data is temporarily stored in the cache. The amount of output data during this phase determines the required cache capacity. During this phase, the input data selector always selects channel 0, representing the external input stream; the output data selector selects channel 1, representing the write cache.

[0067] 3. After the first executor completes its first model layer processing, the external input data has been fully processed, and the cache balance phase begins. During this phase, the cache continuously writes the output of the last executor and continuously reads data for input to the first executor. During this phase, the cache writes and reads are balanced. During this phase, the input data selector always selects path 1, reading data from the cache; the output data selector selects path 1, writing data to the cache.

[0068] 4. After the last executor begins executing the last layer of the model and produces output data, the cache flush phase begins. During this phase, the output of the last executor is not stored in the cache but directly outputs to the external system. Data continues to be read from the cache to supply data to the first executor. During this phase, the input data selector always selects path 1, which reads data from the cache; the output data selector selects path 0, which is the external output stream.

[0069] 5. When the first executor completes all processing tasks, it enters the final phase. During this phase, the cache is cleared and the executor ceases operations, performing only calculations on the remaining data in the executor. This phase ends when all tasks of the last executor are processed, and the neural network model completes its processing tasks. During this phase, the input data selector has no inputs, and the output data selector selects path 0, which represents the external output stream.

[0070] In a possible implementation, when there are multiple actuators between the input data selector and the output data selector, such as Figure 8 As shown, the neural network hardware circuit may further include a first switch (e.g., represented by S1) and a second switch (e.g., represented by S2). The output of the input data selector is connected to the input of each actuator via the first switch, and the input of the output data selector is connected to the output of each actuator via the second switch. The input or output of each actuator can be selected by controlling the state (closed or open) of the first switch and / or the second switch to accommodate various complex scenarios.

[0071] To facilitate control of the first and second switches, the neural network hardware circuit may further include a control module. The control module is configured to control the state (closed or open) of the first and / or second switches according to the configured switch control sequence to select the input or output of each actuator. In some possible implementations, the control module may be the state machine described above. In other possible implementations, the control module and the state machine may be independent modules.

[0072] The first switch and the second switch provided in the embodiment of the present application can be transistor switches, including but not limited to bipolar junction transistor (BJT) switches, MOSFET (Metal-Oxide-Semiconductor Field-Effect Transistor) switches, digital logic transistor switches, Darlington transistor switches, insulated gate bipolar transistors (IGBT), etc.

[0073] The embodiment of the present application also provides a data selection method, which can be applied to the above-mentioned neural network hardware circuit. Figure 9 The data selection method provided in the embodiment of the present application is described.

[0074] S1: In the pre-start phase, the input data selector is controlled to select external input data as an input data source.

[0075] S2: In the cache writing phase, the input data selector is controlled to select external input data as the input data source, and the output data selector is controlled to select the output data of the executor as internal data to be written into the cache.

[0076] S3: In the cache balancing stage, the input data selector is controlled to select internal data as the input data source, and the output data selector is controlled to select the output data of the executor as internal data to be written into the cache.

[0077] S4: In the cache clearing stage, the input data selector is controlled to select internal data as the input data source, and the output data selector is controlled to select the output data of the executor as the external data for output.

[0078] S5: At the end stage, the output data selector is controlled to select the output data of the actuator to be output as external data.

[0079] In one possible implementation, the control of the neural network hardware circuit to select data according to the above steps can be performed by an external control device. Alternatively, the control can be performed by a state machine included in the neural network hardware circuit. The control device includes, but is not limited to, a controller, a control device, or a module or unit with control functions.

[0080] If the neural network hardware circuit includes a state machine, wherein the initial state of the state machine is the pre-startup stage; then the above method further includes: the state machine switches to the cache write stage when receiving a first signal from the executor; the state machine switches to the cache balancing stage when receiving an indication signal from the cache; the state machine switches to the cache clearing stage when receiving a task completion signal from the executor; the state machine switches to the end stage when receiving an empty signal from the cache; and the state machine switches to the pre-startup stage when receiving a second signal from the executor.

[0081] The implementation principle and technical effects of the method embodiment are the same as those of the aforementioned neural network hardware circuit embodiment. For the sake of brief description, for matters not mentioned in the repeated embodiment part, reference can be made to the corresponding content in the aforementioned neural network hardware circuit embodiment.

[0082] An embodiment of the present application also provides a chip, which may include the above-mentioned neural network hardware circuit. The chip may include a GPU (Graphics Processing Unit), a CPU (Central Processing Unit), AI (Artificial Intelligence), an NPU (Neural Network Processing Unit), an ISP (Image Signal Processor), a DPU (Display Processing Unit), a VPU (Video Processing Unit) or a DSP (Digital Signal Processor) data processing core, etc., or the chip may also be a processor chip used in some scenarios such as large-scale data computing.

[0083] The neural network hardware circuit provided by the chip embodiment has the same implementation principle and technical effects as the aforementioned neural network hardware circuit embodiment. For the sake of brief description, for matters not mentioned in the chip embodiment, reference can be made to the corresponding content in the aforementioned neural network hardware circuit embodiment.

[0084] The present application also provides an electronic device, such as Figure 10 As shown, Figure 10 The block diagram of the structure of an electronic device provided by an embodiment of the present application is shown. The electronic device includes: a transceiver, a memory, a communication bus, and a processor. The processor can be the chip mentioned above.

[0085] The transceiver, the memory, and the processor are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these elements can be electrically connected to each other via one or more communication buses or signal lines. The transceiver is used to send and receive data. The memory is used to store computer programs, wherein the computer program includes at least one software function module that can be stored in the memory in the form of software or firmware or solidified in the operating system (OS) of the electronic device. The processor is used to execute the software function module or computer program stored in the memory. The memory can be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0086] The processor may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may include a GPU (Graphics Processing Unit), a CPU (Central Processing Unit), an AI (Artificial Intelligence), an NPU (Neural Network Processing Unit), an ISP (Image Signal Processor), a DPU (Display Processing Unit), a VPU (Video Processing Unit), a DSP (Digital Signal Processor) data processing core, etc., or it may be a processor chip used in some scenarios such as large-scale data computing. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. Alternatively, the processor may be any conventional processor, etc.

[0087] Among them, the above-mentioned electronic devices include but are not limited to mobile phones, tablets, notebooks, computers, etc.

[0088] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referenced to each other.

[0089] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0090] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0091] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A neural network hardware circuit, characterized in that: include: An input data selector, an executor, and an output data selector, wherein the executor is located between the input data selector and the output data selector; The input data selector is configured to select external input data or internal data of the neural network hardware circuit as an input data source according to a first selection control signal; The actuator includes a customized hardware circuit corresponding to at least one layer in the neural network; The output data selector is used to select, according to a second selection control signal, whether to output the output data of the actuator as external data or as the internal data.

2. The neural network hardware circuit according to claim 1, characterized in that: The neural network hardware circuit further includes: a state machine, wherein the state machine is connected to the input data selector and the output data selector respectively; The state machine is configured to send a first selection control signal to the input data selector and send a second selection control signal to the output data selector.

3. The neural network hardware circuit according to claim 2, characterized in that: The neural network hardware circuit further includes: a cache; the states of the state machine include: a pre-start phase, a cache write phase, a cache balance phase, a cache clear phase, and an end phase; Wherein, in the pre-start phase, the input data selector selects external input data as the input data source, and the output data selector has no output; In the cache writing stage, the input data selector selects external input data as the input data source, and the output data selector selects the output data of the executor to be written into the cache as internal data; In the cache balancing stage, the input data selector selects internal data as the input data source, and the output data selector selects to write the output data of the executor as internal data into the cache; In the cache clearing stage, the input data selector selects internal data as the input data source, and the output data selector selects the output data of the executor as the external data output; In the end stage, the input data selector has no input, and the output data selector selects to output the output data of the actuator as external data.

4. The neural network hardware circuit according to claim 3, characterized in that: The state machine is also connected to the cache and the executor; The initial state of the state machine is the pre-start phase; The state machine switches to the cache write phase when receiving the first signal from the executor; The state machine switches to the cache balancing phase when receiving an indication signal from the cache; The state machine switches to the cache clearing phase when receiving a task completion signal from the executor; When the state machine receives an empty signal from the cache, it switches to the end phase; The state machine switches to the pre-start phase when receiving the second signal from the actuator.

5. The neural network hardware circuit according to claim 1, characterized in that: The neural network hardware circuit further includes: a cache, wherein the cache is connected to the input data selector and the output data selector respectively; The cache is used to cache the internal data.

6. The neural network hardware circuit according to claim 5, characterized in that: The buffer memory includes a FIFO memory.

7. The neural network hardware circuit according to any one of claims 1 to 6, characterized in that: There are multiple actuators, and the neural network hardware circuit further includes a first switch and a second switch; The output terminal of the input data selector is connected to the input terminal of each of the actuators through a first switch, and the input terminal of the output data selector is connected to the output terminal of each of the actuators through a second switch.

8. The neural network hardware circuit according to claim 7, characterized in that: The first switch and the second switch are both transistor switches, and the transistor switch includes: one of a BJT switch, an IGBT switch, a MOS tube switch, and a Darlington transistor switch.

9. The neural network hardware circuit according to any one of claims 1 to 6, characterized in that: The neural network corresponding to the neural network hardware circuit includes: a feedforward neural network, a convolutional neural network, a recurrent neural network, a long short-term memory network, a generative adversarial network, a Transformer network, an autoencoder, and a deep residual network.

10. A chip, characterized in that: The method comprises the neural network hardware circuit according to any one of claims 1 to 9.

11. An electronic device, characterized in that: Comprising the chip as claimed in claim 10.

12. A data selection method, characterized in that: Applied to the neural network hardware circuit according to any one of claims 1 to 9, the method comprising: In the pre-start phase, controlling the input data selector to select external input data as an input data source; In the cache writing phase, the input data selector is controlled to select external input data as the input data source, and the output data selector is controlled to select the output data of the executor as internal data to be written into the cache; In the cache balancing stage, controlling the input data selector to select internal data as the input data source, and controlling the output data selector to select output data of the executor as internal data to be written into the cache; In the cache clearing phase, controlling the input data selector to select internal data as the input data source, and controlling the output data selector to select output data of the executor as external data output; In the end stage, the output data selector is controlled to select and output the output data of the actuator as external data.

13. The data selection method according to claim 12, characterized in that: If the neural network hardware circuit includes a state machine, wherein the initial state of the state machine is the pre-startup stage; the method further includes: The state machine switches to the cache write phase when receiving the first signal from the executor; The state machine switches to the cache balancing phase when receiving an indication signal from the cache; The state machine switches to the cache clearing phase when receiving a task completion signal from the executor; When the state machine receives an empty signal from the cache, it switches to the end phase; The state machine switches to the pre-start phase when receiving the second signal from the actuator.

Images