Sigmoid activation function circuit based on linear polyline fitting

Abstract

The present application relates to a sigmoid activation function circuit based on linear broken line fitting, comprising: a data processing module, a compensation module, a first addition module and a second addition module; the data processing module, the first addition module and the second addition module are connected in sequence, and the compensation module is connected with the first addition module; the data processing module inputs and processes input data and outputs to the first addition module; the compensation module inputs and processes input data and outputs corresponding offset to the first addition module; the first addition module accumulates the data output by the data processing module and the offset, and outputs the accumulated data; the second addition module processes the accumulated data to obtain an output value. The present application simplifies the circuit module and introduces the rounding calculation based on the broken line fitting method, effectively reduces the operation power consumption and propagation delay under the condition of ensuring the accuracy, and at the same time realizes smaller hardware resource consumption.

Description

Technical Field

[0001] This invention relates to the technical field of digital logic circuits, and more specifically, to a Sigmoid activation function circuit based on linear piecewise linear fitting. Background Technology

[0002] The Sigmoid activation function involves complex exponential and division operations, making it difficult to implement efficiently in hardware. To address this, many simplified digital logic circuit implementation methods have been proposed, such as lookup table (LUT) based methods, linear piecewise linear fitting methods, and nonlinear piecewise linear fitting methods. LUT-based and nonlinear piecewise linear fitting methods require more hardware resources to achieve high accuracy, while linear piecewise linear fitting methods can effectively balance accuracy and hardware consumption. However, currently, linear piecewise linear fitting methods, while achieving high accuracy, result in significant propagation delays and high power consumption in the digital logic circuits. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a Sigmoid activation function circuit based on linear piecewise linear fitting.

[0004] The technical solution adopted by the present invention to solve its technical problem is: to construct a Sigmoid activation function circuit based on linear piecewise linear fitting, including: a data processing module, a compensation module, a first addition module and a second addition module;

[0005] The data processing module, the first addition module, and the second addition module are connected in sequence, and the compensation module is connected to the first addition module;

[0006] The data processing module receives the input data, processes the input data, and outputs it to the first addition module;

[0007] The compensation module receives the input data, processes the input data, and outputs the corresponding offset to the first addition module;

[0008] The first addition module accumulates the data output by the data processing module and the offset, and then outputs the accumulated data.

[0009] The second addition module performs reduction processing on the accumulated data to obtain the output value.

[0010] In the Sigmoid activation function circuit based on linear piecewise linear fitting described in this invention, the data processing module includes: a preprocessing unit, a shifting unit, and a multiplexing unit;

[0011] The preprocessing unit, the shifting unit, and the multiplexing unit are connected in sequence. The input terminal of the preprocessing unit is connected to the input data, the control terminal of the multiplexing unit is connected to the input data, and the output terminal of the multiplexing unit is connected to the first addition module.

[0012] The preprocessing unit preprocesses the input data and outputs preprocessed data.

[0013] The shifting unit shifts the preprocessed data and outputs the shifted data.

[0014] The multiplexing unit selects the shifted data based on the input data from the control terminal and then outputs the selected data.

[0015] In the Sigmoid activation function circuit based on linear piecewise linear fitting described in this invention, the multiplexing unit includes: a first multiplexer and a second multiplexer;

[0016] The data processing module further includes: an inverting unit;

[0017] The input terminal of the first multiplexer is connected to the output terminal of the shift unit, the output terminal of the first multiplexer is connected to the first input terminal of the first addition module, and the control terminal of the first multiplexer is connected to the control terminal of the second multiplexer and receives the input data.

[0018] The inverting unit is disposed between the input of the second multiplexer and the output of the shifting unit, and the output of the second multiplexer is connected to the second input of the first adder module.

[0019] In the Sigmoid activation function circuit based on linear piecewise linear fitting described in this invention, the inverting unit includes: a first inverter, a second inverter, and a third inverter; the second multiplexer is a five-way multiplexer.

[0020] The input terminals of the first inverter, the second inverter, and the third inverter are all connected to the output terminal of the shift unit. The output terminals of the first inverter, the second inverter, and the third inverter are respectively connected to the first input terminal, the second input terminal, and the fifth input terminal of the second multiplexer. The third and fourth input terminals of the second multiplexer are connected to the output terminal of the shift unit.

[0021] In the Sigmoid activation function circuit based on linear piecewise linear fitting described in this invention, the first addition module includes: a compressor;

[0022] The first input terminal of the compressor is connected to the output terminal of the first multiplexer, the second input terminal of the compressor is connected to the output terminal of the second multiplexer, the third input terminal of the compressor is connected to the output terminal of the compensation module, and the first and second output terminals of the compressor are respectively connected to the second addition module.

[0023] In the Sigmoid activation function circuit based on linear piecewise linear fitting described in this invention, the second addition module is an approximation adder.

[0024] In the Sigmoid activation function circuit based on linear piecewise linear fitting described in this invention, the approximation adder includes: an approximation adder unit and a quasi-adder;

[0025] The first input terminal of the approximation addition unit is connected to the first output terminal of the compressor, the second input terminal of the approximation addition unit is connected to the second output terminal of the compressor, the first output terminal of the approximation addition unit is connected to the output terminal of the Sigmoid activation function circuit, the second output terminal of the approximation addition unit is connected to the carry input terminal of the quasi-value adder, the first input terminal of the quasi-value adder is connected to the first output terminal of the compressor, the second input terminal of the quasi-value adder is connected to the second output terminal of the compressor, and the output terminal of the quasi-value adder is connected to the output terminal of the Sigmoid activation function circuit.

[0026] In the Sigmoid activation function circuit based on linear piecewise linear fitting described in this invention, the approximation addition unit includes: multiple OR gates and one AND gate;

[0027] The multiple OR gates are independent of each other.

[0028] In the Sigmoid activation function circuit based on linear piecewise linear fitting described in this invention, the approximation addition unit includes: multiple buffers;

[0029] The multiple buffers are independent of each other.

[0030] In the Sigmoid activation function circuit based on linear piecewise linear fitting described in this invention, the compensation module performs compensation processing on the first five bits of the input data to obtain the offset.

[0031] The Sigmoid activation function circuit based on linear piecewise linear fitting of the present invention has the following beneficial effects: It includes a data processing module, a compensation module, a first addition module, and a second addition module; the data processing module, the first addition module, and the second addition module are connected sequentially, and the compensation module is connected to the first addition module; the data processing module receives input data, processes the input data, and outputs it to the first addition module; the compensation module receives input data, processes the input data, and outputs the corresponding offset to the first addition module; the first addition module accumulates the data output by the data processing module and the offset, and outputs the accumulated data; the second addition module performs a reduction operation on the accumulated data to obtain the output value. This invention, based on piecewise linear fitting, simplifies the circuit modules and introduces reduction calculations, effectively reducing computational power consumption and propagation delay while ensuring accuracy, and simultaneously achieving lower hardware resource consumption. Attached Figure Description

[0032] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0033] Figure 1 This is a schematic diagram of the circuit structure of the Sigmoid activation function circuit based on linear piecewise linear fitting provided in an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of the structure of an embodiment of the approximation addition unit provided by the present invention;

[0035] Figure 3 This is a schematic diagram of the structure of Embodiment 2 of the approximation addition unit provided by the present invention;

[0036] Figure 4 This is a segmented schematic diagram of the Sigmoid function provided by the present invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] To address the issues of high propagation delay and high power consumption in existing linear piecewise linear fitting methods while maintaining high accuracy, this invention provides a Sigmoid activation function circuit based on linear piecewise linear fitting, which is a digital logic circuit.

[0039] Specifically, the sigmoid activation function (Sigmoid(x)) outputs almost 1 when x > 8, and almost 0 when x < -8. Therefore, this invention mainly focuses on fitting the sigmoid activation function for x ∈ (-8, 8).

[0040] Simulation tests using MATLAB software show that dividing x∈(-8,8) into 18 segments effectively balances fitting accuracy and hardware resource consumption. The specific segmentation is as follows: first, (-8,8) is divided into 16 equal segments of length 1. However, because the derivative changes rapidly in the regions (-1,-2) and (1,2), meaning the curvature of the original function is large, fitting with only a straight line segment would introduce significant errors. Therefore, a broken line segment is added to these two regions for fitting. The segmentation of the Sigmoid function is as follows: Figure 4 As shown by the dashed line in the image.

[0041] Furthermore, MATLAB software is used to fit each sigmoid segment to obtain the corresponding linear analytical parameters, including the slope k and offset b. However, these values ​​are complex decimal numbers, which are not very user-friendly for efficient digital logic circuits. Therefore, in this embodiment of the invention, a linear combination of multiple powers of 2 is used to approximate each parameter. To ensure better implementation of the hardware circuit, it is stipulated that the slope k consists of only one or two powers of 2, i.e., k = 2. n Or k=2 n ±2 m Once the value of k is determined, a corresponding value of b can be found that minimizes the absolute error between the approximate Sigmoid value and the quasi-Sigmoid value.

[0042] The piecewise linear fitting method transforms a function that originally involved complex exponential and division operations into a simple linear function operation, i.e., k*x + b0. As analyzed earlier, k is a linear combination of powers of 2. Therefore, the hardware implementation of k*x can be obtained by shifting x and then performing corresponding addition and subtraction operations; that is, replacing complex multiplication operations with simple and easily implemented shift and addition / subtraction operations. This expression can also be further modified as follows:

[0043] Sigmoid(x) ≈k*x+b0 = k*(x+c)+b (1);

[0044] Where (x+c)∈[0,1), and c is a positive integer. Through this transformation, we can obtain that only the decimal part of the input data x is involved in the calculation, while the integer part is not involved. The integer part only determines the value of k.

[0045] k*(x+c)+b=2 n ±2 m *(x+c)+b

[0046] =2 n *(x+c)±2 m *(x+c)+b

[0047] = var_1±var_2+b (2);

[0048] Where var_1 = 2 n *(x+c), var_2=2 m *(x+c), where b is the offset.

[0049] Therefore, the final calculation of the sigmoid function becomes the sum of three numbers: var_1, var_2, and b.

[0050] Based on the above analysis, this invention designs a Sigmoid activation function circuit based on linear piecewise linear fitting. The specific circuit structure of this Sigmoid activation function circuit is described in reference [reference needed]. Figure 1 .

[0051] Specifically, such as Figure 1 As shown, the Sigmoid activation function circuit based on linear piecewise linear fitting includes: a data processing module 10, a compensation module 20, a first addition module 30, and a second addition module 40.

[0052] The data processing module 10, the first addition module 30, and the second addition module 40 are connected in sequence, and the compensation module 20 is connected to the first addition module 30.

[0053] The data processing module 10 receives input data, processes the input data, and outputs it to the first addition module 30; the compensation module 20 receives input data, processes the input data, and outputs the corresponding offset to the first addition module 30; the first addition module 30 accumulates the data output by the data processing module 10 and the offset, and outputs the accumulated data; the second addition module 40 performs simplification on the accumulated data to obtain the output value.

[0054] Optionally, in this embodiment of the invention, the data processing module 10 includes: a preprocessing unit 11, a shifting unit 12, and a multiplexing unit.

[0055] The preprocessing unit 11, the shifting unit 12, and the multiplexing unit are connected in sequence. The input terminal of the preprocessing unit 11 is connected to the input data, the control terminal of the multiplexing unit is connected to the input data, and the output terminal of the multiplexing unit is connected to the first addition module 30.

[0056] The preprocessing unit 11 preprocesses the input data and then outputs the preprocessed data.

[0057] Specifically, such as Figure 1As shown, x is the input data and y is the output value. The input data x is 16-bit binary data. After receiving the input data x, the preprocessing unit 11 replaces the first 5 bits with 0 and combines them with the last 11 bits of the input data x to form a new 16-bit binary number. This new 16-bit binary data is the preprocessed data.

[0058] After shifting the preprocessed data, shift unit 12 outputs the shifted data.

[0059] Specifically, the shift unit 12 is used to perform left / right shift processing on the new 16-bit binary data output from the preprocessing unit 11, and then output it to the first adder module 30. The number of bits for left / right shift is set by the specific application.

[0060] The multiplexing unit selects the shifted data based on the input data from the control terminal and then outputs the selected data.

[0061] Optionally, in embodiments of the present invention, such as Figure 1 As shown, the multiplexing unit includes a first multiplexer 131 and a second multiplexer 132. Furthermore, the data processing module 10 also includes an inverting unit.

[0062] The input terminal of the first multiplexer 131 is connected to the output terminal of the shift unit 12, the output terminal of the first multiplexer 131 is connected to the first input terminal of the first adder module 30, and the control terminal of the first multiplexer 131 is connected to the control terminal of the second multiplexer 132 and receives input data. The inverting unit is disposed between the input terminal of the second multiplexer 132 and the output terminal of the shift unit 12, and the output terminal of the second multiplexer 132 is connected to the second input terminal of the first adder module 30.

[0063] In one specific embodiment, the inverting unit includes: a first inverter A1, a second inverter A2, and a third inverter A3. The second multiplexer 132 is a five-way selector.

[0064] like Figure 1 As shown, the input terminals of the first inverter A1, the second inverter A2, and the third inverter A3 are all connected to the output terminal of the shift unit 12. The output terminals of the first inverter A1, the second inverter A2, and the third inverter A3 are respectively connected to the first input terminal, the second input terminal, and the fifth input terminal of the second multiplexer 132. The third and fourth input terminals of the second multiplexer 132 are connected to the output terminal of the shift unit 12.

[0065] like Figure 1As shown, the first multiplexer 131 is a six-way multiplexer. The first input of the first multiplexer 131 is the binary number (1 / 16X) shifted 4 bits to the right by the shift unit 12; the second input is the binary number (1 / 2X) shifted 1 bit to the right by the shift unit 12; the third input is the data 0; the fourth input is the unshifted binary number (1X); the fifth input is the binary number (2X) shifted 1 bit to the left by the shift unit 12; and the sixth input is the binary number (4X) shifted 2 bits to the left by the shift unit 12.

[0066] like Figure 1 As shown, the second multiplexer 132 is a five-way multiplexer. The first input of the second multiplexer 132 is a binary number (1X) that has not been shifted and is then inverted by the first inverter A1. The second input of the second multiplexer 132 is a binary number (1 / 4X) that has been shifted 2 bits to the right by the shift unit 12 and then inverted by the second inverter A2. The third input of the second multiplexer 132 is a binary number (1 / 8X) that has been shifted 3 bits to the right by the shift unit 12. The fourth input of the second multiplexer 132 is a binary number (1 / 32X) that has been shifted 5 bits to the right by the shift unit 12. The fifth input of the second multiplexer 132 is a binary number that has been shifted 5 bits to the right by the shift unit 12 and then inverted by the third inverter A3.

[0067] like Figure 1 As shown, the data output by the first multiplexer 131 is var_1, and the data output by the second multiplexer 132 is var_2. var_1 is input to the first input terminal of the first adder module 30, and var_2 is input to the second input terminal of the first adder module 30. The control terminals of the first multiplexer 131 and the second multiplexer 132 are input to bits 11 to 15 of the input data x; that is, the first multiplexer 131 and the second multiplexer 132 select based on bits 11 to 15 of the input data x.

[0068] The compensation module 20 performs compensation processing on the first five bits of the input data to obtain the offset. Specifically, after compensating the first five bits of the input data, the compensation module 20 obtains the corresponding offset, such as... Figure 1 Bias in equation (2). The offset is b in equation (2).

[0069] Optionally, in this embodiment of the invention, the first addition module 30 includes a compressor.

[0070] The first input terminal of the compressor is connected to the output terminal of the first multiplexer 131, the second input terminal of the compressor is connected to the output terminal of the second multiplexer 132, the third input terminal of the compressor is connected to the output terminal of the compensation module 20, and the first and second output terminals of the compressor are respectively connected to the second adder module 40.

[0071] Optionally, in this embodiment of the invention, the compressor can be a 3:2 compressor, which can accumulate var_1, var_2 and b to obtain two values, which are then input into the second addition module 40.

[0072] Optionally, in this embodiment of the invention, the second addition module 40 is an approximation adder. Since the more bits of data an adder has, the longer its propagation delay will be, this embodiment of the invention introduces an approximation adder to reduce the propagation delay.

[0073] The approximation adder includes an approximation adder unit 41 and a quasi-value adder 42.

[0074] The first input terminal of the approximation addition unit 41 is connected to the first output terminal of the compressor, and the second input terminal of the approximation addition unit 41 is connected to the second output terminal of the compressor. The first output terminal of the approximation addition unit 41 (e.g.) Figure 2 In the diagram, s[m-1:0]) is connected to the output of the Sigmoid activation function circuit, and the second output of the approximate addition unit 41 (e.g.) Figure 2 The Cin port in the compressor is connected to the carry input of the quasi-value adder 42. The first input of the quasi-value adder 42 is connected to the first output of the compressor. The second input of the quasi-value adder 42 is connected to the second output of the compressor. The output of the quasi-value adder 42 (i.e., the Cin port in the compressor) is connected to the carry input of the quasi-value adder 42. Figure 2 The port of EA in s[n-1:m] is connected to the output of the Sigmoid activation function circuit.

[0075] Optionally, in a preferred embodiment, the approximation addition unit 41 includes: a plurality of OR gates and an AND gate. The plurality of OR gates are independent of each other.

[0076] Specifically, such as Figure 2As shown, an n-bit reduction adder consists of two parts: a reduction adder unit 41 composed of m OR gates and one AND gate, and a (nm) quasi-adder 42 (EA). The result of the lower m bits is calculated by the m OR gates; the AND gate is used to calculate the AND result of the (nm)-th bit, where the result of the AND gate is used as the carry for the next bit; the (nm)-bit quasi-adder 42 is used to calculate the sum of the higher (nm) bits. Since the m-bit reduction adder unit 41 is composed of OR gates, the bits are independent of each other and there is no cascading relationship. Therefore, the propagation delay of the entire reduction adder is only determined by the quasi-adder 42, which can reduce the propagation delay of the entire reduction adder and also reduce the corresponding power consumption.

[0077] Alternatively, in another preferred embodiment, the approximate addition unit 41 includes a plurality of buffers, wherein the plurality of buffers are independent of each other.

[0078] Specifically, such as Figure 3 As shown, in this embodiment, an n-bit reduction adder also consists of two parts: a reduction adder unit 41 composed of m buffers, and a (nm) quasi-adder 42 (EA). The lower m-bit result reduction adder unit 41 directly selects the value output by one of the buffers, while the value of the (m-1)th bit of the other addend is used as the carry to the higher bits; the (nm)-bit quasi-adder 42 is used to calculate the sum of the higher (nm) bits. Similarly, since the m-bit reduction adder unit 41 is composed of buffers, and the bits are independent of each other without cascading, the propagation delay of the entire reduction adder is only determined by the quasi-adder 42, thus reducing the propagation delay of the entire reduction adder and also reducing the corresponding power consumption.

[0079] This invention enables sigmoid function operations with an average absolute error of only 0.0012, as demonstrated in experiments. Furthermore, it achieves a shorter propagation delay for the sigmoid activation function digital logic circuit, composed entirely of combinational circuits, reaching only 0.9 ns at a 90nm process. A single sigmoid operation at the 90nm process requires only 326.52 μW, resulting in significantly lower power consumption. Simultaneously, the chip implementation is more efficient, requiring less resource consumption and a smaller chip area, thus achieving the goal of minimizing hardware resource consumption.

[0080] The Sigmoid activation function circuit based on linear piecewise linear fitting of the present invention can be applied to logic computing such as field-programmable logic arrays (FPGAs), and can also be applied to the field of emotion (such as happiness, sadness, and grief) recognition, or to the field of artificial intelligence.

[0081] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0082] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0083] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0084] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They do not limit the scope of protection of the present invention. All equivalent changes and modifications made within the scope of the claims of the present invention should fall within the scope of the claims of the present invention.

Claims

1. A Sigmoid activation function circuit based on linear piecewise fitting, characterized by, The application relates to a data processing method and device. The data processing module, the first adding module and the second adding module are sequentially connected, and the compensation module is connected with the first adding module; The data processing module accesses input data, processes the input data and outputs the processed data to the first adding module; The compensation module accesses the input data, processes the input data and outputs corresponding offset to the first adding module; The first adding module accumulatively processes the data output by the data processing module and the offset, and outputs accumulated data; The first adding module comprises a compressor; The second adding module processes the accumulated data to obtain an output value; The second adding module is a rounding adder; The rounding adder comprises a rounding adding unit and a quasi-value adder; The first input end of the rounding adding unit is connected with the first output end of the compressor, the second input end of the rounding adding unit is connected with the second output end of the compressor, the first output end of the rounding adding unit is connected with the output end of a Sigmoid activation function circuit, the second output end of the rounding adding unit is connected with the carry input end of the quasi-value adder, the first input end of the quasi-value adder is connected with the first output end of the compressor, the second input end of the quasi-value adder is connected with the second output end of the compressor, and the output end of the quasi-value adder is connected with the output end of the Sigmoid activation function circuit. The data processing module comprises a preprocessing unit, a shift unit and a multiplexer; 2. The linear piecewise fitting based Sigmoid activation function circuit according to claim 1, wherein, The preprocessing unit, the shift unit and the multiplexer are sequentially connected, the input end of the preprocessing unit accesses the input data, the control end of the multiplexer accesses the input data, and the output end of the multiplexer is connected with the first adding module; The preprocessing unit processes the input data to output preprocessed data; The shift unit shifts the preprocessed data to output the shifted data; The multiplexer selects the shifted data according to the input data accessed by the control end to output the selected data. The multiplexer comprises a first multiplexer and a second multiplexer; 3. The linear piecewise fitting based Sigmoid activation function circuit of claim 2, wherein, The data processing module further comprises an inverter unit; The input end of the first multiplexer is connected with the output end of the shift unit, the output end of the first multiplexer is connected with the first input end of the first adding module, the control end of the first multiplexer is connected with the control end of the second multiplexer and accesses the input data; The inverter unit is arranged between the input end of the second multiplexer and the output end of the shift unit, and the output end of the second multiplexer is connected with the second input end of the first adding module. The inverter unit comprises a first inverter, a second inverter and a third inverter, and the second multiplexer is a five-way selector.

4. The linear piecewise fitting based Sigmoid activation function circuit of claim 3, wherein, ​ The input end of the first inverter, the input end of the second inverter and the input end of the third inverter are connected to the output end of the shift unit, the output end of the first inverter, the output end of the second inverter and the output end of the third inverter are connected to the first input end of the second multiplexer, the second input end of the second multiplexer and the fifth input end of the second multiplexer respectively, and the third input end and the fourth input end of the second multiplexer are connected to the output end of the shift unit.

5. The linear piecewise fitting based Sigmoid activation function circuit of claim 3, wherein, The first input end of the compressor is connected to the output end of the first multiplexer, the second input end of the compressor is connected to the output end of the second multiplexer, the third input end of the compressor is connected to the output end of the compensation module, and the first output end and the second output end of the compressor are connected to the second adding module respectively.

6. The linear piecewise fitting based Sigmoid activation function circuit of claim 1, wherein, The approximate value adding unit comprises a plurality of OR gates and an AND gate. The plurality of OR gates are independent of each other.

7. The linear piecewise fitting based Sigmoid activation function circuit of claim 1, wherein, The approximate value adding unit comprises a plurality of buffers. The plurality of buffers are independent of each other.

8. The linear piecewise fitting based Sigmoid activation function circuit according to any one of claims 1-7, wherein, The compensation module performs compensation processing on the first five data in the input data to obtain the offset.

Images