Activation function processing method and system, storage medium and electronic equipment

Abstract

The invention provides an activation function processing method which comprises the following steps: a control module continuously sends read instructions to an input cache, reads elements in batches and writes the elements into a first module; writing the element into the first cache, and activating a first module ending signal after the element is written into the first cache; sequentially reading elements from the first cache according to the same batch sequence, calling an index unit to calculate an index value, and writing the index value into a second cache according to the same batch sequence; calculating a denominator of an activation function by applying an adder tree and an accumulator, and activating a second module ending signal when accumulation is completed and an accumulated sum is obtained; when a second module ending signal is detected, the index values in the second cache are read according to the same batch sequence, and the output result of the activation function is obtained through calculation according to the accumulated sum. According to the method, the long and short vectors of any number of elements can be processed. The invention further provides an activation function processing system, a computer readable storage medium and electronic equipment, which have the above beneficial effects.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to an activation function processing method, system, storage medium and electronic device. Background Art

[0002] The current activation function is mainly used to process short vectors with a small number of elements, but it is difficult to process long vectors with a large number of elements. It can usually only process vectors with a fixed number of elements (N). Once the hardware has N channels fixed and the number of input vector elements is greater than N, the hardware cannot process it. The more processing channels there are, the larger the area and the higher the cost. Summary of the invention

[0003] The purpose of this application is to provide an activation function processing method, system, storage medium and electronic device, which can realize the processing of input vectors containing any number of elements.

[0004] In order to solve the above technical problems, the present application provides a method for processing an activation function, and the specific technical solution is as follows:

[0005] The control module continuously sends read instructions to the input cache, reads elements of the input vector in batches and writes them into the first module;

[0006] Sending a write instruction to a first cache so as to write the elements into the first cache, and activating a first module end signal after a last batch of elements are written into the first cache;

[0007] When the control module detects the end signal of the first module, the control module sequentially reads elements from the first cache in the same batch order, calls the index unit to calculate the index value, and writes the index value into the second cache in the same batch order;

[0008] Applying the addition tree and the accumulator to calculate the denominator of the activation function, and activating the second module end signal when the last batch of elements are accumulated to obtain the accumulated sum of all elements of the input vector;

[0009] When the control module detects the end signal of the second module, the index value in the second cache is read in the same batch order, and the output result of the activation function is obtained according to the cumulative sum calculation.

[0010] Optionally, the control module continuously sending read instructions to the input buffer includes:

[0011] The control module continuously sends a read address and a read operation enable to the input buffer.

[0012] Optionally, before reading the elements of the input vector in batches and writing them into the first module, the following is further included:

[0013] Determining a channel parallelism; the channel parallelism is based on a bit width configuration of the input buffer;

[0014] Determining the element length of the input vector;

[0015] The maximum number of elements per batch is determined according to the channel parallelism, and the number of batches of the input vectors is determined according to the element length and the channel parallelism.

[0016] Optionally, when reading the elements of the input vector in batches and writing them to the first module, it also includes:

[0017] When the first batch of elements of the input vector are valid, a start signal is pulled high, and when the last batch of elements of the input vector are valid, an end signal is pulled high;

[0018] For each batch of elements, if the elements are valid, the corresponding in-place signal is pulled high; the length of the in-place signal is the same as the channel parallelism.

[0019] Optionally, reading the elements of the input vector in batches and writing them into the first module comprises:

[0020] After the elements of the current batch are written into the first cache, the next batch of elements of the input vector is input into the first module.

[0021] Optionally, when writing the element into the first cache, the method further includes:

[0022] A comparator unit is applied to determine a maximum value among the elements of the current batch; said maximum value is used to calculate an index value in said second module.

[0023] Optionally, reading the elements of the input vector in batches and writing them into the first module comprises:

[0024] writing elements of the input vector in floating point format to the first module;

[0025] Correspondingly, before writing the element into the first cache, the method further includes:

[0026] Converts the element from floating point format to fixed point format.

[0027] The present application also provides an activation function processing system, comprising:

[0028] A control module, configured to continuously send read instructions to the input buffer, read the elements of the input vector in batches and write them into the first module; and to control the second processing module to run when an end signal of the first module is detected; and to control the third processing module to run when an end signal of the second module is detected;

[0029] A first processing module, configured to send a write instruction to a first cache so as to write the elements into the first cache, and to generate a first module end signal after a last batch of elements are written into the first cache;

[0030] A second processing module is used to read elements from the first cache in the same batch order, call the exponential unit to calculate the exponential value, and write the exponential value into the second cache in the same batch order; apply the addition tree and the accumulator to calculate the denominator of the activation function, and generate a second module end signal when the last batch of elements is accumulated to obtain the cumulative sum of all elements of the input vector;

[0031] The third processing module is used to read the exponent values ​​in the second cache in the same batch order, and obtain the output result of the activation function according to the cumulative sum calculation.

[0032] The present application also provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the steps of the above-mentioned method when executed by a processor.

[0033] The present application also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned method when calling the computer program in the memory.

[0034] The present application provides a method for processing an activation function, and the specific technical scheme is as follows: a control module continuously sends read instructions to an input cache, reads elements of an input vector in batches and writes them into a first module; sends write instructions to the first cache so as to write the elements into the first cache, and activates an end signal of the first module after the last batch of elements are written into the first cache; when the control module detects the end signal of the first module, it reads elements from the first cache in sequence in the same batch order, calls an exponential unit to calculate an exponential value, and writes the exponential value into a second cache in the same batch order; an adder tree and an accumulator are used to calculate the denominator of the activation function, and when the last batch of elements is accumulated to obtain the cumulative sum of all elements of the input vector, the end signal of the second module is activated; when the control module detects the end signal of the second module, it reads the exponential value in the second cache in the same batch order, and obtains the output result of the activation function based on the cumulative sum.

[0035] The present application only needs to harden a limited number of data processing channels, such as a maximum parallelism of 2, 4, 8, 16 channels, etc. By introducing the intermediate cache, the first cache and the second cache, and the control module, the input vector is processed in batches. After a certain number of iterations, long and short vectors of any number of elements can be processed. By caching the intermediate results and iteratively calculating the elements of each batch, the calculation of input vectors of arbitrary length can be realized, and it is no longer limited to input vectors of a small number of elements.

[0036] The present application also provides an activation function processing system, a computer-readable storage medium, and an electronic device, which have the above-mentioned beneficial effects and are not described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0038] Figure 1 A flowchart of a method for processing an activation function provided in an embodiment of the present application;

[0039] Figure 2 A processing flow chart of the activation function provided in the embodiment of the present application;

[0040] Figure 3 A timing diagram of the first module provided in an embodiment of the present application;

[0041] Figure 4 A timing diagram of the second module provided in an embodiment of the present application;

[0042] Figure 5 A timing diagram of the third module provided in an embodiment of the present application;

[0043] Figure 6 A schematic diagram of parallel processing provided in an embodiment of the present application;

[0044] Figure 7 A schematic diagram of the structure of an activation function processing system provided in an embodiment of the present application;

[0045] Figure 8 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] The object information involved in this application, including but not limited to the object device information, object personal information, etc., and data, including but not limited to data used for analysis, stored data, displayed data, etc., are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the laws, regulations and standards of relevant countries and regions.

[0048] See also Figure 1 , Figure 1 A flowchart of a method for processing an activation function provided in an embodiment of the present application, the method comprising:

[0049] S101: The control module continuously sends read instructions to the input cache, reads elements of the input vector in batches and writes them into the first module;

[0050] S102: Sending a write instruction to the first cache so as to write the element into the first cache, and activating a first module end signal after the last batch of elements are written into the first cache;

[0051] S103: When the control module detects the end signal of the first module, the control module sequentially reads elements from the first cache in the same batch order, calls the index unit to calculate the index value, and writes the index value into the second cache in the same batch order;

[0052] S104: applying the addition tree and the accumulator to calculate the denominator of the activation function, and activating the second module end signal when the last batch of elements are accumulated to obtain the accumulated sum of all elements of the input vector;

[0053] S105: When the control module detects the end signal of the second module, the index value in the second cache is read in the same batch order, and the output result of the activation function is obtained according to the cumulative sum calculation.

[0054] When an input vector is detected, the control module may send a read instruction to the input cache to read the elements of the input vector in batches. There is no limitation on how to send the read instruction, and a read address and a read operation enable may be continuously sent to the input cache, the read address is used to confirm the address of the input vector in the input cache, and the read operation enable is a control signal for the read operation.

[0055] The batch described in the embodiment of the present application refers to the operation of the elements in the input vector in batches. The number of elements in each batch can be determined according to the channel parallelism, for example, it can be configured as 2, 4, 8, 16, etc. In order to ensure maximum processing efficiency, the number of elements in each batch is usually equal to the channel parallelism. The number of batches can be determined according to the element length and the channel parallelism, which is usually the quotient of the element length and the channel parallelism rounded up.

[0056] The input vector is read from the input buffer to the first module through a read instruction. At this time, the start signal is pulled high when the first batch of elements of the input vector are valid, and the end signal is pulled high when the last batch of elements of the input vector are valid.

[0057] For each batch of elements, if the elements are valid, the corresponding in-place signal is pulled high; the length of the in-place signal is the same as the channel parallelism.

[0058] It can be seen that the start signal and the end signal are both 1 bit, that is, 1 bit, and the number of bits of the in-place signal is the same as the number of elements, and each element has a corresponding in-place signal. After all elements of the input vector output the value of the first module, the end signal is pulled high.

[0059] The elements are input into the first module for comparison to obtain the maximum value of the input vector. Specifically, a comparator can be applied, in which a maximum value obtained by comparing two elements is stored. After all elements of the input vector pass through the comparator, the maximum value of the input vector can be determined.

[0060] In addition, it should be noted that before the elements of the input vector are input to the first module, they are usually in floating point format, such as FP16 format, and the bit width of the corresponding input buffer is 16*N. Half-precision floating point number, namely FP16, is a floating point number representation method used in computer science, which uses 16-bit binary to represent a floating point number. Before the elements are input to the first cache via the comparator, the elements of the input vector can be converted to fixed point format via a data conversion module, such as an FP2FX module (which converts floating point numbers (FP) to fixed point numbers (FX). After all the elements are stored in the first cache in the same batch order, the end signal of the first module can be activated.

[0061] It should be noted that the embodiment of the present application processes the elements of the input vector in batches, and processes them on a first-in-first-out basis in the first module, the second module and the third module, and also follows a first-in-first-out rule during the application of the first cache and the second cache.

[0062] At this time, after the control module detects the end signal of the first module, it starts the operation of the second module. At this time, the elements are read from the first cache in the same batch order, the exponent unit is called to calculate the exponent value, and the exponent value is written to the second cache in the same batch order. At this time, the elements of the input vector are read from the first cache batch by batch, and the maximum value in the comparator above is subtracted when the element passes through the exponent unit, and the exponent value is calculated. The exponent value corresponding to each element is written to the second cache in the same batch order, and the sum is calculated through the addition tree module.

[0063] The addition tree and the accumulator are used to calculate the denominator of the activation function, and when the last batch of elements are accumulated to obtain the cumulative sum of all elements of the input vector, the second module end signal is activated.

[0064] Then, the partial sum of a batch of elements is input into the accumulator to calculate the denominator of the activation function calculation formula. After the last batch of elements of the input vector are accumulated, the end signal of the second module is activated when the cumulative sum of all elements is obtained, and the cumulative sum of all elements of the input vector is obtained.

[0065] It should be noted that the first module end signal and the second module end signal described in the embodiment of the present application are both used to indicate the end of the processing process of the corresponding module. Specifically, they can be generated in real time, or the activation of the signal can be realized by pulling up the corresponding signal.

[0066] After detecting the end signal of the second module, the control module further controls the operation of the third module, reads the index value in the second cache in the same batch order, and the third module is used to calculate the output result of the activation function according to the cumulative sum. Finally, the output result can be input into the output cache.

[0067] The present application only needs to harden a limited number of data processing channels, such as a maximum parallelism of 2, 4, 8, 16 channels, etc. By introducing the intermediate cache, the first cache and the second cache, and the control module, the input vector is processed in batches. After a certain number of iterations, long and short vectors of any number of elements can be processed. By caching the intermediate results and iteratively calculating the elements of each batch, the calculation of input vectors of arbitrary length can be realized, and it is no longer limited to input vectors of a small number of elements.

[0068] In order to better demonstrate the processing of the activation function disclosed in this application, Figure 2 , Figure 2 The processing flow chart of the activation function provided in the embodiment of the present application is as follows:

[0069] The workflow of the first module is as follows:

[0070] Step 1: At the beginning, the control module continuously sends the read address s1_raddr and read enable s1_ren to the input buffer, and reads out the elements of the input vector v1 batch by batch and inputs them into the first module. Assume that each batch has N elements, the input elements are in FP16 format, and the input buffer width is 16*N, where N is the channel parallelism that can be configured according to the input buffer width, which can be configured to 2, 4, 8, or 16.

[0071] When the first batch of elements of an input vector are valid, the start signal s1_first (1 bit) needs to be pulled high. When the last batch of elements are valid, the first module end signal s1_last (1 bit) needs to be pulled high. When a batch of input elements are valid, the in-place signal s1_valid (N bits) needs to be pulled high. Each vector element has a corresponding 1-bit valid bit of the in-place signal s1_valid.

[0072] Step 2: The elements of the input vector v1 enter the processing of the first module batch by batch. The comparator saves the maximum value of the input elements, and the data conversion module converts the input elements into fixed-point format.

[0073] Step 3: The control module sends a write address s1_wen and a write enable s1_waddr to the first cache to store the fixed-point numbers in step 2 into the first cache batch by batch. After the last batch of elements of the input vector v1 is processed, the s1_done signal is pulled high. At this time, the maximum element Zmax of the input vector v1 has been obtained.

[0074] See also Figure 3 , Figure 3 This is a timing diagram of the first module provided in the embodiment of the present application. Figure 3 and subsequent Figure 4 and Figure 5 In the example, N is 16, which means that 16 bits of valid information are input and output. When the hexadecimal representation is 16'hffff, it means that all 16 elements of a batch are valid. Figure 3 It includes the level information corresponding to the reset level, the first batch of input data in stage one, the last batch of input data in stage one, the first module input valid, the first module input data, the first module output valid, the first module output data, the output maximum value valid and the output maximum value.

[0075] The workflow of the second module is as follows:

[0076] After receiving the s1_done signal, the control module starts the operation of the second module, reads out the fixed-point vector elements Zi (Zin_fx signal) of the first cache batch by batch, subtracts the maximum value Zmax (Zmax_fx signal) when passing through the exponential unit, and then calculates the exponential value e^(Zi-Zmax), that is, the signal exp_out. The exponential value e^(Zi-Zmax) is written into the second cache batch by batch, and summed through the addition tree module at the same time, and then the partial sum of a batch of N elements is input into the accumulator to accumulate and calculate the denominator of the activation function softmax. When the last batch of elements of the input vector v1 is accumulated and the cumulative sum of all elements is obtained, the second module end signal s2_done signal is pulled high to obtain the cumulative sum of all elements of the input vector v1.

[0077] See also Figure 4 , Figure 4 This is a timing diagram of the second module provided in an embodiment of the present application. Figure 4 It includes the reset level, the first batch of input data in stage two, the last batch of input data in stage two, the second module input valid, the second module input data, the input maximum value, the second module output valid, the second module output data, the output sum result valid and the output sum result corresponding to each level information.

[0078] The workflow of the third module is as follows:

[0079] After receiving the s2_done signal, the control module starts the operation of the third module, reads out the exponent value (din_a) of the second cache and the sum (din_b) in Acc batch by batch, inputs them into the third module for division, obtains the output result of the activation function softmax, and writes them into the output cache for storage batch by batch.

[0080] See also Figure 5 , Figure 5 This is a timing diagram of the third module provided in an embodiment of the present application. Figure 5 It includes reset level, third module input validity, input numerator, input denominator, division result validity and level information corresponding to the division result.

[0081] Figure 2 The write address s2_waddr and write enable s2_wen sent to the second cache, as well as the in-position signal s2_valid signal are also included. The write address s3_waddr and write enable s3_wen are used to control the module to write the output result of the activation function softmax into the output cache.

[0082] In addition, when the first module of the input vector v1 is completed, the calculation of the second module begins. At this time, the hardware of the first module is idle, and the elements of the second vector v2 can be input for calculation, so that the pipeline processing of the input vector can be realized. Figure 6 As shown, Figure 6 Schematic diagram of parallel processing provided by the embodiment of the present application. Ideally, at the same time, the three processing modules from the first module to the third module simultaneously calculate three input vectors, input vector v1 is calculated by the third module, vector v2 is calculated by the second module, and vector v3 is calculated by the first module, achieving parallel calculation of up to three vectors.

[0083] See also Figure 7 , Figure 7 A schematic diagram of the structure of an activation function processing system provided in an embodiment of the present application, the system comprising:

[0084] A control module, configured to continuously send read instructions to the input buffer, read the elements of the input vector in batches and write them into the first module; and to control the second processing module to run when an end signal of the first module is detected; and to control the third processing module to run when an end signal of the second module is detected;

[0085] A first processing module, configured to send a write instruction to a first cache so as to write the elements into the first cache, and to generate a first module end signal after a last batch of elements are written into the first cache;

[0086] A second processing module is used to read elements from the first cache in the same batch order, call the exponential unit to calculate the exponential value, and write the exponential value into the second cache in the same batch order; apply the addition tree and the accumulator to calculate the denominator of the activation function, and generate a second module end signal when the last batch of elements is accumulated to obtain the cumulative sum of all elements of the input vector;

[0087] The third processing module is used to read the exponent values ​​in the second cache in the same batch order, and obtain the output result of the activation function according to the cumulative sum calculation.

[0088] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0089] The present application also provides an electronic device, see Figure 8 , a structural diagram of an electronic device provided in an embodiment of the present application, such as Figure 8 As shown, a processor 1410 and a memory 1420 may be included.

[0090] Among them, the processor 1410 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1410 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1410 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1410 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1410 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0091] The memory 1420 may include one or more computer-readable storage media, which may be non-transitory. The memory 1420 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 1420 is at least used to store the following computer program 1421, wherein, after the computer program is loaded and executed by the processor 1410, it can implement the relevant steps in the method performed by the electronic device side disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 1420 may also include an operating system 1422 and data 1423, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 1422 may include Windows, Linux, Android, etc.

[0092] In some embodiments, the electronic device may further include a display screen 1430 , an input / output interface 1440 , a communication interface 1450 , a sensor 1460 , a power source 1470 , and a communication bus 1480 .

[0093] certainly, Figure 8 The structure of the electronic device shown in the parallel processing diagram provided in the embodiment of the present application does not constitute a limitation on the electronic device in the embodiment of the present application. In actual applications, the electronic device may include Figure 8More or fewer components, or a combination of certain components, are shown in the parallel processing schematic diagram provided in the embodiments of the present application.

[0094] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the system provided in the embodiment, since it corresponds to the method provided in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0095] Specific examples are used herein to illustrate the principles and implementation methods of the present application, and the description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

[0096] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A method for processing an activation function, characterized in that: include: The control module continuously sends read instructions to the input cache, reads elements of the input vector in batches and writes them into the first module; Sending a write instruction to a first cache so as to write the elements into the first cache, and activating a first module end signal after a last batch of elements are written into the first cache; When the control module detects the end signal of the first module, the control module sequentially reads elements from the first cache in the same batch order, calls the index unit to calculate the index value, and writes the index value into the second cache in the same batch order; Applying the addition tree and the accumulator to calculate the denominator of the activation function, and activating the second module end signal when the last batch of elements are accumulated to obtain the accumulated sum of all elements of the input vector; When the control module detects the end signal of the second module, the index value in the second cache is read in the same batch order, and the output result of the activation function is obtained according to the cumulative sum calculation.

2. The calculation method according to claim 1, characterized in that: The control module continuously sends read instructions to the input buffer including: The control module continuously sends a read address and a read operation enable to the input buffer.

3. The calculation method according to claim 1, characterized in that: Before reading the elements of the input vector in batches and writing them to the first module, it also includes: Determining a channel parallelism; the channel parallelism is based on a bit width configuration of the input buffer; Determining the element length of the input vector; The maximum number of elements per batch is determined according to the channel parallelism, and the number of batches of the input vectors is determined according to the element length and the channel parallelism.

4. The calculation method according to claim 3, characterized in that: When reading the elements of the input vector in batches and writing to the first module, it also includes: When the first batch of elements of the input vector are valid, a start signal is pulled high, and when the last batch of elements of the input vector are valid, an end signal is pulled high; For each batch of elements, if the elements are valid, the corresponding in-place signal is pulled high; the length of the in-place signal is the same as the channel parallelism.

5. The calculation method according to claim 4, characterized in that: The step of reading the elements of the input vector in batches and writing them into the first module comprises: After the elements of the current batch are written into the first cache, the next batch of elements of the input vector is input into the first module.

6. The calculation method according to claim 1, characterized in that: When writing the element into the first cache, the method further includes: A comparator unit is applied to determine a maximum value among the elements of the current batch; said maximum value is used to calculate an index value in said second module.

7. The calculation method according to claim 1, characterized in that: The step of reading the elements of the input vector in batches and writing them into the first module comprises: writing elements of the input vector in floating point format to the first module; Correspondingly, before writing the element into the first cache, the method further includes: Converts the element from floating point format to fixed point format.

8. A processing system for an activation function, characterized in that: include: A control module, used for continuously sending read instructions to the input cache, reading elements of the input vector in batches and writing them into the first module; and for controlling the second processing module to operate when the first module end signal is detected; and for controlling the third processing module to operate when the second module end signal is detected; A first processing module, configured to send a write instruction to a first cache so as to write the elements into the first cache, and to generate a first module end signal after a last batch of elements are written into the first cache; A second processing module is used to read elements from the first cache in the same batch order, call the exponential unit to calculate the exponential value, and write the exponential value into the second cache in the same batch order; apply the addition tree and the accumulator to calculate the denominator of the activation function, and generate a second module end signal when the last batch of elements is accumulated to obtain the cumulative sum of all elements of the input vector; The third processing module is used to read the exponent values ​​in the second cache in the same batch order, and obtain the output result of the activation function according to the cumulative sum calculation.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the activation function processing method according to any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: It includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps of the activation function processing method according to any one of claims 1 to 7 are implemented.

Images