Multifunctional linear convolution accelerator

Abstract

The invention belongs to the field of DSP signal filtering, and particularly relates to a multifunctional linear convolution accelerator and method, and the method comprises the steps that a configuration sub-module receives a starting signal and a configuration signal through a bus, and a first state machine sub-module queries and analyzes the starting signal and a first configuration signal configured by the configuration sub-module and then sends the starting signal and the configuration signal to a second state machine sub-module; starting and configuring the port module through the second state machine sub-module, correspondingly starting and configuring the memory access module and the operation module through the third state machine sub-module and the fourth state machine sub-module after the control module receives an operation starting signal of the port module, and correspondingly starting and configuring the memory access module and the operation module after the control module receives an ending signal of the port module and the memory access module. A reset signal is sent to the port module, the cache module, the memory access module and the operation module, and meanwhile, an end state is written into the state sub-module to wait for next starting; the high-performance hardware acceleration of the digital signal processing algorithm is realized by multiplexing the access logic and the operation logic.

Description

Technical Field

[0001] The invention belongs to the field of DSP signal filtering, and in particular relates to a multifunctional linear convolution accelerator. Background Art

[0002] Digital signal processing technology is widely used in fields such as image processing and speech processing. Through digital signal processing technology, the characteristics of signals can be extracted and analyzed, which plays a very important role in many engineering fields. A digital signal processor is a special integrated circuit designed to quickly execute digital signal processing algorithms. It has a highly optimized hardware structure and can efficiently process large amounts of data. Linear convolution is the integral of the overlapping product of two discrete sequences after flipping and translation over the overlapping length. Based on linear convolution, digital signal processing algorithms such as FIR, cross-correlation and autocorrelation can be implemented.

[0003] However, in existing digital signal processors, due to limited area budget, only hardware acceleration structures can be designed for some digital signal processing algorithms based on linear convolution, resulting in low hardware acceleration performance of other digital signal processing algorithms based on linear convolution. In addition, existing hardware acceleration structures often lack hardware optimization for multiple batches of data, resulting in low data processing efficiency in batch processing scenarios.

[0004] For example, the patent with authorization announcement number CN113554163B discloses a convolutional neural network accelerator, including: multiple multiplication-addition trees for performing parallel operations on multi-channel feature maps, and a scaling operator and an adder connected in series after each multiplication-addition tree; the multiplication-addition tree is used to perform multiplication-addition operations on quantized weight data and quantized feature data of the feature map of one of the corresponding channels to obtain the multiplication-addition operation results of the feature map of the corresponding channel; the scaling operator is used to perform scaling operations on the multiplication-addition operation results of the feature map of the corresponding channel to obtain the scaling operation results of the feature map of the corresponding channel; the adder is used to perform zero point adjustment on the scaling operation results of the feature map of the corresponding channel; the result output by the adder is used as the quantized convolution result to provide an operation architecture for the quantized convolutional neural network.

[0005] In order to solve the above problems, the present invention provides a multifunctional linear convolution accelerator. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention proposes a multifunctional linear convolution accelerator, comprising: a configuration submodule receives a start signal and a configuration signal through a bus, a first state machine submodule starts and configures a port module through a second state machine submodule after querying and parsing the start signal and the first configuration signal configured by the configuration submodule, and after the control module receives the operation start signal of the port module, a memory access module and an operation module are correspondingly started and configured through a third state machine submodule and a fourth state machine submodule, and after the control module receives the end signal of the port module and the memory access module, a reset signal is sent to the port module, the cache module, the memory access module and the operation module, and an end state is written to the state submodule, waiting for the next start; the present invention realizes high-performance hardware acceleration of digital signal processing algorithms by multiplexing memory access logic and operation logic.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A multifunctional linear convolution accelerator, comprising: a control module, a port module, a cache module, a memory access module, a calculation module and a bus;

[0009] The control module includes a first state machine submodule, a configuration submodule and a state submodule;

[0010] The port module includes a second state machine submodule; the operation module includes a third state machine submodule; the memory access module includes a fourth state machine submodule;

[0011] A first state machine submodule, used to control the startup and configuration of the port module, the cache module, the memory access module and the operation module according to the received startup signal and the first configuration signal;

[0012] The first configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, the number of source data 2 batches, whether the batches of data correspond, whether source data 2 is flipped, and whether source data 2 is conjugated;

[0013] A configuration submodule, used for receiving a start signal and a configuration signal through a bus;

[0014] A status submodule, used for sending the working status of the accelerator through the bus;

[0015] The configuration submodule receives a start signal and a configuration signal through the bus. After querying and parsing the start signal and the first configuration signal configured by the configuration submodule, the first state machine submodule starts and configures the port module through the second state machine submodule. After the control module receives the operation start signal of the port module, the memory access module and the operation module are correspondingly started and configured through the third state machine submodule and the fourth state machine submodule. After the control module receives the end signal of the port module and the memory access module, a reset signal is sent to the port module, the cache module, the memory access module and the operation module, and the end state is written to the state submodule, waiting for the next start.

[0016] Specifically, the port module also includes an input port submodule and an output port submodule;

[0017] A second state machine submodule, used for receiving and analyzing a start signal and a second configuration signal sent by the control module;

[0018] An input port submodule, used for storing source data into a cache module through a bus;

[0019] The output port submodule is used to send the result data in the cache module to the bus through the bus;

[0020] The second configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, and the number of source data 2 batches;

[0021] The workflow of each sub-module of the port module includes:

[0022] The start signal and the second configuration signal sent by the control module are received and analyzed by the second state machine submodule. After analyzing the start signal and the second configuration signal, the second state machine submodule starts and configures the input port submodule and the output port submodule according to the analyzed second configuration signal.

[0023] Specifically, the cache module includes a source data 1 ping area, a source data 1 pong area, a source data 2 area, a result data ping area and a result data pong area; the steps of the cache module performing data caching include:

[0024] A1. If source data 1 does not need ping-pong storage, the input port submodule stores source data 1 in the source data 1 ping-pong area in the cache module through the bus. After source data 1 is stored, source data 2 is stored in the source data 2 area of ​​the cache module through the bus until source data 2 is stored.

[0025] A2. If the source data 1 needs ping-pong storage, the input port submodule stores part of the source data 1 into the source data 1 ping-pong area in the cache module through the bus;

[0026] A3, when the source data 1 ping-pong area is full, the source data 2 is stored in the source data 2 area in the cache module through the bus, and when the source data 2 is stored, the remaining source data 1 is ping-pong stored in the source data 1 area of ​​the cache module through the bus;

[0027] A4. After the source data 2 is stored, the second state machine submodule sends a calculation start signal to the control module. At the same time, the output port submodule ping-pong accesses the result data in the result data area in the cache module and sends it to the bus. After all the result data are sent, the second state machine submodule sends an end signal to the control module, waiting for the reset of the control module.

[0028] Specifically, the source data 1 ping area, source data 1 pong area, source data 2 area, result data ping area and result data pong area are all composed of k SRAMs and have the same data storage method, where k is the operation parallelism in the operation module, and the bit width of the SRAM is twice the bit width of the processed data.

[0029] Specifically, the memory access module also includes a source data 1 distribution submodule, a source data 2 distribution submodule and a result data distribution submodule;

[0030] The fourth state machine submodule is used to receive and parse the start signal and the third configuration signal of the control module. After parsing the start signal and the third configuration signal, the fourth state machine submodule starts and configures the source data 1 distribution submodule, the source data 2 distribution submodule and the result data distribution submodule;

[0031] The third configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, the number of source data 2 batches, whether each batch of data corresponds, and whether source data 2 is flipped.

[0032] Specifically, the steps of performing data access by the memory access module include:

[0033] B1. If the batches of data in the third configuration signal parsed by the fourth state machine submodule do not correspond, each batch of data of source data 1 and each batch of data of source data 2 are accessed and sent in a corresponding manner; if the batches of data in the third configuration signal correspond, each batch of data of source data 1 is accessed and sent in a corresponding manner only with the same batch of data of source data 2;

[0034] B2. If the source data 2 in the third configuration signal parsed by the fourth state machine submodule is not flipped, the sliding window accesses the source data 2 in the cache module in order from the beginning to the end and calculates and sends it; if the source data 2 is flipped, the sliding window accesses the source data 2 in the cache module in reverse order from the end to the beginning and calculates and sends it;

[0035] B3, when the data sent by B1 or B2 to the operation module is completed, the operation result of the operation module is received by the result data distribution submodule and stored in the result data ping-pong area or pong area of ​​the cache module;

[0036] B4. After the operation results are stored, the fourth state machine submodule sends an end signal to the control module and waits for the control module to be reset.

[0037] Specifically, the operation module also includes an input control submodule, an output control submodule, and a multiplication-accumulation tree submodule;

[0038] The third state machine submodule is used to receive and parse the start signal and the fourth configuration signal of the control module. After parsing the start signal and the fourth configuration signal, the third state machine submodule starts and configures the input control submodule, the output control submodule and the multiplication-accumulation tree submodule according to the parsed fourth configuration signal;

[0039] The fourth configuration signal includes whether the source data 2 is conjugated.

[0040] Specifically, the steps of the operation module operation include:

[0041] C1. Set the operation parallelism of the multiplication-accumulation tree in the operation module to k, the input control submodule selects the path between the k input ports and the multiplication-accumulation tree, and the output control submodule selects the path between the multiplication-accumulation tree and the output port;

[0042] C2. If the source data 2 in the fourth configuration signal parsed by the third state machine submodule is conjugated, the input control submodule will first conjugate the source data 2 from the memory access module, and then send it to the multiplication-accumulation tree submodule, and obtain the operation result of each sliding window through the tree structure in the multiplication-accumulation tree submodule, and send the operation result back to the memory access module through the output control module.

[0043] A multifunctional linear convolution acceleration method, comprising the steps of:

[0044] S1, the control module starts and configures the port module after receiving the first configuration signal through the bus, and after the port module receives and analyzes the start signal and the second configuration signal of the control module, if the source data 1 does not need ping-pong storage, the input port submodule first stores the source data 1 to the source data 1 ping-pong area of ​​the cache module through the bus;

[0045] S2, after the source data 1 is stored, the source data 2 is stored in the source data 2 area of ​​the cache module through the bus until the source data 2 is stored;

[0046] S3, if the source data 1 needs ping-pong storage, the input port submodule first stores part of the source data 1 to the source data 1 ping-pong area of ​​the cache module through the bus, and when the source data 1 ping-pong area is full, the source data 2 is stored to the source data 2 area in the cache module through the bus until the source data 2 is completely stored;

[0047] S4, after the storage of source data 2 is completed, the second state machine submodule sends a calculation start signal to the control module. After the calculation start signal is sent, if source data 1 needs ping-pong storage, the input port submodule bus ping-pongs the remaining source data 1 to the source data 1 area in the cache module, and the output port submodule ping-pongs the result data in the result data area in the cache module and sends it to the bus. After all the result data are sent, the second state machine submodule sends an end signal to the control module, waiting for the reset of the control module.

[0048] S5. After receiving the operation start signal from the port module, the control module starts and configures the memory access module and the operation module through the third configuration signal and the fourth configuration signal;

[0049] S6, after receiving the operation start signal and the third configuration signal from the control module, the memory access module accesses the source data and coefficients in a sliding window manner, and sends the overlapping part of the window and the data k in parallel to the operation module; if the overlapping part is less than k, it is padded with zeros and sent in k parallel; if the overlapping part exceeds k, it is sent in k parallel in multiple batches, and the remainder is padded with zeros and sent in k parallel to the operation module;

[0050] S7, after receiving the operation start signal and the fourth configuration signal from the control module, the operation module operates the data from each sliding window of the memory access module and sends the operation result of each sliding window to the memory access module;

[0051] S8, when the operation module completes the operation, the result data distribution submodule receives the operation result of the operation module and performs storage switching of the result data ping-pong area or pong area according to the current operation result points and the ping-pong storage boundary of the result data area in the cache module;

[0052] S9. When all result data are stored, the memory access module sends an end signal to the control module and waits for the control module to be reset.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] In view of the deficiencies in the prior art, the present invention realizes high-performance hardware acceleration of commonly used linear convolution-based digital signal processing algorithms with a lower hardware area by multiplexing memory access logic and operation logic. In addition, the multifunctional linear convolution accelerator simultaneously supports multiple batch processing operations, thereby improving the data processing efficiency in multiple batch operation scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is an overall architecture diagram of a multifunctional linear convolution accelerator according to Embodiment 1 of the present invention;

[0056] Figure 2 This is a schematic diagram of data storage in the cache module source data 1 ping area according to embodiment 1 of the present invention;

[0057] Figure 3 This is a schematic diagram of the linear convolution, FIR, cross-correlation and autocorrelation algorithms of Example 1 of the present invention;

[0058] Figure 4 Schematic diagram of the structure of the multiplication-accumulation tree module in the operation module of embodiment 1 of the present invention. DETAILED DESCRIPTION

[0059] Example 1

[0060] See also Figure 1 , an embodiment provided by the present invention: a multifunctional linear convolution accelerator, comprising: a control module, a port module, a cache module, a memory access module, a calculation module and a bus;

[0061] Control module: used to control the startup and configuration of each module in the accelerator;

[0062] Port module: used to control data transmission between the bus and cache module;

[0063] Cache module: used to cache source data and calculation results;

[0064] Memory access module: used to access the source data of the cache module and send it to the operation module, and receive the operation results of the operation module and store them in the cache module;

[0065] Operation module: receives source data from the memory access module and performs operations in various ways.

[0066] The control module includes a first state machine submodule, a configuration submodule and a state submodule;

[0067] The port module includes a second state machine submodule, an input port submodule and an output port submodule; the operation module includes a third state machine submodule, an input control submodule, an output control submodule and a multiplication-accumulation tree submodule; the memory access module includes a fourth state machine submodule, a source data 1 distribution submodule, a source data 2 distribution submodule and a result data distribution submodule;

[0068] A first state machine submodule, used to control the startup and configuration of the port module, the cache module, the memory access module and the operation module according to the received startup signal and the first configuration signal;

[0069] The first configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, the number of source data 2 batches, whether the batches of data correspond, whether source data 2 is flipped, and whether source data 2 is conjugated;

[0070] A configuration submodule, used for receiving a start signal and a configuration signal through a bus;

[0071] A status submodule, used for sending the working status of the accelerator through the bus;

[0072] The configuration submodule receives a start signal and a configuration signal through the bus. After querying and parsing the start signal and the first configuration signal configured by the configuration submodule, the first state machine submodule starts and configures the port module through the second state machine submodule. After the control module receives the operation start signal of the port module, the memory access module and the operation module are correspondingly started and configured through the third state machine submodule and the fourth state machine submodule. After the control module receives the end signal of the port module and the memory access module, a reset signal is sent to the port module, the cache module, the memory access module and the operation module, and the end state is written to the state submodule, waiting for the next start.

[0073] A second state machine submodule, used for receiving and analyzing a start signal and a second configuration signal sent by the control module;

[0074] An input port submodule, used for storing source data into a cache module through a bus;

[0075] The output port submodule is used to send the result data in the cache module to the bus through the bus;

[0076] The second configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, and the number of source data 2 batches;

[0077] Furthermore, in this embodiment, the workflow of each submodule of the port module includes:

[0078] The start signal and the second configuration signal sent by the control module are received and analyzed by the second state machine submodule. After analyzing the start signal and the second configuration signal, the second state machine submodule starts and configures the input port submodule and the output port submodule according to the analyzed second configuration signal.

[0079] Further, in this embodiment, the cache module includes a source data 1 ping area, a source data 1 pong area, a source data 2 area, a result data ping area, and a result data pong area;

[0080] Furthermore, in this embodiment, the step of the cache module performing data caching includes:

[0081] A1. If source data 1 does not need ping-pong storage, the input port submodule stores source data 1 in the source data 1 ping-pong area in the cache module through the bus. After source data 1 is stored, source data 2 is stored in the source data 2 area of ​​the cache module through the bus until source data 2 is stored.

[0082] Furthermore, the ping-pong storage in this embodiment is a commonly used data flow control processing technique, which is mainly used to efficiently process and transmit data flows. Ping-pong storage achieves seamless buffering and processing of data by isochronously allocating the input data flow to two data buffers and switching between the two buffers. This method is particularly suitable for pipeline processing and can ensure the continuity of data flow and the real-time processing.

[0083] A2. If the source data 1 needs ping-pong storage, the input port submodule stores part of the source data 1 into the source data 1 ping-pong area in the cache module through the bus;

[0084] A3, when the source data 1 ping-pong area is full, the source data 2 is stored in the source data 2 area in the cache module through the bus, and when the source data 2 is stored, the remaining source data 1 is ping-pong stored in the source data 1 area of ​​the cache module through the bus;

[0085] A4. After the source data 2 is stored, the second state machine submodule sends a calculation start signal to the control module. At the same time, the output port submodule ping-pong accesses the result data in the result data area in the cache module and sends it to the bus. After all the result data are sent, the second state machine submodule sends an end signal to the control module, waiting for the reset of the control module.

[0086] For further information, see Figure 1 and Figure 2 In this embodiment, the source data 1 ping area, the source data 1 pong area, the source data 2 area, the result data ping area and the result data pong area are all composed of k SRAMs and have the same data storage method, where k is the operation parallelism in the operation module, and the bit width of the SRAM is twice the bit width of the processed data.

[0087] Further, in order to better illustrate the data storage process, this embodiment takes source data 1 as an example to perform data storage, including:

[0088] First, the first batch of data of source data 1 is sequentially stored from address 0 of SRAM with sequence number 0 to address 0 of SRAM with sequence number k-1;

[0089] If address 0 of k SRAMs are all full, the remaining data will be sequentially stored in address 1 of k SRAMs, and so on. If the first batch of source data 1 is completely stored, the second batch of data will continue to be stored until source data 1 is completely stored.

[0090] Furthermore, in this embodiment, taking source data 1 as an example, detailed steps for data storage include:

[0091] S101, after receiving source data 1, the input port submodule stores it in the source data 1 ping-pong area of ​​the cache module; the source data 1 ping-pong area is composed of k SRAMs, each of which has a certain storage capacity.

[0092] S102, store the first batch of data: store the first batch of data of source data 1 from SRAM with sequence number 0, and the address starts from 0;

[0093] For example, assuming k=4, the source data 1 has two batches of data, each batch has 12 elements, then the first 8 elements of the first batch are first stored sequentially in address 0 of SRAM0~SRAM3, and then the remaining 4 elements of the first batch are sequentially stored in address 1 of SRAM0~SRAM1, and then the first 4 elements of the second batch are sequentially stored in address 1 of SRAM2~SRAM3, and finally the remaining 8 elements of the second batch are sequentially stored in address 2 of SRAM0~SRAM3.

[0094] Ping-pong storage: If source data 1 requires ping-pong storage, when the ping-pong area of ​​source data 1 is full, the next source data 2 will be stored in source data 2 area, and then the remaining source data 1 will be stored in source data 1 pong area; this alternating storage ensures continuous data processing.

[0095] For example, it is assumed that the source data 1 has two batches of data, each batch of data has 10 elements, and k=4, that is, the ping-pong area and pong area of ​​the source data 1 each have 4 SRAMs (SRAM0-SRAM3), and the depth of the SRAM is 2; the storage process of the source data 1 is as follows:

[0096] First, the first 8 elements of the first batch are sequentially stored in address 0 of source data 1 ping-pong area SRAM0~SRAM3, and then the remaining 2 elements of the first batch are sequentially stored in address 1 of source data 1 ping-pong area SRAM0, and then the first 6 elements of the second batch are sequentially stored in address 1 of source data 1 ping-pong area SRAM1~SRAM3, and finally the remaining 4 elements of the second batch are sequentially stored in address 0 of source data 1 ping-pong area SRAM0~SRAM1.

[0097] This process ensures seamless buffering and continuous processing of data by isochronously allocating the input data stream to two data buffers (ping-pong area and pong area) and switching between the two. In particular, when source data 1 requires ping-pong storage, data can be efficiently switched between the ping-pong area and the pong area of ​​source data 1, reducing the interruption time of data processing and memory access conflicts.

[0098] This design adapts to different data types by adjusting the storage method of SRAM. The bit width of each SRAM is twice the bit width of the processing data. In this way, the system can maintain high throughput and low latency while processing large amounts of data, significantly improving the overall performance and reliability of the signal processing system.

[0099] Further, the fourth state machine submodule in this embodiment is used to receive and parse the start signal and the third configuration signal of the control module. After parsing the start signal and the third configuration signal, the fourth state machine submodule starts and configures the source data 1 distribution submodule, the source data 2 distribution submodule and the result data distribution submodule;

[0100] Furthermore, in this embodiment, the third configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, the number of source data 2 batches, whether each batch of data corresponds, and whether source data 2 is flipped.

[0101] Furthermore, in this embodiment, the step of performing data access by the memory access module includes:

[0102] B1. If the batches of data in the third configuration signal parsed by the fourth state machine submodule do not correspond, each batch of data of source data 1 and each batch of data of source data 2 are accessed and sent in a corresponding manner; if the batches of data in the third configuration signal correspond, each batch of data of source data 1 is accessed and sent in a corresponding manner only with the same batch of data of source data 2;

[0103] B2. If the source data 2 in the third configuration signal parsed by the fourth state machine submodule is not flipped, the sliding window will slide from the beginning to the end to access the source data 2 in the cache module and calculate and send it. If the source data 2 is flipped, the sliding window will slide from the end to the beginning to access the source data 2 in the cache module in reverse order and calculate and send it.

[0104] Further, in order to more clearly illustrate steps B1-B2, in this embodiment, the specific steps of data access are performed by taking the case where the batches of data are not corresponding and the source data 2 is not flipped as an example, including:

[0105] B3, assuming that the fourth state machine submodule receives and parses the third configuration signal, and obtains the parsed third configuration signal that the batches of data are not corresponding and the source data 2 is not flipped;

[0106] B4. According to the parsed third configuration signal, the source data 1 distribution submodule and the source data 2 distribution submodule are started and configured through the fourth state machine submodule;

[0107] B5. Assume that the source data 1 cached in the cache module is p batches of N-point vectors, and the source data 2 is q batches of M-point vectors. Use the configured source data 1 distribution submodule to access and calculate and send the first batch of source data 1 in the source data area of ​​the cache module. At the same time, use the configured source data 2 distribution submodule to access and calculate and send the first batch of source data 2 in the source data area of ​​the cache module.

[0108] Further, in this embodiment, the steps of using the configured source data 1 distribution submodule to access, calculate and send the first batch of source data 1 in the source data area of ​​the cache module include:

[0109] B511, set a sliding window 1 with a length of M in the source data 1 distribution module, and use the overlapping position of the end of the sliding window 1 and the beginning of the first batch of data of the source data 1 as the starting position, and the overlapping position of the beginning of the sliding window 1 and the end of the first batch of data of the source data 1 as the ending position, slide the sliding window 1 N+M-1 times, sequentially access the overlapping part of each sliding window 1 and the first batch of data of the source data 1 and send it to the operation module;

[0110] B512, for each sliding window 1, divide the length of the overlapping part of the sliding window 1 and the first batch of data of the source data 1 by k, perform g times of k parallel access on the divisible part, and send the accessed data to the operation module for k parallel operations, and at the same time, perform 1 parallel access of the remainder length on the remainder part and send the accessed data to the operation module for 1 time of k parallel operations after filling 0;

[0111] B513, if the source data 1 needs ping-pong storage, the source data 1 distribution module performs access switching of the source data 1 ping-pong area or pong area according to the number of source data 1 points, the number of source data 1 batches, and the storage boundary of the source data 1 ping-pong area and the pong area;

[0112] Further, in this embodiment, the steps of using the configured source data 2 distribution submodule to access, calculate and send the first batch of source data 2 in the source data area of ​​the cache module include:

[0113] B521, set a sliding window 2 with a length of N in the source data 2 distribution module, slide the sliding window 2 N+M-1 times according to the method of B511, access the overlapping part of each sliding window 2 and the first batch of data of the source data 2 in reverse order and send it to the operation module;

[0114] B522, when the data access and operation of the first batch of data of source data 1 and the first batch of data of source data 2 are completed, the first batch of data of source data 1 and the second batch of data of source data 2 are accessed according to the method of B511-B513 and B521;

[0115] B6, when the data access and operation of the first batch of data of source data 1 and each batch of data of source data 2 are completed, the second batch of data of source data 1 and each batch of data of source data 2 are accessed in the same manner as B511-B513 and B521-B522, until each batch of data of source data 1 and each batch of data of source data 2 complete the corresponding data access and operation;

[0116] B7, when the operation module is completed, the result data distribution submodule receives the operation result of the operation module and performs storage switching of the result data ping-pong area or pong area according to the current operation result points and the ping-pong storage boundary of the result data area in the cache module;

[0117] B8. When all result data are stored, the memory access module sends an end signal to the control module and waits for the control module to be reset.

[0118] Further, according to the above-mentioned process B1-B8, the specific steps of implementing the source data access of the FIR algorithm in this embodiment include:

[0119] B11, assuming that the third configuration signal parsed by the fourth state machine submodule is to fix the selection of source data 2 not to be flipped and source data 2 not to be conjugated, then source data 2 is regarded as q groups of M-order filter coefficients;

[0120] B111, when the selected source data 2 batches of data do not correspond, each batch of data of source data 1 is subjected to q times of M-order FIR filtering, and a total of p*q times of FIR filtering is performed;

[0121] B112, when the corresponding batches of data are selected, each batch of source data 1 will only perform one FIR operation with the corresponding batch of source data 2;

[0122] For example, in order to further illustrate the FIR algorithm to access source data, as shown in FIG. Figure 3 As shown, the specific steps of the FIR algorithm for source data access include:

[0123] Suppose source data 1 is a vector of N points A = [x[0], x[1], ..., x[N-1]] and source data 2 is a vector of M points B = [h[0], h[1], ..., h[M-1]];

[0124] Furthermore, assuming that N and M in this embodiment are both 5, a clearer example of the FIR algorithm performing source data access is:

[0125] Q1. Assume that source data 1 is a batch of 5-point vectors A = [x[0], x[1], ..., x[4]], and source data 2 is a batch of 5-point vectors B = [h[0], h[1], ..., h[4]], and 5+5-1 calculation results will be obtained under the FIR, cross-correlation and autocorrelation algorithms. The operation parallelism of the multiplication-accumulation tree of the operation module is 4. The cache module includes source data 1 ping-pong area, source data 1 pong area, source data 2 area, result data ping-pong area and result data pong area, and each area is composed of 4 SRAMs;

[0126] Q2, first access the data of the first sliding window: access x[0] in source data area 1 and h[0] in source data area 2, fill in 0 to [x[0],0,0,0] and [h[0],0,0,0], and then send 4 in parallel to the calculation module;

[0127] Q3, then perform the second sliding window: access x[0], x[1] and h[1], h[0], fill in 0 to [x[0], x[1], 0, 0] and [h[1], h[0], 0, 0], and then send the 4 in parallel to the calculation module;

[0128] Q4, perform the third sliding window: access x[0], x[1], x[2] and h[2], h[1], h[0], fill in 0s to [x[0], x[1], x[2], 0] and [h[2], h[1], h[0], 0], and then send the 4 in parallel to the computing module.

[0129] Q5, fourth, perform the fourth sliding window: access x[0], x[1], x[2], x[3] and h[3], h[2], h[1], h[0], and send 4 in parallel to the calculation module.

[0130] Q6, perform the fifth sliding window: first access x[0], x[1], x[2], x[3] and h[4], h[3], h[2], h[1], and send 4 in parallel to the calculation module; then access x[4] and h[0], fill in 0s to [x[4], 0, 0, 0] and [h[0], 0, 0, 0] and send them to the calculation module.

[0131] Q7, perform the sixth sliding window: access x[1], x[2], x[3], x[4] and h[4], h[3], h[2], h[1], and send 4 in parallel to the computing module.

[0132] Q8, perform the seventh sliding window: access x[2], x[3], x[4] and h[4], h[3], h[2], fill in 0s to [x[2], x[3], x[4], 0] and [h[4], h[3], h[2], 0], and then send the 4 in parallel to the computing module.

[0133] Q9, perform the 8th sliding window: access x[3], x[4] and h[4], h[3], fill in 0s to [x[3], x[4], 0, 0] and [h[4], h[3], 0, 0], and then send 4 in parallel to the computing module.

[0134] Q10. Finally, perform the 9th sliding window: access x[4] and h[4], fill in 0s to [x[4],0,0,0] and [h[4],0,0,0], and then send the 4 to the computing module in parallel.

[0135] B12. If the cross-correlation algorithm is to be implemented, the source data 2 flip and the source data 2 conjugate are fixedly selected, and the source data 1 and the source data 2 are regarded as two cross-correlated vectors. At the same time, B111-B113 are repeated to implement the configuration options and operations of whether each batch of data corresponds;

[0136] Furthermore, specific examples of source data access to implement the cross-correlation algorithm in this embodiment include:

[0137] First, the data of the first sliding window is accessed: x[0] in source data area 1 and h[4] in source data area 2 are accessed, and 0s are added to [x[0],0,0,0] and [h[4],0,0,0], and then the 4 are sent to the computing module in parallel.

[0138] Next, perform the second sliding window: access x[0], x[1] and h[3], h[4], fill in 0s to [x[0], x[1], 0, 0] and [h[3], h[4], 0, 0], and then send the 4 to the computing module in parallel.

[0139] Third, perform the third sliding window: access x[0], x[1], x[2] and h[2], h[3], h[4], fill in 0s to [x[0], x[1], x[2], 0] and [h[2], h[3], h[4], 0], and then send the 4 to the computing module in parallel.

[0140] Fourth, perform the fourth sliding window: access x[0], x[1], x[2], x[3] and h[1], h[2], h[3], h[4], and send 4 in parallel to the computing module.

[0141] Fifth, perform the fifth sliding window: first access x[0], x[1], x[2], x[3] and h[0], h[1], h[2], h[3], and send 4 in parallel to the calculation module; then access x[4] and h[4], fill in 0s to [x[4], 0, 0, 0] and [h[4], 0, 0, 0], and then send 4 in parallel to the calculation module.

[0142] Sixth, perform the sixth sliding window: access x[1], x[2], x[3], x[4] and h[0], h[1], h[2], h[3], and send 4 in parallel to the computing module.

[0143] Seventh, perform the 7th sliding window: access x[2], x[3], x[4] and h[0], h[1], h[2], fill in 0s to [x[2], x[3], x[4], 0] and [h[0], h[1], h[2], 0], and then send the 4 to the computing module in parallel.

[0144] Eighth, perform the 8th sliding window: access x[3], x[4] and h[0], h[1], fill in 0s to [x[3], x[4], 0, 0] and [h[0], h[1], 0, 0], and then send the 4 to the computing module in parallel.

[0145] Finally, perform the 9th sliding window: access x[4] and h[0], fill in 0s to [x[4],0,0,0] and [h[0],0,0,0], and then send 4 in parallel to the computing module.

[0146] B15. If you want to implement the autocorrelation algorithm, then select the corresponding batches of data and configure the same options as the cross-correlation algorithm. However, the source data 1 and source data 2 sent to the port module through the bus need to be the same data, that is, vector A and vector B must be the same vector.

[0147] The fourth state machine submodule in this embodiment can dynamically configure the source data 1 distribution submodule, the source data 2 distribution submodule and the result data distribution submodule according to the received start signal and the third configuration signal. This means that the system can flexibly adjust the data processing flow according to different operation requirements and configuration parameters, thereby improving the flexibility and adaptability of the system. For example, when performing FIR filtering operations, it can be determined whether to perform multiple filtering or only one filtering according to whether to select each batch of data, so that different operation requirements can be met and operation efficiency can be optimized. Secondly, through the sliding window mechanism and parallel access strategy, the system can efficiently process complex mathematical operations, such as FIR filtering, cross-correlation and autocorrelation. The sliding window sequentially slides from the beginning to the end to access the data in the cache module, which can not only ensure the continuity and consistency of data access, but also improve the speed of data processing through parallel processing technology. For example, in the FIR algorithm, after the data accessed by the sliding window each time is processed by filling 0, k parallel transmission to the calculation module can be realized, which greatly improves the throughput and processing capacity of the calculation module.

[0148] Further, in this embodiment, the third state machine submodule is used to receive and parse the start signal and the fourth configuration signal of the control module. After parsing the start signal and the fourth configuration signal, the third state machine submodule starts and configures the input control submodule, the output control submodule and the multiplication-accumulation tree submodule according to the parsed fourth configuration signal;

[0149] Furthermore, in this embodiment, the multiplication-accumulation tree submodule includes a multiplication-accumulation tree, see Figure 4 , when k = 4, the specific operation process of the multiplication-accumulation tree is:

[0150] Assume that the four complex source data 1 and complex source data 2 accessed in parallel from the source data area in the cache module are (src1_1, src1_2, src1_3, src1_4) and (src2_1, src2_2, src2_3, src2_4) in sequence;

[0151] Construct the multiply-accumulate input pairs (src1_1, src2_1), (src1_2, src2_2), (src1_3, src2_3), and (src1_4, src2_4) using (src1_1, src1_2, src1_3, src1_4) and (src2_1, src2_2, src2_3, src2_4);

[0152] Multiply the input pairs (src1_1, src2_1) and (src1_2, src2_2) and add them together to get the result 1. Multiply the input pairs (src1_3, src2_3) and (src1_4, src2_4) and add them together to get the result 2.

[0153] Add result 1 and result 2 to obtain result 3, and then add result 3 to itself to obtain the final output result of the multiplication-accumulation tree;

[0154] The fourth configuration signal includes whether the source data 2 is conjugated.

[0155] Furthermore, the steps of computing by the computing module in this embodiment include:

[0156] C1. Set the operation parallelism of the multiplication-accumulation tree in the operation module to k, the input control submodule selects the path between the k input ports and the multiplication-accumulation tree, and the output control submodule selects the path between the multiplication-accumulation tree and the output port;

[0157] C2. If the source data 2 in the fourth configuration signal parsed by the third state machine submodule is conjugated, the input control submodule will first conjugate the source data 2 from the memory access module, and then send it to the multiplication-accumulation tree submodule, and obtain the operation result of each sliding window through the tree structure in the multiplication-accumulation tree submodule, and send the operation result back to the memory access module through the output control module.

[0158] Example 2

[0159] Another embodiment provided by the present invention is a multifunctional linear convolution acceleration method, comprising the following steps:

[0160] S1, the control module starts and configures the port module after receiving the first configuration signal through the bus, and after the port module receives and analyzes the start signal and the second configuration signal of the control module, if the source data 1 does not need ping-pong storage, the input port submodule first stores the source data 1 to the source data 1 ping-pong area of ​​the cache module through the bus;

[0161] S2, after the source data 1 is stored, the source data 2 is stored in the source data 2 area of ​​the cache module through the bus until the source data 2 is stored;

[0162] S3, if the source data 1 needs ping-pong storage, the input port submodule first stores part of the source data 1 to the source data 1 ping-pong area of ​​the cache module through the bus, and when the source data 1 ping-pong area is full, the source data 2 is stored to the source data 2 area in the cache module through the bus until the source data 2 is completely stored;

[0163] S4, after the storage of source data 2 is completed, the second state machine submodule sends a calculation start signal to the control module. After the calculation start signal is sent, if source data 1 needs ping-pong storage, the input port submodule bus ping-pongs the remaining source data 1 to the source data 1 area in the cache module, and the output port submodule ping-pongs the result data in the result data area in the cache module and sends it to the bus. After all the result data are sent, the second state machine submodule sends an end signal to the control module, waiting for the reset of the control module.

[0164] S5. After receiving the operation start signal from the port module, the control module starts and configures the memory access module and the operation module through the third configuration signal and the fourth configuration signal;

[0165] S6. After receiving the operation start signal and the third configuration signal from the control module, the memory access module accesses the source data and coefficients in a sliding window manner, and sends the overlapping part of the window and data k in parallel to the operation module; if the overlapping part is less than k, it is padded with zeros and sent k in parallel; if the overlapping part exceeds k, it is divided into multiple batches and sent k in parallel, and the remainder is padded with zeros and sent k in parallel to the operation module.

[0166] Furthermore, in order to explain the S1-S6 method in more detail in this embodiment, the exemplary steps are as follows:

[0167] S7. Let source data 1 be a vector of N points A = [x[0], x[1], ..., x[N-1]] and source data 2 be a vector of M points B =

[0168] [h[0], h[1], ..., h[M-1]], and in this embodiment, N and M are both set to 5, then source data 1 is a batch of 5-point vectors A = [x[0], x[1], ..., x[4]], and source data 2 is a batch of 5-point vectors B = [h[0], h[1], ..., h[4]];

[0169] Furthermore, in this embodiment, the FIR, cross-correlation and autocorrelation algorithms perform a sliding window access process as follows: Figure 3 , and Figure 3 The steps of the corresponding algorithm when accessing memory in a sliding window manner are shown through examples of specific values ​​of N and M. The specific process is as follows:

[0170] Further, according to the parameters set in S7, in this embodiment, 5+5-1 calculation results will be obtained when accessing memory under FIR, cross-correlation and autocorrelation algorithms, the operation parallelism of the multiplication-accumulation tree of the operation module is 4, and the cache module includes a source data 1 ping area, a source data 1 pong area, a source data 2 area, a result data ping area and a result data pong area, and each area is composed of 4 SRAMs;

[0171] S8, after receiving the operation start signal and the third configuration signal from the control module, the memory access module accesses the source data and coefficients in a sliding window manner, and sends the overlapping part of the window and the data to the operation module in parallel by 4, if the overlapping part is less than 4, it is padded with zeros and sent in parallel by 4, if the overlapping part exceeds 4, it is sent in parallel by 4 in multiple batches, and the remainder is padded with zeros and sent in parallel by 4 to the operation module;

[0172] Furthermore, in this embodiment, the source data and coefficients are accessed in a sliding window manner, where the sliding window includes sliding windows corresponding to the FIR algorithm, the cross-correlation algorithm, and the autocorrelation algorithm;

[0173] Exemplarily, if the source data access of the FIR algorithm is to be implemented, the specific steps corresponding to the sliding window access include:

[0174] Q2, first access the data of the first sliding window: access x[0] in source data area 1 and h[0] in source data area 2, fill in 0 to [x[0],0,0,0] and [h[0],0,0,0], and then send 4 in parallel to the calculation module;

[0175] Q3, then perform the second sliding window: access x[0], x[1] and h[1], h[0], fill in 0 to [x[0], x[1], 0, 0] and [h[1], h[0], 0, 0], and then send the 4 in parallel to the calculation module;

[0176] Q4, perform the third sliding window: access x[0], x[1], x[2] and h[2], h[1], h[0], fill in 0s to [x[0], x[1], x[2], 0] and [h[2], h[1], h[0], 0], and then send the 4 in parallel to the computing module.

[0177] Q5, fourth, perform the fourth sliding window: access x[0], x[1], x[2], x[3] and h[3], h[2], h[1], h[0], and send 4 in parallel to the calculation module.

[0178] Q6, perform the fifth sliding window: first access x[0], x[1], x[2], x[3] and h[4], h[3], h[2], h[1], and send 4 in parallel to the calculation module; then access x[4] and h[0], fill in 0s to [x[4], 0, 0, 0] and [h[0], 0, 0, 0] and send them to the calculation module.

[0179] Q7, perform the sixth sliding window: access x[1], x[2], x[3], x[4] and h[4], h[3], h[2], h[1], and send 4 in parallel to the computing module.

[0180] Q8, perform the seventh sliding window: access x[2], x[3], x[4] and h[4], h[3], h[2], fill in 0s to [x[2], x[3], x[4], 0] and [h[4], h[3], h[2], 0], and then send the 4 in parallel to the computing module.

[0181] Q9, perform the 8th sliding window: access x[3], x[4] and h[4], h[3], fill in 0s to [x[3], x[4], 0, 0] and [h[4], h[3], 0, 0], and then send 4 in parallel to the computing module.

[0182] Q10. Finally, perform the 9th sliding window: access x[4] and h[4], fill in 0s to [x[4],0,0,0] and [h[4],0,0,0], and then send the 4 to the computing module in parallel.

[0183] Exemplarily, if the source data access of the cross-correlation algorithm is to be implemented, the specific steps of the corresponding sliding window access include:

[0184] First, the data of the first sliding window is accessed: x[0] in source data area 1 and h[4] in source data area 2 are accessed, and 0s are added to [x[0],0,0,0] and [h[4],0,0,0], and then the 4 are sent to the computing module in parallel.

[0185] Next, perform the second sliding window: access x[0], x[1] and h[3], h[4], fill in 0s to [x[0], x[1], 0, 0] and [h[3], h[4], 0, 0], and then send the 4 to the computing module in parallel.

[0186] Third, perform the third sliding window: access x[0], x[1], x[2] and h[2], h[3], h[4], fill in 0s to [x[0], x[1], x[2], 0] and [h[2], h[3], h[4], 0], and then send the 4 to the computing module in parallel.

[0187] Fourth, perform the fourth sliding window: access x[0], x[1], x[2], x[3] and h[1], h[2], h[3], h[4], and send 4 in parallel to the computing module.

[0188] Fifth, perform the fifth sliding window: first access x[0], x[1], x[2], x[3] and h[0], h[1], h[2], h[3], and send 4 in parallel to the calculation module; then access x[4] and h[4], fill in 0s to [x[4], 0, 0, 0] and [h[4], 0, 0, 0], and then send 4 in parallel to the calculation module.

[0189] Sixth, perform the sixth sliding window: access x[1], x[2], x[3], x[4] and h[0], h[1], h[2], h[3], and send 4 in parallel to the computing module.

[0190] Seventh, perform the 7th sliding window: access x[2], x[3], x[4] and h[0], h[1], h[2], fill in 0s to [x[2], x[3], x[4], 0] and [h[0], h[1], h[2], 0], and then send the 4 to the computing module in parallel.

[0191] Eighth, perform the 8th sliding window: access x[3], x[4] and h[0], h[1], fill in 0s to [x[3], x[4], 0, 0] and [h[0], h[1], 0, 0], and then send the 4 to the computing module in parallel.

[0192] Finally, perform the 9th sliding window: access x[4] and h[0], fill in 0s to [x[4],0,0,0] and [h[0],0,0,0], and then send 4 in parallel to the computing module.

[0193] Exemplarily, if the source data access of the autocorrelation algorithm is to be implemented, the corresponding batches of data are fixedly selected, and the same options as the cross-correlation algorithm are configured, except that vector A and vector B must be the same vector.

[0194] S9, after receiving the operation start signal and the fourth configuration signal from the control module, the operation module operates the data from each sliding window of the memory access module and sends the operation result of each sliding window to the memory access module;

[0195] Furthermore, in this embodiment, if the FIR algorithm is to be implemented, the input control submodule directly sends the source data 1 and the source data 2 to the multiplication-accumulation tree operation; if the cross-correlation or autocorrelation algorithm is to be implemented, the input control module first takes the conjugate of the source data 2, and then sends the source data 1 and the source data 2 to the multiplication-accumulation tree operation;

[0196] S9, when the operation module completes the operation, the result data distribution submodule receives the operation result of the operation module and performs storage switching of the result data ping-pong area or pong area according to the current operation result points and the ping-pong storage boundary of the result data area in the cache module;

[0197] S10, when all result data are stored, the memory access module sends an end signal to the control module, and waits for the control module to be reset;

[0198] S11. After receiving the end signals from the port module and the memory access module, the first status submodule sends a reset signal to each module, writes the end status to the status submodule, and waits for the next startup.

[0199] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in the field may also change, modify, replace and modify the above-mentioned embodiments without departing from the scope of protection of the purpose of the present invention and the claims, and all of these are within the protection of the present invention.

[0200] If the disclosed technical solution involves personal information, the product using the disclosed technical solution has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the disclosed technical solution involves sensitive personal information, the product using the disclosed technical solution has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. A multifunctional linear convolution accelerator, characterized in that: include: Control module, port module, cache module, memory access module, operation module and bus; The control module includes a first state machine submodule, a configuration submodule and a state submodule; The port module includes a second state machine submodule; the operation module includes a third state machine submodule; the memory access module includes a fourth state machine submodule; The first state machine submodule is used to control the startup and configuration of the port module, the cache module, the memory access module and the operation module according to the received startup signal and the first configuration signal; The first configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, the number of source data 2 batches, whether the batches of data correspond, whether the source data 2 is flipped, and whether the source data 2 is conjugated; The configuration submodule is used to receive a start signal and a configuration signal through a bus; The state submodule is used to send the working state of the accelerator through the bus; The configuration submodule receives a start signal and a configuration signal through a bus. After querying and parsing the start signal and the first configuration signal configured by the configuration submodule, the first state machine submodule starts and configures the port module through the second state machine submodule. After the control module receives the operation start signal of the port module, the memory access module and the operation module are correspondingly started and configured through the third state machine submodule and the fourth state machine submodule. After the control module receives the end signal of the port module and the memory access module, a reset signal is sent to the port module, the cache module, the memory access module and the operation module, and the end state is written to the state submodule, waiting for the next start.

2. A multifunctional linear convolution accelerator as claimed in claim 1, characterized in that: The port module also includes an input port submodule and an output port submodule; The second state machine submodule is used to receive and analyze the start signal and the second configuration signal sent by the control module; The input port submodule is used to store the source data into the cache module through the bus; The output port submodule is used to send the result data in the cache module to the bus via the bus; The second configuration signal includes source data 1 point number, source data 1 batch number, source data 2 point number, source data 2 batch number; The workflow of each submodule of the port module includes: The second state machine submodule receives and parses the start signal and the second configuration signal sent by the control module. After parsing the start signal and the second configuration signal, the second state machine submodule starts and configures the input port submodule and the output port submodule according to the parsed second configuration signal.

3. A multifunctional linear convolution accelerator as claimed in claim 2, characterized in that: The cache module includes a source data 1 ping area, a source data 1 pong area, a source data 2 area, a result data ping area and a result data pong area; The steps of the cache module performing data caching include: A1. If the source data 1 does not need ping-pong storage, the input port submodule stores the source data 1 in the source data 1 ping-pong area in the cache module through the bus, and after the storage of the source data 1 is completed, the source data 2 is stored in the source data 2 area of ​​the cache module through the bus until the storage of the source data 2 is completed; A2. If the source data 1 needs ping-pong storage, the input port submodule stores part of the source data 1 into the source data 1 ping-pong area in the cache module through the bus; A3, when the source data 1 ping-pong area is full, the source data 2 is stored in the source data 2 area in the cache module through the bus, and when the source data 2 is stored, the remaining source data 1 is ping-pong stored in the source data 1 area of ​​the cache module through the bus; A4. After the source data 2 is stored, the second state machine submodule sends a calculation start signal to the control module, and at the same time, the output port submodule ping-pong accesses the result data in the result data area in the cache module and sends it to the bus. After all the result data are sent, the second state machine submodule sends an end signal to the control module, waiting for the control module to be reset.

4. A multifunctional linear convolution accelerator as claimed in claim 3, characterized in that: The source data 1 ping area, source data 1 pong area, source data 2 area, result data ping area and result data pong area are all composed of k SRAMs and have the same data storage method, where k is the operation parallelism in the operation module, and the bit width of the SRAM is twice the bit width of the data processed by the operation module.

5. A multifunctional linear convolution accelerator as claimed in claim 4, characterized in that: The memory access module also includes a source data 1 distribution submodule, a source data 2 distribution submodule and a result data distribution submodule; The fourth state machine submodule is used to receive and parse the start signal and the third configuration signal of the control module. After parsing the start signal and the third configuration signal, the fourth state machine submodule starts and configures the source data 1 distribution submodule, the source data 2 distribution submodule and the result data distribution submodule; The third configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, the number of source data 2 batches, whether each batch of data corresponds, and whether source data 2 is flipped.

6. A multifunctional linear convolution accelerator as claimed in claim 5, characterized in that: The steps of the memory access module performing data access include: B1. If the batches of data in the third configuration signal parsed by the fourth state machine submodule do not correspond, each batch of data of source data 1 and each batch of data of source data 2 are accessed and sent in a corresponding manner; if the batches of data in the third configuration signal correspond, each batch of data of source data 1 is accessed and sent in a corresponding manner only with the same batch of data of source data 2; B2. If the source data 2 in the third configuration signal parsed by the fourth state machine submodule is not flipped, the sliding window accesses the source data 2 in the cache module in order from the beginning to the end and calculates and sends it; if the source data 2 is flipped, the sliding window accesses the source data 2 in the cache module in reverse order from the end to the beginning and calculates and sends it; B3, when the data sent by B1 or B2 to the operation module is completed, the operation result of the operation module is received by the result data distribution submodule and stored in the result data ping-pong area or pong area of ​​the cache module; B4. After the operation results are stored, the fourth state machine submodule sends an end signal to the control module and waits for the control module to be reset.

7. A multifunctional linear convolution accelerator as claimed in claim 6, characterized in that: The operation module also includes an input control submodule, an output control submodule, and a multiplication-accumulation tree submodule; The third state machine submodule is used to receive and parse the start signal and the fourth configuration signal of the control module. After parsing the start signal and the fourth configuration signal, the third state machine submodule starts and configures the input control submodule, the output control submodule and the multiplication-accumulation tree submodule according to the parsed fourth configuration signal; The fourth configuration signal includes whether the source data 2 is conjugated.

8. A multifunctional linear convolution accelerator as claimed in claim 7, characterized in that: The steps of the operation module operation include: C1, setting the operation parallelism of the multiplication-accumulation tree in the operation module to k, the input control submodule gates the paths between the k input ports and the multiplication-accumulation tree, and the output control submodule gates the paths between the multiplication-accumulation tree and the output port; C2. If the source data 2 in the fourth configuration signal parsed by the third state machine submodule is conjugated, the input control submodule will first conjugate the source data 2 from the memory access module, and then send it to the multiplication-accumulation tree submodule, and obtain the operation result of each sliding window through the tree structure in the multiplication-accumulation tree submodule, and send the operation result back to the memory access module through the output control module.

9. A multifunctional linear convolution acceleration method, which is used to implement a multifunctional linear convolution accelerator according to any one of claims 1 to 8, characterized in that the steps include: S1, the control module starts and configures the port module after receiving the first configuration signal through the bus, and after the port module receives and analyzes the start signal and the second configuration signal of the control module, if the source data 1 does not need ping-pong storage, the input port submodule first stores the source data 1 to the source data 1 ping-pong area of ​​the cache module through the bus; S2, after the source data 1 is stored, the source data 2 is stored in the source data 2 area of ​​the cache module through the bus until the source data 2 is stored; S3, if the source data 1 needs ping-pong storage, the input port submodule first stores part of the source data 1 to the source data 1 ping-pong area of ​​the cache module through the bus, and when the source data 1 ping-pong area is full, the source data 2 is stored to the source data 2 area in the cache module through the bus until the source data 2 is completely stored; S4, after the storage of source data 2 is completed, the second state machine submodule sends a calculation start signal to the control module. After the calculation start signal is sent, if source data 1 needs ping-pong storage, the input port submodule bus ping-pongs the remaining source data 1 to the source data 1 area in the cache module, and the output port submodule ping-pongs the result data in the result data area in the cache module and sends it to the bus. After all the result data are sent, the second state machine submodule sends an end signal to the control module, waiting for the reset of the control module. S5. After receiving the operation start signal from the port module, the control module starts and configures the memory access module and the operation module through the third configuration signal and the fourth configuration signal; S6, after receiving the operation start signal and the third configuration signal from the control module, the memory access module accesses the source data and coefficients in a sliding window manner, and sends the window and data overlapping part k to the operation module in parallel; If the overlapped part is less than k, it is padded with zeros and sent in k parallels. If the overlapped part exceeds k, it is sent in k parallels in multiple batches, and the remainder is padded with zeros and sent in k parallels to the operation module. S7, after receiving the operation start signal and the fourth configuration signal from the control module, the operation module operates the data from each sliding window of the memory access module and sends the operation result of each sliding window to the memory access module; S8, when the operation module completes the operation, the result data distribution submodule receives the operation result of the operation module and performs storage switching of the result data ping-pong area or pong area according to the current operation result points and the ping-pong storage boundary of the result data area in the cache module; S9. When all result data are stored, the memory access module sends an end signal to the control module and waits for the control module to be reset.

Images