A multifunctional linear convolution accelerator

Abstract

The present invention belongs to the field of DSP signal filtering, and in particular relates to a multifunctional linear convolution accelerator and method, comprising: a configuration submodule receiving a start signal and a configuration signal via a bus; the first state machine submodule, after querying and parsing the start signal and the first configuration signal configured by the configuration submodule, starts and configures a port module via a second state machine submodule; after a control module receives an operation start signal of the port module, starts and configures a memory access module and an operation module correspondingly via a third state machine submodule and a fourth state machine submodule; after the control module receives an end signal of the port module and the memory access module, sends a reset signal to the port module, the cache module, the memory access module and the operation module, and writes an end state to the state submodule, waiting for the next start; the present invention realizes high-performance hardware acceleration of digital signal processing algorithms by multiplexing memory access logic and operation logic.

Description

Technical Field

[0001] The invention belongs to the field of DSP signal filtering, and in particular relates to a multifunctional linear convolution accelerator. Background Art

[0002] Digital signal processing technology is widely used in fields such as image processing and speech processing. It can be used to extract and analyze signal features, playing a crucial role in many engineering fields. A digital signal processor (DSP) is a specialized integrated circuit designed to rapidly execute digital signal processing algorithms. Its highly optimized hardware architecture allows for efficient processing of large amounts of data. Linear convolution is the integration of the product of the overlapping parts of two discrete sequences, flipped and translated, over the overlap length. Based on linear convolution, digital signal processing algorithms such as FIR, cross-correlation, and autocorrelation can be implemented.

[0003] However, in existing digital signal processors, due to limited area budgets, hardware acceleration structures can only be designed for some digital signal processing algorithms based on linear convolution, resulting in low hardware acceleration performance of other digital signal processing algorithms based on linear convolution. Moreover, existing hardware acceleration structures often lack hardware optimization for multiple batches of data, resulting in low data processing efficiency in batch processing scenarios.

[0004] For example, the patent with authorization announcement number CN113554163B discloses a convolutional neural network accelerator, including: multiple multiplication-addition trees for performing parallel operations on multi-channel feature maps, and a scaling operator and adder connected in series after each multiplication-addition tree; the multiplication-addition tree is used to perform multiplication-addition operations on quantized weight data and quantized feature data of the feature map of one of the corresponding channels to obtain the multiplication-addition operation result of the feature map of the corresponding channel; the scaling operator is used to perform a scaling operation on the multiplication-addition operation result of the feature map of the corresponding channel to obtain the scaling operation result of the feature map of the corresponding channel; the adder is used to perform zero point adjustment on the scaling operation result of the feature map of the corresponding channel; the result output by the adder is used as the quantized convolution result to provide an operation architecture for the quantized convolution neural network.

[0005] In order to solve the above problems, the present invention provides a multifunctional linear convolution accelerator. Summary of the Invention

[0006] In response to the shortcomings of the existing technology, the present invention proposes a multifunctional linear convolution accelerator, including: a configuration submodule receives a start signal and a configuration signal through a bus; after the first state machine submodule queries and parses the start signal and the first configuration signal configured by the configuration submodule, it starts and configures the port module through the second state machine submodule; after the control module receives the operation start signal of the port module, it starts and configures the memory access module and the operation module accordingly through the third state machine submodule and the fourth state machine submodule; after the control module receives the end signal of the port module and the memory access module, it sends a reset signal to the port module, the cache module, the memory access module and the operation module, and writes the end state to the state submodule, waiting for the next startup; the present invention realizes high-performance hardware acceleration of digital signal processing algorithms by multiplexing memory access logic and operation logic.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A multifunctional linear convolution accelerator, comprising: a control module, a port module, a cache module, a memory access module, an operation module and a bus;

[0009] The control module includes a first state machine submodule, a configuration submodule and a state submodule;

[0010] The port module includes a second state machine submodule; the operation module includes a third state machine submodule; the memory access module includes a fourth state machine submodule;

[0011] A first state machine submodule, configured to control the startup and configuration of the port module, the cache module, the memory access module, and the operation module according to the received startup signal and the first configuration signal;

[0012] The first configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, the number of source data 2 batches, whether the batches of data correspond, whether source data 2 is flipped, and whether source data 2 is conjugated;

[0013] A configuration submodule, configured to receive a start signal and a configuration signal via a bus;

[0014] A status submodule, used to send the working status of the accelerator through the bus;

[0015] The configuration submodule receives the start signal and the configuration signal through the bus. After querying and parsing the start signal and the first configuration signal configured by the configuration submodule, the first state machine submodule starts and configures the port module through the second state machine submodule. After the control module receives the operation start signal of the port module, the memory access module and the operation module are started and configured accordingly through the third state machine submodule and the fourth state machine submodule. After the control module receives the end signal of the port module and the memory access module, a reset signal is sent to the port module, the cache module, the memory access module and the operation module, and the end status is written to the state submodule, waiting for the next startup.

[0016] Specifically, the port module also includes an input port submodule and an output port submodule;

[0017] A second state machine submodule, configured to receive and analyze a startup signal and a second configuration signal sent by the control module;

[0018] An input port submodule, configured to store source data into a cache module via a bus;

[0019] The output port submodule is used to send the result data in the cache module to the bus through the bus;

[0020] The second configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, and the number of source data 2 batches;

[0021] The workflow of each sub-module of the port module includes:

[0022] The second state machine submodule receives and parses the start signal and the second configuration signal sent by the control module. After parsing the start signal and the second configuration signal, the second state machine submodule starts and configures the input port submodule and the output port submodule according to the parsed second configuration signal.

[0023] Specifically, the cache module includes a source data 1 ping area, a source data 1 pong area, a source data 2 area, a result data ping area, and a result data pong area; the steps of performing data caching by the cache module include:

[0024] A1. If source data 1 does not require ping-pong storage, the input port submodule stores source data 1 in the source data 1 ping-pong area of ​​the cache module via the bus. After source data 1 is stored, source data 2 is stored in the source data 2 area of ​​the cache module via the bus until source data 2 is stored.

[0025] A2. If the source data 1 requires ping-pong storage, the input port submodule stores part of the source data 1 into the source data 1 ping-pong area in the cache module through the bus;

[0026] A3, after the source data 1 ping-pong area is full, the source data 2 is stored in the source data 2 area of ​​the cache module through the bus. After the storage of the source data 2 is completed, the remaining source data 1 is ping-pong stored in the source data 1 area of ​​the cache module through the bus;

[0027] A4. After the source data 2 is stored, the second state machine submodule sends a calculation start signal to the control module. At the same time, the output port submodule ping-pong accesses the result data in the result data area in the cache module and sends it to the bus. After all the result data are sent, the second state machine submodule sends an end signal to the control module and waits for the control module to be reset.

[0028] Specifically, the source data 1 ping area, the source data 1 pong area, the source data 2 area, the result data ping area and the result data pong area are all composed of k SRAMs and have the same data storage method, where k is the operation parallelism in the operation module, and the bit width of the SRAM is twice the bit width of the processed data.

[0029] Specifically, the memory access module also includes a source data 1 distribution submodule, a source data 2 distribution submodule and a result data distribution submodule;

[0030] a fourth state machine submodule, configured to receive and parse the start signal and the third configuration signal from the control module; after parsing the start signal and the third configuration signal, the fourth state machine submodule starts and configures the source data 1 distribution submodule, the source data 2 distribution submodule, and the result data distribution submodule;

[0031] The third configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, the number of source data 2 batches, whether each batch of data corresponds, and whether source data 2 is flipped.

[0032] Specifically, the steps of performing data access by the memory access module include:

[0033] B1. If the batches of data in the third configuration signal analyzed by the fourth state machine submodule do not correspond, then each batch of data in source data 1 and each batch of data in source data 2 are accessed and calculated and sent accordingly; if the batches of data in the third configuration signal correspond, then each batch of data in source data 1 is accessed and calculated and sent only accordingly with the same batch of data in source data 2;

[0034] B2. If the source data 2 in the third configuration signal parsed by the fourth state machine submodule is not flipped, the sliding window accesses the source data 2 in the cache module in a sliding order from the beginning to the end, calculates and sends the data; if the source data 2 is flipped, the sliding window accesses the source data 2 in the cache module in a sliding order from the end to the beginning, calculates and sends the data;

[0035] B3, when the data sent by B1 or B2 to the operation module is completed, the result of the operation module is received by the result data distribution submodule and stored in the result data ping-pong area or pong area of ​​the cache module;

[0036] B4. After the calculation results are stored, the fourth state machine submodule sends an end signal to the control module and waits for the control module to be reset.

[0037] Specifically, the operation module also includes an input control submodule, an output control submodule, and a multiplication-accumulation tree submodule;

[0038] The third state machine submodule is configured to receive and parse the start signal and the fourth configuration signal of the control module. After parsing the start signal and the fourth configuration signal, the third state machine submodule starts and configures the input control submodule, the output control submodule, and the multiplication-accumulation tree submodule according to the parsed fourth configuration signal.

[0039] The fourth configuration signal includes whether the source data 2 is conjugated.

[0040] Specifically, the steps of the operation module include:

[0041] C1. Set the operation parallelism of the multiplication-accumulation tree in the operation module to k, the input control submodule selects the paths between the k input ports and the multiplication-accumulation tree, and the output control submodule selects the paths between the multiplication-accumulation tree and the output port;

[0042] C2. If the source data 2 in the fourth configuration signal parsed by the third state machine sub-module is conjugated, the input control sub-module will first conjugate the source data 2 from the memory access module, and then send it to the multiplication-accumulation tree sub-module. The operation result of each sliding window is obtained through the tree structure within the multiplication-accumulation tree sub-module, and the operation result is sent back to the memory access module through the output control module.

[0043] A multifunctional linear convolution acceleration method, comprising the following steps:

[0044] S1: After receiving a first configuration signal via a bus, the control module starts and configures the port module. After receiving and parsing the start signal and the second configuration signal from the control module, if the source data 1 does not require ping-pong storage, the input port submodule first stores the source data 1 in the source data 1 ping-pong area of ​​the cache module via the bus.

[0045] S2, after the storage of source data 1 is completed, source data 2 is stored in the source data 2 area of ​​the cache module through the bus until the storage of source data 2 is completed;

[0046] S3. If source data 1 requires ping-pong storage, the input port submodule first stores part of source data 1 into the source data 1 ping-pong area of ​​the cache module through the bus. When the source data 1 ping-pong area is full, source data 2 is stored into the source data 2 area of ​​the cache module through the bus until the storage of source data 2 is completed.

[0047] S4, after the storage of source data 2 is completed, the second state machine submodule sends a calculation start signal to the control module. After the calculation start signal is sent, if source data 1 requires ping-pong storage, the input port submodule bus ping-pongs the remaining source data 1 to the source data 1 area in the cache module, and the output port submodule ping-pongs the result data in the result data area in the cache module and sends it to the bus. After all the result data are sent, the second state machine submodule sends an end signal to the control module and waits for the control module to be reset.

[0048] S5. After receiving the operation start signal from the port module, the control module starts and configures the memory access module and the operation module through the third configuration signal and the fourth configuration signal;

[0049] S6. After receiving the operation start signal and the third configuration signal from the control module, the memory access module accesses the source data and coefficients in a sliding window manner, and sends the overlapping portion k of the window and the data to the operation module in parallel; if the overlapping portion is less than k, it is padded with zeros and sent in k parallel batches; if the overlapping portion exceeds k, it is sent in k parallel batches in multiple batches, and the remainder is padded with zeros and sent in k parallel batches to the operation module;

[0050] S7. After receiving the calculation start signal and the fourth configuration signal from the control module, the calculation module calculates the data from each sliding window of the memory access module and sends the calculation result of each sliding window to the memory access module;

[0051] S8, when the operation module completes the operation, the result data distribution submodule receives the operation result of the operation module and switches the storage of the result data to the ping-pong area or the pong area according to the current operation result points and the ping-pong storage boundary of the result data area in the cache module;

[0052] S9. When all result data are stored, the memory access module sends an end signal to the control module and waits for the control module to be reset.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] The present invention addresses the deficiencies of the existing technology by multiplexing memory access logic and operation logic to achieve high-performance hardware acceleration of commonly used linear convolution-based digital signal processing algorithms with a lower hardware area. In addition, the multifunctional linear convolution accelerator simultaneously supports multiple batch operations, improving data processing efficiency in multi-batch operation scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a diagram of the overall architecture of a multifunctional linear convolution accelerator according to Example 1 of the present invention;

[0056] Figure 2 This is a schematic diagram of data storage in the cache module source data 1 ping area according to embodiment 1 of the present invention;

[0057] Figure 3 This is a schematic diagram of the linear convolution, FIR, cross-correlation and autocorrelation algorithms in Example 1 of the present invention;

[0058] Figure 4 Schematic diagram of the structure of the multiplication-accumulation tree module in the operation module of Example 1 of the present invention. DETAILED DESCRIPTION

[0059] Example 1

[0060] See also Figure 1 , an embodiment provided by the present invention: a multifunctional linear convolution accelerator, comprising: a control module, a port module, a cache module, a memory access module, a calculation module and a bus;

[0061] Control module: used to control the startup and configuration of each module in the accelerator;

[0062] Port module: used to control data transmission between the bus and cache module;

[0063] Cache module: used to cache source data and calculation results;

[0064] Memory access module: used to access the source data of the cache module and send it to the calculation module, and at the same time receive the calculation results of the calculation module and store them in the cache module;

[0065] Operation module: receives source data from the access module and performs operations in various ways.

[0066] The control module includes a first state machine submodule, a configuration submodule and a state submodule;

[0067] The port module includes a second state machine submodule, an input port submodule, and an output port submodule; the operation module includes a third state machine submodule, an input control submodule, an output control submodule, and a multiplication-accumulation tree submodule; the memory access module includes a fourth state machine submodule, a source data 1 distribution submodule, a source data 2 distribution submodule, and a result data distribution submodule;

[0068] A first state machine submodule, configured to control the startup and configuration of the port module, the cache module, the memory access module, and the operation module according to the received startup signal and the first configuration signal;

[0069] The first configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, the number of source data 2 batches, whether the batches of data correspond, whether source data 2 is flipped, and whether source data 2 is conjugated;

[0070] A configuration submodule, configured to receive a start signal and a configuration signal via a bus;

[0071] A status submodule, used to send the working status of the accelerator through the bus;

[0072] The configuration submodule receives the start signal and the configuration signal through the bus. After querying and parsing the start signal and the first configuration signal configured by the configuration submodule, the first state machine submodule starts and configures the port module through the second state machine submodule. After the control module receives the operation start signal of the port module, the memory access module and the operation module are started and configured accordingly through the third state machine submodule and the fourth state machine submodule. After the control module receives the end signal of the port module and the memory access module, a reset signal is sent to the port module, the cache module, the memory access module and the operation module, and the end status is written to the state submodule, waiting for the next startup.

[0073] A second state machine submodule, configured to receive and analyze a startup signal and a second configuration signal sent by the control module;

[0074] An input port submodule, configured to store source data into a cache module via a bus;

[0075] The output port submodule is used to send the result data in the cache module to the bus through the bus;

[0076] The second configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, and the number of source data 2 batches;

[0077] Furthermore, in this embodiment, the workflow of each submodule of the port module includes:

[0078] The second state machine submodule receives and parses the start signal and the second configuration signal sent by the control module. After parsing the start signal and the second configuration signal, the second state machine submodule starts and configures the input port submodule and the output port submodule according to the parsed second configuration signal.

[0079] Furthermore, in this embodiment, the cache module includes a source data 1 ping area, a source data 1 pong area, a source data 2 area, a result data ping area, and a result data pong area;

[0080] Furthermore, in this embodiment, the steps of the cache module performing data caching include:

[0081] A1. If source data 1 does not require ping-pong storage, the input port submodule stores source data 1 in the source data 1 ping-pong area of ​​the cache module via the bus. After source data 1 is stored, source data 2 is stored in the source data 2 area of ​​the cache module via the bus until source data 2 is stored.

[0082] Furthermore, the ping-pong buffering technique used in this embodiment is a commonly used data flow control technique, primarily used to efficiently process and transmit data streams. Ping-pong buffering achieves seamless data buffering and processing by isochronously distributing the input data stream to two data buffers and switching between them. This method is particularly well-suited for pipeline processing, ensuring data flow continuity and real-time processing.

[0083] A2. If the source data 1 requires ping-pong storage, the input port submodule stores part of the source data 1 into the source data 1 ping-pong area in the cache module through the bus;

[0084] A3, after the source data 1 ping-pong area is full, the source data 2 is stored in the source data 2 area of ​​the cache module through the bus. After the storage of the source data 2 is completed, the remaining source data 1 is ping-pong stored in the source data 1 area of ​​the cache module through the bus;

[0085] A4. After the source data 2 is stored, the second state machine submodule sends a calculation start signal to the control module. At the same time, the output port submodule ping-pong accesses the result data in the result data area in the cache module and sends it to the bus. After all the result data are sent, the second state machine submodule sends an end signal to the control module and waits for the control module to be reset.

[0086] Further, see Figure 1 and Figure 2 In this embodiment, the source data 1 ping area, the source data 1 pong area, the source data 2 area, the result data ping area, and the result data pong area are all composed of k SRAMs and have the same data storage method, where k is the operation parallelism in the operation module, and the bit width of the SRAM is twice the bit width of the processed data.

[0087] Furthermore, in order to better illustrate the data storage process, this embodiment uses source data 1 as an example to perform data storage, including:

[0088] First, the first batch of data of source data 1 is stored sequentially from address 0 of SRAM with sequence number 0 to address 0 of SRAM with sequence number k-1;

[0089] If the address 0 of the k SRAMs is full, the remaining data will be stored sequentially in the address 1 of the k SRAMs. Similarly, if the first batch of source data 1 is stored, the second batch of data will continue to be stored until the source data 1 is stored.

[0090] Furthermore, in this embodiment, taking source data 1 ping area as an example, detailed steps for data storage include:

[0091] S101 , after receiving source data 1 , the input port submodule stores it in the source data 1 ping-pong area of ​​the cache module; the source data 1 ping-pong area is composed of k SRAMs, each of which has a certain storage capacity.

[0092] S102, store the first batch of data: store the first batch of source data 1 starting from SRAM with sequence number 0, with address starting from 0;

[0093] For example, assuming k=4, the source data 1 has two batches of data, each with 12 elements. Then the first 8 elements of the first batch are sequentially stored in address 0 of SRAM0~SRAM3, and the remaining 4 elements of the first batch are sequentially stored in address 1 of SRAM0~SRAM1. Then the first 4 elements of the second batch are sequentially stored in address 1 of SRAM2~SRAM3, and finally the remaining 8 elements of the second batch are sequentially stored in address 2 of SRAM0~SRAM3.

[0094] Ping-pong storage: If source data 1 requires ping-pong storage, when the ping-pong area of ​​source data 1 is full, the next source data 2 will be stored in source data 2 area, and then the remaining source data 1 will be stored in source data 1 pong area; this alternating storage ensures continuous data processing.

[0095] For example, assume that source data 1 has two batches of data, each with 10 elements, and k=4. That is, the ping-pong area and pong area of ​​source data 1 each have 4 SRAMs (SRAM0-SRAM3), and the SRAM depth is 2. The storage process of source data 1 is as follows:

[0096] First, the first 8 elements of the first batch are sequentially stored in address 0 of source data 1 ping-pong area SRAM0~SRAM3, and then the remaining 2 elements of the first batch are sequentially stored in address 1 of source data 1 ping-pong area SRAM0, and then the first 6 elements of the second batch are sequentially stored in address 1 of source data 1 ping-pong area SRAM1~SRAM3, and finally the remaining 4 elements of the second batch are sequentially stored in address 0 of source data 1 ping-pong area SRAM0~SRAM1.

[0097] This process ensures seamless data buffering and continuous processing by isochronously allocating the input data stream to two data buffers (the ping-pong area and the pong area) and switching between them. In particular, when source data 1 requires ping-pong storage, data can be efficiently switched between the ping-pong and pong areas, reducing data processing interruptions and memory access conflicts.

[0098] This design adapts the SRAM storage method to different data types. The bit width of each SRAM is twice the bit width of the data being processed. This allows the system to process large amounts of data while maintaining high throughput and low latency, significantly improving the overall performance and reliability of the signal processing system.

[0099] Furthermore, in this embodiment, the fourth state machine submodule is configured to receive and parse the start signal and the third configuration signal of the control module. After parsing the start signal and the third configuration signal, the fourth state machine submodule starts and configures the source data 1 distribution submodule, the source data 2 distribution submodule, and the result data distribution submodule.

[0100] Furthermore, in this embodiment, the third configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, the number of source data 2 batches, whether each batch of data corresponds, and whether source data 2 is flipped.

[0101] Furthermore, in this embodiment, the steps of performing data access by the memory access module include:

[0102] B1. If the batches of data in the third configuration signal analyzed by the fourth state machine submodule do not correspond, then each batch of data in source data 1 and each batch of data in source data 2 are both accessed and calculated and sent accordingly; if the batches of data in the third configuration signal correspond, then each batch of data in source data 1 is only accessed and calculated and sent accordingly with the same batch of data in source data 2;

[0103] B2. If the source data 2 in the third configuration signal parsed by the fourth state machine sub-module is not flipped, the sliding window slides sequentially from the beginning to the end to access the source data 2 in the cache module and calculates and sends it; if the source data 2 is flipped, the sliding window slides in reverse order from the end to the beginning to access the source data 2 in the cache module and calculates and sends it.

[0104] Furthermore, in order to more clearly illustrate steps B1-B2, in this embodiment, the specific steps of data access are as follows, taking the case where the batches of data do not correspond and the source data 2 is not flipped as an example:

[0105] B3. Assume that the fourth state machine submodule receives and parses the third configuration signal, and obtains the parsed third configuration signal as follows: each batch of data does not correspond and the source data 2 does not flip;

[0106] B4. Based on the parsed third configuration signal, the fourth state machine submodule is started and configured with respect to the source data 1 distribution submodule and the source data 2 distribution submodule;

[0107] B5. Assume that the source data 1 cached in the cache module is p batches of N-point vectors, and the source data 2 is q batches of M-point vectors. Use the configured source data 1 distribution submodule to access, calculate, and send the first batch of source data 1 in the source data area of ​​the cache module. Simultaneously, use the configured source data 2 distribution submodule to access, calculate, and send the first batch of source data 2 in the source data area of ​​the cache module.

[0108] Furthermore, in this embodiment, the steps of using the configured source data 1 distribution submodule to access, calculate, and send the first batch of source data 1 in the source data area of ​​the cache module include:

[0109] B511. Set a sliding window 1 of length M in the source data 1 distribution module, and use the overlapping position of the end of sliding window 1 and the beginning of the first batch of source data 1 as the starting position, and the overlapping position of the beginning of sliding window 1 and the end of the first batch of source data 1 as the ending position. Slide sliding window 1 N+M-1 times, sequentially access the overlapping part of each sliding window 1 with the first batch of source data 1, and send it to the calculation module;

[0110] B512. For each sliding window 1, divide the length of the overlapping portion between sliding window 1 and the first batch of data of source data 1 by k, perform g parallel accesses on the divisible portion, and send the accessed data to the calculation module for k parallel operations. Simultaneously, perform one parallel access on the remainder portion for the remainder length, and after padding with zeros, send the accessed data to the calculation module for k parallel operations.

[0111] B513. If the source data 1 requires ping-pong storage, the source data 1 distribution module switches access to the ping-pong area or the pong area of ​​the source data 1 according to the number of source data 1 points, the number of source data 1 batches, and the storage boundaries of the ping-pong area and the pong area of ​​the source data 1.

[0112] Furthermore, in this embodiment, the steps of using the configured source data 2 distribution submodule to access, calculate, and send the first batch of source data 2 in the source data area of ​​the cache module include:

[0113] B521. Set a sliding window 2 of length N in the source data 2 distribution module. Slide the sliding window 2 N+M-1 times according to the method of B511. Access the overlapping part of each sliding window 2 and the first batch of source data 2 in reverse order and send it to the calculation module.

[0114] B522: When the data access and operation of the first batch of data of source data 1 and the first batch of data of source data 2 are completed, the first batch of data of source data 1 and the second batch of data of source data 2 are accessed according to B511-B513 and B521;

[0115] B6. When the data access and calculation and sending of the first batch of data of source data 1 and each batch of data of source data 2 are completed, the second batch of data of source data 1 and each batch of data of source data 2 are accessed in the same manner as B511-B513 and B521-B522 until the corresponding data access and calculation and sending of each batch of data of source data 1 and each batch of data of source data 2 are completed;

[0116] B7, when the operation module completes the operation, the result data distribution submodule receives the operation result of the operation module and switches the storage of the result data to the ping-pong area or the pong area according to the current operation result points and the ping-pong storage boundary of the result data area in the cache module;

[0117] B8. When all result data are stored, the memory access module sends an end signal to the control module and waits for the control module to be reset.

[0118] Furthermore, according to the above-mentioned process B1-B8, the specific steps of implementing the source data access of the FIR algorithm in this embodiment include:

[0119] B11. Assuming that the third configuration signal parsed by the fourth state machine submodule is to fix the selection of source data 2 not to be flipped and source data 2 not to be conjugated, then source data 2 is regarded as q groups of M-order filter coefficients;

[0120] B111. When the selected source data 2 batches do not correspond, each batch of source data 1 is subjected to q M-order FIR filtering times, and a total of p*q FIR filtering times;

[0121] B112. When selecting the correspondence between each batch of data, each batch of source data 1 will only perform one FIR operation with the corresponding batch of source data 2;

[0122] For example, in order to further illustrate the FIR algorithm to access source data, as shown in FIG. Figure 3 As shown, the specific steps of the FIR algorithm for source data access include:

[0123] Suppose source data 1 is a vector of N points A = [x[0], x[1], ..., x[N-1]] and source data 2 is a vector of M points B = [h[0], h[1], ..., h[M-1]];

[0124] Furthermore, assuming that N and M in this embodiment are both 5, a clearer example of the FIR algorithm performing source data access is:

[0125] Q1. Assume that source data 1 is a batch of 5-point vectors A = [x[0], x[1], ..., x[4]], and source data 2 is a batch of 5-point vectors B = [h[0], h[1], ..., h[4]]. Under the FIR, cross-correlation, and autocorrelation algorithms, 5+5-1 calculation results will be obtained. The parallelism of the multiplication-accumulation tree of the operation module is 4. The cache module contains the source data 1 ping area, the source data 1 pong area, the source data 2 area, the result data ping area, and the result data pong area. Each area is composed of 4 SRAMs.

[0126] Q2, first access the data of the first sliding window: access x[0] in source data area 1 and h[0] in source data area 2, fill 0 to [x[0],0,0,0] and [h[0],0,0,0], and then send 4 in parallel to the calculation module;

[0127] Q3, then perform the second sliding window: access x[0], x[1] and h[1], h[0], fill in 0 to [x[0], x[1], 0, 0] and [h[1], h[0], 0, 0], and then send the 4 to the calculation module in parallel;

[0128] Q4, the third sliding window is performed: access x[0], x[1], x[2] and h[2], h[1], h[0], fill in 0 to [x[0], x[1], x[2], 0] and [h[2], h[1], h[0], 0], and then send the 4 to the calculation module in parallel.

[0129] Q5, the fourth sliding window is performed: access x[0], x[1], x[2], x[3] and h[3], h[2], h[1], h[0], and 4 are sent to the calculation module in parallel.

[0130] Q6, the fifth sliding window is performed: first access x[0], x[1], x[2], x[3] and h[4], h[3], h[2], h[1], and send 4 in parallel to the calculation module; then access x[4] and h[0], fill in 0 to [x[4], 0, 0, 0] and [h[0], 0, 0, 0] and send them to the calculation module.

[0131] Q7, the sixth sliding window is performed: access x[1], x[2], x[3], x[4] and h[4], h[3], h[2], h[1], and 4 are sent to the calculation module in parallel.

[0132] Q8, the seventh sliding window is performed: access x[2], x[3], x[4] and h[4], h[3], h[2], fill in 0 to [x[2], x[3], x[4], 0] and [h[4], h[3], h[2], 0], and then send 4 in parallel to the calculation module.

[0133] Q9, the eighth sliding window is performed: access x[3], x[4] and h[4], h[3], fill in 0 to [x[3], x[4], 0, 0] and [h[4], h[3], 0, 0], and then send 4 in parallel to the calculation module.

[0134] Q10. Finally, perform the 9th sliding window: access x[4] and h[4], fill in 0 to [x[4],0,0,0] and [h[4],0,0,0], and then send 4 in parallel to the calculation module.

[0135] B12. To implement the cross-correlation algorithm, select source data 2 flip and source data 2 conjugate, and treat source data 1 and source data 2 as two cross-correlated vectors. Repeat B111-B113 to implement the corresponding configuration options and operations for each batch of data.

[0136] Furthermore, specific examples of source data access to implement the cross-correlation algorithm in this embodiment include:

[0137] First, the data of the first sliding window is accessed: x[0] in source data area 1 and h[4] in source data area 2 are accessed, 0 is added to [x[0],0,0,0] and [h[4],0,0,0], and then the 4 are sent to the calculation module in parallel.

[0138] Then, perform the second sliding window: access x[0], x[1] and h[3], h[4], fill in 0 to [x[0], x[1], 0, 0] and [h[3], h[4], 0, 0], and then send the 4 to the calculation module in parallel.

[0139] The third sliding window is performed: x[0], x[1], x[2] and h[2], h[3], h[4] are accessed, 0 is added to [x[0], x[1], x[2], 0] and [h[2], h[3], h[4], 0], and the 4 are sent to the calculation module in parallel.

[0140] Fourth, perform the fourth sliding window: access x[0], x[1], x[2], x[3] and h[1], h[2], h[3], h[4], and send 4 in parallel to the calculation module.

[0141] The fifth sliding window is performed: first access x[0], x[1], x[2], x[3] and h[0], h[1], h[2], h[3], and send 4 in parallel to the calculation module; then access x[4] and h[4], fill them with 0 to [x[4], 0, 0, 0] and [h[4], 0, 0, 0], and send 4 in parallel to the calculation module.

[0142] Sixth, perform the 6th sliding window: access x[1], x[2], x[3], x[4] and h[0], h[1], h[2], h[3], and send 4 in parallel to the calculation module.

[0143] Seventh, perform the 7th sliding window: access x[2], x[3], x[4] and h[0], h[1], h[2], fill in 0 to [x[2], x[3], x[4], 0] and [h[0], h[1], h[2], 0], and then send the 4 to the calculation module in parallel.

[0144] The eighth sliding window is performed: x[3], x[4] and h[0], h[1] are accessed, 0 is added to [x[3], x[4], 0, 0] and [h[0], h[1], 0, 0], and the 4 are sent to the calculation module in parallel.

[0145] Finally, perform the 9th sliding window: access x[4] and h[0], fill in 0 to [x[4],0,0,0] and [h[0],0,0,0], and then send 4 in parallel to the calculation module.

[0146] B15. To implement the autocorrelation algorithm, select the corresponding data batches and configure the same options as the cross-correlation algorithm. However, the source data 1 and source data 2 sent to the port module through the bus must be the same data, that is, vector A and vector B must be the same vector.

[0147] The fourth state machine submodule in this embodiment can dynamically configure the source data 1 distribution submodule, the source data 2 distribution submodule, and the result data distribution submodule based on the received start signal and the third configuration signal. This means that the system can flexibly adjust the data processing flow according to different computational requirements and configuration parameters, thereby improving the system's flexibility and adaptability. For example, when performing an FIR filtering operation, the system can decide whether to perform multiple filtering operations or only one filtering operation based on whether to select each batch of data. This can not only meet different computational requirements but also optimize computational efficiency. Secondly, through the sliding window mechanism and parallel access strategy, the system can efficiently process complex mathematical operations such as FIR filtering, cross-correlation, and autocorrelation. The sliding window sequentially slides from the beginning to the end to access the data in the cache module, not only ensuring the continuity and consistency of data access, but also increasing the speed of data processing through parallel processing technology. For example, in the FIR algorithm, the data accessed by the sliding window each time can be sent to the computing module in parallel after being padded with zeros, which greatly improves the throughput and processing power of the computing module.

[0148] Furthermore, in this embodiment, the third state machine submodule is configured to receive and parse the start signal and the fourth configuration signal of the control module. After parsing the start signal and the fourth configuration signal, the third state machine submodule starts and configures the input control submodule, the output control submodule, and the multiplication-accumulation tree submodule according to the parsed fourth configuration signal.

[0149] Furthermore, in this embodiment, the multiplication-accumulation tree submodule includes a multiplication-accumulation tree, see Figure 4 , when k = 4, the specific operation process of the multiplication-accumulation tree is:

[0150] Assume that the four complex source data 1 and the four complex source data 2 accessed in parallel from the source data area in the cache module are (src1_1, src1_2, src1_3, src1_4) and (src2_1, src2_2, src2_3, src2_4) in sequence;

[0151] Construct the multiply-accumulate input pairs (src1_1, src2_1), (src1_2, src2_2), (src1_3, src2_3), and (src1_4, src2_4) using (src1_1, src1_2, src1_3, src1_4) and (src2_1, src2_2, src2_3, src2_4).

[0152] Multiply the input pairs (src1_1, src2_1) and (src1_2, src2_2) and add them together to get the result 1. Multiply the input pairs (src1_3, src2_3) and (src1_4, src2_4) and add them together to get the result 2.

[0153] Add result 1 and result 2 to get result 3, and then add result 3 to itself to get the final output of the multiplication-accumulation tree;

[0154] The fourth configuration signal includes whether the source data 2 is conjugated.

[0155] Furthermore, the steps of the operation module in this embodiment include:

[0156] C1. Set the operation parallelism of the multiplication-accumulation tree in the operation module to k, the input control submodule selects the paths between the k input ports and the multiplication-accumulation tree, and the output control submodule selects the paths between the multiplication-accumulation tree and the output port;

[0157] C2. If the source data 2 in the fourth configuration signal parsed by the third state machine sub-module is conjugated, the input control sub-module will first conjugate the source data 2 from the memory access module, and then send it to the multiplication-accumulation tree sub-module. The operation result of each sliding window is obtained through the tree structure within the multiplication-accumulation tree sub-module, and the operation result is sent back to the memory access module through the output control module.

[0158] Example 2

[0159] Another embodiment provided by the present invention is a multifunctional linear convolution acceleration method, comprising the following steps:

[0160] S1: After receiving a first configuration signal via a bus, the control module starts and configures the port module. After receiving and parsing the start signal and the second configuration signal from the control module, if the source data 1 does not require ping-pong storage, the input port submodule first stores the source data 1 in the source data 1 ping-pong area of ​​the cache module via the bus.

[0161] S2, after the storage of source data 1 is completed, source data 2 is stored in the source data 2 area of ​​the cache module through the bus until the storage of source data 2 is completed;

[0162] S3. If source data 1 requires ping-pong storage, the input port submodule first stores part of source data 1 into the source data 1 ping-pong area of ​​the cache module through the bus. When the source data 1 ping-pong area is full, source data 2 is stored into the source data 2 area of ​​the cache module through the bus until the storage of source data 2 is completed.

[0163] S4, after the storage of source data 2 is completed, the second state machine submodule sends a calculation start signal to the control module. After the calculation start signal is sent, if source data 1 requires ping-pong storage, the input port submodule bus ping-pongs the remaining source data 1 to the source data 1 area in the cache module, and the output port submodule ping-pongs the result data in the result data area in the cache module and sends it to the bus. After all the result data are sent, the second state machine submodule sends an end signal to the control module and waits for the control module to be reset.

[0164] S5. After receiving the operation start signal from the port module, the control module starts and configures the memory access module and the operation module through the third configuration signal and the fourth configuration signal;

[0165] S6. After receiving the operation start signal and the third configuration signal from the control module, the memory access module accesses the source data and coefficients in a sliding window manner, and sends the overlapping part k of the window and data to the operation module in parallel; if the overlapping part is less than k, it is padded with zeros and then sent in parallel k times; if the overlapping part exceeds k, it is divided into multiple batches and sent in parallel k times, and the remainder is padded with zeros and then sent in parallel k times to the operation module.

[0166] Furthermore, in order to explain the S1-S6 method in more detail in this embodiment, the exemplary steps are as follows:

[0167] S7. Let source data 1 be a vector of N points A = [x[0], x[1], ..., x[N-1]] and source data 2 be a vector of M points B =

[0168] [h[0], h[1], ..., h[M-1]], and in this embodiment, N and M are both set to 5, then the source data 1 is a batch of 5-point vectors A = [x[0], x[1], ..., x[4]], and the source data 2 is a batch of 5-point vectors B = [h[0], h[1], ..., h[4]];

[0169] Furthermore, in this embodiment, the FIR, cross-correlation and autocorrelation algorithms perform a sliding window memory access process as follows: Figure 3 , and Figure 3 The steps of the corresponding algorithm when accessing memory in a sliding window manner are demonstrated through examples of specific values ​​of N and M. The specific process is as follows:

[0170] Furthermore, according to the parameters set in S7, in this embodiment, 5+5-1 calculation results are obtained when accessing memory under the FIR, cross-correlation, and autocorrelation algorithms. The computational parallelism of the multiplication-accumulation tree of the operation module is 4. The cache module includes a source data 1 ping area, a source data 1 pong area, a source data 2 area, a result data ping area, and a result data pong area. Each area is composed of 4 SRAMs.

[0171] S8, after receiving the operation start signal and the third configuration signal from the control module, the memory access module accesses the source data and coefficients in a sliding window manner, and sends the overlapping part of the window and the data to the operation module in parallel by 4. If the overlapping part is less than 4, it is padded with zeros and sent in parallel by 4. If the overlapping part exceeds 4, it is sent in parallel by 4 in multiple batches, and the remainder is padded with zeros and sent in parallel by 4.

[0172] Furthermore, in this embodiment, the source data and coefficients are accessed in a sliding window manner, where the sliding window includes sliding windows corresponding to the FIR algorithm, the cross-correlation algorithm, and the autocorrelation algorithm;

[0173] For example, if the source data access of the FIR algorithm is to be implemented, the specific steps corresponding to the sliding window access include:

[0174] Q2, first access the data of the first sliding window: access x[0] in source data area 1 and h[0] in source data area 2, fill 0 to [x[0],0,0,0] and [h[0],0,0,0], and then send 4 in parallel to the calculation module;

[0175] Q3, then perform the second sliding window: access x[0], x[1] and h[1], h[0], fill in 0 to [x[0], x[1], 0, 0] and [h[1], h[0], 0, 0], and then send the 4 to the calculation module in parallel;

[0176] Q4, the third sliding window is performed: access x[0], x[1], x[2] and h[2], h[1], h[0], fill in 0 to [x[0], x[1], x[2], 0] and [h[2], h[1], h[0], 0], and then send the 4 to the calculation module in parallel.

[0177] Q5, the fourth sliding window is performed: access x[0], x[1], x[2], x[3] and h[3], h[2], h[1], h[0], and 4 are sent to the calculation module in parallel.

[0178] Q6, the fifth sliding window is performed: first access x[0], x[1], x[2], x[3] and h[4], h[3], h[2], h[1], and send 4 in parallel to the calculation module; then access x[4] and h[0], fill in 0 to [x[4], 0, 0, 0] and [h[0], 0, 0, 0] and send them to the calculation module.

[0179] Q7, the sixth sliding window is performed: access x[1], x[2], x[3], x[4] and h[4], h[3], h[2], h[1], and 4 are sent to the calculation module in parallel.

[0180] Q8, the seventh sliding window is performed: access x[2], x[3], x[4] and h[4], h[3], h[2], fill in 0 to [x[2], x[3], x[4], 0] and [h[4], h[3], h[2], 0], and then send 4 in parallel to the calculation module.

[0181] Q9, the eighth sliding window is performed: access x[3], x[4] and h[4], h[3], fill in 0 to [x[3], x[4], 0, 0] and [h[4], h[3], 0, 0], and then send 4 in parallel to the calculation module.

[0182] Q10. Finally, perform the 9th sliding window: access x[4] and h[4], fill in 0 to [x[4],0,0,0] and [h[4],0,0,0], and then send 4 in parallel to the calculation module.

[0183] For example, if the source data access of the cross-correlation algorithm is to be implemented, the specific steps of the corresponding sliding window access include:

[0184] First, the data of the first sliding window is accessed: x[0] in source data area 1 and h[4] in source data area 2 are accessed, 0 is added to [x[0],0,0,0] and [h[4],0,0,0], and then the 4 are sent to the calculation module in parallel.

[0185] Then, perform the second sliding window: access x[0], x[1] and h[3], h[4], fill in 0 to [x[0], x[1], 0, 0] and [h[3], h[4], 0, 0], and then send the 4 to the calculation module in parallel.

[0186] The third sliding window is performed: x[0], x[1], x[2] and h[2], h[3], h[4] are accessed, 0 is added to [x[0], x[1], x[2], 0] and [h[2], h[3], h[4], 0], and the 4 are sent to the calculation module in parallel.

[0187] Fourth, perform the fourth sliding window: access x[0], x[1], x[2], x[3] and h[1], h[2], h[3], h[4], and send 4 in parallel to the calculation module.

[0188] The fifth sliding window is performed: first access x[0], x[1], x[2], x[3] and h[0], h[1], h[2], h[3], and send 4 in parallel to the calculation module; then access x[4] and h[4], fill them with 0 to [x[4], 0, 0, 0] and [h[4], 0, 0, 0], and send 4 in parallel to the calculation module.

[0189] Sixth, perform the 6th sliding window: access x[1], x[2], x[3], x[4] and h[0], h[1], h[2], h[3], and send 4 in parallel to the calculation module.

[0190] Seventh, perform the 7th sliding window: access x[2], x[3], x[4] and h[0], h[1], h[2], fill in 0 to [x[2], x[3], x[4], 0] and [h[0], h[1], h[2], 0], and then send the 4 to the calculation module in parallel.

[0191] The eighth sliding window is performed: x[3], x[4] and h[0], h[1] are accessed, 0 is added to [x[3], x[4], 0, 0] and [h[0], h[1], 0, 0], and the 4 are sent to the calculation module in parallel.

[0192] Finally, perform the 9th sliding window: access x[4] and h[0], fill in 0 to [x[4],0,0,0] and [h[0],0,0,0], and then send 4 in parallel to the calculation module.

[0193] For example, to implement source data access for the autocorrelation algorithm, the corresponding batches of data are fixedly selected, and the same options as the cross-correlation algorithm are configured, except that vector A and vector B must be the same vector.

[0194] S9. After receiving the calculation start signal and the fourth configuration signal from the control module, the calculation module calculates the data from each sliding window of the memory access module and sends the calculation result of each sliding window to the memory access module;

[0195] Furthermore, in this embodiment, if the FIR algorithm is to be implemented, the input control submodule directly sends source data 1 and source data 2 to the multiplication-accumulation tree operation; if the cross-correlation or autocorrelation algorithm is to be implemented, the input control module first takes the conjugate of source data 2, and then sends source data 1 and source data 2 to the multiplication-accumulation tree operation;

[0196] S9, when the operation module completes the operation, the result data distribution submodule receives the operation result of the operation module and switches the storage of the result data to the ping-pong area or the pong area according to the current operation result points and the ping-pong storage boundary of the result data area in the cache module;

[0197] S10, when all result data are stored, the memory access module sends an end signal to the control module and waits for the control module to be reset;

[0198] S11. After receiving the end signals from the port module and the memory access module, the first status submodule sends a reset signal to each module and writes the end status to the status submodule, waiting for the next startup.

[0199] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also change, modify, replace and modify the above-mentioned embodiments without departing from the scope of protection of the purpose of the present invention and the claims, and all of these are protected by the present invention.

[0200] If the technical solution disclosed herein involves personal information, the product using the technical solution disclosed herein has clearly informed the individual of the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution disclosed herein involves sensitive personal information, the product using the technical solution disclosed herein has obtained the individual's separate consent before processing the sensitive personal information and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, a clear and prominent sign is set to inform that the individual has entered the personal information collection scope and that personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information. The personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. A multifunctional linear convolution accelerator, characterized in that: include: Control module, port module, cache module, memory access module, operation module and bus; The control module includes a first state machine submodule, a configuration submodule and a state submodule; The port module includes a second state machine submodule; the operation module includes a third state machine submodule; the memory access module includes a fourth state machine submodule; The first state machine submodule is configured to control the startup and configuration of the port module, the cache module, the memory access module, and the operation module according to the received startup signal and the first configuration signal; The first configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, the number of source data 2 batches, whether the batches of data correspond, whether source data 2 is flipped, and whether source data 2 is conjugated; The configuration submodule is used to receive a start signal and a configuration signal via a bus; The status submodule is used to send the working status of the accelerator through the bus; The configuration submodule receives a start signal and a configuration signal through a bus. After querying and parsing the start signal and the first configuration signal configured by the configuration submodule, the first state machine submodule starts and configures the port module through the second state machine submodule. After the control module receives the operation start signal of the port module, the memory access module and the operation module are correspondingly started and configured through the third state machine submodule and the fourth state machine submodule. After the control module receives the end signal of the port module and the memory access module, a reset signal is sent to the port module, the cache module, the memory access module and the operation module, and an end state is written to the state submodule, waiting for the next start.

2. The multifunctional linear convolution accelerator according to claim 1, wherein: The port module also includes an input port submodule and an output port submodule; The second state machine submodule is configured to receive and analyze the startup signal and the second configuration signal sent by the control module; The input port submodule is used to store the source data into the cache module via the bus; The output port submodule is used to send the result data in the cache module to the bus via the bus; The second configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, and the number of source data 2 batches; The workflow of each submodule of the port module includes: The second state machine submodule receives and parses the start signal and the second configuration signal sent by the control module. After parsing the start signal and the second configuration signal, the second state machine submodule starts and configures the input port submodule and the output port submodule according to the parsed second configuration signal.

3. The multifunctional linear convolution accelerator according to claim 2, wherein: The cache module includes a source data 1 ping area, a source data 1 pong area, a source data 2 area, a result data ping area and a result data pong area; The steps of the cache module performing data caching include: A1. If the source data 1 does not require ping-pong storage, the input port submodule stores the source data 1 in the source data 1 ping-pong area of ​​the cache module via the bus. After the storage of the source data 1 is completed, the input port submodule stores the source data 2 in the source data 2 area of ​​the cache module via the bus until the storage of the source data 2 is completed. A2. If the source data 1 requires ping-pong storage, the input port submodule stores part of the source data 1 into the source data 1 ping-pong area in the cache module via the bus; A3, when the source data 1 ping-pong area is full, the source data 2 is stored in the source data 2 area of ​​the cache module via the bus, and when the storage of the source data 2 is completed, the remaining source data 1 is ping-pong stored in the source data 1 area of ​​the cache module via the bus; A4. After the source data 2 is stored, the second state machine submodule sends a calculation start signal to the control module. At the same time, the output port submodule ping-pong accesses the result data in the result data area in the cache module and sends it to the bus. After all the result data are sent, the second state machine submodule sends an end signal to the control module and waits for the control module to be reset.

4. The multifunctional linear convolution accelerator according to claim 3, wherein: The source data 1 ping area, source data 1 pong area, source data 2 area, result data ping area and result data pong area are all composed of k SRAMs and have the same data storage method, where k is the operation parallelism in the operation module, and the bit width of the SRAM is twice the bit width of the data processed by the operation module.

5. The multifunctional linear convolution accelerator according to claim 4, characterized in that: The memory access module also includes a source data 1 distribution submodule, a source data 2 distribution submodule and a result data distribution submodule; The fourth state machine submodule is configured to receive and parse the start signal and the third configuration signal of the control module. After parsing the start signal and the third configuration signal, the fourth state machine submodule starts and configures the source data 1 distribution submodule, the source data 2 distribution submodule, and the result data distribution submodule; The third configuration signal includes the number of source data 1 points, the number of source data 1 batches, the number of source data 2 points, the number of source data 2 batches, whether each batch of data corresponds, and whether source data 2 is flipped.

6. The multifunctional linear convolution accelerator according to claim 5, characterized in that: The steps of the memory access module performing data access include: B1. If the batches of data in the third configuration signal analyzed by the fourth state machine submodule do not correspond, then each batch of data in source data 1 and each batch of data in source data 2 are accessed and calculated and sent accordingly; if the batches of data in the third configuration signal correspond, then each batch of data in source data 1 is accessed and calculated and sent only accordingly with the same batch of data in source data 2; B2. If the source data 2 in the third configuration signal parsed by the fourth state machine submodule is not flipped, the sliding window accesses the source data 2 in the cache module in a sliding order from the beginning to the end, calculates and sends the data; if the source data 2 is flipped, the sliding window accesses the source data 2 in the cache module in a sliding order from the end to the beginning, calculates and sends the data; B3, when the data sent by B1 or B2 to the operation module is completed, the result of the operation module is received by the result data distribution submodule and stored in the result data ping-pong area or pong area of ​​the cache module; B4. After the calculation results are stored, the fourth state machine submodule sends an end signal to the control module and waits for the control module to be reset.

7. The multifunctional linear convolution accelerator according to claim 6, characterized in that: The operation module further includes an input control submodule, an output control submodule, and a multiplication-accumulation tree submodule; The third state machine submodule is configured to receive and parse the start signal and the fourth configuration signal of the control module. After parsing the start signal and the fourth configuration signal, the third state machine submodule starts and configures the input control submodule, the output control submodule, and the multiplication-accumulation tree submodule according to the parsed fourth configuration signal. The fourth configuration signal includes whether the source data 2 is conjugated.

8. The multifunctional linear convolution accelerator according to claim 7, characterized in that: The steps of the operation module operation include: C1. Setting the computational parallelism of the multiplication-accumulation tree in the computation module to k, the input control submodule strobing the paths between the k input ports and the multiplication-accumulation tree, and the output control submodule strobing the paths between the multiplication-accumulation tree and the output port; C2. If the source data 2 in the fourth configuration signal parsed by the third state machine sub-module is conjugated, the input control sub-module will first conjugate the source data 2 from the memory access module, and then send it to the multiplication-accumulation tree sub-module. The operation result of each sliding window is obtained through the tree structure within the multiplication-accumulation tree sub-module, and the operation result is sent back to the memory access module through the output control module.

9. A multifunctional linear convolution acceleration method, which is used to implement a multifunctional linear convolution accelerator according to any one of claims 1 to 8, characterized in that the steps include: S1: After receiving a first configuration signal via a bus, the control module starts and configures the port module. After receiving and parsing the start signal and the second configuration signal from the control module, if the source data 1 does not require ping-pong storage, the input port submodule first stores the source data 1 in the source data 1 ping-pong area of ​​the cache module via the bus. S2, after the storage of source data 1 is completed, source data 2 is stored in the source data 2 area of ​​the cache module through the bus until the storage of source data 2 is completed; S3. If source data 1 requires ping-pong storage, the input port submodule first stores part of source data 1 into the source data 1 ping-pong area of ​​the cache module through the bus. When the source data 1 ping-pong area is full, source data 2 is stored into the source data 2 area of ​​the cache module through the bus until the storage of source data 2 is completed. S4, after the storage of source data 2 is completed, the second state machine submodule sends a calculation start signal to the control module. After the calculation start signal is sent, if source data 1 requires ping-pong storage, the input port submodule bus ping-pongs the remaining source data 1 to the source data 1 area in the cache module, and the output port submodule ping-pongs the result data in the result data area in the cache module and sends it to the bus. After all the result data are sent, the second state machine submodule sends an end signal to the control module and waits for the control module to be reset. S5. After receiving the operation start signal from the port module, the control module starts and configures the memory access module and the operation module through the third configuration signal and the fourth configuration signal; S6. After receiving the operation start signal and the third configuration signal from the control module, the memory access module accesses the source data and coefficients in a sliding window manner, and sends the overlapping portion k of the window and the data to the operation module in parallel; If the overlapping part is less than k, it will be padded with zeros and sent in k parallels. If the overlapping part exceeds k, it will be sent in k parallels in multiple batches, and the remainder will be padded with zeros and sent in k parallels to the operation module. S7. After receiving the calculation start signal and the fourth configuration signal from the control module, the calculation module calculates the data from each sliding window of the memory access module and sends the calculation result of each sliding window to the memory access module; S8, when the operation module completes the operation, the result data distribution submodule receives the operation result of the operation module and switches the storage of the result data to the ping-pong area or the pong area according to the current operation result points and the ping-pong storage boundary of the result data area in the cache module; S9. When all result data are stored, the memory access module sends an end signal to the control module and waits for the control module to be reset.

Images