Method for solving multiple parallel read-write access conflicts of memory and artificial intelligence chip

Abstract

The invention provides a memory multi-parallel read-write access conflict solution method, a chip storage circuit and a computing device, the memory multi-parallel read-write access conflict solution comprises a plurality of arithmetic logic units, a storage management module and a memory, sending a parallel read-write access request to the memory through a storage management module, and obtaining data required by matrix operation; the memory receives parallel read-write access requests from the plurality of arithmetic logic units and provides data required by matrix operation for the plurality of arithmetic logic units, and the memory is configured to comprise a specific number of storage sub-modules according to the degree of parallelism of data access of the plurality of arithmetic logic units, the specific number is a prime number. According to the technical scheme, in the data parallel read-write access process, access conflicts can be reduced, and the parallel efficiency of data read-write is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence memory, in particular to a method for solving memory multi-parallel read-write access conflict, a chip memory circuit and a computing device. BACKGROUND

[0002] The demand for large-scale scientific and engineering calculations in social life is growing, and simply relying on the method of improving the operation speed of CPU has far failed to keep up with the growth rate of computing demand, so parallel computing has emerged to provide another way of thinking to solve this problem.

[0003] In the field of artificial intelligence, there are a large number of matrix operations and convolution operations and other heavy computing operations. Therefore, it is a common method to split the matrix into a large number of small matrices and then distribute them to a large number of arithmetic logic units to perform parallel operations, such as the similar parallel processing method used in GPU. An important problem in such parallel computing is the read-write problem of storage space.

[0004] In order to more clearly describe the problem, the following will use a simplified matrix multiplication as an example to describe the problem, see Figure 1 In the example in the above, the size of matrix A and matrix B is 4x4, and when multiplying matrices, the 4 elements of each row of matrix A are multiplied by the 4 elements of each column of matrix B and added together. If the above operation is performed in series, the execution order is: read the 4 elements of the first row of matrix A from the storage; read the 4 elements of the first column of matrix B from the storage; perform the multiplication and accumulation operation of the 4 elements of the first row of matrix A and the 4 elements of the first column of matrix B in turn; read the 4 elements of the second column of matrix B from the storage; perform the multiplication and accumulation operation of the 4 elements of the first row of matrix A and the 4 elements of the second column of matrix B in turn; read the 4 elements of the third column of matrix B from the storage; perform the multiplication and accumulation operation of the 4 elements of the first row of matrix A and the 4 elements of the third column of matrix B in turn; read the 4 elements of the fourth column of matrix B from the storage; perform the multiplication and accumulation operation of the 4 elements of the first row of matrix A and the 4 elements of the fourth column of matrix B in turn; read the 4 elements of the second row of matrix A from the storage; read the 4 elements of the first column of matrix B from the storage; perform the multiplication and accumulation operation of the 4 elements of the second row of matrix A and the 4 elements of the first column of matrix B in turn; and so on, until the matrix multiplication operation is completed.

[0005] If the operation of multiplying matrix A and matrix B is split into four arithmetic logic units (ALUs) and executed in parallel, with each ALU performing only the multiplication and addition of a single row of matrix A with four columns of matrix B, the specific steps are as follows: Simultaneously read four elements from the first, second, third, and fourth rows of matrix A from storage, and send the first row to the first ALU, the second row to the second, the third row to the third, and the fourth row to the fourth ALU; read the four elements from the first column of matrix B from storage and send them to the four ALUs respectively; each of the four ALUs executes the multiplication and addition of a single row of matrix A with the first column of matrix B in parallel; read the four elements from the second column of matrix B from storage and send them to the four ALUs respectively; each of the four ALUs executes the multiplication and addition of a single row of matrix A with the second column of matrix B in parallel; and so on.

[0006] As can be seen from the calculation process, parallel computing significantly increases computation speed by adding arithmetic logic units. However, this introduces the issue of parallel read / write operations in data storage. For example, matrix A might be stored in memory in multiple ways... Figure 2 As shown in (a), the data is stored sequentially in SRAM. Specifically, the data storage space consists of four SRAMs, each with 1024 rows, and each row stores one data element. The four data elements of the first row of matrix A are stored in the first row of the four SRAMs, the four data elements of the second row of matrix A are stored in the second row of the four SRAMs, and so on. During serial execution, only one data element needs to be read from the storage space each time, and this process is repeated four times to read all the data from one row of matrix A for subsequent calculations. During parallel computation, four data elements need to be read from the storage space each time, such as the first element of each row of matrix A. However, the problem is that since the first element of each row of matrix A is stored in SRAM 0, it is impossible to read multiple data elements at once.

[0007] The simplest solution to this problem is to split conflicting access requests into multiple accesses, allowing only one request to access the data at a time when data access conflicts occur. However, this method will inevitably lead to a decrease in read and write access performance. If no adjustments are made to the data storage location, splitting multiple concurrent read or write requests that need to access the same storage block into multiple accesses, allowing only one request to access the data at a time, will also result in performance degradation.

[0008] Therefore, a technical solution is needed that can reduce concurrent access conflicts while ensuring access efficiency. Summary of the Invention

[0009] The application aims to provide a method for solving memory multi-parallel read-write access conflict and a computing device, which can reduce parallel access conflict under the premise of ensuring access efficiency.

[0010] According to an aspect of the application, a data stream artificial intelligence chip is provided, comprising a plurality of arithmetic logic units, a storage management module and a memory, wherein,

[0011] When the plurality of arithmetic logic units operate in parallel, the storage management module sends a parallel read-write access request to the memory to obtain data required for operation;

[0012] The memory receives the parallel read-write access request and provides data required for operation to the plurality of arithmetic logic units,

[0013] According to the parallel degree of data access of the plurality of arithmetic logic units, the memory is configured to include a specific number of storage sub-modules, and the specific number is a prime number.

[0014] According to some embodiments, the prime number is greater than the parallel degree.

[0015] According to some embodiments, the storage management module is configured to:

[0016] Obtain a data interval of parallel read-write access;

[0017] Judge the relationship between the data interval and the number of storage sub-modules;

[0018] If the data interval is an integer multiple of the storage sub-modules, store data offset;

[0019] Parallel read-write access data required for operation.

[0020] According to some embodiments, the data offset storage comprises:

[0021] Group data, and divide data blocks in each group according to the data interval of reading data according to the parallel degree, and each data block includes an integer multiple of the storage sub-modules;

[0022] When storing the data blocks in the group, each data block is sequentially stored after being shifted left or right by one storage sub-module relative to the previous data block.

[0023] According to another aspect of the application, a method for solving memory multi-parallel read-write access conflict is provided, comprising:

[0024] According to the parallel degree of data access, divide the memory into a specific number of storage sub-modules, and the specific number is a prime number and the prime number is greater than the parallel degree;

[0025] data intervals for obtaining parallel read-write access;

[0026] determining a relationship between the data intervals and the number of storage sub-modules;

[0027] storing data offsets if the data intervals are integer multiples of the storage sub-modules;

[0028] data required for parallel read-write access operations.

[0029] According to some embodiments, the storing of data offsets comprises:

[0030] grouping data, dividing data blocks according to data intervals for reading data in parallelism in each group, and including integer multiples of the storage sub-modules in each data block;

[0031] when storing data blocks in a group, sequentially storing each data block after storing one storage sub-module to the left or right of the previous data block.

[0032] According to some embodiments, the storing of data offsets further comprises:

[0033] if the data intervals are not integer multiples of the storage sub-modules, directly sequentially storing.

[0034] According to another aspect of the present application, there is provided a chip storage circuit comprising the data flow artificial intelligence chip according to any one of the preceding aspects.

[0035] According to another aspect of the present application, there is provided a storage system comprising a storage controller and a memory, wherein the storage controller performs the method according to any one of the preceding aspects.

[0036] According to another aspect of the present application, there is provided a computer program product comprising a computer program which, when executed by a processor, performs the method according to any one of the preceding aspects.

[0037] According to another aspect of the present application, there is provided a computing device comprising:

[0038] a processor; and

[0039] a memory storing a computer program which, when executed by the processor, causes the processor to perform the method according to any one of the preceding aspects.

[0040] According to another aspect of the present application, there is provided a non-transitory computer readable storage medium having stored thereon computer readable instructions which, when executed by a processor, cause the processor to perform the method according to any one of the preceding aspects.

[0041] According to an embodiment of the present application, by dividing the memory into a certain number of storage sub-modules, and the certain number is a prime number. When the data access mode is randomly distributed, by selecting a prime number greater than the parallelism as the number of sub-modules, the data can be more evenly dispersed, reducing the access pressure of the same sub-module, reducing data parallel read-write access conflicts, thereby improving the overall access efficiency and parallelism.

[0042] According to some embodiments, by comparing the data interval with the number of storage sub-modules, it is determined whether the current data access mode may have read-write access conflicts, and according to the determination result, the data offset storage strategy is selected to further reduce the access conflicts and improve the parallel efficiency of data read-write.

[0043] It should be understood that the above general description and the following detailed description are only exemplary and do not limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows.

[0045] Figure 1 A simple matrix multiplication diagram is shown.

[0046] Figure 2 A diagram showing a conventional storage method is shown.

[0047] Figure 3 A diagram showing the structure of an apparatus according to an example data flow artificial intelligence chip is shown.

[0048] Figure 4 A diagram showing a comparison of different storage sub-module numbers in various scenarios according to an example is shown.

[0049] Figure 5 A flowchart showing a method for solving multiple parallel read-write access conflicts of a memory according to an example is shown.

[0050] Figure 6 A block diagram of a computing device according to an example embodiment of the present application is shown. DETAILED DESCRIPTION

[0051] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in various forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present application will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. Like reference numerals refer to like elements throughout the drawings, and thus repeated description thereof will be omitted.

[0052] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of the invention. However, those skilled in the art will recognize that the technical solutions of the invention can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of the invention.

[0053] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0054] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0055] It should be understood that although the terms first, second, third, etc., may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first component discussed below may be referred to as the second component without departing from the teachings of the present invention. As used herein, the term "and / or" includes all combinations of any one and more of the associated listed items.

[0056] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0057] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of exemplary embodiments, and the modules or processes in the drawings are not necessarily essential for implementing the present invention, and therefore cannot be used to limit the scope of protection of the present invention.

[0058] See Figure 2(a) According to the example above, during parallel processing, four data items need to be read from the storage space each time, such as the first element of each row of matrix A. Since the first element of each row of matrix A is stored in SRAM 0, it is impossible to read multiple data items at once.

[0059] The simplest solution to this problem is to split conflicting access requests into multiple requests, allowing only one request at a time when data access conflicts occur. However, this method will degrade read and write performance. Another solution is to add a certain offset when storing data, see [link to relevant documentation]. Figure 2 As shown in (b), when storing the second row, it is offset to the right by one SRAM storage; when storing the third row, it is offset to the right by two SRAM storage; and for the fourth row, it is offset to the right by three SRAM storage. However, this offset storage method is strongly related to the data format.

[0060] See Figure 2 As shown in (c), if matrix A has 5 columns, then each time the first element of the 4 rows is read, one data point is read from each SRAM, eliminating the need for offset storage. Similarly, see... Figure 2 As shown in (d), if matrix A has 6 columns, then when reading the first element of each of the 4 rows, the same problem will occur where the data to be read at the same time is in the same SRAM.

[0061] In summary, if no adjustments are made to the data storage location, splitting multiple concurrent read or write requests into multiple accesses, with only one request allowed at a time, will lead to performance degradation. If a storage location offset approach is adopted, it may require different methods or offsets depending on the data format, resulting in numerous scenarios to analyze and complex designs.

[0062] To address this issue, this invention proposes a solution and computing device for resolving parallel read / write access conflicts in memory, which reduces parallel access conflicts while maintaining access efficiency. According to an embodiment, by selecting a prime number greater than the degree of parallelism as the number of sub-modules, parallel read / write access conflicts are reduced, improving memory access efficiency. By comparing the data interval with the number of storage sub-modules, it is determined whether read / write access conflicts are likely to occur in the current data access mode. Based on the determination result, a data offset storage strategy is selected to further reduce access conflicts and improve the parallel efficiency of data read / write.

[0063] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention.

[0064] Figure 3The diagram illustrates a device structure of a dataflow artificial intelligence chip according to an example.

[0065] See Figure 3 According to an example embodiment, a schematic diagram of a data flow artificial intelligence chip device structure is provided. The data flow artificial intelligence chip includes: multiple arithmetic logic units 01, a storage management module 03, and a memory 02. When the multiple arithmetic logic units 01 perform operations in parallel, the storage management module 03 sends a parallel read / write access request to the memory 02 to obtain the data required for the operation.

[0066] The memory 02 receives the parallel read / write access request and provides the data required for computation to the plurality of arithmetic logic units 01. The memory is configured to include a specific number of storage sub-modules, which are prime numbers, based on the degree of parallelism of the data access by the plurality of arithmetic logic units. The prime number is greater than the degree of parallelism.

[0067] According to some embodiments, the memory 02 is responsible for storing large amounts of data, particularly datasets used for computation. The storage management module 03 can efficiently handle concurrent read and write requests from multiple arithmetic logic units 01. The multiple arithmetic logic units 01 are core computing components on the chip, each capable of independently performing mathematical operations, particularly optimized for operations such as multiplication and addition. They work in parallel to accelerate large-scale data processing tasks.

[0068] According to some embodiments, when the plurality of arithmetic logic units 01 need to perform operations, they send read / write access requests to the memory 02 in parallel, requesting the necessary data. Upon receiving these requests, the memory 02, through its built-in access control mechanism, can respond to multiple requests simultaneously, efficiently distributing data to the requesting arithmetic logic units 01, ensuring immediate data availability, and reducing computational waiting time. Figure 4 The diagram illustrates the comparison of different storage sub-module selections based on the number of storage sub-modules in various example scenarios.

[0069] Regarding the selection of the number of storage submodules, matrix multiplication will still be used as an example below (see...). Figure 1 (Example in Chinese). In this example, the number of rows in matrix A and the number of columns in matrix B are both fixed at 4. The advantages of setting the number of storage sub-modules (SRAM) to 5 are compared with the traditional setting of 4, with the number of columns in matrix A and the number of rows in matrix B being 4, 5, 6, 8, 10, and 16.

[0070] See Figure 4In example (a), assuming matrix A has 4 columns, the storage methods of matrix A in 4 SRAMs and 5 SRAMs are shown in the following diagrams (here, offset storage is not performed initially; colors are used to distinguish each row of data in the matrix: pink represents the data in the first row, blue represents the data in the second row, yellow represents the data in the third row, green represents the data in the fourth row, and red text marks the first element of each row). In comparison, in the 4 SRAM scheme, the first element of each row of matrix A is stored in SRAM 0, making it impossible to read 4 data points in one clock cycle; while in the 5 SRAM scheme, the first element of each row of matrix A is stored in different SRAMs, allowing 4 data points to be read in one clock cycle.

[0071] See Figure 4 In example (b), assuming matrix A has 5 columns, the storage methods of matrix A in 4 SRAMs and 5 SRAMs are shown in the figure below. In comparison, in the 4 SRAM scheme, the first element of each row of matrix A is stored in a different SRAM, allowing 4 data entries to be read in one clock cycle; while in the 5 SRAM scheme, the first element of each row of matrix A is stored in SRAM 0, making it impossible to read 4 data entries in one clock cycle.

[0072] See Figure 4 In example (c), assuming matrix A has 6 columns, the storage methods of matrix A in 4 SRAMs and 5 SRAMs are shown in the figure below. Comparing the two, in the 4 SRAM scheme, the first element of each row of matrix A is stored in SRAM 0 and SRAM 2, making it impossible to read 4 data points in one clock cycle; while in the 5 SRAM scheme, the first element of each row of matrix A is stored in different SRAMs, allowing 4 data points to be read in one clock cycle.

[0073] See Figure 4 In example (d), assuming matrix A has 8 columns, the storage methods of matrix A in 4 SRAMs and 5 SRAMs are shown in the figure below. In comparison, in the 4 SRAM scheme, the first element of each row of matrix A is stored in SRAM 0, making it impossible to read 4 data points in one clock cycle; while in the 5 SRAM scheme, the first element of each row of matrix A is stored in different SRAMs, allowing 4 data points to be read in one clock cycle.

[0074] See Figure 4In the example in (e), assuming matrix A has 10 columns, the storage methods of matrix A in 4 SRAMs and 5 SRAMs are shown in the figure below. Comparing the two, in the 4 SRAM scheme, the first element of each row of matrix A is stored in both SRAM 0 and SRAM 2, making it impossible to read 4 data points in one clock cycle; while in the 5 SRAM scheme, the first element of each row of matrix A is stored in SRAM 0, also making it impossible to read 4 data points in one clock cycle.

[0075] See Figure 4 In example (f), assuming matrix A has 16 columns, the storage methods of matrix A in 4 SRAMs and 5 SRAMs are shown in the figure below. In comparison, in the 4 SRAM scheme, the first element of each row of matrix A is stored in SRAM 0, making it impossible to read 4 data points in one clock cycle; while in the 5 SRAM scheme, the first element of each row of matrix A is stored in different SRAMs, allowing 4 data points to be read in one clock cycle.

[0076] In summary, with the 4-SRAM scheme, it's impossible to read four data points in one clock cycle when the number of columns in matrix A is 4, 6, 8, 10, or 16. With the 5-SRAM scheme, this problem only occurs when the number of columns in matrix A is 5 or 10. Clearly, setting the number of SRAMs to a prime number reduces the probability of not being able to read four data points in parallel within one clock cycle. Mathematically, with the 4-SRAM scheme, SRAM read / write conflicts occur whenever the number of columns in matrix A is any even number greater than or equal to 4, preventing the parallel reading of four data points within one clock cycle. With the 5-SRAM scheme, SRAM read / write conflicts only occur when the number of columns in matrix A is a multiple of 5.

[0077] Furthermore, regardless of whether a 4-SRAM or 5-SRAM scheme is used, SRAM access conflicts will occur. Assuming the number of data items read / written in parallel is M, and the number of storage submodules is P, typically the M data items are equally spaced. For example, when reading the first element of each row of a matrix in parallel, the interval between the first elements of each row is the number of columns in the matrix. When the data interval is an integer multiple of the number of storage submodules, i.e., the interval is N*P (N is a multiple), offset storage is performed. According to some embodiments, data can be divided into M*P*N groups, and within each group, data blocks can be divided according to the data interval N*P of the parallel data reads. Offset storage is performed within each group, with each data block offset to the left or right of the previous data block by one storage submodule. This resolves the SRAM access conflict problem.

[0078] The following example, using matrix A with 10 columns, illustrates how to resolve SRAM access conflicts.Figure 4 As shown in (g), the data is divided into only one group, with a group size of M*P*N = 4*5*2. Each pair of rows forms a block, with a block size of N*P = 2*5. Each block is stored offset one position relative to the previous block. In the diagram, pink rows are not offset; blue rows are offset one position to the right, so data "10" and "15" are offset from SRAM 0 to SRAM 1. Similarly, data "14" and "19" are offset one position to the right, but since there is no SRAM 5, they are placed in SRAM 0. Yellow rows are offset two positions; green rows are offset three positions. After this offset processing, during matrix operations, the parallel reading of data "0", "10", "20", and "30" will appear in different SRAMs, thus avoiding conflicts caused by simultaneous access to the same SRAM.

[0079] See Figure 4 As shown in (h), if matrix A has a size of 8 rows and 10 columns, and each calculation still reads the first element of the first 4 rows of the matrix in parallel, and after the data of the first four rows is calculated in parallel, the data of the last four rows is calculated in parallel, then its offset storage method is as follows: the matrix is ​​divided into two groups, each group having a size of M*P*N=4*5*2, and each group is divided into 4 data segments, each data segment having a size of N*P=2*5, and each data segment is offset by one position relative to the previous data segment.

[0080] In summary, the storage management module 03 divides the memory 02 into a prime number of storage sub-modules with a parallelism similar to that of the arithmetic logic units (ALUs). By subdividing the memory into a number of sub-modules with a parallelism similar to that of the ALUs, data access requests can be allocated and managed more efficiently. This ensures that each sub-module receives the load as evenly as possible during high-concurrency access, reducing access conflicts and waiting times.

[0081] In some implementations, choosing prime numbers as the number of submodules is based on considerations of reducing data conflicts and improving memory access efficiency. In multiprocessor systems, if the number of submodules is composite, some submodules may become "hot spots," leading to uneven data distribution and access bottlenecks. Using prime numbers as the number of submodules can distribute access requests more evenly, significantly reducing potential concurrent access conflicts.

[0082] According to some implementations, ensuring that the number of submodules is greater than the degree of parallelism allows for some expansion space to accommodate potential future increases in parallel processing needs without significantly increasing hardware complexity. This memory configuration strategy reflects a deep understanding of data locality, parallel efficiency, and scalability in high-performance computing, helping to maximize the computing speed and resource utilization of dataflow AI chips.

[0083] The storage management module is configured to: obtain the data interval for parallel read / write access; determine the relationship between the data interval and the number of storage sub-modules; if the data interval is an integer multiple of the number of storage sub-modules, offset the data storage; and access the data required for parallel read / write operations.

[0084] According to some embodiments, the storage management module obtains the number of data items accessed in parallel at each moment within a specific time period. The obtained data interval is compared with the number of sub-modules in the storage. The module analyzes whether the data access pattern can efficiently match the current storage structure. When the data interval is an integer multiple of the number of storage sub-modules, read / write access conflicts may exist. In this case, a data offset storage strategy can be implemented, where data is pre-grouped according to the access pattern and stored in a circular or interleaved distribution among the storage sub-modules. This reduces access conflicts and improves the parallel efficiency of data read and write operations.

[0085] According to some implementations, if the data interval is not an integer multiple of the storage submodule, it indicates that the current data access mode does not completely match the storage submodule division. In this case, no access conflict will occur, and parallel read / write access can be performed directly using sequential storage, that is, the data is stored in the submodule according to its natural order. This configuration strategy can simplify data management and access logic, reduce the overhead of data rearrangement, and improve the efficiency of parallel read / write access.

[0086] According to some embodiments, based on the above analysis and adjusted data layout, the multiple arithmetic logic units initiate read and write requests to the memory in parallel, efficiently obtain or update the data required for the operation, reduce access latency, and improve the overall throughput of the operation.

[0087] Data offset storage includes: grouping data, dividing each group into data blocks according to the data interval for parallel data reading, and each data block including an integer multiple of the data from the storage submodule;

[0088] When storing data blocks within a group, each data block is stored by shifting to the left or right by one storage sub-module relative to the previous data block.

[0089] According to some embodiments, the entire dataset is divided into several groups according to certain rules (e.g., by matrix rows or columns, or according to computational requirements). Each group represents the data set required for one or a series of consecutive operations. Within each group, the data is further divided into data blocks according to the degree of parallelism (i.e., the number of arithmetic logic units working simultaneously). The amount of data contained in each data block is exactly the amount of input data required by all units in one parallel operation, and this amount is an integer multiple of the number of storage sub-modules, ensuring that the data can be evenly and efficiently distributed to each sub-module.

[0090] According to some embodiments, for data blocks within each group, the first data block is stored in a starting storage submodule in a conventional manner. Subsequent data blocks are stored starting from the submodule following the last submodule of the previous block, i.e., the starting position is successively offset to the left or right by one storage submodule. This approach ensures a balanced distribution of data across different submodules and also reduces access conflicts, as parallel data requests are directed to different submodules. This process is repeated until all data blocks within the group have been stored according to the above rules. By using this offset storage strategy, combined with the sequential storage of data within data blocks, the system can maximize the utilization of the parallel access capabilities of storage submodules while maintaining relatively simple data access logic, reducing potential access bottlenecks, and effectively improving the parallelism and efficiency of data access.

[0091] Figure 5 The flowchart illustrates a solution to a memory multi-parallel read / write access conflict based on an example.

[0092] See Figure 5 A solution to memory multi-parallel read / write access conflicts according to an example embodiment includes:

[0093] In S501, the memory is divided into a specific number of storage sub-modules according to the parallelism of data access, wherein the specific number is a prime number and the prime number is greater than the parallelism.

[0094] According to some embodiments, the memory is rationally divided into multiple sub-modules based on the degree of parallelism of data access. Choosing a prime number greater than the degree of parallelism as the number of sub-modules helps optimize data distribution and reduce access conflicts.

[0095] In S503, the data interval for parallel read / write access is obtained.

[0096] Based on some embodiments, the number of data access intervals in parallel read and write operations during actual system operation is analyzed and collected to understand the data parallel read and write access pattern and provide a basis for subsequent optimization strategies.

[0097] In S505, the relationship between the data interval and the number of storage submodules is determined.

[0098] According to some embodiments, the number of data access intervals obtained is compared with the number of storage sub-modules, and the data layout strategy is adopted based on the comparison.

[0099] In S507, if the data interval is an integer multiple of the storage submodule, the data is stored with an offset.

[0100] In some implementations, if the data interval is an integer multiple of the number of submodules, a data offset storage strategy is adopted. This strategy, by interleaving data among submodules, can more evenly distribute access pressure and reduce potential access conflicts. If it is not an integer multiple, a sequential storage method is adopted to maintain the natural order of the data, reduce data management pressure, reduce the overhead of data reorganization, and improve the efficiency of parallel read and write access.

[0101] In S509, parallel read and write operations access the data required for computation.

[0102] According to some embodiments, based on the storage layout optimized by the above methods, efficient parallel read and write access operations are performed to ensure fast and conflict-free access to the data required for computation, thereby improving overall computational performance.

[0103] If the data interval is an integer multiple of the storage submodule, the data is stored with an offset. For example, if the data interval is an integer multiple of the storage submodule, the data is grouped, and each group is divided into data blocks according to the degree of parallelism. Each data block includes an integer multiple of the specified number of data. When storing data blocks within a group, the starting position is shifted one storage submodule to the right relative to the starting position of the previous data block, and then stored sequentially.

[0104] According to some embodiments, when the data interval is an integer multiple of the storage submodule, the data to be stored is grouped according to certain rules, such as by matrix rows or columns, to ensure that each group of data is logically related to each other, which facilitates subsequent parallel processing.

[0105] According to some embodiments, within each data group, the data is further divided into multiple data blocks based on the degree of parallelism. The amount of data contained in each data block is ensured to be an integer multiple of the number of storage sub-modules, to ensure that the data can be evenly distributed among the sub-modules, preparing for parallel computing.

[0106] According to some embodiments, for data blocks within each group, a "right shift" strategy is used to allocate storage locations. For example, each data block is shifted to the right by one storage sub-module relative to the previous data block and stored cyclically. This ensures that access pressure can be effectively distributed and potential sub-module access conflicts can be reduced even during high-parallel access.

[0107] If the data interval is an integer multiple of the storage submodule, the data is stored with an offset; otherwise, it is stored sequentially.

[0108] According to some embodiments, if the data interval is not an integer multiple of the storage submodule, a sequential storage strategy is adopted, that is, the data is directly stored in the storage submodule according to the natural order of the data without additional data block offset, which simplifies the storage logic and reduces the complexity of data preprocessing.

[0109] According to some embodiments, the data management and storage strategies in the present invention can significantly improve the efficiency and response speed of data stream AI chips when processing large-scale parallel computing, ensuring the efficiency and smoothness of data access.

[0110] According to some embodiments, the present invention can be applied to the design of chip storage circuits. By applying the data flow artificial intelligence chip as described above, the parallel read and write access efficiency of the chip storage circuit can be improved, so that the chip storage circuit can run smoothly and respond quickly when processing large-scale parallel operations.

[0111] According to some embodiments, the present invention can be applied to the design of storage systems. The storage controller executes the configuration strategy and storage method described above, which can effectively distribute access pressure and reduce potential sub-module access conflicts.

[0112] According to some embodiments, the present invention can also be applied to the design of a computer program product, wherein when the computer program is executed by a processor, it implements the configuration strategy and storage method described above, maximizes the use of the parallel access capability of the hardware storage module, reduces potential access bottlenecks, and effectively improves the parallelism and efficiency of data access.

[0113] Figure 6 A block diagram of a computing device according to an exemplary embodiment of the present invention is shown.

[0114] like Figure 6 As shown, the computing device 30 includes a processor 12 and a memory 14. The computing device 30 may also include a bus 22, a network interface 16, and an I / O interface 18. The processor 12, memory 14, network interface 16, and I / O interface 18 can communicate with each other via the bus 22.

[0115] Processor 12 may include one or more general-purpose CPUs (Central Processing Units), microprocessors, or application-specific integrated circuits, for executing relevant program instructions. According to some embodiments, computing device 30 may also include a high-performance display adapter (GPU) 20 for accelerating processor 12.

[0116] Memory 14 may include a machine-readable medium in the form of volatile memory, such as random access memory (RAM), read-only memory (ROM), and / or cache memory. Memory 14 is used to store one or more programs containing instructions, as well as data. Processor 12 may read the instructions stored in memory 14 to perform the methods described above according to embodiments of the present invention.

[0117] The computing device 30 can also communicate with one or more networks via the network interface 16. The network interface 16 can be a wireless network interface.

[0118] Bus 22 can include address bus, data bus, control bus, etc. Bus 22 provides a path for exchanging information between components.

[0119] It should be noted that, in specific implementations, the computing device 30 may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the device described above may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0120] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), network storage devices, cloud storage devices, or any type of medium or device suitable for storing instructions and / or data.

[0121] This invention also provides a computer program product comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments.

[0122] Those skilled in the art will clearly understand that the technical solutions of the present invention can be implemented by means of software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware capable of independently performing or cooperating with other components to perform a specific function, wherein the hardware may be, for example, a field-programmable gate array (FPGA), an integrated circuit, etc.

[0123] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0124] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0125] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0126] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0127] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0128] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention.

[0129] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0130] Exemplary embodiments of the present invention have been specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, arrangements, or implementations described herein; rather, the present invention is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended provisions.

Claims

1. A dataflow artificial intelligence chip, characterized in that, The application relates to a data stream artificial intelligence chip. The data stream artificial intelligence chip comprises a plurality of arithmetic logic units, a storage management module and a memory, wherein When the plurality of arithmetic logic units operate in parallel, the storage management module sends a parallel read-write access request to the memory to obtain data required for operation; The memory receives the parallel read-write access request and provides the plurality of arithmetic logic units with data required for operation, According to the parallel degree of data access of the plurality of arithmetic logic units, the memory is configured to comprise a specific number of storage sub-modules, and the specific number is a prime number.

2. The dataflow artificial intelligence chip according to claim 1, wherein, The prime number is greater than the parallel degree.

3. The dataflow artificial intelligence chip of claim 1, wherein, The storage management module is configured to: obtain a data interval of parallel read-write access; determine the relationship between the data interval and the number of storage sub-modules; if the data interval is an integer multiple of the number of storage sub-modules, store data in a shifted manner; and parallel read-write access data required for operation.

4. The dataflow artificial intelligence chip according to claim 3, wherein, The data is stored in a shifted manner, comprising: grouping data, dividing data blocks in each group according to the data interval of parallel read data, and including an integer multiple of the number of storage sub-modules in each data block; when storing the data blocks in the group, each data block is sequentially stored after being stored in a shifted manner by one storage sub-module to the left or right relative to the previous data block.

5. A method for resolving memory multi-concurrent read-write access conflicts, the method comprising: The application relates to a data stream artificial intelligence chip. According to the parallel degree of data access, the memory is divided into a specific number of storage sub-modules, and the specific number is a prime number and the prime number is greater than the parallel degree. Case number: obtain a data interval of parallel read-write access; determine the relationship between the data interval and the number of storage sub-modules; if the data interval is an integer multiple of the number of storage sub-modules, store data in a shifted manner; and parallel read-write access data required for operation.

6. The method of claim 5, wherein, The data is stored in a shifted manner, comprising: grouping data, dividing data blocks in each group according to the data interval of parallel read data, and including an integer multiple of the number of storage sub-modules in each data block; when storing the data blocks in the group, each data block is sequentially stored after being stored in a shifted manner by one storage sub-module to the left or right relative to the previous data block.

7. A chip memory circuit, characterized by The application relates to a data stream artificial intelligence chip.

8. A storage system, characterized by, The application relates to a data stream artificial intelligence chip. The application relates to a data stream artificial intelligence chip.

9. A computer program product, characterised in that, The application relates to a data stream artificial intelligence chip. The application relates to a data stream artificial intelligence chip.

10. A computing device, comprising: The application relates to a data stream artificial intelligence chip. The application relates to a data stream artificial intelligence chip. The application relates to a data stream artificial intelligence chip. The application relates to a data stream artificial intelligence chip. The application relates to a data stream artificial intelligence chip.

Images