Memory merging device, chip product, computer device and data reading method

Abstract

The application discloses a memory merging device, a chip product, a computer device and a data reading method, and relates to the chip technical field.The memory merging device comprises a request arbitration module, an instruction control module, a data collection module and a data storage module.The request arbitration module sends control information to the instruction control module based on a data reading instruction sent by a first request party under the condition that Q merging requests corresponding to the data reading instruction meet the condition;the instruction control module reads the data corresponding to the Q merging requests in multiple batches from the data storage module according to the control information and stores the data in the data collection module; and the data collection module provides the full-amount data corresponding to the data reading instruction to the first request party after collecting the full-amount data.The application helps to improve the flexibility of data reading.

Description

Technical Field

[0001] This application relates to the field of chip technology, and in particular to a memory merging device, chip product, computer equipment, and data reading method. Background Technology

[0002] In modern parallel computing chip designs such as GPUs (Graphics Processing Units) and AI (Artificial Intelligence), memory merging devices are an essential component.

[0003] Currently, memory merging devices lack flexibility when reading data. Summary of the Invention

[0004] This application provides a memory merging device, a chip product, a computer device, and a data reading method. The technical solutions provided by this application include the following aspects.

[0005] According to one aspect of the embodiments of this application, a memory merging apparatus is provided, the memory merging apparatus comprising: a request arbitration module, an instruction control module, and a data storage module;

[0006] The request arbitration module is used to generate Q merge requests corresponding to the data read instruction sent by the first requester. Each merge request is used to read data in the same row belonging to the data storage module, where Q is a positive integer.

[0007] The arbitration request module is also used to send control information to the instruction control module when the Q merging requests meet the conditions.

[0008] The instruction control module is used to read the data corresponding to the Q merge requests from the data storage module in multiple batches according to the control information.

[0009] According to one aspect of the embodiments of this application, a chip product is provided, the chip product including the memory merging device as described above.

[0010] According to one aspect of the embodiments of this application, a computer device is provided, the computer device including the memory merging apparatus as described above.

[0011] According to one aspect of the embodiments of this application, a data reading method applied to a memory merging apparatus is provided, the memory merging apparatus comprising: a request arbitration module, an instruction control module, and a data storage module;

[0012] The method includes:

[0013] The arbitration module generates Q merge requests corresponding to the data read instruction sent by the first requester. Each merge request is used to read data in the same row belonging to the data storage module, where Q is a positive integer.

[0014] If the Q merging requests meet the conditions, the arbitration request module sends control information to the instruction control module.

[0015] The instruction control module reads the data corresponding to the Q merge requests from the data storage module in multiple batches according to the control information.

[0016] The technical solution provided in this application can bring the following beneficial effects:

[0017] If the conditions are met for the Q merge requests corresponding to the data read command, the data corresponding to the Q merge requests is divided into multiple batches for reading. This method of reading the data corresponding to the Q merge requests from the data storage module in multiple batches, rather than reading all the data corresponding to the Q merge requests at once, helps to improve the flexibility of data reading. Attached Figure Description

[0018] Figure 1 This is a structural block diagram of a memory merging device provided in one possible implementation of this application;

[0019] Figure 2 This is a structural block diagram of the memory merging device provided in another possible implementation of this application;

[0020] Figure 3 This is a schematic diagram of the request identifier corresponding to the merge request provided in one possible implementation of this application;

[0021] Figure 4 This is a schematic diagram of instruction metadata provided in one possible implementation of this application;

[0022] Figure 5 This is a schematic diagram of instruction dispatch information provided in one possible implementation of this application;

[0023] Figure 6 This is a schematic diagram of the information in the information buffer of the storage block provided in one possible implementation of this application;

[0024] Figure 7 This is a flowchart of a data reading method applied to a memory merging device, provided in one possible implementation of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0026] The memory merging device has both coalescing and caching functions. Upstream of the memory merging device, there are typically multiple requesters, each with a parallel execution unit. The M threads executed by this unit at one time are called an instruction. It's important to note that M is the maximum capacity of an instruction; an instruction may only be valid for a few threads. This application only considers read operations that access memory. This operation requires downstream data to be returned to the requester. The M threads request different memory addresses but have the same length.

[0027] The request sent by the requester is called the initial thread request (init_thread_request), and the request after merging is called the coalesced_cache_request. The granularity of the merging module is generally equal to the cache line size, for example, assuming the cache line size is R bytes.

[0028] An initial thread request (i.e., one instruction) will correspond to Q merge cache requests depending on the merging of M data storage addresses. The value of Q is different for each instruction. The data read by each thread may fall within two cache lines (i.e., across cache lines). For example, if the data storage address is 0x1078 and the length is 32 bytes, R is 128 bytes. Therefore, one thread can correspond to a maximum of 2 merge cache requests, and M threads can correspond to a maximum of 2M merge cache requests.

[0029] When a requester issues a read request, if the requester demands data, the memory merging device needs to collect data from M threads before returning it to the requester. Clearly, the current memory merging device lacks flexibility when reading data.

[0030] Furthermore, current memory merging devices also suffer from a deadlock problem caused by insufficient resources: if the addresses of the M threads of an instruction, after merging, result in Q merged cache requests, and the number of merged cache requests in the same block (bank) and the same set (set) of the cache is greater than the way number (U) of the cache function module, then the cache in the request direction cannot receive all the merged cache requests, and the cache in the return direction cannot return to the requester because not all the read data has been collected, thus causing a deadlock.

[0031] To resolve the deadlock issue mentioned above, a common approach is to ensure that the number Q of merged buffer requests corresponding to any instruction is less than or equal to the number of paths U of the cache module, thus avoiding deadlock. Alternatively, additional hardware resources can be allocated specifically to address the deadlock problem.

[0032] Since the number of cache paths is limited by hardware logic timing, it generally does not exceed 16 or 32. Therefore, the first solution is not always applicable. The second solution requires additional dedicated hardware resources and can only be utilized when Q is greater than U, resulting in low hardware resource utilization.

[0033] Therefore, this application proposes a simpler and more practical memory merging device, which can not only improve the flexibility of data reading, but also solve the aforementioned deadlock problem.

[0034] Please refer to Figure 1 The diagram illustrates a structural block diagram of a memory merging apparatus provided in one possible implementation of this application. The memory merging apparatus 10 includes: a request arbitration module 11, an instruction control module 12, and a data storage module 14.

[0035] Arbitration module 11 is used to generate Q merge requests corresponding to the data read instruction sent by the first requester. Each merge request is used to read data in the same row belonging to data storage module 14, where Q is a positive integer.

[0036] The first requester can be any requester. The upstream of the memory merging device 10 can include one or more requesters, and the first requester can be any one of these requesters. Each requester generally has a parallel execution unit, and the M thread tasks executed by the parallel execution unit at one time can be called an instruction. When the instruction sent by the requester is used to read data from the data storage module 14 of the memory merging device 10, each requester sends an instruction to read data corresponding to up to M single requests from the data storage module 14. Here, M is a positive integer. That is, the above M threads correspond one-to-one with M single requests. The value of M can be preset, which depends on the number of threads executing tasks in parallel in the parallel execution unit of the requester, such as M=4, 6, or 8, etc., which is not limited in this embodiment. It should be noted that M here is the maximum capability of an instruction. An instruction may have all M threads effective, or it may only have a few of the M threads effective.

[0037] Each individual request has a corresponding data storage address and data length. The data storage address indicates the storage location of the data requested in the data storage module 14; the data length indicates the length or amount of data requested. Typically, the M individual requests mentioned above correspond to different data storage addresses but have the same data length.

[0038] In some embodiments, the arbitration module 11 is used to receive a data read instruction sent by the first requester. The data read instruction is used to read data corresponding to X individual requests from the data storage module 14, where X is a positive integer. Based on the data storage addresses corresponding to the X individual requests, Q merge requests corresponding to the data read instruction are generated. It should be understood that the aforementioned X individual requests are the individual requests actually triggered by the data read instruction; therefore, X is a positive integer less than or equal to M.

[0039] After receiving the data read instruction, the arbitration module 11 generates Q merge requests corresponding to the data read instruction based on the data storage addresses corresponding to the X individual requests and the merging granularity. The merging granularity is typically a row in the data storage module 14. The arbitration module 11 merges multiple individual requests that need to read data within the same row of the data storage module 14, generating a single merge request. Therefore, each merge request is used to read data belonging to the same row of the data storage module 14.

[0040] Furthermore, for a single request, the data it requests to read may reside in one row of the data storage module 14, or it may reside in two rows of the data storage module 14, such as two adjacent rows, i.e., there is a cross-row situation. Therefore, the data requested to be read in a single request may be split into two merge requests, with each merge request reading the data from the aforementioned two rows respectively.

[0041] By generating Q merged requests corresponding to the data read instructions based on the data storage addresses corresponding to the X individual requests of the data read instruction, the request merging function is realized. This merges multiple individual requests that access the same row of data, which helps to simplify the data read process and improve data read efficiency.

[0042] The arbitration request module 11 is also used to send control information to the instruction control module 12 if the Q merge requests meet the conditions. Optionally, the control information is used to instruct the instruction control module 12 to read the data corresponding to the Q merge requests from the data storage module 14 in multiple batches.

[0043] Optionally, the above conditions relate to at least one of the following: the number of merge requests, and the location of the data to be read in the data storage module 14.

[0044] For example, the above conditions include: the number of merge requests exceeds a preset threshold. This preset threshold can be set based on the hardware performance of the memory merging device or empirical values, and this application does not limit it in this regard.

[0045] When there are a large number of merge requests, reading the data corresponding to Q merge requests from the data storage module 14 in multiple batches, instead of reading all the data corresponding to Q merge requests at once, helps to improve the flexibility of data reading.

[0046] In some embodiments, the above conditions may be pre-defined conditions used to determine whether there is a deadlock risk in the data read instructions. Deadlock risk refers to the inability to read all the data corresponding to Q merge requests in one go.

[0047] Data storage module 14 may include one or more storage banks, and each storage bank may include one or more sets. Optionally, the above condition includes: among Q merge requests, the number of merge requests falling into the same set within the same storage bank of the data storage module is greater than the way number U of the data storage module, where U is a positive integer. If the number of merge requests falling into the same set within the same storage bank of the data storage module 14 is greater than the way number U of the data storage module 14, that is, if the above condition is met for Q merge requests, it indicates that there is a risk of deadlock in the data read instructions.

[0048] When there is a deadlock risk in the data read command, the data storage module 14 cannot receive all the merge requests in the requesting direction, and in the return direction, it cannot return to the first requester because the full amount of data corresponding to the data read command has not been collected. To solve this problem, in this embodiment, when the request arbitration module 11 detects a deadlock risk in the data read command, it sends control information to the command control module 12. The command control module 12 then reads the data corresponding to Q merge requests from the data storage module 14 in multiple batches according to the control information.

[0049] The arbitration module 11 can divide the Q merge requests corresponding to the data read instruction into multiple batches. The number of merge requests in each batch that fall in the same group of the same storage block in the data storage module 14 is not greater than (i.e. less than or equal to) the number of paths U, which can avoid deadlock problems.

[0050] In one possible implementation, the request arbitration module 11 sends control information to the instruction control module 12, which instructs the instruction control module 12 to read data corresponding to Q merge requests from the data storage module 14 in multiple batches. Optionally, the request arbitration module 11 sends a single control message to the instruction control module 12, which instructs the instruction control module 12 to read data corresponding to Q merge requests from the data storage module 14 in multiple batches.

[0051] In another possible implementation, the arbitration request module 11 can send multiple control messages to the instruction control module 12. Each control message is used to read the data corresponding to a batch of merge requests. Furthermore, these multiple control messages are sent sequentially. After sending one control message, the data corresponding to the batch of merge requests for that control message is read first. After reading is completed, the next control message is sent, and so on, until all the data corresponding to the Q merge requests for the data reading instruction has been read.

[0052] In practical applications, whether the arbitration request module 11 sends one control message or multiple control messages to the instruction control module 12 can be pre-configured or determined by the capabilities of the arbitration request module 11. This application does not limit this.

[0053] The instruction control module 12 is used to read data corresponding to Q merge requests from the data storage module 14 in multiple batches according to control information. Optionally, the instruction control module 12 is also used to store the read merge request data into the data collection module 13.

[0054] When the arbitration module 11 sends a control message to the instruction control module 12, the instruction control module 12, after receiving the control message, reads the data corresponding to Q merge requests from the data storage module 14 in multiple batches and stores them in the data collection module 13.

[0055] When the arbitration module 11 sends multiple control messages to the instruction control module 12, the instruction control module 12, upon receiving each control message, reads the data corresponding to a batch of merge requests for that control message from the data storage module 14 and stores it in the data collection module 13. Then, upon receiving the next control message, it continues to read the data corresponding to a batch of merge requests for that control message from the data storage module 14 and store it in the data collection module 13. This process is repeated until all data corresponding to the Q merge requests for the data read instruction has been read.

[0056] Optionally, such as Figure 1 As shown, the memory merging device 10 also includes a data collection module 13. The data collection module 13 is used to provide the full data corresponding to the data read instruction to the first requester after collecting the full data corresponding to the data read instruction. The full data corresponding to the data read instruction includes data corresponding to Q merging requests.

[0057] In addition, the aforementioned Q merge requests can be sent by the request arbitration module 11 to the data storage module 14, and the data storage module 14 stores the data corresponding to each merge request into the data collection module 13.

[0058] By employing the above method, after collecting all the data corresponding to the data reading command, feedback is sent to the first requester, ensuring the integrity of the data reading.

[0059] The technical solution provided in this application provides that, when the conditions are met for the Q merge requests corresponding to the data reading instruction, the data corresponding to the Q merge requests is divided into multiple batches for reading. This method of reading the data corresponding to the Q merge requests from the data storage module in multiple batches, rather than reading all the data corresponding to the Q merge requests at once, helps to improve the flexibility of data reading.

[0060] In addition, if the number of merge requests falling within the same group of the same storage block in the data storage module is greater than the number of paths U of the data storage module, that is, if there is a risk of deadlock in the data read instructions, the deadlock problem can be avoided by reading data in multiple batches, with each batch including the number of merge requests falling within the same group of the same storage block in the data storage module not greater than (i.e. less than or equal to) the number of paths U.

[0061] Please refer to Figure 2 The diagram illustrates a structural block diagram of a memory merging apparatus provided in another possible implementation of this application. The memory merging apparatus 10 includes: a request arbitration module 11, an instruction control module 12, a data collection module 13, and a data storage module 14.

[0062] The functions of the arbitration request module 11, the instruction control module 12, the data collection module 13, and the data storage module 14 can be found in the descriptions in the above embodiments, and will not be repeated here.

[0063] like Figure 2 As shown, assume there are N requesters in the system, where N is a positive integer. For ease of description, assume that each instruction from each requester is a read operation task involving M threads. An initial thread request, or instruction, contains the data storage addresses corresponding to M individual requests, but some threads may be invalid. In some implementations, the interface has thread-active information to identify which threads are valid. The aforementioned interface can be understood as a physical or logical communication channel between the requester and the memory merging device 10, dynamically identifying the validity of each thread through thread-active information.

[0064] In the memory merging device 10, in the request direction, any data read instruction sent by any of the N requesters first enters the request arbitration module 11. The request coalesce unit in the request arbitration module 11 implements the merging function. It can be a dedicated request coalesce unit for each requester, or a shared request coalesce unit for all N requesters. If each requester has its own dedicated request coalesce unit, the N request coalesce units precede the arbitration unit; if all N requesters share a single request coalesce unit, that unit follows the arbitration unit. The arbitration unit coordinates the access permissions of multiple requesters to downstream resources. For example, when multiple requesters simultaneously initiate data read instructions, the arbitration unit can use a priority strategy (such as fixed priority, round-robin, etc.) to determine which requester's data read instruction will be processed first. Alternatively, if each requester has its own dedicated request coalesce unit, a single request is first merged into a merged request before entering the arbitration unit. This helps reduce the arbitration load (because the number of merged requests Q is generally less than the original number of threads M) and improves arbitration efficiency. If N requesters share a single request merging unit, the arbitration unit first selects the requester, and then the request enters the shared request merging unit for merging. This helps save hardware resources (only one request merging unit is needed), but the arbitration unit needs to handle more fine-grained requests.

[0065] Meanwhile, the arbitration request module 11 also includes one or more hash algorithm units. The main function of the hash algorithm unit is to obtain information such as the storage block identifier (bank_id) and group identifier (set_id) based on the storage address corresponding to an input merge request (generally the data storage address corresponding to the merge request to reduce the number of accesses by the hash algorithm unit). This determines which storage block and group in the data storage module 14 the merge request should access through the hash algorithm, thereby reducing storage block / group conflicts in the data storage module 14. Furthermore, in this embodiment, it is assumed that the number of storage blocks included in the data storage module 14 is B, where B is a positive integer.

[0066] like Figure 3 As shown, a merge request (denoted as coalesced_cache_request) sent from the arbitration module 11 to the data storage module 14 corresponds to a unique request identifier (denoted as cache_req_id). The cache_req_id mainly includes: requester identifier (denoted as requestor_id), instruction identifier (denoted as instruction_id), storage block identifier (denoted as bank_id), and sub-request identifier (denoted as sub_req_id). Specifically, the requester identifier indicates the first requester, i.e., the identifier information of the first requester. The requester identifier distinguishes which requester (a total of N requesters) the data read instruction corresponding to the merge request originates from. The instruction identifier indicates the data read instruction, i.e., the identifier information of the data read instruction. The instruction identifier distinguishes which instruction was issued by the requester corresponding to the merge request. The storage block identifier indicates the storage block to be accessed by the merge request, indicating which storage block in the data storage module 14 the merge request will be sent to. The sub-request identifier is the identification information of the merge request, used to distinguish the various merge requests that access the data storage module 14 in the data read instruction. The sub-request identifier ranges from 0 to 2M-1.

[0067] In some embodiments, the arbitration request module 11 is used to send control information to the instruction control module 12 when Q merge requests meet the conditions, or when a deadlock risk is detected in the data read instruction. Optionally, the control information includes: instruction meta information (denoted as instruction_meta_info) corresponding to the data read instruction and instruction dispatch information (denoted as instruction_dispath_info) corresponding to the data read instruction.

[0068] The instruction metadata corresponding to the data read instruction is used to indicate the location of the data corresponding to at least one single request of the data read instruction in the data storage module 14.

[0069] The instruction dispatch information corresponding to the data read instruction is used to indicate that Q merge requests meet the above conditions.

[0070] Optionally, when the above conditions are used to determine whether there is a deadlock risk in the data read instruction, the instruction dispatch information corresponding to the data read instruction is used to indicate whether there is a deadlock risk in the data read instruction.

[0071] In one possible implementation, when the arbitration module 11 sends a control message to the instruction control module 12, the control message includes instruction metadata indicating the location of the data corresponding to X individual requests of the data read instruction in the data storage module 14. Here, the X individual requests are all the individual requests corresponding to the data read instruction.

[0072] In another possible implementation, when the arbitration module 11 sends multiple control messages to the instruction control module 12, each control message includes instruction metadata indicating the location of the data corresponding to a portion of a single request for a data read instruction in the data storage module 14. This portion of a single request can be a part of all single requests corresponding to a data read instruction, such as a single request corresponding to a batch merging request.

[0073] In the above manner, when the arbitration module 11 determines that the conditions for Q merging requests are met, it sends the instruction element information and instruction dispatch information corresponding to the data reading instruction to the instruction control module 12. On the one hand, this enables the instruction control module 12 to know that the conditions for the current Q merging requests are met. On the other hand, it enables the instruction control module 12 to know the storage location of the data to be read by the data reading instruction when the conditions for the Q merging requests are met, thereby ensuring the accuracy and completeness of subsequent data collection.

[0074] like Figure 4 As shown, the instruction metadata corresponding to the data read instruction includes: instruction length (denoted as instruction_length) and M thread information (denoted as thread_info).

[0075] The instruction length indicates the data length corresponding to a single request. Specifically, it represents the data length shared by M single requests corresponding to a data read instruction.

[0076] There are M thread information entries, each indicating the position of the data corresponding to a single request in a row of the data storage module 14. M is a positive integer and represents the maximum number of single requests.

[0077] It should be understood that M is the maximum number of single requests. The number of single requests indicated by the instruction metadata included in the control information can be less than M or equal to M. If the number of single requests indicated by the instruction metadata included in the control information is less than M, some of the thread information in the above M thread information can be represented by default values ​​to indicate a single request that is not actually to be read.

[0078] By including the instruction length and M thread information in the instruction metadata, the location of the data corresponding to each individual request of the data read instruction in the data storage module 14 can be clearly and accurately indicated.

[0079] Optionally, the instruction metadata corresponding to the data read instruction also includes an instruction end flag (denoted as instruction_end_flag). The instruction end flag indicates whether the current instruction metadata is the last instruction metadata of the data read instruction. If the data corresponding to the same single request is located in two rows of the data storage module 14, the data read instruction corresponds to two instruction metadata; otherwise, the data read instruction corresponds to one instruction metadata.

[0080] Each data read instruction corresponds to one or two instruction metadata entries. Taking the data read instruction sent by the first requester as an example, among the M single requests corresponding to the data read instruction, if the data corresponding to the same single request is located in two rows of the data storage module 14, the data read instruction corresponds to two instruction metadata entries; otherwise (i.e., if the data corresponding to the same single request is not located in two rows of the data storage module 14), the data read instruction corresponds to one instruction metadata entry.

[0081] If a data read instruction corresponds to one instruction metadata entry, then the instruction end marker in that instruction metadata entry is a first value. If a data read instruction corresponds to two instruction metadata entries, then the instruction end marker in the first instruction metadata entry is a second value, and the instruction end marker in the second instruction metadata entry is the first value. The first and second values ​​are different. For example, the first value is 1, and the second value is 0.

[0082] Since the data corresponding to a single request may be stored across multiple rows, each data read instruction corresponds to one or two instruction metadata entries. By using the instruction end marker, the number of instruction metadata entries can be clearly and accurately indicated, avoiding the omission of instruction metadata entries.

[0083] Optionally, such as Figure 4As shown, each thread's information includes: thread active flag (denoted as thread_active_flag), request base identifier (denoted as cache_req_basic_id), starting byte offset (denoted as start_byte_offset), and valid length (denoted as valid_length).

[0084] The thread validity flag indicates whether the thread is valid in this instruction metadata. For example, in the initial thread request, one of the M threads may be invalid; or, if only one thread in a data read instruction requests data that spans multiple rows, then in the second instruction metadata corresponding to the data read instruction, the thread validity flags for the other threads that do not span multiple rows will be invalid. The request base identifier can include the storage block identifier and sub-request identifier from the above request identifiers, indicating the merge request corresponding to this thread. The starting byte offset indicates the starting byte position of this thread within this row. The valid length indicates the valid length of this thread within this row.

[0085] In some embodiments, for the i-th single request in at least one of the above single requests, if the data corresponding to the i-th single request is located in the k-th and k+1-th rows of the data storage module 14, where i is a positive integer and k is a positive integer, then: in the first instruction metadata corresponding to the data read instruction, the i-th thread information is used to indicate the position of the data corresponding to the i-th single request in the k-th row. Specifically, the starting byte offset and effective length in the i-th thread information are used to indicate the position of the data corresponding to the i-th single request in the k-th row; in the second instruction metadata corresponding to the data read instruction, the i-th thread information is used to indicate the position of the data corresponding to the i-th single request in the k+1-th row. Specifically, the starting byte offset and effective length in the i-th thread information are used to indicate the position of the data corresponding to the i-th single request in the k+1-th row; where the i-th thread information is the thread information corresponding to the i-th single request.

[0086] Using the above method, when the data corresponding to a single request is stored across rows, the position of the data corresponding to that single request in the two rows can be clearly and accurately indicated.

[0087] Each instruction also corresponds to an instruction dispatch record. For example... Figure 5 As shown, the instruction dispatch information corresponding to the data read instruction includes: indicator flag, requester identifier (requestor_id), instruction identifier (instruction_id), and B block request counts (denoted as bank_req_cnt).

[0088] An indicator flag is used to indicate that Q merge requests meet the conditions. Optionally, the indicator flag is 1 bit. For example, when the indicator flag takes a first value, it indicates that Q merge requests meet the conditions; when the indicator flag takes a second value, it indicates that Q merge requests do not meet the conditions; wherein the first value and the second value are different. For example, the first value is 1 and the second value is 0.

[0089] The arbitration request module 11 is also used to set the value of the indicator flag to a first value if it is determined that Q merger requests meet the conditions, and to set the value of the indicator flag to a second value if it is determined that Q merger requests do not meet the conditions.

[0090] Optionally, when the above conditions are used to determine whether there is a deadlock risk in the data read instruction, the above indicator flag can also be called the instruction deadlock flag (denoted as instruction_deadlock_flag), used to indicate that there is a deadlock risk in the data read instruction. Optionally, the instruction deadlock flag is 1 bit. For example, when the instruction deadlock flag takes the first value, it indicates that there is a deadlock risk; when the instruction deadlock flag takes the second value, it indicates that there is no deadlock risk; wherein the first value and the second value are different. For example, the first value is 1 and the second value is 0.

[0091] The arbitration request module 11 is also used to set the value of the instruction deadlock flag to a first value when a deadlock risk is detected in the data read instruction, the first value being used to indicate that there is a deadlock risk in the data read instruction; and to set the value of the instruction deadlock flag to a second value when no deadlock risk is detected in the data read instruction, the second value being used to indicate that there is no deadlock risk in the data read instruction.

[0092] By setting the value of the indicator flag in the arbitration module 11, it is possible to clearly and intuitively reflect whether the Q merge requests meet the conditions, such as whether there is a risk of deadlock.

[0093] The requester identifier is used to indicate the first requester. The instruction identifier is used to indicate the data read instruction. The requester identifier and the instruction identifier are unique identifiers for a single instruction. Since the instruction metadata and instruction dispatch information are in one-to-one correspondence, it is not necessary to store the requester identifier and instruction identifier in the instruction metadata.

[0094] The B block request counts are used to indicate the number of merge requests received by the B storage blocks included in the data storage module 14, where B is a positive integer. For each of the B storage blocks, there is a corresponding block request count to identify the number of merge requests for which a data read instruction falls. M single requests can correspond to a maximum of 2M merge requests. These 2M merge requests may fall within the same storage block, so the maximum representation range of the block request counts needs to be able to represent 2M. For an instruction, the sum of its B block request counts equals Q; the B block request counts are used in subsequent logic to determine whether all the data corresponding to this instruction is ready.

[0095] By including indicator flags, requester identifiers, instruction identifiers, and B block request counts in the instruction dispatch information, the indicator flags can clearly and intuitively reflect whether the Q merge requests meet the conditions, such as whether there is a risk of deadlock. The requester identifiers and instruction identifiers can effectively distinguish between data read instructions from different requesters and different data read instructions from the same requester. The B block request counts can be used in subsequent logic to determine whether all the data corresponding to this instruction is ready, thus fully ensuring the accuracy and completeness of data reading.

[0096] In some embodiments, when a data read instruction corresponds to one instruction metadata entry, the data read instruction corresponds to one instruction dispatch entry. When a data read instruction corresponds to two instruction metadata entries, the data read instruction corresponds to two instruction dispatch entries, wherein the two instruction dispatch entries include one valid instruction dispatch entry and one invalid instruction dispatch entry.

[0097] Optionally, the instruction control module 12 includes a first storage unit and a second storage unit. The first storage unit stores instruction metadata, and the second storage unit stores instruction dispatch information. Optionally, the first storage unit is constructed using SRAM (Static Random-Access Memory) because it has a larger number of bits per layer. Optionally, the second storage unit is constructed using register resources to facilitate operation by other module logic.

[0098] By employing the above method, the storage containers for instruction metadata and instruction dispatch information can be made to have the same depth, i.e., the depths of the first and second storage units are identical. This facilitates the storage and retrieval of instruction metadata and instruction dispatch information, and also helps reduce the amount of information in the instruction metadata; for example, it eliminates the need to include the requester identifier and instruction identifier within the instruction metadata.

[0099] In some embodiments, the instruction dispatch information corresponding to the data read instruction further includes: a valid flag (valid_flag) to indicate whether the current instruction dispatch information is valid.

[0100] Since an instruction corresponds to at most two instruction metadata entries (two levels in the first memory unit) and only one valid instruction dispatch entry (one level in the second memory unit), a 1-bit validity flag is used in the instruction dispatch entry to indicate whether the current level's instruction dispatch information is valid. If an instruction corresponds to two levels of instruction metadata, then one level of instruction dispatch information in the corresponding second memory unit is valid, and the other is invalid. The validity flag in a valid instruction dispatch entry indicates validity, while the validity flag in an invalid instruction dispatch entry indicates invalidity.

[0101] By designing a valid flag in the instruction dispatch information, the validity of the instruction dispatch information can be effectively distinguished, making the storage containers of instruction metadata and instruction dispatch information have the same depth, thereby simplifying the storage and retrieval of the above two types of information.

[0102] The following section will use two examples to introduce and explain instruction metadata and instruction dispatch information.

[0103] Example 1: For instance, M=4, R=32 bytes, cache bank number B=4, way number U=4. Of the M threads for one instruction, only threads 0, 1, and 2 are valid, while thread 3 is invalid; the M data storage addresses (memory addresses) are 0x1000, 0x1004, 0x1008, and the invalid address, respectively; the shared request length for the M threads is 4 bytes. We will find that the three valid memory addresses + request length all fall within the same 32-byte cache line, and none of them cross cache lines. Therefore, this instruction corresponds to one instruction_meta_info entry, instruction_end_flag = 1; instruction_length = 4.

[0104] The four thread_info values ​​are as follows:

[0105] thread_active_flag = 1, cache_req_basic_id is assumed to be 0x10, start_byte_offset = 0, valid_length = 4;

[0106] thread_active_flag = 1, cache_req_basic_id is the same as thread0, which is 0x10, start_byte_offset = 0x4, valid_length = 4;

[0107] thread_active_flag = 1, cache_req_basic_id is the same as thread0, which is 0x10, start_byte_offset = 0x8, valid_length = 4;

[0108] thread_active_flag = 0, cache_req_basic_id, start_byte_offset, and valid_length are all default values.

[0109] This instruction corresponds to one instruction_meta_info, so instruction_dispath_info only occupies 1 layer and valid_flag is valid. The number Q of coalesced_cache_requests after merging is 1, Q < U, so instruction_deadlock_flag = 0. The sum of B bank_req_cnt is equal to 1.

[0110] Example 2: For example, M=4, R=32 bytes, cache bank number B=4, way number U=4. All M threads of one instruction are valid, the M data storage addresses (memory address) are 0x101C, 0x201D, 0x301E and 0x4000 respectively, and the request length shared by the M threads is 8 bytes. We find that three out of the four memoryaddr + request length entries span cache lines. Specifically, 0x101C + 8 bytes occupies cache lines 0x1000 and 0x1020, with each cache line having a valid length of 4 bytes; 0x201D + 8 bytes occupies cache lines 0x2000 and 0x2020, with valid lengths of 3 bytes and 5 bytes respectively; 0x301E + 8 bytes occupies cache lines 0x3000 and 0x3020, with valid lengths of 2 bytes and 6 bytes respectively; and 0x4000 + 8 bytes occupies one cache line 0x4000. Therefore, this instruction corresponds to two instruction_meta_info entries.

[0111] The first instruction_meta_info has instruction_end_flag = 0 and instruction_length = 8. The four thread_info entries are as follows:

[0112] thread_active_flag = 1, cache_req_basic_id is assumed to be 0x10, start_byte_offset = 0x1C, valid_length = 4;

[0113] thread_active_flag = 1, cache_req_basic_id is assumed to be 0x20, start_byte_offset = 0x1D, valid_length = 3;

[0114] thread_active_flag = 1, cache_req_basic_id is assumed to be 0x30, start_byte_offset = 0x1E, valid_length = 2;

[0115] thread_active_flag = 1, cache_req_basic_id is assumed to be 0x40, start_byte_offset = 0x0, valid_length = 8.

[0116] The second instruction_meta_info has instruction_end_flag = 1 and instruction_length = 8. The four thread_info entries are as follows:

[0117] thread_active_flag = 1, cache_req_basic_id is assumed to be 0x11, start_byte_offset = 0x0, valid_length = 4;

[0118] thread_active_flag = 1, cache_req_basic_id is assumed to be 0x21, start_byte_offset= 0x0, valid_length = 5;

[0119] thread_active_flag = 1, cache_req_basic_id is assumed to be 0x31, start_byte_offset = 0x0, valid_length = 6;

[0120] thread_active_flag = 0, cache_req_basic_id, start_byte_offset, and valid_length are all default values.

[0121] This instruction corresponds to two `instruction_meta_info` entries, therefore `instruction_dispath_info` occupies two levels, with `valid_flag` indicating validity and invalidity respectively. The merged number of `coalesced_cache_request`s is Q = 7. If the number of these Q `coalesced_cache_request`s falling within the same bank and set is greater than U, then `instruction_deadlock_flag` = 1; otherwise, `instruction_deadlock_flag` = 0. The sum of the B `bank_req_cnt` entries equals 7.

[0122] Instruction metadata and instruction dispatch information are stored in the first and second storage units of the instruction control module 12. The first storage unit is a 1-read-1-write SRAM, meaning that it can read one layer and write one layer at a time within the same cycle.

[0123] In some embodiments, the arbitration request module 11 is further configured to, for any one of the Q merge requests, increment the block request count corresponding to the first storage block by 1 when the merge request is sent to the first storage block in the data storage module 14. The instruction control module 12 is further configured to, after receiving a data ready signal corresponding to the merge request sent by the first storage block, decrement the block request count corresponding to the first storage block by 1. The data ready signal indicates that the data corresponding to the merge request is in a readable state. A description of this process is provided below. By employing the above method to maintain the block request count values ​​corresponding to each storage block in the data storage module 14, it is possible to ensure that all data corresponding to the Q merge requests of the data read instruction is read, avoiding omissions.

[0124] In some embodiments, the arbitration request module 11 is further configured to: enter an authorization lock state when it is determined that Q merging requests meet the conditions, the authorization lock state being a state where only the data read instruction is processed; and exit the authorization lock state after the data collection module 13 has collected the full amount of data corresponding to the data read instruction.

[0125] In the arbitration request module 11, if it is determined that Q merge requests meet the conditions, such as determining that there is a deadlock risk with the data read instruction, then the grant of the arbitration unit needs to be retained for the first requester. This ensures that all Q merge requests corresponding to the first requester's data read instruction are sent to the downstream storage block, and that the data corresponding to the Q merge requests is sent to the data collection module 13 before the grant is released. This method ensures that all data for a single data read instruction is completely read. See the following description for specific processing details.

[0126] like Figure 6As shown, for each storage block, the information buffer (denoted as isched buffer) stores the merge request and corresponding auxiliary information sent from upstream. Each layer of information in the buffer mainly contains 1 bit of valid information (denoted as valid), a request identifier (denoted as cache_req_id), a group identifier (set id), a way identifier (wayid), and 1 bit of data ready information (denoted as data_ready). If the data storage module 14 has sector functionality, a sector need bit (denoted as sector_need_bit) is also required to indicate the sector information needed for the merge request. The request identifier is the {requestor identifier (denoted as requestor_id), instruction identifier (denoted as instruction_id), storage block identifier (denoted as bank_id), sub-request identifier (denoted as sub_req_id)} information described above. A merge request must allocate a cacheline; the set id and way id are the corresponding cacheline information. The data readiness information indicates whether the data in the corresponding cacheline is currently valid or ready. If the cache is hit, the data readiness information is 1 (indicating that it is valid or ready). If the cache is missed, it needs to wait until the read data is returned to the storage block before it can be set to 1.

[0127] Multiple merge requests may correspond to the same cacheline, and conversely, one cacheline may correspond to multiple layers in the information buffer. Once data in one cacheline is ready, the data readiness information of all layers in the corresponding information buffer will be set to 1.

[0128] After the first layer of data ready information in the information buffer is set to 1, the data ready signal generation logic will be triggered, and a data ready signal (denoted as data_ready_signal) will be sent to the instruction control module 12. This information must include at least the {requestor identifier (denoted as requestor_id) and instruction identifier (denoted as instruction_id)} information.

[0129] The instruction control module 12 can simultaneously receive data ready signal information from B storage blocks. Based on the received {requestor identifier (denoted as requestor_id)} information, it matches it with the instruction dispatch information stored in the valid layer of the second storage unit, and decrements the request count of B blocks in the matching instruction dispatch information by 1. If the request count of B blocks is found to be reduced to 0, it indicates that all data of all threads of the data reading instruction is ready and stored in the downstream storage blocks. At this point, the data collection finish detection & read data control logic in instruction control module 12 will trigger two actions simultaneously. One action is to send a data read signal (denoted as read_cache_data_signal) to the read data control module of the B storage blocks. This data read signal must include at least the {requestor_id, instruction_id} information. The other action is to read the corresponding level 1 or level 2 instruction metadata from the first storage unit and send the read instruction metadata to the data collection module 13. After these two actions are completed, the instruction metadata and instruction dispatch information corresponding to the data read instruction in the first and second storage units of instruction control module 12 can be released. If multiple instructions meet the data readiness conditions, the data collection finish detection & read data control logic will select one instruction to trigger the above two actions.

[0130] After receiving a data read signal, the read control module of the storage block searches for all merge requests corresponding to {requestor_id, instruction_id} in the information buffer to obtain cacheline information (set id / way id information), and then initiates a read cacheline operation to the data control module. After passing through the data pipeline, the cacheline data is retrieved from the Data SRAM and sent to the data collection module 13.

[0131] The data collection module 13 receives cacheline data from B storage blocks. The module can generally set up a data pack buffer to pack the cacheline data together in a certain format according to the instruction metadata and then return it to the corresponding requester.

[0132] The value of Q corresponding to an instruction will exceed the pipeline level of the bankid / setid that the request arbitration module 11 can monitor for merging cache requests. Therefore, the request arbitration module 11 will likely need many clock cycles to detect whether the indicator flag of an instruction (taking the instruction deadlock flag as an example) needs to be set to 1 (setting it to 1 indicates a deadlock risk). When the request arbitration module 11 sends the first merge request of an instruction downstream, it needs to simultaneously allocate a layer of space in the second storage unit, set the valid flag of the instruction dispatch information to 1 (indicating that the information of the current layer is valid), assign values ​​to {requestor_id, instruction_id}, and set the instruction deadlock flag to its initial value of 0. When the request arbitration module 11 sends any merge request downstream, it needs to synchronously update the value of the B block request count in the instruction dispatch information corresponding to that instruction in the second storage unit (i.e., increment by 1).

[0133] In some embodiments, the arbitration request module 11, after entering the authorization lock state, sends a first number of merge requests corresponding to the data read instruction to the corresponding storage block in the data storage module 14, wherein the first number is less than or equal to the number of paths U of the data storage module 14. The instruction control module 12, when determining that the data corresponding to the first number of merge requests is in a readable state, sends a data read signal to the corresponding storage block in the data storage module 14, the data read signal being used to request the reading of the data corresponding to the merge request. The data collection module 13 is used to collect the data corresponding to the first number of merge requests sent by the storage blocks in the data storage module 14.

[0134] In a batch, by reading the data corresponding to the first number of merge requests, it is possible to avoid the number of merge requests processed in a single session exceeding the number of paths U of the data storage module 14, thereby preventing deadlock problems from occurring.

[0135] In addition, the instruction control module 12 can determine which merge requests' corresponding data are in a readable state and which merge requests' corresponding data are not yet in a readable state based on the data ready signals sent by the storage blocks in the data storage module 14.

[0136] In this way, for each batch of merge requests, if it is confirmed that all of the data can be read, the data collection module 13 is notified to collect the data corresponding to the merge requests of that batch, thus ensuring the integrity of the data collection corresponding to each batch of merge requests.

[0137] In some embodiments, the data collection module 13 is further configured to send an indication message to the request arbitration module 11 after collecting data corresponding to a first number of merge requests. This indication message indicates that the data corresponding to the first number of merge requests has been collected. The request arbitration module 11 is further configured to, if there are unprocessed merge requests among the Q merge requests corresponding to the data read instruction, re-execute the step of sending the first number of merge requests corresponding to the data read instruction to the corresponding storage block in the data storage module 14. The request arbitration module 11 is further configured to exit the authorization lock state when all Q merge requests corresponding to the data read instruction have been processed.

[0138] Since the Q merge requests corresponding to the data read instruction are generated by the request arbitration module 11, the request arbitration module 11 knows the number of merge requests, and the data collection module 13 can enable the request arbitration module 11 to determine the number of merge requests for which the corresponding data has been collected by sending instruction information to the request arbitration module 11. By comparing the number of merge requests for which the corresponding data has been collected with the total number Q of merge requests corresponding to the data read instruction, the request arbitration module 11 can accurately determine whether all of the above Q merge requests have been processed.

[0139] By using the above method, in scenarios where the full data corresponding to a single data read instruction is read in multiple batches, it can be ensured that the full data corresponding to the single data read instruction is completely read.

[0140] If the arbitration module 11 detects a deadlock risk in the data read instruction, it sets the instruction deadlock flag to 1 and blocks the arbitration logic in the arbitration module 11 (i.e., enters the authorization lock state). It also needs to immediately write the instruction metadata of the data read instruction already cached in the arbitration module 11 to the first storage unit. At this point, the instruction metadata may be incomplete, as some thread information may not be stored. To simplify the process, the instruction control module 12 waits until all instructions preceding the data read instruction have been completed (data returned to the corresponding requester). Then, the data collection completion detection & data read control logic selects the data read instruction (provided that the request counts of B blocks have been reduced to 0, satisfying the data readiness condition). Similar to the above process, the data collection completion detection & data read control logic in the instruction control module 12 triggers two actions simultaneously: sending a data read signal to the storage block, and then the storage block returns cacheline data to the data collection module 13. Once the data collection module 13 detects that all data for the data read instruction has been received, it will notify the request arbitration module 11 in the requesting direction to unlock the arbitration logic. However, this batch of data cannot be returned to the first requester immediately because subsequent data has not yet been collected. After the arbitration logic is unlocked, the next batch of merge requests corresponding to the data read instruction is sent to the storage block, and the above process is repeated. The first and second storage units in the instruction control module 12 reallocate space to store the information of the data read instruction once, and finally, this batch of cacheline data is also moved to the data collection module 13. To simplify the process, if the request arbitration module 11 detects that the instruction deadlock flag of the data read instruction has been assigned a value of 1, that is, there is a deadlock risk, then after all the merge requests corresponding to the data read instruction are sent to the downstream storage block, the arbitration logic is temporarily blocked again. The arbitration logic in the request arbitration module 11 is unlocked only after multiple batches of cacheline data for the data read instruction have been moved to the data collection module 13. The deadlock risk described above can be easily resolved using the above method; moreover, no additional hardware storage resources are required, and the request and response (data reading) paths are the same regardless of whether the instruction deadlock flag is 1, resulting in high utilization of hardware resources.

[0141] Furthermore, in the above method, the B storage blocks are independent, so the instruction control module 12 needs to be placed outside each storage block, resulting in a relatively large interaction latency between the instruction control module 12 and the B storage blocks. In some embodiments, the B storage blocks can be integrated into one module, using a module with B hit detection / data control sub-modules to achieve parallel caching capabilities; in this case, the functionality of the instruction control module 12 can also be implemented in this module, thus reducing the interaction latency between the instruction control module 12 and the B storage blocks.

[0142] The following are embodiments of the method of this application. For details not disclosed in the embodiments of the method of this application, please refer to the embodiments above.

[0143] Please refer to Figure 7 This illustrates a flowchart of a data reading method applied to a memory merging apparatus, as provided in one possible implementation of this application. The components of this memory merging apparatus can be found in the description of the embodiments above. Figure 7 As shown, the method may include at least one of the following steps 710 to 730.

[0144] Step 710: The arbitration module requests that, based on the data read instruction sent by the first requester, generate Q merge requests corresponding to the data read instruction. Each merge request is used to read data from the same row belonging to the data storage module, where Q is a positive integer.

[0145] Step 720: If the Q merge requests meet the conditions, the arbitration module requests to send control information to the instruction control module. Optionally, the control information instructs the instruction control module to read the data corresponding to the Q merge requests from the data storage module in multiple batches.

[0146] Step 730: The instruction control module reads the data corresponding to the Q merge requests from the data storage module in multiple batches according to the control information.

[0147] In some embodiments, the above conditions include: The number of merge requests falling within the same group of the same storage block in the data storage module out of the Q merge requests is greater than the number of paths U of the data storage module, where U is a positive integer. The number of merge requests falling within the same group of the same storage block in each of the multiple batches is less than or equal to the number of paths U.

[0148] In some embodiments, the control information includes: instruction metadata corresponding to the data read instruction and instruction dispatch information corresponding to the data read instruction. The instruction metadata corresponding to the data read instruction is used to indicate the location of data corresponding to at least one single request of the data read instruction in the data storage module. The instruction dispatch information corresponding to the data read instruction is used to indicate that Q merge requests meet the conditions.

[0149] In some embodiments, the instruction metadata corresponding to the data read instruction includes: instruction length and M thread information.

[0150] Command length indicates the length of data corresponding to a single request.

[0151] There are M thread information entries, each indicating the position of the data corresponding to a single request in a row of the data storage module. M is a positive integer and represents the maximum number of requests per request.

[0152] In some embodiments, the instruction metadata corresponding to a data read instruction further includes an instruction end flag, used to indicate whether the current instruction metadata is the last instruction metadata of the data read instruction. Wherein, if the data corresponding to the same single request is located in two rows of the data storage module, the data read instruction corresponds to two instruction metadata; otherwise, the data read instruction corresponds to one instruction metadata.

[0153] In some embodiments, for the i-th single request in at least one single request, if the data corresponding to the i-th single request is located in the k-th and k+1-th rows of the data storage module, where i is a positive integer and k is a positive integer, then: in the first instruction metadata corresponding to the data read instruction, the i-th thread information is used to indicate the position of the data corresponding to the i-th single request in the k-th row; in the second instruction metadata corresponding to the data read instruction, the i-th thread information is used to indicate the position of the data corresponding to the i-th single request in the k+1-th row; wherein, the i-th thread information is the thread information corresponding to the i-th single request.

[0154] In some embodiments, the instruction dispatch information corresponding to the data read instruction includes: an indication flag, a requester identifier, an instruction identifier, and a B-block request count.

[0155] Indicator flags are used to indicate that Q merge requests meet the conditions.

[0156] The requester identifier is used to indicate the first requester.

[0157] Command identifier, used to indicate data read commands.

[0158] The B block request count indicates the number of merge requests received by each of the B storage blocks included in the data storage module, where B is a positive integer.

[0159] In some embodiments, the method further includes: when the arbitration request module determines that Q merger requests meet the conditions, setting the value of the indicator flag to a first value; when the arbitration request module determines that Q merger requests do not meet the conditions, setting the value of the indicator flag to a second value; wherein the first value and the second value are different.

[0160] In some embodiments, the method further includes: for any one of the Q merge requests, when the request arbitration module sends the merge request to the first storage block in the data storage module, incrementing the value of the block request count corresponding to the first storage block by 1; after receiving the data ready signal corresponding to the merge request sent by the first storage block, the instruction control module decrements the value of the block request count corresponding to the first storage block by 1, wherein the data ready signal is used to indicate that the data corresponding to the merge request is in a readable state.

[0161] In some embodiments, when a data read instruction corresponds to one instruction metadata, the data read instruction corresponds to one instruction dispatch information; when a data read instruction corresponds to two instruction metadata, the data read instruction corresponds to two instruction dispatch information, wherein the two instruction dispatch information includes one valid instruction dispatch information and one invalid instruction dispatch information.

[0162] In some embodiments, the instruction dispatch information corresponding to the data read instruction further includes: a validity flag, used to indicate whether the current instruction dispatch information is valid.

[0163] In some embodiments, the method further includes: the instruction control module storing the data corresponding to the Q merge requests into the data collection module; after collecting the full data corresponding to the data read instruction, the data collection module provides the full data to the first requester, the full data including the data corresponding to the Q merge requests.

[0164] In some embodiments, the method further includes: the arbitration request module entering an authorization lock state when it determines that Q merge requests meet the conditions; the authorization lock state is a state where only the data read instruction is processed. The arbitration request module exits the authorization lock state after the data collection module has collected all the data corresponding to the data read instruction.

[0165] In some embodiments, the method further includes: after entering the authorization lock state, the request arbitration module sends a first number of merge requests corresponding to the data read instruction to the corresponding storage block in the data storage module, wherein the first number is less than or equal to the number of paths U. When the instruction control module determines that the data corresponding to the first number of merge requests is in a readable state, it sends a data read signal to the corresponding storage block in the data storage module, the data read signal being used to request the reading of the data corresponding to the merge request. The data collection module collects the data corresponding to the first number of merge requests sent by the storage blocks in the data storage module.

[0166] In some embodiments, the method further includes: after collecting data corresponding to a first number of merge requests, the data collection module sends an indication message to the request arbitration module, the indication message indicating that the data corresponding to the first number of merge requests has been collected. If there are unprocessed merge requests among the Q merge requests, the request arbitration module re-executes the step of sending the first number of merge requests corresponding to the data read instruction to the corresponding storage block in the data storage module. Once all Q merge requests have been processed, the request arbitration module exits the authorization lock state.

[0167] An exemplary embodiment of this application also provides a chip product, which includes the memory merging device described above. Optionally, the chip product may be a GPU chip, an AI chip, a TPU (Tensor Processing Unit) chip, an NPU (Neural Processing Unit) chip, etc.

[0168] An exemplary embodiment of this application also provides a computer device including the memory merging apparatus described above. Optionally, the computer device may be a personal computer, workstation, game console, and some mobile devices (such as tablet computers, smartphones, etc.), or it may be an in-vehicle terminal device, smart home device, smart TV, smart robot, etc., or it may be a server, server cluster, artificial intelligence computing cluster, cloud computing cluster, etc., wherein the artificial intelligence computing cluster may also be simply referred to as an intelligent computing cluster or smart computing cluster, and this application does not limit it in this regard.

[0169] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0170] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A memory merging device, characterized in that, The memory merging device includes: a request arbitration module, an instruction control module, and a data storage module; The request arbitration module is used to generate Q merge requests corresponding to the data read instruction sent by the first requester. Each merge request is used to read data in the same row belonging to the data storage module, where Q is a positive integer. The request arbitration module is also used to send control information to the instruction control module when the Q merge requests meet the conditions. The conditions are used to determine whether there is a deadlock risk in the data reading instruction. The deadlock risk means that the data corresponding to the Q merge requests cannot be read in one go. The instruction control module is used to read the data corresponding to the Q merge requests from the data storage module in multiple batches according to the control information.

2. The memory merging apparatus according to claim 1, characterized in that, The condition includes: among the Q merge requests, the number of merge requests falling into the same group of the same storage block in the data storage module is greater than the number of paths U of the data storage module, where U is a positive integer; The number of merge requests in each of the multiple batches that fall within the same group of the same storage block in the data storage module is less than or equal to the number of paths U.

3. The memory merging apparatus according to claim 1, characterized in that, The control information includes: instruction metadata corresponding to the data read instruction and instruction dispatch information corresponding to the data read instruction; The instruction metadata corresponding to the data read instruction is used to indicate the location of the data corresponding to at least one single request of the data read instruction in the data storage module; The instruction dispatch information corresponding to the data read instruction is used to indicate that the Q merge requests meet the conditions.

4. The memory merging apparatus according to claim 3, characterized in that, The instruction metadata corresponding to the data read instruction includes: Instruction length, used to indicate the data length corresponding to the single request; There are M thread information entries, each indicating the position of the data corresponding to a single request in a row of the data storage module. M is a positive integer and is the maximum number of single requests.

5. The memory merging apparatus according to claim 4, characterized in that, The instruction metadata corresponding to the data read instruction also includes: The instruction end flag is used to indicate whether the current instruction metadata is the last instruction metadata of the data read instruction; Wherein, if the data corresponding to the same single request is located in two rows of the data storage module, the data read instruction corresponds to two instruction metadata entries; otherwise, the data read instruction corresponds to one instruction metadata entry.

6. The memory merging apparatus according to claim 5, characterized in that, For the i-th single request in the at least one single request, if the data corresponding to the i-th single request is located in the k-th and (k+1)-th rows of the data storage module, where i is a positive integer and k is a positive integer, then: In the first instruction metadata corresponding to the data reading instruction, the i-th thread information is used to indicate the position of the data corresponding to the i-th single request in the k-th row; In the second instruction metadata corresponding to the data reading instruction, the i-th thread information is used to indicate the position of the data corresponding to the i-th single request in the (k+1)-th row; The i-th thread information is the thread information corresponding to the i-th single request.

7. The memory merging apparatus according to claim 3, characterized in that, The instruction dispatch information corresponding to the data read instruction includes: An indicator flag is used to indicate that the Q merge requests meet the condition; Requester identifier, used to indicate the first requester; An instruction identifier is used to indicate the data read instruction; B block request counts are used to indicate the number of merge requests received by each of the B storage blocks included in the data storage module, where B is a positive integer.

8. The memory merging apparatus according to claim 7, characterized in that, The arbitration request module is also used for: If it is determined that the Q merge requests meet the conditions, the value of the indicator flag is set to a first value; If it is determined that the Q merge requests do not meet the conditions, the value of the indicator flag is set to the second value; The first value and the second value are different.

9. The memory merging apparatus according to claim 7, characterized in that, The request arbitration module is further configured to, for any one of the Q merge requests, increment the value of the block request count corresponding to the first storage block by 1 when the merge request is sent to the first storage block in the data storage module; The instruction control module is further configured to decrement the value of the block request count corresponding to the first storage block by 1 after receiving the data ready signal corresponding to the merge request sent by the first storage block. The data ready signal is used to indicate that the data corresponding to the merge request is in a readable state.

10. The memory merging apparatus according to claim 3, characterized in that, When the data read instruction corresponds to a piece of instruction metadata, the data read instruction corresponds to a piece of instruction dispatch information; When the data read instruction corresponds to two pieces of instruction metadata, the data read instruction corresponds to two pieces of instruction dispatch information, wherein the two pieces of instruction dispatch information include one valid instruction dispatch information and one invalid instruction dispatch information.

11. The memory merging apparatus according to claim 10, characterized in that, The instruction dispatch information corresponding to the data read instruction also includes: A validity flag is used to indicate whether the current instruction dispatch information is valid.

12. The memory merging apparatus according to any one of claims 1 to 11, characterized in that, The memory merging device further includes: a data collection module; The instruction control module is also used to store the data corresponding to the Q merge requests into the data collection module; The data collection module is used to provide the full data to the first requester after collecting the full data corresponding to the data read instruction. The full data includes the data corresponding to the Q merge requests.

13. The memory merging apparatus according to claim 12, characterized in that, The arbitration request module is also used for: If it is determined that the Q merge requests meet the conditions, the system enters an authorization lock state, which is a state where only the data read instruction is processed. After the data collection module has collected all the data corresponding to the data read instruction, it exits the authorization lock state.

14. The memory merging apparatus according to claim 13, characterized in that, The request arbitration module is used to send a first number of merge requests corresponding to the data read instruction to the corresponding storage block in the data storage module after entering the authorized lock state, wherein the first number is less than or equal to the number of paths U of the data storage module; The instruction control module is used to send a data read signal to the corresponding storage block in the data storage module when it is determined that the data corresponding to the first number of merge requests are all in a readable state. The data read signal is used to request to read the data corresponding to the merge request. The data collection module is used to collect the data corresponding to the first number of merge requests sent by the storage blocks in the data storage module.

15. The memory merging apparatus according to claim 14, characterized in that, The data collection module is further configured to send an indication message to the request arbitration module after collecting the data corresponding to the first number of merge requests, the indication message being used to indicate that the data corresponding to the first number of merge requests has been collected. The request arbitration module is further configured to, in the event that there are unprocessed merge requests among the Q merge requests, execute again the step of sending the first number of merge requests corresponding to the data reading instruction to the corresponding storage block in the data storage module; The arbitration request module is also used to exit the authorization lock state after all Q merge requests have been processed.

16. The memory merging apparatus according to any one of claims 1 to 11, characterized in that, The arbitration request module is used for: The system receives the data reading instruction sent by the first requester, wherein the data reading instruction is used to read X data corresponding to single requests from the data storage module, where X is a positive integer; Based on the data storage addresses corresponding to the X individual requests, the Q merge requests corresponding to the data read instruction are generated.

17. A chip product, characterized in that, The chip product includes a memory merging device as described in any one of claims 1 to 16.

18. A computer device, characterized in that, The computer device includes a memory merging apparatus as described in any one of claims 1 to 16.

19. A data reading method applied to a memory merging device, characterized in that, The memory merging device includes: a request arbitration module, an instruction control module, and a data storage module; The method includes: The arbitration module generates Q merge requests corresponding to the data read instruction sent by the first requester. Each merge request is used to read data in the same row belonging to the data storage module, where Q is a positive integer. If the Q merge requests meet the conditions, the request arbitration module sends control information to the instruction control module. The conditions are used to determine whether there is a deadlock risk in the data reading instruction. The deadlock risk means that the data corresponding to the Q merge requests cannot be read in one go. The instruction control module reads the data corresponding to the Q merge requests from the data storage module in multiple batches according to the control information.

Images