A method for scheduling a cached request and an artificial intelligence chip

Abstract

The application provides a cache request scheduling method and an artificial intelligence chip, and relates to the technical field of artificial intelligence chips.The method comprises the following steps: querying a scheduling buffer based on a first memory address carried by a first data request; when a target virtual link of the same memory address is queried, adding a first scheduling record of the first data request on the target virtual link, and the first scheduling record comprises a request index of a second request relied on by the first data request, so that the first data request is executed after the second data request, and data requests on different virtual links are executed in parallel, the same address order is ensured, and different address disorder is realized, and the function is correct. The length of the virtual link in the scheduling buffer is not a fixed length, but can be continuously extended backward under the condition that the scheduling buffer is not full, so that cache access congestion is effectively reduced; and corresponding resources are allocated to the scheduling record of the first data request from the remaining resources of the scheduling buffer, so that resource waste is reduced.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence chip technology, and in particular to a cache request scheduling method and an artificial intelligence chip. Background Technology

[0002] In artificial intelligence chips, caches often adopt a relaxed memory consistency model to improve efficiency; specifically, the memory consistency model improves efficiency by out-of-order reading and writing at different addresses and keeping the order of reading and writing at the same address.

[0003] In related technologies, Last Level Cache (LLC) uses queues to achieve out-of-order processing of requests with different addresses and order preservation of requests with the same address. Specifically, a separate queue is set up for each memory address in the Last Level Cache. Requests with the same memory address are placed in the same queue, thus preserving the order of requests with the same address. Requests with different memory addresses are distributed to different queues, thus achieving out-of-order processing of requests with different addresses.

[0004] When using queues to implement out-of-order processing of different addresses and order preservation of the same address, if the queue for a certain request's memory address is full, that request and subsequent requests will enter a waiting state, causing cache access blockage. In fact, other queues in the cache may have a lot of free space, which leads to resource waste and affects the overall access performance of the cache. Summary of the Invention

[0005] This application provides a cache request scheduling method and an artificial intelligence chip, which can improve the overall access performance of the cache by using virtual links in the scheduling buffer to achieve order preservation for the same address and out-of-order execution for different addresses.

[0006] On one hand, embodiments of this application provide a cached request scheduling method, the method comprising:

[0007] Receive a first data request, the first data request including a first request index and a first memory address;

[0008] Create an initial scheduling record corresponding to the first request index in the scheduling buffer;

[0009] When a target virtual link corresponding to the first memory address exists in the scheduling buffer, and the first data request is a write request, a second request index is added to the initial scheduling record, and the initial scheduling record is marked as a new tail record of the target virtual link to obtain the first scheduling record;

[0010] The second request index is a request index of a second data request on which the first data request depends.

[0011] Optionally, before the adding the second request index in the initial scheduling record and marking the initial scheduling record as a new tail record of the target virtual link to obtain the first scheduling record, the method further comprises:

[0012] Determining a second scheduling record marked as a tail record in the target virtual link.

[0013] Obtaining the second request index from the second scheduling record.

[0014] Optionally, after the obtaining the second request index from the second scheduling record, the method further comprises:

[0015] Marking the second scheduling record as a non-tail record.

[0016] Optionally, the initial scheduling record comprises a conflict slot, a conflict enable bit and a tail mark.

[0017] The adding the second request index in the initial scheduling record and marking the initial scheduling record as a new tail record of the target virtual link to obtain the first scheduling record comprises:

[0018] Setting the conflict slot as the second request index, setting the conflict enable bit as a first conflict state, and setting the tail mark as a first tail state to obtain the first scheduling record.

[0019] The first conflict state indicates that there is a dependency relationship with other request indexes in the virtual link, and the first tail state indicates a tail record of the virtual link.

[0020] Optionally, the method further comprises:

[0021] When there is a target virtual link corresponding to the first memory address in the scheduling buffer, and the first data request is a read request, adding the second request index in the initial scheduling record to obtain a third scheduling record.

[0022] Optionally, the initial scheduling record comprises a conflict slot, a conflict enable bit and a tail mark.

[0023] The adding the second request index in the initial scheduling record to obtain the third scheduling record comprises:

[0024] Setting the conflict slot as the second request index, setting the conflict enable bit as a first conflict state, and setting the tail mark as a second tail state to obtain the third scheduling record.

[0025] The second conflict state indicates that there is no dependency relationship with other request indexes in the virtual link, and the first tail state indicates a tail record of the virtual link.

[0026] Optionally, the method further comprises:

[0027] When the target virtual link corresponding to the first memory address does not exist in the scheduling buffer, and the first data request is a write request, a new virtual link corresponding to the first memory address is created;

[0028] The initial scheduling record is marked as a tail record of the new virtual link to obtain a fourth scheduling record.

[0029] Optionally, the initial scheduling record comprises a conflict slot, a conflict enable bit and a tail mark.

[0030] The initial scheduling record is marked as a tail record of the new virtual link to obtain a fourth scheduling record, comprising:

[0031] The conflict slot is set to be empty, the conflict enable bit is set to be in a second conflict state, and the tail mark is set to be in a first tail state to obtain the fourth scheduling record.

[0032] The second conflict state indicates that there is no dependency relationship with other request indexes in the virtual link, and the first tail state indicates a tail record of the virtual link.

[0033] Optionally, the method further comprises:

[0034] When the target virtual link corresponding to the first memory address does not exist in the scheduling buffer, and the first data request is a read request, a default state of the initial scheduling record is maintained.

[0035] The default state comprises that the conflict slot is empty, the conflict enable bit is in a second conflict state, and the tail mark is in a second tail state, and the second tail state indicates a non-tail record of the virtual link.

[0036] In one aspect, an artificial intelligence chip is provided, comprising:

[0037] a processing core, a cache and a main memory.

[0038] The cache is configured to execute the above-mentioned request scheduling method of the cache.

[0039] On one hand, embodiments of this application provide a computer device, including a memory, an artificial intelligence chip, and a computer program stored on the memory and executable on the artificial intelligence chip. When the artificial intelligence chip executes the computer program, it implements the steps of the above-mentioned cache request scheduling method.

[0040] In this embodiment, the cache receives and processes the first data request sent by the core, and then queries the scheduling buffer based on the first memory address carried by the first data request. When a target virtual link with the same memory address is found, a first scheduling record of the first data request is added to the target virtual link. The first scheduling record includes the request index of the second request that the first data request depends on. This ensures that the first data request is executed after the second data request. At the same time, data requests on different virtual links in the scheduling buffer are executed in parallel, achieving order preservation for the same address and out-of-order execution for different addresses, thus ensuring correct functionality.

[0041] Secondly, the length of each virtual link in the scheduling buffer is not fixed, but can be continuously extended as long as the entire scheduling buffer is not full. Therefore, compared to the queue being full and causing requests to enter a waiting state, using virtual links can effectively reduce cache access congestion.

[0042] In addition, when the first data request is received, the corresponding resources are allocated from the remaining resources of the entire scheduling buffer for the scheduling record of the first data request. This makes full use of the resources of the scheduling buffer, reduces resource waste, and improves the overall access performance of the cache. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1A A flowchart illustrating a method for preserving the order of elements at the same address and reordering elements at different addresses, provided in an embodiment of this application;

[0045] Figure 1B A flowchart illustrating a method for preserving the order of elements at the same address and reordering elements at different addresses, provided in an embodiment of this application;

[0046] Figure 2 This is a schematic diagram of the structure of an artificial intelligence chip provided in an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of the structure of an artificial intelligence chip provided in an embodiment of this application;

[0048] Figure 4 A flowchart of a request scheduling method of a cache provided by an embodiment of the present application is shown in FIG. 1.

[0049] Figure 5A A structure diagram of a virtual link provided by an embodiment of the present application is shown in FIG. 2.

[0050] Figure 5B A structure diagram of a virtual link provided by an embodiment of the present application is shown in FIG. 3.

[0051] Figure 5C A structure diagram of a virtual link provided by an embodiment of the present application is shown in FIG. 4.

[0052] Figure 6 A structure diagram of a virtual link provided by an embodiment of the present application is shown in FIG. 5.

[0053] Figure 7A A structure diagram of a virtual link provided by an embodiment of the present application is shown in FIG. 6.

[0054] Figure 7B A structure diagram of a virtual link provided by an embodiment of the present application is shown in FIG. 7.

[0055] Figure 8 A structure diagram of a virtual link provided by an embodiment of the present application is shown in FIG. 8.

[0056] Figure 9 A structure diagram of a computer device provided by an embodiment of the present application is shown in FIG. 9. DETAILED DESCRIPTION

[0057] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0058] The terms "first", "second", and the like in the present application are only used for descriptive purposes, and should not be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. In addition, in the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0059] In the related art, in the LLC, different address reordering and same address ordering are realized by using queues; that is, a queue is separately set for each memory address in the last level cache, and requests of the same memory address are placed in the same queue, so that the same address ordering is realized; requests of different memory addresses are distributed to different queues, so that the different address reordering is realized.

[0060] When different address reordering and same address ordering are realized by using queues, the following two cases exist:

[0061] Case 1: When a certain requested memory address hits a full queue, and other queues have a large amount of free space, the request and subsequent requests are in a waiting state.

[0062] For example, referring to Figure 1A , three queues are set in the LLC, namely queue 1, queue 2, and queue 3, and each queue can store 4 requests of the same memory address.

[0063] When request 1 is received, it is assumed that the memory address of request 1 hits queue 1, and queue 1 is full, at which time request 1 is in a waiting state; if request 2 is received after request 1, the memory address of request 2 hits queue 2, even if queue 2 is free, request 2 is in a waiting state, and so on. This not only causes cache access congestion, but also causes resource (such as free queues queue 2, queue 3, etc.) waste, thereby affecting the overall access performance of the cache.

[0064] Case 2: Due to the limitation of LLC area, the number of queues set in the LLC is small; in this way, it is easy to appear that the memory address of the request does not hit any queue, in this case, the request and subsequent requests are in a waiting state.

[0065] For example, referring to Figure 1B , three queues are set in the LLC, namely queue 1, queue 2, and queue 3, and each queue can store 4 requests of the same memory address.

[0066] When request 1 is received, it is assumed that there is no queue in queue 1, queue 2, and queue 3 that hits the memory address of request 1, at which time request 1 is in a waiting state; if request 2 is received after request 1, the memory address of request 2 hits queue 2, even if queue 2 is free, request 2 is in a waiting state, and so on. This method has poor scalability and can cause cache access congestion and resource waste, thereby affecting the overall access performance of the cache.

[0067] Therefore, the present application provides a cache request scheduling method, which realizes same address in-order and different address out-of-order by constructing a virtual link in the scheduling buffer, thereby improving the overall access performance of the cache.

[0068] Before introducing the cache request scheduling method in the present application, a structure diagram of an artificial intelligence chip to which the embodiments of the present application are applicable is introduced.

[0069] Referring to Figure 2The artificial intelligence chip 200 at least includes a processing core 201, a cache 202, and a main memory 203. The processing core 201 has the ability of artificial intelligence related calculation, such as matrix multiplication, convolution operation, and the like, which are common calculation operations in an artificial intelligence model.

[0070] The cache 202 is a small and fast storage area between the processing core 201 and the main memory 203, used to temporarily store data or instructions that the processing core 201 or other hardware devices are likely to frequently access in the near future, with the purpose of speeding up access and reducing access delay. The main memory 203 is the next level of the cache 202, and compared with the cache 202, the main memory 203 generally has a larger capacity but a slower speed.

[0071] The cache 202 includes multiple levels of caches. The request scheduling method of the cache in the present application is applicable to LLC in the multiple levels of caches, and is also applicable to caches of other levels. The present application does not make specific limitations in this regard.

[0072] In some embodiments, referring to Figure 3 The cache 202 includes a tag memory 301, a hit-miss check unit 302, a cache memory 303, and a schedule buffer 304.

[0073] Specifically, the tag memory 301 is used to save tag information of each cache line in a plurality of cache sets. A cache set is a logical grouping composed of a plurality of cache lines. A cache line is the smallest unit of storing data in the cache 202, and is also the smallest operation unit of the cache 202.

[0074] A cache line includes data and tag information. The tag information includes a tag, a valid bit, and a dirty bit. The tag is used to identify the address range of the data in the main memory. The valid bit indicates whether the data is valid. The dirty bit indicates whether the data is modified, i.e., whether it is consistent with the main memory.

[0075] The cache memory 303 saves the data in each cache line. The tag information saved by the tag memory 301 has a one-to-one correspondence with the data saved by the cache memory 303.

[0076] In the embodiments of the present application, the cache 202 receives a first data request sent by the processing core 201. The first data request carries a first request index and a first memory address. The hit-miss check unit 302 reads the tag information of each cache line from the tag memory 301. Then, the hit-miss check unit 302 performs hit-miss check on the first memory address of the first data request based on the read tag information.

[0077] When the first memory address matches the tag information of a cache line, it is determined that the first data request hits the cache line; when there is no tag information matching the first memory address, a cache line is allocated for the first data request, and the memory address of the cache line is written into the tag storage 301.

[0078] The scheduling buffer 304 can store a plurality of scheduling records, each of which includes at least the following elements: a request index, a hazard slot, a hazard_en bit, and a tail marker. The plurality of scheduling records in the scheduling buffer 304 form one or more virtual links.

[0079] Each virtual link includes at least one scheduling record of a data request, which corresponds to the same memory address and is executed in sequence to achieve the same address ordering. Data requests associated with different virtual links correspond to different memory addresses, and data requests associated with different virtual links can be executed in parallel to achieve different address out-of-order.

[0080] In this application, the first data request can be a read request or a write request.

[0081] After the hit check unit 302 performs the hit check, the scheduling buffer 304 creates an initial scheduling record corresponding to the first request index.

[0082] When there is a target virtual link corresponding to the first memory address in the scheduling buffer 304, and the first data request is a write request, a second request index is added to the initial scheduling record, and the initial scheduling record is marked as a new tail record of the target virtual link to obtain a first scheduling record.

[0083] When there is a target virtual link corresponding to the first memory address in the scheduling buffer 304, and the first data request is a read request, a second request index is added to the initial scheduling record to obtain a third scheduling record.

[0084] The second request index is the request index of a second data request on which the first data request depends, which ensures that the first data request is executed after the second data request, and the data requests on different virtual links in the scheduling buffer 304 are executed in parallel to achieve same address ordering and different address out-of-order, ensuring correct function.

[0085] Since the length of the virtual link in the scheduling buffer 304 is not a fixed length, but can be continuously extended backwards under the condition that the entire scheduling buffer is not full, compared to the case where the queue is full and the request enters a waiting state, using a virtual link can effectively reduce the probability of cache access congestion.

[0086] Meanwhile, the corresponding resource is allocated for the scheduling record of the first data request from the remaining resources of the whole scheduling buffer, so that the resources of the scheduling buffer can be fully utilized, resource waste is reduced, and the overall access performance of the cache is improved.

[0087] In some embodiments, when the target virtual link corresponding to the first memory address does not exist in the scheduling buffer 304, and the first data request is a write request, a new virtual link corresponding to the first memory address is directly created; then the initial scheduling record is marked as the tail record of the new virtual link to obtain a fourth scheduling record; without waiting for the queue to be released, the scalability is effectively improved, and the cache access congestion is reduced.

[0088] Moreover, when the virtual link with the same address does not exist in the scheduling buffer 304, the corresponding resource is allocated from the remaining resources of the whole scheduling buffer to create a new virtual link, so that the resources of the scheduling buffer can be fully utilized, resource waste is reduced, and the overall access performance of the cache is improved.

[0089] In some embodiments, when the target virtual link corresponding to the first memory address does not exist in the scheduling buffer 304, and the first data request is a read request, the default state of the initial scheduling record is maintained, and without waiting for the queue to be released, the scalability is effectively improved, and the cache access congestion is reduced.

[0090] In some embodiments, if the first data request is a read request, the data of the cache line in the cache memory 303 is read, which can be previously read from the main memory 203; then, a request response is returned to the processing core 201, and the request response carries the read data.

[0091] If the first data request is a write request, the target data carried by the write request is written into the storage area corresponding to the cache line in the cache memory 303, and a request response is returned to the processing core 201, and the request response indicates that the writing is completed.

[0092] In addition to the above structure, the artificial intelligence chip 200 in the application can also include other structures, which are not limited in the application.

[0093] The artificial intelligence chip 200 can be a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a domain specific architecture (DSA), etc.

[0094] The following is based on Figure 2 and3 The architecture diagram of the artificial intelligence chip is shown, and a flow of a request scheduling method is specifically introduced, see Figure 4 The method is executed by a cache in the artificial intelligence chip, and the method comprises the following steps:

[0095] Step 401, receiving a first data request.

[0096] Specifically, the first data request can be any data request sent by a processing core to the cache. The first data request carries a first request index and a first memory address; the first request index is used to uniquely identify the first data request; and the first memory address is used to perform hit check on the first data request. The first data request can be a read request or a write request.

[0097] Step 402, creating an initial scheduling record corresponding to the first request index in the scheduling buffer.

[0098] Specifically, in the scheduling buffer, each scheduling record comprises at least the following elements: a request index, a conflict slot, a conflict enable bit and a tail marker.

[0099] The tail marker is used to identify whether it is a tail record of a virtual link; in actual application, the tail marker comprises a first tail state and a second tail state. The first tail state indicates a tail record of a virtual link; and the second tail state indicates a non-tail record of a virtual link. In the embodiment of the present application, the scheduling record in the first tail state is usually a scheduling record corresponding to a write request, that is, the tail record of the virtual link is a scheduling record of a write request.

[0100] The tail marker can correspond to different values, and the meaning of the tail marker is different when the values are different. For example, when the tail marker is 1, it indicates the first tail state; and when the state bit of the cache group is 0, it indicates the second tail state.

[0101] The conflict slot saves the request index of the dependent data request, that is, the request index of the data request in the hazard relationship, wherein the hazard relationship can be various forms, which are not limited in the present application.

[0102] In order to more clearly explain the hazard relationship, one of the forms is taken as an example for illustrative description: since the write request usually changes the data of the memory address, the write request is usually regarded as having a hazard relationship by other data requests; that is, the data request of the same memory address entering later will regard the request index of the tail record (that is, the scheduling record of the write request) as having a hazard relationship; for example, write after write (Write After Write, abbreviated as WAW) and read after write (Read After Write, abbreviated as RAW).

[0103] The conflict enable bit is used to identify whether there is a dependency relationship with other request indexes in the virtual link; in actual application, the conflict enable bit includes: a first conflict state and a second conflict state; the first conflict state indicates that there is a dependency relationship with other request indexes in the virtual link; and the second conflict state indicates that there is no dependency relationship with other request indexes in the virtual link.

[0104] The conflict enable bit can correspond to different values, and when the values are different, the meanings corresponding to the conflict enable bit are also different. For example, when the conflict enable bit is 1, the first conflict state is indicated; and when the state bit of the cache group is 0, the second conflict state is indicated.

[0105] After the initial scheduling record corresponding to the first request index is created, the default state of the initial scheduling record includes: the conflict slot is empty, the conflict enable bit is the second conflict state, and the tail marker is the second tail state. Of course, the default state of the initial scheduling record can also be other forms, and the present application does not make specific limitation thereto.

[0106] Step 403, when the target virtual link corresponding to the first memory address exists in the scheduling buffer, and the first data request is a write request, a second request index is added in the initial scheduling record, and the initial scheduling record is marked as the new tail record of the target virtual link, to obtain a first scheduling record.

[0107] Specifically, the second request index is the request index of a second data request dependent on the first data request; the second scheduling record corresponding to the second request index, and the second scheduling record has been added in the target virtual link. The second request index is added in the initial scheduling record of the first data request, to identify that the first data request depends on the second data request corresponding to the same memory address; and the initial scheduling record is marked as the new tail record of the target virtual link, to identify that the scheduling record of the first data request is the latest tail record of the target virtual link.

[0108] Since the first data request depends on the second data request, and the first data request and the second data request correspond to the same virtual link (i.e. the same memory address), in the execution process, the second data request is executed first, and the first data request is executed after the execution of the second data request, so that the same address order preservation is realized.

[0109] In the embodiments of the present application, the first data request sent by the cache receiving processing core is received, and then the scheduling buffer is queried based on the first memory address carried by the first data request; when the target virtual link of the same memory address is queried, the first scheduling record of the first data request is added on the target virtual link, wherein the first scheduling record includes the request index of the second request relied on by the first data request, so as to ensure that the first data request is executed after the second data request, and at the same time, the data requests on different virtual links in the scheduling buffer are executed in parallel, the same address order preservation and different address disorder are realized, and the function correctness is ensured.

[0110] Secondly, the length of each virtual link in the scheduling buffer is not a fixed length, but can be continuously extended backward under the condition that the entire scheduling buffer is not full, therefore, compared with the request entering the waiting state caused by the full queue, the virtual link can effectively reduce the probability of cache access congestion.

[0111] In addition, when the first data request is received, the corresponding resource is allocated for the scheduling record of the first data request from the remaining resources of the entire scheduling buffer, so that the resources of the scheduling buffer can be fully utilized, the resource waste is reduced, and the overall access performance of the cache is improved.

[0112] In some embodiments, the second request index is added in the initial scheduling record, and the initial scheduling record is marked as a new tail record of the target virtual link, before the first scheduling record is obtained, the second scheduling record marked as a tail record in the target virtual link is determined; and then the second request index is obtained from the second scheduling record.

[0113] Specifically, before the first scheduling record is marked as a new tail record of the target virtual link, the old tail record of the target virtual link is the second scheduling record; therefore, before the first scheduling record is obtained, each scheduling record in the target virtual link is scanned; when the second scheduling record with the first tail state is scanned, the second request index is read from the second scheduling record; wherein the second scheduling record is the scheduling record of the second data request relied on by the first data request; and the second request index is the request index of the second data request relied on by the first data request.

[0114] After the second request index is read from the second scheduling record, the conflict slot in the initial scheduling record is set as the second request index, the conflict enable bit in the initial scheduling record is set as the first conflict state, and the tail mark in the initial scheduling record is set as the first tail state, to obtain the first scheduling record.

[0115] At this time, the new tail record of the target virtual link has been updated to the first scheduling record; accordingly, the second scheduling record is marked as a non-tail record, that is, the tail mark in the second scheduling record is adjusted to the second tail state.

[0116] For example, referring to Figure 5A , the scheduling buffer is configured to include a virtual link 1 (corresponding to the memory address 1), and the virtual link 1 includes a scheduling record 1 with a request index of index 1. In the scheduling record 1, the tail flag is 1 (i.e., the scheduling record 1 is a tail record of the virtual link 1), the conflict enable bit is 0 (i.e., the scheduling record 1 has no dependency on other request indexes in the virtual link 1), and the conflict slot bit is empty.

[0117] For example, referring to Figure 5B , when a write request 2 with a request index of index 2 is received, the write request 2 carries the memory address 1 (i.e., hits the virtual link 1), and thus a scheduling record 2 with index 2 is added to the virtual link 1. In the scheduling record 2, the tail flag is 1, the conflict enable bit is 1 (i.e., the scheduling record 2 has a dependency on other request indexes in the virtual link 1), and the conflict slot bit is 1 (i.e., the scheduling record 2 depends on index 1). Meanwhile, the tail flag in the scheduling record 1 is adjusted to 0.

[0118] For example, referring to Figure 5C , when a write request 0 with a request index of index 0 is received, the write request 0 carries the memory address 1 (i.e., hits the virtual link 1), and thus a scheduling record 0 with index 0 is added to the virtual link 1. In the scheduling record 0, the tail flag is 1, the conflict enable bit is 1, and the conflict slot bit is 2 (i.e., the scheduling record 0 depends on index 2). Meanwhile, the tail flag in the scheduling record 2 is adjusted to 0.

[0119] In some embodiments, when there is no target virtual link corresponding to the first memory address in the scheduling buffer, and the first data request is a write request, a new virtual link corresponding to the first memory address is created; and then the initial scheduling record is marked as a tail record of the new virtual link to obtain a fourth scheduling record.

[0120] Specifically, the tail flag in the initial scheduling record is set to a first tail state, i.e., the initial scheduling record is marked as a tail record of the new virtual link; since the new virtual link does not contain other scheduling records, the initial scheduling record has no other request index that depends on it, based on which the conflict slot bit in the initial scheduling record is set to empty, and the conflict enable bit in the initial scheduling record is set to a second conflict state to obtain the fourth scheduling record.

[0121] For example, referring to Figure 5C , the scheduling buffer is configured to include a virtual link 1 (corresponding to the memory address 1), and the virtual link 1 includes a scheduling record 0 with a request index of index 0, a scheduling record 1 with a request index of index 1, and a scheduling record 2 with a request index of index 2.

[0122] For example, referring to Figure 6When the write request 5 with index 5 is received, the write request 5 carries the memory address 2 (i.e. no virtual link is hit), and thus, the virtual link 2 is created, and then the scheduling record 5 with index 5 is added to the virtual link 2. In the scheduling record 5, the tail flag is 1, the conflict enable bit is 0, and the conflict slot is empty.

[0123] In the embodiments of the present application, when a write request is received, if there is no virtual link with the same address in the scheduling buffer, a new virtual link can be created in the scheduling buffer in time, and the scheduling record of the write request is recorded as the tail record of the new virtual link, without waiting for the queue to be released, so as to reduce the cache access congestion, and improve the scalability.

[0124] Secondly, when there is no virtual link with the same address in the scheduling buffer, the corresponding resource is allocated from the remaining resources in the whole scheduling buffer to create a new virtual link, so as to fully utilize the resources of the scheduling buffer, reduce the resource waste, and further improve the overall access performance of the cache.

[0125] In some embodiments, when there is a target virtual link corresponding to the first memory address in the scheduling buffer, and the second data request is a read request, the second request index is added in the initial scheduling record to obtain a third scheduling record.

[0126] Specifically, when there is a target virtual link corresponding to the first memory address in the scheduling buffer, each scheduling record in the target virtual link is scanned; when a second scheduling record with a tail flag of a first tail state is scanned, the second request index is read from the second scheduling record.

[0127] The conflict enable bit of the initial scheduling record is set to a first conflict state, the conflict slot of the initial scheduling record is set to the second request index (indicating the request index dependent on the second data request), and the tail flag of the initial scheduling record is set to a second tail state (i.e. the scheduling record of the read request does not enter the virtual link), to obtain the third scheduling record.

[0128] For example, referring to Figure 6 It is assumed that the scheduling buffer includes a virtual link 1 (corresponding to a memory address 1) and a virtual link 2 (corresponding to a memory address 2), the virtual link 1 includes a scheduling record 0 with an index 0, a scheduling record 1 with an index 1, and a scheduling record 2 with an index 2. The virtual link 2 includes a scheduling record 5 with an index 5.

[0129] For example, referring to Figure 7A, when a read request 6 with index 6 is received, the read request 6 carries the memory address 2 (i.e., hits the virtual link 2), and thus, a scheduling record 6 with index 6 is added on the virtual link 2. In the scheduling record 6, the tail flag is 0, the conflict enable bit is 1, and the conflict slot bit is 5 (i.e., depends on index 5). At this time, the tail record of the virtual link 2 is still the scheduling record 5.

[0130] Referring to Figure 7B , when a write request 3 with index 3 is received, the write request 3 carries the memory address 2 (i.e., hits the virtual link 2), and thus, a scheduling record 3 with index 3 is added on the virtual link 2. In the scheduling record 3, the tail flag is 1, the conflict enable bit is 1, and the conflict slot bit is 5 (i.e., depends on index 5). At this time, the tail flag in the scheduling record 5 is adjusted to 0.

[0131] In the embodiments of the present application, when a read request is received, if a virtual link of the same memory address is found, a third scheduling record of the first data request is added on the virtual link, and the third scheduling record includes a request index of a second request on which the first data request depends, so as to ensure the correctness of the read request.

[0132] Secondly, the scheduling record of the read request can be directly mounted on the virtual link of the same memory address, without updating the tail flag (i.e., without entering the virtual link), thereby reducing resource consumption and reducing the probability of cache access blockage.

[0133] In some embodiments, when the target virtual link corresponding to the first memory address does not exist in the scheduling buffer, and the first data request is a read request, the default state of the initial scheduling record is maintained; the default state includes: the conflict slot bit is empty, the conflict enable bit is the second conflict state, and the tail flag is the second tail state.

[0134] Specifically, when the target virtual link corresponding to the first memory address does not exist in the scheduling buffer, and the first data request is a read request, a new virtual link does not need to be constructed, but the initial scheduling record of the first data request is directly saved in the scheduling buffer.

[0135] For example, referring to Figure 6 , it is assumed that the scheduling buffer includes a virtual link 1 (corresponding to a memory address 1) and a virtual link 2 (corresponding to a memory address 2), the virtual link 1 includes: a scheduling record 1 with index 1, a scheduling record 2 with index 2, and a scheduling record 0 with index 0. The virtual link 2 includes: a scheduling record 5 with index 5.

[0136] Referring to Figure 8When a read request 4 with index 4 is received, this read request 4 carries memory address 3 (i.e., no virtual link is hit). Here, there is no need to create a new virtual link; the scheduling record 4 of read request 4 is directly saved. In scheduling record 4, the tail mark is 0, the conflict enable bit is 0, and the conflict slot is empty.

[0137] In this embodiment of the application, when a read request is received, if there is no virtual link with the same address in the scheduling buffer, the scheduling record of the read request can be stored separately in the scheduling buffer without waiting for the queue to be released, thereby reducing cache access blocking and improving scalability.

[0138] Based on the same technical concept, embodiments of this application provide a computer device, such as... Figure 9 As shown, it includes at least one artificial intelligence chip 200 and a memory 901 connected to at least one artificial intelligence chip 200. In this embodiment, the specific connection medium between the artificial intelligence chip 200 and the memory 901 is not limited. Figure 9 Taking the connection between the AI ​​chip 200 and the memory 901 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0139] In this embodiment, the memory 901 stores instructions that can be executed by at least one artificial intelligence chip 200. By executing the instructions stored in the memory 901, the at least one artificial intelligence chip 200 can perform the steps of the above-mentioned cached request scheduling method.

[0140] Among them, the artificial intelligence chip 200 is the control center of the computer device. It can connect to various parts of the computer device through various interfaces and lines. By running or executing instructions stored in the memory 901 and calling data stored in the memory 901, it can realize cached request scheduling.

[0141] Optionally, the artificial intelligence chip 200 may include one or more processing units. The artificial intelligence chip 200 may integrate an application processor and a modem processor, wherein the application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the artificial intelligence chip 200. In some embodiments, the artificial intelligence chip 200 and the memory 901 may be implemented on the same chip; in some embodiments, they may be implemented separately on independent chips.

[0142] The artificial intelligence chip 200 can be a general processor, such as a central processing unit (CPU), a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0143] The memory 901 is a non-volatile computer readable storage medium, which can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 901 can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read only memory (PROM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. The memory 901 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer device, but is not limited thereto. The memory 901 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used to store program instructions and / or data.

[0144] Based on the same inventive concept, the embodiments of the present application provide a computer readable storage medium storing a computer program executable by a computer device, which, when running on the computer device, causes the computer device to execute the steps of the above-mentioned cache request scheduling method.

[0145] Those skilled in the art will appreciate that embodiments of the present application can be devised for a method, or for a computer program product. Accordingly, the present application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0146] The present application is described in reference to the drawings, which are as follows:

[0147] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart or block diagram block or blocks.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or block diagram block or blocks.

[0149] While preferred embodiments of the application have been described, those skilled in the art will appreciate that other modifications than those specifically described can be made within the scope of the present application. Accordingly, no limitation is intended based on the description as recited in the patent claims. It is therefore intended that the scope of the application be determined by the following claims and their equivalents.

[0150] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method of scheduling requests for a cache, characterized by, Comprising: receiving a first data request, the first data request comprising a first request index and a first memory address; creating an initial scheduling record corresponding to the first request index in a scheduling buffer; when there is a target virtual link corresponding to the first memory address in the scheduling buffer and the first data request is a write request, adding a second request index in the initial scheduling record, and marking the initial scheduling record as a new tail record of the target virtual link to obtain a first scheduling record; wherein the second request index is a request index of a second data request on which the first data request depends.

2. The method of claim 1, wherein, Before the adding the second request index in the initial scheduling record and marking the initial scheduling record as the new tail record of the target virtual link to obtain the first scheduling record, further comprising: determining a second scheduling record marked as a tail record in the target virtual link; obtaining the second request index from the second scheduling record.

3. The method of claim 2, wherein, After the obtaining the second request index from the second scheduling record, further comprising: marking the second scheduling record as a non-tail record.

4. The method of claim 1, wherein, The initial scheduling record comprises a conflict slot, a conflict enable bit and a tail mark; The adding the second request index in the initial scheduling record and marking the initial scheduling record as the new tail record of the target virtual link to obtain the first scheduling record comprises: setting the conflict slot as the second request index, setting the conflict enable bit as a first conflict state, and setting the tail mark as a first tail state to obtain the first scheduling record; wherein the first conflict state indicates that there is a dependency relationship with other request indexes in a virtual link, and the first tail state indicates a tail record of a virtual link.

5. The method of claim 1, wherein, Further comprising: when there is a target virtual link corresponding to the first memory address in the scheduling buffer and the first data request is a read request, adding the second request index in the initial scheduling record to obtain a third scheduling record.

6. The method of claim 5, wherein, The initial scheduling record comprises a conflict slot, a conflict enable bit and a tail mark; The adding the second request index in the initial scheduling record to obtain the third scheduling record comprises: setting the conflict slot as the second request index, setting the conflict enable bit as a first conflict state, and setting the tail mark as a second tail state to obtain the third scheduling record; wherein the first conflict state indicates that there is a dependency relationship with other request indexes in a virtual link, and the second tail state indicates a non-tail record of a virtual link.

7. The method of any one of claims 1 to 6, wherein, Further comprising: when there is no target virtual link corresponding to the first memory address in the scheduling buffer and the first data request is a write request, creating a new virtual link corresponding to the first memory address; marking the initial scheduling record as a tail record of the new virtual link to obtain a fourth scheduling record.

8. The method of claim 7, wherein, The initial scheduling record comprises a conflict slot, a conflict enable bit and a tail mark; The marking the initial scheduling record as the tail record of the new virtual link to obtain the fourth scheduling record comprises: setting the conflict slot to be empty, setting the conflict enable bit to be a second conflict state, and setting the tail flag to be a first tail state, to obtain the fourth scheduling record; wherein the second conflict state indicates that there is no dependency relationship with other request indexes in the virtual link, and the first tail state indicates a tail record of the virtual link.

9. The method of claim 8, wherein, Further comprising: when there is no target virtual link corresponding to the first memory address in the scheduling buffer, and the first data request is a read request, keeping a default state of the initial scheduling record; the default state includes that the conflict slot is empty, the conflict enable bit is the second conflict state, and the tail flag is a second tail state; and the second tail state indicates a non-tail record of the virtual link.

10. An artificial intelligence chip, characterized by, Comprising: a processing core, a cache, and a main memory; the cache is configured to perform the method of any one of claims 1 to 9.

Images