Instruction cache optimization method, device, storage medium and program product

By obtaining the historical instruction fetching number and arbitration priority of the thread bundle, the processing flow of instruction cache is optimized, which solves the problem of poor synchronization of thread bundle execution progress in multi-core processors, and improves the performance and resource utilization of instruction caches.

CN118171612BActive Publication Date: 2024-11-08北京壁仞科技开发有限公司 +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410598217.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-14
Publication Date
2024-11-08
Estimated Expiration
2044-05-14

AI Technical Summary

Technical Problem

In a multi-core processor architecture, poor synchronization of execution progress of multiple thread bundles leads to a degradation of instruction cache performance. The prior art has poor arbitration control effect through the instruction scheduling module, which affects the performance of instruction cache.

Method used

By obtaining the historical number of instructions fetches of multiple thread bundles and determining priority based on negative correlation, the arbitration module selects a thread bundle with high priority from the thread bundle to execute the instruction fetch request, and merges requests of the same instruction address to improve the locality of the instruction cache Sex and hit rate.

Benefits of technology

It improves the synchronization of execution progress between thread bundles, improves the performance of instruction cache, reduces duplicate access to the outside world, and optimizes hardware resource utilization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118171612B_ABST
    Figure CN118171612B_ABST
Patent Text Reader

Abstract

The embodiment of the present application provides an instruction cache optimization method, device, storage medium and program product, which relates to the field of chip design verification technology. The method includes: determining the priorities of the multiple thread bundles based on the historical number of instruction fetches of the multiple thread bundles, and the historical number of instruction fetches is negatively correlated with the priority. Then, based on the priorities of the multiple thread bundles, at least one thread bundle with arbitration permission is determined from the multiple thread bundles, so that the thread bundle with slow execution progress obtains arbitration permission and obtains the corresponding target instruction first, so that the execution progress of the multiple thread bundles is gradually unified, thereby improving the locality and hit rate of the instruction cache, and then improving the performance of the instruction cache. Since the execution progress between thread bundles is more synchronized, other resources such as data cache can also be fully utilized, reducing repeated external accesses, while avoiding the problem of a single thread bundle executing too slowly and other thread bundles waiting, thereby improving the utilization rate of hardware resources.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of chip design verification technology, and in particular to an instruction cache optimization method, device, storage medium and program product. Background Art

[0002] In order to improve the performance of the entire chip system, a cache component is designed between the processor and the external memory to solve the problem of the operating frequency of the processor accessing data and instructions. The multi-core processor architecture usually adopts a multi-level cache structure. The first-level cache includes: instruction cache (I-Cache) and data cache (D-Cache); the second-level cache is designed as a multi-core shared cache.

[0003] The instruction cache (I-Cache) is a memory close to the processor. When executing a program, the processor will frequently fetch instructions from the instruction cache and execute them. When the processor executes programs of multiple thread warps in parallel, the synchronization of the execution progress of multiple thread warps will affect the performance of the instruction cache.

[0004] In the related art, the instruction scheduling module performs fair arbitration, sequentially fetches instructions of each thread warp from the instruction cache, and transmits the instructions to the execution unit for execution, so as to control the synchronization of the execution progress of multiple thread warps. However, when the execution progress between thread warps is controlled by the instruction scheduling module downstream of the instruction cache, the control effect is poor, which in turn affects the performance of the instruction cache. Summary of the invention

[0005] Embodiments of the present application provide an instruction cache optimization method, device, storage medium, and program product for synchronizing the execution progress between thread warps, thereby improving the performance of the instruction cache.

[0006] On the one hand, an embodiment of the present application provides an instruction cache optimization method, including:

[0007] Obtain instruction fetch requests for multiple warps;

[0008] Determining priorities of the plurality of thread warps based on historical instruction fetch counts of the plurality of thread warps, wherein the historical instruction fetch counts are negatively correlated with the priorities;

[0009] determining at least one warp permitted by arbitration from among the plurality of warps based on respective priorities of the plurality of warps;

[0010] An instruction fetch request of the at least one thread warp is executed to obtain a corresponding target instruction.

[0011] On the one hand, an embodiment of the present application provides an instruction cache optimization device, including:

[0012] An acquisition module, used for acquiring instruction fetch requests of multiple thread warps;

[0013] A processing module, configured to determine the priority of each of the plurality of thread warps based on the historical number of instruction fetches of each of the plurality of thread warps, wherein the historical number of instruction fetches is negatively correlated with the priority;

[0014] an arbitration module, configured to determine at least one thread warp permitted by arbitration from the plurality of thread warps based on respective priorities of the plurality of thread warps;

[0015] The execution module is used to execute the instruction fetch request of the at least one thread warp and obtain the corresponding target instruction.

[0016] Optionally, the arbitration module is specifically used for:

[0017] If the historical instruction fetch numbers of the plurality of thread warps include a historical instruction fetch number that reaches a count upper limit, setting the priority of the thread warp corresponding to the historical instruction fetch number that reaches the count upper limit as a blocking priority;

[0018] The warp corresponding to the blocking priority is excluded from the plurality of warps, and at least one warp permitted by arbitration is determined from the remaining warps according to respective priorities of the remaining warps.

[0019] Optionally, the historical number of instruction fetches of each of the multiple thread warps is recorded through a dynamic synchronization matrix, wherein the dynamic synchronization matrix includes: multiple recording areas, each recording area corresponds to a thread warp; the position of the recording mark in each recording area is used to characterize the historical number of instruction fetches of the corresponding thread warp.

[0020] Optionally, the processing module is further used for:

[0021] If the record mark in the record area moves to the highest bit in the record area, the number of historical instruction fetches of the thread warp corresponding to the record area reaches the upper limit of the count.

[0022] Optionally, the processing module is further used for:

[0023] When none of the recording marks in the plurality of recording areas is located at the lowest bit, the recording marks in the plurality of recording areas are respectively moved one bit toward the lowest bit.

[0024] Optionally, the execution module is specifically used to:

[0025] When there are multiple instruction fetch requests with the same instruction address among the instruction fetch requests of the at least one thread warp, merging the multiple instruction fetch requests into a target request;

[0026] The target request is executed, and a corresponding target instruction is obtained from an instruction cache or an upper-level cache.

[0027] Optionally, the execution module is specifically used to:

[0028] For the instruction fetch request of the at least one thread warp, the following operations are performed respectively:

[0029] Execute an instruction fetch request of a thread warp and obtain the corresponding target instruction from the instruction cache or the upper cache.

[0030] On the one hand, an embodiment of the present application provides a computer device, including a memory, a processor chip, and a computer program stored in the memory and executable on the processor chip, wherein the processor chip implements the steps of the above-mentioned instruction cache optimization method when executing the computer program.

[0031] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program executable by a computer device. When the computer program runs on the computer device, the computer device executes the steps of the above-mentioned instruction cache optimization method.

[0032] On the one hand, an embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer device, the computer device executes the steps of the above-mentioned instruction cache optimization method.

[0033] In the embodiment of the application, the priorities of the multiple thread warps are determined based on the historical number of instructions fetched by each of the multiple thread warps, wherein the historical number of instructions fetched is negatively correlated with the priority. Then, based on the priorities of the multiple thread warps, at least one thread warp with arbitration permission is determined from the multiple thread warps, so that the thread warp with slow execution progress can obtain arbitration permission and corresponding target instructions first, so that the execution progress of the multiple thread warps is gradually unified, thereby improving the locality and hit rate of the instruction cache, and further improving the performance of the instruction cache.

[0034] Secondly, since the execution progress between thread warps is more synchronized, other resources such as data cache can be fully utilized, reducing repeated external access. In addition, since the execution progress between thread warps is more synchronized, the problem of a single thread warp executing too slowly and other thread warps waiting for a long time is avoided, thereby improving the utilization of hardware resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1 A schematic diagram of the structure of a processor chip provided in an embodiment of the present application;

[0037] Figure 2 A flow chart of an instruction cache optimization method provided in an embodiment of the present application Figure 1 ;

[0038] Figure 3 Schematic diagram of the counting method based on the dynamic synchronization matrix provided in the embodiment of the present application Figure 1 ;

[0039] Figure 4 Schematic diagram of the counting method based on the dynamic synchronization matrix provided in the embodiment of the present application Figure 2 ;

[0040] Figure 5 Schematic diagram of the counting method based on the dynamic synchronization matrix provided in the embodiment of the present application Figure 3 ;

[0041] Figure 6 Schematic diagram of the counting method based on the dynamic synchronization matrix provided in the embodiment of the present application Figure 4 ;

[0042] Figure 7 Schematic diagram of the counting method based on the dynamic synchronization matrix provided in the embodiment of the present application Figure 5 ;

[0043] Figure 8 A flow chart of an instruction cache optimization method provided in an embodiment of the present application Figure 2 ;

[0044] Fig. 9 Schematic diagram of the counting method based on the dynamic synchronization matrix provided in the embodiment of the present application Figure 6 ;

[0045] Fig.10 A schematic diagram of the structure of an instruction cache optimization device provided in an embodiment of the present application;

[0046] Fig.11 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and beneficial effects of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0048] refer to Figure 1 , which is a structural diagram of a processor chip applicable to an embodiment of the present application. The processor chip 100 includes at least: multiple computing cores 101, an instruction cache 102, and an upper-level cache 103.

[0049] The instruction cache 102 simultaneously serves multiple computing cores 101. The multiple computing cores 101 can execute programs of multiple warps in parallel. A warp refers to multiple threads that are bound together and executed synchronously.

[0050] The program of each thread warp runs on a computing core 101. Of course, a computing core 101 can also execute the programs of multiple thread warps in parallel, which is not specifically limited in this application.

[0051] In some cases, when the programs of multiple warps are the same and the execution progress of the multiple warps is also the same, the instruction cache 102 can achieve the best performance.

[0052] The instruction cache 102 is a temporary memory, which has a smaller capacity than the upper cache 103 but a faster exchange speed. The instructions in the instruction cache 102 are obtained from the upper cache 103, and these instructions are to be accessed by the computing core 101 in a short time. In this way, when the computing core 101 calls an instruction, it can bypass the upper cache 103 and call it directly from the instruction cache 102, thereby speeding up the reading speed.

[0053] The upper level cache 103 may be a secondary cache in a multi-level cache structure, or may be a video memory, etc.

[0054] In addition to the above-mentioned structure, the processor chip 100 in the present application may also include other structures, which are not specifically limited in the present application.

[0055] The processor chip 100 may be: a central processing unit (CPU), a graphics processing unit (GPU), a general-purpose computing on graphics processing units (GPGPU), a domain-specific architecture (DSA), etc.

[0056] In actual applications, when a processor executes programs of multiple thread warps in parallel, the synchronization of the execution progress of multiple thread warps will affect the performance of the instruction cache. The more synchronized the execution progress of multiple thread warps is, the better the memory access locality of the instruction cache is, so the performance of the instruction cache is better. The more dispersed the execution progress of multiple thread warps is, the worse the memory access locality of the instruction cache is, so the performance of the instruction cache is worse.

[0057] In the related art, fair arbitration is performed through an instruction scheduling module, instructions of each thread warp are sequentially fetched from an instruction cache, and the instructions are sent to an execution unit for execution, so as to control the synchronization of the execution progress of multiple thread warps.

[0058] However, when the execution unit actually executes the instructions of each thread bundle, the execution time is not the same. For example, some instructions are delayed for a long time during execution due to some operations, resulting in a long execution time for these instructions. Some instructions are executed quickly, that is, the execution time is short. Since the execution time of the instructions of each thread bundle is different, the execution progress of each thread bundle gradually disperses. In other words, when the execution progress between thread bundles is controlled by the instruction scheduling module downstream of the instruction cache, the control effect is poor, which in turn affects the performance of the instruction cache.

[0059] In view of this, this application is based on Figure 1 The processor chip architecture diagram shown in FIG. 1 provides a flow chart of an instruction cache optimization method, such as Figure 2 As shown, the process of the method is executed by a processor chip and includes the following steps:

[0060] Step 201 : obtaining instruction fetch requests of multiple warps.

[0061] Specifically, a warp refers to multiple threads that are bound together and executed synchronously. Multiple warps executed in parallel on a processor chip send instruction fetch requests to the instruction cache. Multiple warps can run on one computing core in a processor chip or on multiple computing cores in a processor chip.

[0062] Step 202 : determining the priorities of the plurality of thread warps based on the historical instruction fetch counts of the plurality of thread warps, wherein the historical instruction fetch counts are negatively correlated with the priority.

[0063] Specifically, at the instruction cache entry, the historical instruction fetch counts of each of the multiple thread warps are queried, wherein the historical instruction fetch counts of the thread warps are used to characterize the execution progress of the thread warps. The more historical instruction fetch counts of the thread warps, the faster the execution progress of the thread warps; accordingly, the lower the priority assigned to the thread warps. The fewer historical instruction fetch counts of the thread warps, the slower the execution progress of the thread warps; accordingly, the higher the priority assigned to the thread warps.

[0064] A plurality of priorities are set in advance based on the negative correlation between the number of historical instruction fetches and the priority, each historical priority corresponds to a range of instruction fetches, and each range of instruction fetches includes one or more historical numbers of instruction fetches. After determining the number of historical instruction fetches of a thread warp, a matching range of instruction fetches is determined from the plurality of ranges of instruction fetches based on the historical number of instruction fetches of the thread warp. Then, the priority corresponding to the matching range of instruction fetches is used as the priority of the thread warp.

[0065] In some embodiments, the historical number of instruction fetches of each of the plurality of thread warps is recorded by a dynamic synchronization matrix, wherein the dynamic synchronization matrix comprises: a plurality of recording areas, each recording area corresponding to a thread warp; and the position of a recording mark in each recording area is used to characterize the historical number of instruction fetches of the corresponding thread warp.

[0066] Specifically, each recording area in the dynamic synchronization matrix can be one or more rows in the dynamic synchronization matrix, or one or more columns in the dynamic synchronization matrix, or an area obtained by other division methods. The recording mark can be a symbol such as a character or a number. In the reset phase, the values ​​of each position in the dynamic synchronization matrix are set to 0. As the number of historical instruction fetches of the thread warp gradually increases, the recording mark in the recording area corresponding to the thread warp gradually moves from the lowest bit to the highest bit, and the number of bits moved each time can be set according to actual conditions.

[0067] For example, if Figure 3 As shown, a dynamic synchronization matrix is ​​set to record the historical number of instruction fetches of the four warps. Each row of the dynamic synchronization matrix corresponds to a warp. Each row of the dynamic synchronization matrix includes 8 bits (from right to left, the 0th bit to the 7th bit, the 0th bit is the lowest bit, and the 7th bit is the highest bit). The four warps are: warp 0, warp 1, warp 2, and warp 3, and the record mark is 1.

[0068] During the reset phase, the values ​​of each position in the dynamic synchronization matrix are set to 0.

[0069] When warp 2 and warp 3 obtain arbitration permission, the 0th bit of the row corresponding to warp 2 and warp 3 in the dynamic synchronization matrix is ​​set to 1. That is, the record mark (1) is placed in the lowest bit of the row corresponding to warp 2 and warp 3.

[0070] Next, when warp 0 and warp 3 obtain arbitration permission, bit 0 of the row corresponding to warp 0 in the dynamic synchronization matrix is ​​set to 1. In the row corresponding to warp 3, the record mark (1) is shifted left by 1 bit.

[0071] In some embodiments, if the record mark in the record area moves to the highest bit in the record area, the number of historical instruction fetches of the warp corresponding to the record area reaches the upper limit of the count.

[0072] Specifically, the upper limit value of the count is associated with the window size of the dynamic synchronization matrix, and the upper limit value of the count can be set according to the properties of the hardware used to implement the dynamic synchronization matrix.

[0073] It should be noted that when the record mark moves to the highest bit in the record area, even if the thread warp corresponding to the record area subsequently obtains arbitration permission, the record mark of the record area cannot move to a higher bit but remains at the highest bit.

[0074] For example, if Figure 4 As shown, a dynamic synchronization matrix is ​​set to record the historical number of instruction fetches of the four warps. Each row of the dynamic synchronization matrix corresponds to a warp. Each row of the dynamic synchronization matrix includes 8 bits (from right to left, the 0th bit to the 7th bit, the 0th bit is the lowest bit, and the 7th bit is the highest bit). The four warps are: warp 0, warp 1, warp 2, and warp 3, and the record mark is 1.

[0075] In the dynamic synchronization matrix, the record mark (1) of thread warp 3 has moved to the highest bit, that is, the historical number of instructions fetched by thread warp 3 has reached the upper limit of the count.

[0076] Assume that when warp 3 obtains arbitration permission, the record mark (1) of warp 3 is no longer moved.

[0077] In the embodiment of the present application, multiple thread warps are synchronized in one window, which can effectively improve the locality and hit rate of the instruction cache, thereby improving the performance of the instruction cache.

[0078] In some embodiments, when none of the recording marks in the plurality of recording areas is located at the least significant bit, the recording marks in the plurality of recording areas are respectively moved one bit toward the least significant bit.

[0079] Specifically, when multiple recording areas all contain recording marks (i.e., they all start counting), and none of the multiple recording marks are located at the lowest bit. At this point, the lowest bit of the multiple recording areas no longer works for subsequent counting. However, the size of the recording area is limited. When the recording mark in the recording area moves to the highest bit, the recording area can no longer continue to count. In order to ensure the consistency of multiple recording areas when counting, and to dynamically expand the size of the recording area, the present application proposes that when none of the recording marks in the multiple recording areas are located at the lowest bit, the recording marks in the multiple recording areas are respectively moved one bit toward the lowest bit. In this way, for the recording area where the recording mark moves to the highest bit, counting can continue; for the recording area where the recording mark does not move to the highest bit, one extra bit is used for counting.

[0080] In practical applications, when none of the record marks in the multiple record areas is located at the lowest bit, the record marks in the multiple record areas can be moved one bit toward the lowest bit. Alternatively, when none of the record marks in the multiple record areas is located at the lowest bit and there is a record mark in the record area that has moved to the highest bit, the record marks in the multiple record areas can be moved one bit toward the lowest bit. This application makes specific limitations on this.

[0081] For example, if Figure 5 As shown, a dynamic synchronization matrix is ​​set to record the historical number of instruction fetches of the four warps. Each row of the dynamic synchronization matrix corresponds to a warp. Each row of the dynamic synchronization matrix includes 8 bits (from right to left, the 0th bit to the 7th bit, the 0th bit is the lowest bit, and the 7th bit is the highest bit). The four warps are: warp 0, warp 1, warp 2, and warp 3, and the record mark is 1.

[0082] In the dynamic synchronization matrix, the record mark (1) of warp 3 moves to the highest bit, that is, the historical instruction fetch count of warp 3 reaches the upper limit of the count. The record marks (1) of warp 2 and warp 1 move to the first bit. The record mark (1) of warp 0 is at the 0th bit.

[0083] Assuming that warp 0 obtains arbitration permission, in the row corresponding to warp 0, the record mark (1) is shifted left by 1 bit, that is, the record mark (1) is moved to the first bit.

[0084] At this time, the lowest bits of the four recording areas are all 0, so the recording marks (1) of the four recording areas are all moved one bit to the right. In this way, the recording mark (1) of thread bundle 3 is moved to the 6th bit (no longer in the highest bit); the recording marks (1) of thread bundle 0, thread bundle 1 and thread bundle 2 are moved to the 0th bit.

[0085] In the embodiment of the present application, when the record marks in multiple record areas are not located at the lowest bit, the record marks in the multiple record areas are respectively moved one bit toward the lowest bit, thereby realizing dynamic expansion of the record area and saving resource consumption.

[0086] Step 203 : determining at least one thread warp that is permitted by arbitration from the multiple thread warps based on the respective priorities of the multiple thread warps.

[0087] Specifically, the higher the priority of a warp, the higher the probability of obtaining arbitration permission; correspondingly, the lower the priority of a warp, the lower the probability of obtaining arbitration permission. The priorities corresponding to multiple warps in this application can be completely different or partially the same.

[0088] The number of thread warps that are allowed to obtain arbitration permission each time (per beat) is determined based on hardware resources (i.e., cache resources). When there are multiple thread warps that are allowed to obtain arbitration permission each time (per beat), thread warps with higher priorities are given priority to obtain arbitration permission. After thread warps with higher priorities obtain arbitration permission and there are still remaining hardware resources, thread warps with lower priorities can obtain arbitration permission.

[0089] In some embodiments, for a warp whose historical instruction fetch count reaches an upper limit, the present application proposes at least several arbitration methods:

[0090] Method 1: If the historical instruction fetch counts of the plurality of thread warps include the historical instruction fetch count that reaches the upper limit of the count, the priority of the thread warp corresponding to the historical instruction fetch count is set as the blocking priority. The thread warp corresponding to the blocking priority is excluded from the plurality of thread warps, and at least one thread warp permitted by arbitration is determined from the remaining thread warps according to the respective priorities of the remaining thread warps.

[0091] Specifically, there may be one or more thread warps corresponding to the blocking priority. The instruction cache blocks the thread warp at the blocking priority, that is, the thread warp is not allowed to obtain arbitration permission, and the remaining thread warps obtain arbitration permission according to their priority.

[0092] Method 2: For a thread warp whose historical instruction fetch count reaches the upper limit, the priority of the thread warp is determined based on the negative correlation between the historical instruction fetch count and the priority. Then, at least one thread warp permitted by arbitration is determined from the multiple thread warps according to their respective priorities.

[0093] Specifically, the instruction cache still allows a thread warp whose historical instruction fetch count reaches the upper limit of the count value to obtain arbitration permission, but the priority of the thread warp in obtaining arbitration permission is the lowest.

[0094] In some embodiments, two synchronization control modes, strong synchronization mode and weak synchronization mode, can be set. When the synchronization control mode is set to strong synchronization mode, the thread warp whose historical instruction fetch count reaches the upper limit value of the count is arbitrated in the above-mentioned method 1. When the synchronization control mode is set to weak synchronization mode, the thread warp whose historical instruction fetch count reaches the upper limit value of the count is arbitrated in the above-mentioned method 2.

[0095] For example, see Figure 6 , set the dynamic synchronization matrix to record the historical instruction fetch counts of the four warps. Each row of the dynamic synchronization matrix corresponds to a warp. Each row of the dynamic synchronization matrix includes 8 bits (from right to left, from bit 0 to bit 7, bit 0 is the lowest bit, and bit 7 is the highest bit). The four warps are: warp 0, warp 1, warp 2, and warp 3, and the record mark is 1.

[0096] When the synchronization control mode is set to the strong synchronization mode, the priority setting rules are as follows: when the record mark (1) is located at the 0th to 3rd bit, it corresponds to the first priority; when the record mark (1) is located at the 4th to 6th bit, it corresponds to the second priority; when the record mark (1) is located at the 7th bit, it corresponds to the blocking priority.

[0097] In the dynamic synchronization matrix, the record mark (1) of thread warp 3 is located at the 7th position, so the priority of thread warp 3 is the blocking priority.

[0098] The record mark (1) of the thread bundle 2 and the thread bundle 1 is located at the first position, and the record mark (1) of the thread bundle 0 is located at the 0th position, so the priorities of the thread bundle 0, the thread bundle 1, and the thread bundle 2 are all the first priority.

[0099] The instruction cache blocks thread warp 3, and then arbitration is performed according to the respective priorities of thread warp 0, thread warp 1, and thread warp 2.

[0100] Assuming that warp 0 obtains arbitration permission, in the row corresponding to warp 0, the record mark (1) is shifted left by 1 bit, that is, the record mark (1) is moved to the first bit.

[0101] See also Figure 7 , set the dynamic synchronization matrix to record the historical instruction fetch counts of the four warps. Each row of the dynamic synchronization matrix corresponds to a warp. Each row of the dynamic synchronization matrix includes 8 bits (from right to left, from bit 0 to bit 7, bit 0 is the lowest bit, and bit 7 is the highest bit). The four warps are: warp 0, warp 1, warp 2, and warp 3, and the record mark is 1.

[0102] When the synchronization control mode is set to the weak synchronization mode, the priority setting rule is: when the record mark (1) is located at the 0th to 3rd position, it corresponds to the first priority; when the record mark (1) is located at the 4th to 7th position, it corresponds to the second priority.

[0103] In the dynamic synchronization matrix, the record mark (1) of thread warp 3 is located at the 7th position, so the priority of thread warp 3 is the second priority.

[0104] The record mark (1) of the thread bundle 2 and the thread bundle 1 is located at the first position, and the record mark (1) of the thread bundle 0 is located at the 0th position, so the priorities of the thread bundle 0, the thread bundle 1, and the thread bundle 2 are all the first priority.

[0105] Arbitration is performed according to the respective priorities of warp 0, warp 1, warp 2 and warp 3, wherein the priorities of warp 0, warp 1 and warp 2 are higher than the priority of warp 3. Therefore, warp 0, warp 1 and warp 2 have a higher probability of obtaining arbitration permission.

[0106] Assume that warp 3 obtains arbitration permission, but the record mark (1) corresponding to warp 3 is already at the highest position, so the record mark (1) is no longer moved.

[0107] In an embodiment of the present application, for a thread warp whose historical instruction fetch count reaches an upper limit, a blocking priority is set to perform arbitration blocking, which can effectively control the progress of multiple thread warps within a window, thereby improving the locality and hit rate of the instruction cache, and further improving the performance of the instruction cache.

[0108] Step 204: execute an instruction fetch request of at least one warp to obtain a corresponding target instruction.

[0109] Specifically, an instruction fetch request of at least one warp is placed in a request processing queue, and the instruction fetch requests in the request processing queue are subsequently executed in sequence to obtain corresponding target instructions, and the target instructions are returned to the corresponding warp for execution.

[0110] In an embodiment of the present application, the priorities of the multiple thread bundles are determined based on the historical number of instructions fetched by each of the multiple thread bundles, wherein the historical number of instructions fetched is negatively correlated with the priority. Then, based on the respective priorities of the multiple thread bundles, at least one thread bundle with arbitration permission is determined from the multiple thread bundles, so that the thread bundle with slow execution progress can obtain arbitration permission and corresponding target instructions in priority, so that the execution progress of the multiple thread bundles is gradually unified, thereby improving the locality and hit rate of the instruction cache, and then improving the performance of the instruction cache. Secondly, since the execution progress between thread bundles is more synchronized, other resources such as data cache can also be fully utilized, reducing repeated external access. In addition, since the execution progress between thread bundles is more synchronized, the problem of a single thread bundle executing too slowly and other thread bundles waiting for a long time is avoided, thereby improving the utilization rate of hardware resources.

[0111] It should be noted that the present application can also combine the inter-thread progress control executed in the upstream instruction cache described in the previous steps 201-204 with the inter-thread progress control of the downstream instruction scheduling module. Through the joint control of upstream and downstream, the dispersion of execution between thread warps can be minimized, the locality of memory access can be improved, and the best performance can be achieved for the instruction cache.

[0112] In some embodiments, for an instruction fetch request of at least one warp, the following operations are performed respectively: executing an instruction fetch request of a warp, and obtaining a corresponding target instruction from an instruction cache or an upper-level cache.

[0113] Specifically, for each thread warp that obtains arbitration permission, the instruction cache is queried based on the instruction address in the instruction fetch request of the thread warp. When the instruction cache contains the corresponding target instruction, the target instruction is returned to the thread warp. When the instruction cache does not contain the corresponding target instruction, the corresponding target instruction is obtained from the upper cache and returned to the thread warp for execution.

[0114] Meanwhile, in the dynamic synchronization matrix, if the record mark corresponding to the warp is not located at the highest position, the record mark corresponding to the warp is shifted left by one position. If the record mark corresponding to the warp is already located at the highest position, the record mark corresponding to the warp is not moved.

[0115] In some embodiments, when there are multiple instruction fetch requests with the same instruction address in the instruction fetch requests of at least one warp, the multiple instruction fetch requests are merged into one target request. The target request is executed to obtain a corresponding target instruction from the instruction cache or the upper cache.

[0116] Specifically, when there are multiple instruction fetch requests with the same instruction address, it means that these multiple instruction fetch requests essentially fetch the same instruction. Therefore, the multiple instruction fetch requests are merged into a target request, which carries indication information, and the indication information is used to characterize the multiple instruction fetch requests of the synthesized target request.

[0117] Next, the instruction cache is queried based on the instruction address in the target request. When the instruction cache contains the corresponding target instruction, the target instruction is obtained from the instruction cache. When the instruction cache does not contain the corresponding target instruction, the corresponding target instruction is obtained from the upper cache.

[0118] After obtaining the target instruction, multiple instruction fetch requests for synthesizing the target request and the thread warps that send the multiple instruction fetch requests are determined based on the indication information carried by the target request, and the obtained target instructions are returned to the corresponding thread warps respectively.

[0119] Meanwhile, in the dynamic synchronization matrix, for each of the plurality of warps, if the record mark corresponding to the warp is not located at the highest bit, the record mark corresponding to the warp is shifted left by one bit. If the record mark corresponding to the warp is already located at the highest bit, the record mark corresponding to the warp is not moved.

[0120] In the embodiment of the present application, by controlling the priorities of multiple thread bundles, thread bundles with slow execution progress can obtain arbitration permission and corresponding target instructions first, thereby gradually improving the execution progress of multiple thread bundles. As the synchronization of thread bundles improves, the probability of request merging increases, that is, the probability of multiple instruction fetch requests with the same instruction address increases. When there are multiple instruction fetch requests with the same instruction address, the multiple instruction fetch requests are merged into one target request, and the target request is executed to obtain the corresponding target instruction from the instruction cache or the upper cache, thereby reducing the number of requests processed by the instruction cache, thereby improving the performance of the instruction cache.

[0121] In order to better explain the embodiments of the present application, an instruction cache optimization method flow provided by the embodiments of the present application is introduced below in combination with a specific implementation scenario. The method flow can be executed by the instruction cache.

[0122] See also Figure 8 ,The entry of the instruction cache includes : dynamic synchronization matrix, polling arbitration module 1, polling arbitration module 2, arbitration output module, request merging module, request processing queue.,The synchronization control modes supported by the instruction cache include : strong synchronization mode and weak synchronization mode.

[0123] The dynamic synchronization matrix is ​​set to record the historical instruction counts of the four thread warps. Each row of the dynamic synchronization matrix corresponds to a thread warp. Each row of the dynamic synchronization matrix includes 8 bits (from right to left, from bit 0 to bit 7, bit 0 is the lowest bit, and bit 7 is the highest bit). The four thread warps are: thread warp 0, thread warp 1, thread warp 2, and thread warp 3, and the record mark is 1. The dynamic synchronization matrix is ​​as follows Fig. 9 As shown, the record mark (1) of the warp 3 is located at the 7th position, the record marks (1) of the warp 2 and the warp 1 are located at the 1st position, and the record mark (1) of the warp 0 is located at the 0th position.

[0124] Assuming that the synchronization control mode is configured as the strong synchronization mode, the priority setting rule is: when the record mark (1) is located at the 0th to 3rd bit, it corresponds to the first priority; when the record mark (1) is located at the 4th to 6th bit, it corresponds to the second priority; when the record mark (1) is located at the 7th bit, it corresponds to the blocking priority.

[0125] The four thread warps send instruction fetch requests to the instruction cache respectively, and request masks at the instruction cache entry. Specifically, the polling arbitration module 1 obtains the first priority mask from the dynamic synchronization matrix, that is, all data from bit 0 to bit 3 in the dynamic synchronization matrix. Fig. 9 It can be seen that at this time, the record marks (1) of the thread warp 0, thread warp 1, and thread warp 2 are all located in the first priority mask, so the priorities of the thread warp 0, thread warp 1, and thread warp 2 are all the first priority.

[0126] Polling arbitration module 1 arbitrates thread bundle 0, thread bundle 1, and thread bundle 2. Assuming that thread bundle 0 and thread bundle 1 obtain arbitration permission, the record mark (1) corresponding to thread bundle 0 and thread bundle 1 is shifted left by 1 bit in the dynamic synchronization matrix. For details, see Fig. 9 .

[0127] The polling arbitration module 2 obtains the second priority mask from the dynamic synchronization matrix, that is, all data from the 4th to the 7th bit in the dynamic synchronization matrix. Fig. 9 It can be seen that at this time, the record mark (1) of thread warp 3 is located in the second priority mask and is located at the highest bit (bit 7), so the priority of thread warp 3 is set to the blocking priority. Polling arbitration module 2 blocks thread warp 3, that is, polling arbitration module 2 does not output the instruction fetch request of thread warp 3.

[0128] The polling arbitration module 1 outputs the first instruction fetch request of thread warp 0 and the second instruction fetch request of thread warp 1 to the request merging module through the arbitration output module. Since the instruction addresses in the first instruction fetch request and the second instruction fetch request are the same, the request merging module merges the first instruction fetch request and the second instruction fetch request into a target request and puts the target request into the request processing queue.

[0129] The target request is taken out from the request processing queue, and the instruction cache is queried based on the instruction address in the target request. When the instruction cache contains the corresponding target instruction, the target instruction is obtained from the instruction cache. When the instruction cache does not contain the corresponding target instruction, the corresponding target instruction is obtained from the upper cache.

[0130] Finally, the target instruction is returned to thread warp 0 and thread warp 1 for execution respectively.

[0131] In the embodiment of the present application, by controlling the progress between thread warps in the upstream instruction cache, thread warps with slow execution progress can obtain arbitration permission and corresponding target instructions first, thereby gradually unifying the execution progress of multiple thread warps, thereby improving the locality and hit rate of the instruction cache, and thus improving the performance of the instruction cache. Secondly, since the execution progress between thread warps is more synchronized, other resources such as data cache can also be fully utilized, reducing repeated external access.

[0132] Based on the same technical concept, the present application embodiment provides a structural diagram of an instruction cache optimization device, such as Fig.10 As shown, the instruction cache optimization device 1000 includes:

[0133] An acquisition module 1001 is used to acquire instruction fetch requests of multiple thread warps;

[0134] A processing module 1002 is configured to determine the priority of each of the plurality of thread warps based on the historical number of instruction fetches of each of the plurality of thread warps, wherein the historical number of instruction fetches is negatively correlated with the priority;

[0135] An arbitration module 1003, configured to determine at least one thread warp permitted by arbitration from the plurality of thread warps based on respective priorities of the plurality of thread warps;

[0136] The execution module 1004 is configured to execute the instruction fetch request of the at least one thread warp and obtain a corresponding target instruction.

[0137] Optionally, the arbitration module 1003 is specifically used for:

[0138] If the historical instruction fetch numbers of the plurality of thread warps include a historical instruction fetch number that reaches a count upper limit, setting the priority of the thread warp corresponding to the historical instruction fetch number that reaches the count upper limit as a blocking priority;

[0139] The warp corresponding to the blocking priority is excluded from the plurality of warps, and at least one warp permitted by arbitration is determined from the remaining warps according to respective priorities of the remaining warps.

[0140] Optionally, the historical number of instruction fetches of each of the multiple thread warps is recorded through a dynamic synchronization matrix, wherein the dynamic synchronization matrix includes: multiple recording areas, each recording area corresponds to a thread warp; the position of the recording mark in each recording area is used to characterize the historical number of instruction fetches of the corresponding thread warp.

[0141] Optionally, the processing module 1002 is further configured to:

[0142] If the record mark in the record area moves to the highest bit in the record area, the number of historical instruction fetches of the thread warp corresponding to the record area reaches the upper limit of the count.

[0143] Optionally, the processing module 1002 is further configured to:

[0144] When none of the recording marks in the plurality of recording areas is located at the lowest bit, the recording marks in the plurality of recording areas are respectively moved one bit toward the lowest bit.

[0145] Optionally, the execution module 1004 is specifically used for:

[0146] When there are multiple instruction fetch requests with the same instruction address among the instruction fetch requests of the at least one thread warp, merging the multiple instruction fetch requests into a target request;

[0147] The target request is executed, and a corresponding target instruction is obtained from an instruction cache or an upper-level cache.

[0148] Optionally, the execution module 1004 is specifically used for:

[0149] For the instruction fetch request of the at least one thread warp, the following operations are performed respectively:

[0150] Execute an instruction fetch request of a thread warp and obtain the corresponding target instruction from the instruction cache or the upper cache.

[0151] Based on the historical number of instructions fetched by each of the multiple thread warps, the priorities of the multiple thread warps are determined, wherein the historical number of instructions fetched is negatively correlated with the priorities. Then, based on the priorities of the multiple thread warps, at least one thread warp that is permitted by arbitration is determined from the multiple thread warps, so that the thread warp with slow execution progress can obtain arbitration permission and corresponding target instructions in priority, so that the execution progress of the multiple thread warps is gradually unified, thereby improving the locality and hit rate of the instruction cache, and further improving the performance of the instruction cache.

[0152] Secondly, since the execution progress between thread warps is more synchronized, other resources such as data cache can be fully utilized, reducing repeated external access. In addition, since the execution progress between thread warps is more synchronized, the problem of a single thread warp executing too slowly and other thread warps waiting for a long time is avoided, thereby improving the utilization of hardware resources.

[0153] Based on the same technical concept, the embodiment of the present application provides a computer device, such as Fig.11 As shown, it includes at least one processor chip 100 and a memory 1101 connected to the at least one processor chip 100. The specific connection medium between the processor chip 100 and the memory 1101 is not limited in the embodiment of the present application. Fig.11 For example, the processor chip 100 and the memory 1101 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, and the like.

[0154] In an embodiment of the present application, the memory 1101 stores instructions that can be executed by at least one processor chip 100. By executing the instructions stored in the memory 1101, at least one processor chip 100 can perform the steps of the above-mentioned instruction cache optimization method.

[0155] Among them, the processor chip 100 is the control center of the computer device, and various interfaces and lines can be used to connect various parts of the computer device, and the instruction cache can be optimized by running or executing instructions stored in the memory 1101 and calling data stored in the memory 1101. Optionally, the processor chip 100 may include one or more processing units, and the processor chip 100 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor chip 100. In some embodiments, the processor chip 100 and the memory 1101 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on independent chips.

[0156] The processor chip 100 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0157] The memory 1101 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 1101 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, and the like. The memory 1101 is any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer device, but is not limited thereto. The memory 1101 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0158] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program executable by a computer device. When the computer program runs on the computer device, the computer device executes the steps of the above-mentioned instruction cache optimization method.

[0159] Based on the same inventive concept, an embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer device, the computer device executes the steps of the above-mentioned instruction cache optimization method.

[0160] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0161] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer device or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0162] These computer program instructions may also be stored in a computer readable memory capable of directing a computer device or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0163] These computer program instructions can also be loaded onto a computer device or other programmable data processing device so that a series of operation steps are executed on the computer device or other programmable device to produce a process implemented by the computer device, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0164] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0165] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for optimizing instruction cache, characterized in that: At least for GPU or GPGPU, including: Obtaining instruction fetch requests of a plurality of thread warps executed in parallel, wherein the plurality of thread warps have the same program; In the instruction cache entry, query the historical instruction fetch counts of each of the plurality of thread warps, where the historical instruction fetch counts of each of the plurality of thread warps are recorded by a dynamic synchronization matrix, wherein the dynamic synchronization matrix includes: a plurality of recording areas, each recording area corresponding to a thread warp; a position of a recording mark in each recording area, used to represent the historical instruction fetch count of the corresponding thread warp; Determining priorities of the plurality of thread warps based on historical instruction fetch counts of the plurality of thread warps, wherein the historical instruction fetch counts are negatively correlated with the priorities, and the historical instruction fetch counts of the thread warps are used to characterize execution progress of the thread warps; determining at least one warp permitted by arbitration from among the plurality of warps based on respective priorities of the plurality of warps; An instruction fetch request of the at least one warp is executed, and a corresponding target instruction is obtained from an instruction cache or an upper-level cache.

2. The method according to claim 1, characterized in that The step of determining at least one thread warp permitted by arbitration from the plurality of thread warps based on respective priorities of the plurality of thread warps comprises: If the historical instruction fetch numbers of the plurality of thread warps include a historical instruction fetch number that reaches a count upper limit, setting the priority of the thread warp corresponding to the historical instruction fetch number that reaches the count upper limit as a blocking priority; The warp corresponding to the blocking priority is excluded from the plurality of warps, and at least one warp permitted by arbitration is determined from the remaining warps according to respective priorities of the remaining warps.

3. The method according to claim 1, characterized in that Also includes: If the record mark in the record area moves to the highest bit in the record area, the number of historical instruction fetches of the thread warp corresponding to the record area reaches the upper limit of the count.

4. The method according to claim 1, characterized in that Also includes: When none of the recording marks in the plurality of recording areas is located at the lowest bit, the recording marks in the plurality of recording areas are respectively moved one bit toward the lowest bit.

5. The method according to any one of claims 1 to 4, characterized in that: The executing the instruction fetch request of the at least one thread warp, obtaining a corresponding target instruction from an instruction cache or an upper-level cache, includes: When there are multiple instruction fetch requests with the same instruction address among the instruction fetch requests of the at least one thread warp, merging the multiple instruction fetch requests into a target request; The target request is executed, and a corresponding target instruction is obtained from an instruction cache or an upper-level cache.

6. The method according to any one of claims 1 to 4, characterized in that: The executing the instruction fetch request of the at least one thread warp, obtaining a corresponding target instruction from an instruction cache or an upper-level cache, includes: For the instruction fetch request of the at least one thread warp, the following operations are performed respectively: Execute an instruction fetch request of a thread warp and obtain the corresponding target instruction from the instruction cache or the upper cache.

7. A computer device comprising a memory, a processor chip, and a computer program stored in the memory and executable on the processor chip, characterized in that: When the processor chip executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium, characterized in that: It stores a computer program executable by a computer device. When the computer program is run on the computer device, the computer device executes the steps of the method according to any one of claims 1 to 6.

9. A computer program product, characterized in that The computer program product comprises a computer program stored on a computer-readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer device, the computer device is caused to execute the steps of the method according to any one of claims 1 to 6.