Request processing method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the invention provides a request processing method and device, electronic equipment and a readable storage medium. The method comprises the steps that in the current program execution process, the memory access hit times of all memory segments in a target cache are obtained; for any memory segment, under the condition that the memory access hit frequency of the memory segment is not higher than a preset frequency threshold value, updating a record item of the memory segment into a first record value in a preset filtering record table; the filtering record table comprises a plurality of record items, and one record item corresponds to one memory segment. And receiving a memory access request, and obtaining a record item corresponding to a memory access address of the memory access request in the filtering record table as a to-be-filtered record item. And under the condition that the to-be-filtered record item is the first record value, sending the memory access request to a next-level memory of the target cache. And the cache utilization efficiency is optimized.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a request processing method, device, electronic device and readable storage medium. Background Art

[0002] With the development of computer technology, more and more computer systems have a growing demand for on-chip cache capacity. However, due to the limitations of power consumption and area, it is impossible to design a cache capacity that is too large. Therefore, it is necessary to use the limited cache capacity to improve system performance and improve cache utilization efficiency.

[0003] In related technologies, performance is often improved by data prefetching, that is, preloading data that may be used by the program in the future into the cache. However, this method is highly dependent on the accuracy of the prefetching algorithm, so there may be a situation where the prefetched data is not actually accessed, which further causes cache resource tension and reduces cache utilization efficiency. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a request processing method, device, electronic device and readable storage medium to solve the problem of how to improve cache utilization efficiency. The specific technical solution is as follows: In a first aspect of the present invention, a request processing method is provided, the method comprising: During the current program execution, the number of memory access hits of each memory segment in the target cache is obtained; For any of the memory segments, when the number of memory access hits of the memory segment is not higher than a preset number threshold, updating the record item of the memory segment in a preset filtering record table to a first record value; the filtering record table includes a plurality of record items, one record item corresponding to one memory segment; Receiving a memory access request, and obtaining a record item corresponding to a memory access address of the memory access request in the filtering record table as a record item to be filtered; When the record item to be filtered is the first record value, the memory access request is sent to a next-level storage of the target cache.

[0005] In a second aspect of the present invention, a request processing device is provided, the device comprising: A first acquisition module is used to acquire the number of memory access hits of each memory segment in the target cache during the execution of the current program; A first updating module is used for updating, for any of the memory segments, a record item of the memory segment to a first record value in a preset filtering record table when the number of memory access hits of the memory segment is not higher than a preset number threshold; the filtering record table includes a plurality of record items, one record item corresponding to one memory segment; A second acquisition module, configured to receive a memory access request, and acquire a record item corresponding to a memory access address of the memory access request in the filtering record table as a record item to be filtered; The first sending module is used to send the memory access request to the next level storage of the target cache when the record item to be filtered is the first record value.

[0006] In a third aspect of the present invention, there is also provided an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; The processor is used to implement the method described in the first aspect when executing the program stored in the memory.

[0007] In a fourth aspect of the implementation of the present invention, a computer-readable storage medium is further provided, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method described in the first aspect above.

[0008] In a fifth aspect of the implementation of the present invention, there is also provided a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method described in the first aspect above.

[0009] The request processing method provided by the embodiment of the present invention obtains the number of memory access hits of each memory segment in the target cache during the current program execution process. For any of the memory segments, when the number of memory access hits of the memory segment is not higher than the preset number threshold, the record item of the memory segment is updated to a first record value in a preset filtering record table; the filtering record table contains multiple record items, one record item corresponds to one memory segment. A memory access request is received, and the record item corresponding to the memory access address of the memory access request in the filtering record table is obtained as a record item to be filtered. When the record item to be filtered is the first record value, the memory access request is sent to the next level storage of the target cache. In this way, the embodiment of the present invention can filter the memory access request based on the actual utilization of each memory segment by setting the filtering record table and setting the value of the record item based on the number of memory access hits of the memory segment, bypassing part of the memory access to the next level storage without passing through the target cache, greatly alleviating the tension of cache resources and improving cache utilization efficiency. Moreover, through the filtering record table, the filtering strategy can be adjusted according to the actual memory access situation, which can be applied to the memory access requirements of various different applications. At the same time, the cache optimization management method provided by the embodiment of the present invention does not need to modify the cache architecture design itself, and only needs to be implemented through filtering the record table, so the implementation cost is relatively low. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0011] Figure 1 A flowchart of a request processing method according to an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a request processing system in an embodiment of the present invention; Figure 3 A schematic diagram of a workflow of a request processing system in an embodiment of the present invention; Figure 4 is a schematic diagram of the structure of a request processing device in an embodiment of the present invention; Figure 5 Schematic diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] The terms "first", "second", etc. in the specification and claims of the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable when appropriate, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, the term "and / or" in the specification and claims is used to describe the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In the embodiments of the present invention, the term "multiple" refers to two or more, and other quantifiers are similar.

[0014] Figure 1 FIG. 1 is a flowchart of a request processing method according to an embodiment of the present invention. Figure 1 As shown, the method includes: Step 101: During the execution of the current program, the number of memory access hits of each memory segment in the target cache is obtained.

[0015] Step 102: for any of the memory segments, when the number of memory access hits of the memory segment is not higher than a preset number threshold, update the record item of the memory segment in a preset filtering record table to a first record value; the filtering record table includes multiple record items, and one record item corresponds to one memory segment.

[0016] Step 103: Receive a memory access request, and obtain a record item corresponding to the memory access address of the memory access request in the filtering record table as a record item to be filtered.

[0017] Step 104: When the record item to be filtered is the first record value, send the memory access request to the next level storage of the target cache.

[0018] With respect to the above steps 101 to 104, the embodiment of the present invention may be applied to any computer system, for example, a computer system that may include multi-scenario applications, such as a system including graph processing applications, database applications, neural network training applications, etc., and the embodiment of the present invention does not impose any limitation on this.

[0019] Among them, cache refers to a temporary storage located between the processor and the memory, which serves as an intermediate layer to facilitate data exchange between the memory and the processor (Central Processing Unit, CPU), that is, the processor accesses the memory through the cache. The above-mentioned target cache can be any level of cache in the system. It should be noted that the on-chip cache is usually multi-level, for example, there is generally a level 1 cache (L1 cache), a level 2 cache (L2 cache), a level 3 cache (L3 cache), and a level 4 cache (L4 cache) in some special scenarios, and there is often no next level of cache after the L4 / L3 cache, so L4 or L3 is usually also called the last level cache (Last Level Cache), and the storage of the next level of the Last Level Cache is often a memory controller. Based on this, the target cache in the embodiment of the present invention can be L1 cache, or L2 cache, L3 cache or L4 cache, and can be set according to actual needs, and the embodiment of the present invention does not limit this.

[0020] Among them, the above-mentioned current program refers to the application currently executed by the processor. Accordingly, the above-mentioned memory access request can be sent by the processor or the application for reading data or writing data. The above-mentioned memory segment refers to the memory space corresponding to the specified address range. A memory segment in the embodiment of the present invention can correspond to a memory page, or it can also be obtained by randomly dividing the memory space, or it can also be obtained by dividing according to the actual access range of the current program. It can be set according to actual needs, and the embodiment of the present invention does not limit this.

[0021] The memory access address may be carried in a memory access request, which is used to instruct writing data into or reading data from the memory access address. The memory access address may be obtained through a specified field in the memory access request.

[0022] Specifically, in order to improve memory access efficiency, part of the memory data is usually stored in the cache. On this basis, the cache unit in the cache is mapped to part of the memory address, that is, the data of part of the memory address exists in the cache at the same time. The above-mentioned cache unit refers to a storage unit in the cache, which can be a cache line or a cache block, or a cache area corresponding to one or more memory pages, which can be set according to actual needs, and the embodiment of the present invention does not limit this. On this basis, the above-mentioned memory segment can be a memory space corresponding to a cache unit, and of course it can also be a memory space corresponding to multiple cache units, and the embodiment of the present invention does not limit this.

[0023] A memory access hit refers to a memory access address hitting a cache unit in the cache. In the case of a memory access hit, the corresponding data can be accessed through the cache. Correspondingly, in the case of a memory access miss, the corresponding data often needs to be accessed through the next level of storage.

[0024] Accordingly, the above-mentioned memory access hit count refers to the number of times the accessed data happens to exist in the target cache when executing a memory access request. On this basis, the more times a memory address hits in the cache, the higher the utilization rate of the corresponding cache item, so that the cache can be used more effectively to improve memory access efficiency. Correspondingly, the fewer times a memory address hits in the cache, the lower the utilization rate of the corresponding cache item, resulting in a waste of cache resources.

[0025] Specifically, the above-mentioned number of memory access hits can be implemented by sampling and evaluation. Specifically, three counters can be configured for each memory segment, and the default value is 0. Once a cache item is detected to be valid (valid=1), a counter of the memory segment that matches the cache item tag (tag) is increased by 1, and then each time the cache item hits, the counter is increased by 1. In this case, one counter corresponds to one cache item, and each memory segment can record a maximum of three cache item hits in the filter record table. The embodiment of the present invention can further set a number threshold. If two or more of the three counters corresponding to a memory segment exceed the number threshold, it can be determined that the cache utilization of the memory segment is high.

[0026] Specifically, in the embodiment of the present invention, the number of memory access hits of each memory segment in the target cache can be obtained when the preset sampling conditions are met. The preset sampling conditions can be set by yourself, for example, the preset sampling conditions can be set to a fixed time after the last update of the filter record table. The fixed time can be set by hardware or configured by software. For example, a counter can be set by hardware, and the counter is set to 0 after the filter record table is updated. The counter is increased by 1 every clock cycle. When the counter value reaches the set value, it can be determined that the fixed time has been reached, and the preset sampling condition is met. The set value can be set by hardware logic or configured by software from a specific interface to a specified register, and the embodiment of the present invention does not limit this. Alternatively, the preset sampling condition can also be that the number of memory access hits of a certain memory segment fluctuates greatly, which can be set by yourself according to actual needs, and the embodiment of the present invention does not limit this.

[0027] The filtering record table may be pre-configured, and the filtering record table may be in the form of a bitmap. The form of the filtering record table may be set according to actual needs, and the embodiment of the present invention does not limit this. Specifically, there are multiple record items in the filtering record table, and one record item may correspond to one memory segment.

[0028] Among them, the initial value of the record item corresponding to each memory segment in the above-mentioned filter record table can be set according to actual needs. For example, the filter record table can be set to filter the memory access of a certain application, or it can be set to filter the memory access of other applications except a certain application. It can be set according to actual needs. Alternatively, a bitmap can be used to record one bit of filtering information for each page in the memory access address range, so that the record item corresponding to the page to be filtered can be set to 1, indicating that the cache utilization rate of the page is low, that is, the memory access request of the address range corresponding to the page is filtered, and the record item corresponding to the page that does not need to be filtered is set to 0, that is, the memory access request of the address range corresponding to the page is not filtered.

[0029] In one case, you can divide several memory access address boundaries and filter the memory access requests of some address ranges. For example, if an application has a high utilization rate of memory access and the address ranges corresponding to the application are all mapped to the target cache, then the record items of the memory segments corresponding to the address range of the application in the target record items can be set to reserved values ​​(non-filtered values). In another case, if an application has strong temporal locality, such as a loop in the program, then the record items of the memory segments corresponding to the address range accessed by the loop can be set to reserved values ​​(non-filtered values), indicating that no filtering is required.

[0030] Specifically, after the preset sampling conditions are met, the number of memory access hits of each memory segment after the last update of the filter record table can be obtained. At this time, the memory access request can be stopped, so that the filter record table can be updated again before receiving a new memory access request. Specifically, the filter record table can be updated in a variety of ways, for example: coarse-grained, such as at intervals of a certain time period, and the overall update is based on the obtained number of memory access hits of each memory segment. Alternatively, it can also be fine-grained, such as updating immediately after obtaining the number of memory access hits of some memory segments.

[0031] Optionally, the embodiment of the present invention may further include: S21. When the number of memory access hits of the memory segment is higher than the preset number threshold, update the record item of the memory segment to a second record value in the filter record table.

[0032] Specifically, the record item of the memory segment can be updated to the first record value or the second record value according to whether the number of memory access hits is higher than the preset number threshold. When the number of memory access hits of the memory segment is not higher than the preset number threshold, it indicates that the utilization rate of the corresponding cache item is low. Correspondingly, when the number of memory access hits of the cache unit is higher than the preset number threshold, it indicates that the utilization rate of the corresponding cache item is high.

[0033] The first record value and the second record value are different values ​​and can be set by yourself. For example, the first record value can be 1 and the second record value can be 0. Of course, the first record value can also be 0 and the second record value can be 1, which is not limited in the embodiment of the present invention. Therefore, the embodiment of the present invention can quickly determine the utilization rate of the corresponding cache item of the access address in the target cache according to the value of the record item in the filter record table.

[0034] Specifically, the tag of the cache line contained in the memory segment corresponding to the record item can be compared with the tag in the memory access address. If they are equal, the memory segment and the memory access address are in a mapping relationship. The corresponding record item can be used as the record item to be filtered.

[0035] The next level storage refers to the lower level memory adjacent to the target cache. For example, if the target cache is L2 cache, the next level storage is L3 cache. If the target cache is Last Level Cache, the next level storage is a memory controller, and so on. Specifically, when the record item to be filtered is the first record value, it indicates that the utilization rate of the cache item corresponding to the memory access address is low. In order to improve the resource utilization rate of the target cache, the memory access request can be sent to the next level storage of the target cache for memory access.

[0036] Optionally, the embodiment of the present invention may further include: Step 104: When the record item to be filtered is the second record value, send the memory access request to the target cache.

[0037] Specifically, when the record item to be filtered is the second record value mentioned above, it indicates that the cache item corresponding to the memory access address has a high utilization rate of the target cache. At this time, in order to improve the memory access efficiency of the memory access address, the memory access request can be sent to the target cache for memory access without request filtering.

[0038] In summary, the request processing method provided by the embodiment of the present invention obtains the number of memory access hits of each memory segment in the target cache during the current program execution process. For any of the memory segments, when the number of memory access hits of the memory segment is not higher than the preset number threshold, the record item of the memory segment is updated to the first record value in the preset filtering record table; the filtering record table contains multiple record items, and one record item corresponds to one memory segment. Receive a memory access request, and obtain the record item corresponding to the memory access address of the memory access request in the filtering record table as the record item to be filtered. When the record item to be filtered is the first record value, send the memory access request to the next level storage of the target cache. In this way, the embodiment of the present invention can filter the memory access request based on the actual utilization of each memory segment by setting the filtering record table and setting the value of the record item based on the number of memory access hits of the memory segment, bypass part of the memory access to the next level storage without passing through the target cache, greatly alleviate the tension of cache resources, and improve cache utilization efficiency. And through the filtering record table, the filtering strategy can be adjusted according to the actual memory access situation, which can be applied to the memory access requirements of various different applications. At the same time, the cache optimization management method provided by the embodiment of the present invention does not need to modify the cache architecture design itself, and only needs to be implemented through filtering the record table, so the implementation cost is relatively low.

[0039] At the same time, the embodiment of the present invention can collect the number of memory access hits of each memory segment, and can dynamically update the record items in the filter record table according to the number of memory access hits, thereby improving the flexibility of cache optimization and the diversity of scenarios.

[0040] Optionally, any of the memory segments includes at least two memory units, and the operation of obtaining the number of memory access hits of each memory segment in the target cache may specifically include: S31. Obtain the number of memory access hits of each memory unit included in each memory segment in the target cache.

[0041] The embodiment of the present invention may further include: S32. For any of the memory segments, select at least one sampling memory unit from the memory units included in the memory segment.

[0042] S33: When the proportion of the number of memory units in the memory segment whose memory access hit times are higher than the preset times threshold is not greater than the preset proportion threshold, determine that the memory access hit times of the memory segment is not higher than the preset times threshold.

[0043] The above-mentioned memory unit may be an address range corresponding to a cache line. Accordingly, a memory segment may include at least two memory units. On this basis, a memory segment may correspond to two or more cache lines.

[0044] Specifically, in order to further simplify the hardware logic of request processing, the embodiment of the present invention can set fewer record items in the filter record table. Specifically, a memory segment can include at least two memory units, and accordingly, a method similar to sampling evaluation (Set Dueling) can be used to determine whether the number of memory access hits of the memory segment in the target cache is not higher than a preset number threshold.

[0045] Specifically, in some cases, a memory segment contains a large number of memory cells. If the number of memory access hits of all memory cells is obtained, the data processing amount is large and the processing efficiency is low. In an embodiment of the present invention, at least one sampling memory cell can be selected from the memory cells contained in the memory segment. Specifically, if the memory segment contains M memory cells, N memory cells can be selected as sampling memory cells. Among them, N is less than M, and M and N are both positive integers. Exemplarily, when the number of memory cells contained in the memory segment is greater than 3, 3 memory cells can be selected as sampling memory cells. Of course, other values ​​can also be set according to the number of memory cells contained in the memory segment, and the embodiment of the present invention does not limit this. The number of memory access hits of each sampling memory cell contained in each memory segment in the target cache can be obtained respectively. Accordingly, for any memory segment, if the proportion of the number of sampling memory cells in the memory segment whose number of memory access hits is higher than the preset number threshold is not greater than the preset proportion threshold, it can be determined that the number of memory access hits of the memory segment is not higher than the preset number threshold. The above-mentioned number ratio refers to the ratio of the number of sampled memory units in which the number of memory access hits in the sampled memory units is higher than the preset number threshold to the total number of sampled memory units. The above-mentioned preset ratio threshold can be set by oneself, and can be set to 50%, 60%, 75%, etc., which is not limited in the embodiment of the present invention. It can be understood that in order to improve the cache utilization efficiency, the above-mentioned preset ratio threshold is usually not less than 50%.

[0046] Correspondingly, when the proportion of the number of memory cells in the memory segment whose memory access hit times are higher than the preset times threshold is greater than the preset proportion threshold, it is determined that the memory access hit times of the memory segment are higher than the preset times threshold.

[0047] Furthermore, the embodiment of the present invention can determine whether the number of memory access hits of the memory segment is not higher than the preset number threshold through a preset ratio threshold by means of sampling evaluation, which can reduce the number of entries in the filter record table to a certain extent, thereby further simplifying the logical flow of request processing and improving request processing efficiency.

[0048] Optionally, before the above operation of updating the record item of the memory segment to the first record value in the preset filter record table, the embodiment of the present invention may further include: S41: Clear inconsistent state data in the target cache.

[0049] Among them, the above-mentioned inconsistent state data refers to the cache item data in the target cache that is inconsistent with the next level storage. Specifically, before updating the filter record table, the inconsistent state data in the target cache needs to be cleared first. Specifically, since there may be access requests that were previously sent to the cache and were bypassed to the next level storage after updating the filter record table, it is necessary to write the cache data corresponding to these requests in the target cache to the next level storage. That is: if after the last update of the filter record table, there is a record item of the memory segment corresponding to a cache line updated from the second record value to the first record value, then the memory access request of the memory address corresponding to the cache line is updated from unfiltered to filtered. In this case, in order to ensure cache consistency and avoid inaccurate subsequent memory access data, the inconsistent state data in the target cache can be cleared, that is: the current data stored in the cache line is written to the next level storage, and at the same time, the valid bit of the cache line is updated to invalid, so that the accuracy of subsequent memory access can be guaranteed.

[0050] The valid bit is the valid bit of the cache block, which is used to indicate whether the data of the cache block is valid. The valid bit of the cache block can be set to 0 to update the valid bit to invalid. In this way, the cache block can be released to recycle cache resources. Furthermore, the released cache block can be used for read and write access to memory segments with high memory access frequency according to actual needs.

[0051] In the embodiment of the present invention, the inconsistent state data in the target cache is cleared before the record item of the memory segment is updated to the first record value in the preset filter record table. In this way, cache consistency can be guaranteed after the filter strategy of some memory segments is updated, and the accuracy of the data accessed can be further guaranteed.

[0052] Optionally, the above operation of clearing inconsistent data in the target cache may specifically include: S51. For any cache block in the target cache, when the next-level data is inconsistent with the current data stored in the cache block, the current data stored in the cache block is written to the storage address corresponding to the cache block in the next-level storage, and the valid bit of the cache block is updated to invalid; the next-level data is the data stored at the storage address corresponding to the cache block in the next-level storage.

[0053] Among them, the above-mentioned next-level data refers to the data of the next-level storage corresponding to the cache block. When the next-level data is inconsistent with the current data stored in the cache block, it indicates that there is a cache inconsistency in the target cache. The current data stored in the cache block may have been written before and has not yet been synchronized to the next-level storage. At this time, the embodiment of the present invention can execute the above-mentioned step S51 to ensure cache consistency.

[0054] Specifically, the dirty bit of the cache block or the page rewrite flag (dirty bit) can be used to determine whether the next-level data is consistent with the current data stored in the cache block. The dirty bit is used to indicate whether the data in the cache block is consistent with the next-level storage. Dirty bit = 0 means non-dirty, indicating that it is consistent with the next-level storage, and dirty bit = 1 means dirty, indicating that it is inconsistent with the next-level storage.

[0055] In the embodiment of the present invention, when the next-level data is inconsistent with the current data stored in the cache block, the current data stored in the cache block is written to the storage address corresponding to the cache block in the next-level storage, and the valid bit of the cache block is updated to invalid; the next-level data is the data stored at the storage address corresponding to the cache block in the next-level storage. In this way, the operation of writing data to the next-level storage can be performed only when there is an inconsistency, avoiding unnecessary invalid writing, and reducing the amount of data processing to a certain extent.

[0056] It should be noted that in recent years, with the development of data centers and big data applications and the increase in the number of cores in mainstream computer systems, the system's demand for on-chip cache capacity has gradually increased in chip architecture design. At the same time, traditional cache implementations are limited by power consumption and area, and the cache capacity cannot be designed to be too large. In order to make full use of limited cache capacity to improve system performance, related technologies usually use cache prefetching, cache locking, cache partitioning, etc. And the above solutions usually rely on the cooperation of software and hardware.

[0057] Among them, cache prefetching loads data that the program may use in the future into the cache. In data-intensive applications, a reasonable prefetching strategy can significantly reduce memory access latency and improve program performance. However, if the cache prefetching algorithm is not designed reasonably and the prefetched data is not actually accessed, this part of the prefetched data will occupy limited cache space, further causing cache resource tension. In addition, each prefetching strategy often requires independent hardware logic implementation, making it difficult to support multiple solutions at the same time to adapt to the memory access requirements of applications in multiple scenarios.

[0058] Cache locks lock the data that may be frequently used by the program in the cache, ensuring that hot data can be accessed as quickly as possible. However, cache locks require modifications to the cache architecture design, and similar to cache prefetching, there is a problem with the accuracy of hot data prediction. If data that is locked in the cache for a long time is not accessed again, it will cause a waste of cache resources.

[0059] Cache partitioning divides the cache into multiple areas, and accesses with certain memory access characteristics are assigned to specified areas to accelerate applications with specific memory access characteristics. However, cache partitioning often requires modifications to the cache architecture design, and the acceleration portability for specific applications is poor, making it difficult to adapt to the memory access requirements of applications in multiple scenarios.

[0060] In the embodiment of the present invention, some memory access requests can be bypassed through a filtering mechanism, and some memory access requests are sent directly to the next level storage without passing through the target cache, thereby optimizing cache utilization efficiency.

[0061] Optionally, Figure 2 FIG. 1 is a schematic diagram of a structure of a request processing system in an embodiment of the present invention. Figure 2 As shown, it may include a diversion module, a filtering strategy table, a feedback information processing module, and a consistency management module, and its external interface may be used to connect to a cache and a next-level storage. Figure 2 The cache in refers to the target cache, which can be any level of cache in the memory system. The filtering strategy table corresponds to the above filtering record table.

[0062] The diversion module searches for the corresponding entry in the filtering strategy table according to the address of each memory access request. If the content of the entry indicates that the memory access request of the current address needs to be filtered, the diversion module bypasses the current memory access request and sends it directly to the next level of storage without passing through the cache. Otherwise, the diversion module sends the current memory access request to the cache. The diversion module will bypass the access request with low cache efficiency and determine whether to filter the access request by searching the corresponding entry in the filtering strategy table. This can be implemented by a simple first-in-first-out queue (FIFO).

[0063] The filter policy table can be dynamically updated by the feedback information processing module. During the program execution, the feedback information processing module has been collecting information such as cache utilization, such as cache hits. When the utilization of cache items in a certain address range has been low over the past period of time, the corresponding table items in the filter policy table are updated to achieve adaptive adjustment of the filter policy. In addition, after updating the filter policy table, there is a possibility that the access requests previously sent to the cache are bypassed to the next level of storage, so it is necessary to write the cache items corresponding to these requests to the next level of storage. This function is implemented by the consistency management module.

[0064] At the same time, considering that the number of entries in the filtering policy table may be large when the filtering policy table is configured in a fine-grained manner, the feedback information processing module should try to simplify the hardware logic for calculating the cache efficiency of each entry in the filtering policy table. Therefore, sampling evaluation (Set Dueling) and other methods can be used to collect cache utilization efficiency information.

[0065] There are also many ways to implement the consistency management module, and its implementation scheme is related to the update strategy of the filter policy table. For example, if the update strategy of the filter policy table is to check the records of the feedback information processing module at intervals and update the entire table, the method of refreshing the cache by the consistency management module can be similar to the traditional cache flush method. In addition, if the update of the filter policy table only updates some items, it can be considered to refresh only the cache items corresponding to the newly added items that need to be filtered. The specific implementation can use tag matching and other methods to clear some items.

[0066] Optionally, Figure 3 FIG. 1 is a schematic diagram of a workflow of a request processing system in an embodiment of the present invention. Figure 3 As shown, the update of the filter policy table can be started through software configuration or hardware setting. The diversion module is suspended and no new memory access requests are received. Specifically, the suspension can be implemented using a state machine. The state machine jumps to the suspended state, in which the diversion module no longer receives new memory access requests. The hardware logic can pull down the ready signal of the corresponding receive request to achieve suspension. Furthermore, the feedback information processing module collects the cache utilization information of each table item in the filter policy table, and gives the value to be updated for each table item according to the sampling evaluation strategy. Before the filter policy table is updated, the consistency management module clears the cache items of the current cache to ensure consistency. Further, the filter policy table is updated, and the diversion module is woken up to start receiving new memory access requests.

[0067] For example, taking the target cache as the last level cache (LLC), Figure 2 The cache in is LLC, and the next level storage is the memory controller. If the last level cache capacity is 8MB, the memory capacity is 4GB, and the filter policy table records 1 bit of filter information for each page access, assuming the page size is 4KB, the filter policy table can record (4GB / 4KB)*1bit = 1Mb of information.

[0068] First, the default value of the filter policy table is all 0, that is, all memory access requests go through LLC. After a period of memory access, the feedback information processing module records a batch of LLC utilization information. Then, the filter policy table is updated to the value to be updated calculated by the feedback information processing module. At this time, the filter policy table is adaptively configured once, and the suspended diversion module can be awakened. According to the new filter policy table, the memory access request is sent to LLC, or directly bypassed to the memory controller.

[0069] Optionally, the feedback information processing module can also be independent of the software and implemented entirely by hardware logic to adaptively modify the filtering strategy table. In the embodiment of the present invention, it is not necessary to modify the architecture design of the cache itself, but only to collect some information of the cache when the program is running, such as cache hit count.

[0070] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0071] Figure 4 FIG. 1 is a schematic diagram of a structure of a request processing device in an embodiment of the present invention. Figure 4 As shown, the device 20 may include: A first acquisition module 201 is used to acquire the number of memory access hits of each memory segment in the target cache during the current program execution; A first updating module 202 is configured to update, for any of the memory segments, a record item of the memory segment to a first record value in a preset filtering record table when the number of memory access hits of the memory segment is not higher than a preset number threshold; the filtering record table includes a plurality of record items, one record item corresponding to one memory segment; A second acquisition module 203 is used to receive a memory access request and acquire a record item corresponding to the memory access address of the memory access request in the filtering record table as a record item to be filtered; The first sending module 204 is configured to send the memory access request to a next-level storage of the target cache when the record item to be filtered is the first record value.

[0072] Optionally, the device further comprises: The second updating module is used to update the record item of the memory segment to a second record value in the filtering record table when the number of memory access hits of the memory segment is higher than the preset number threshold.

[0073] Optionally, the device further comprises: The second sending module is used to send the memory access request to the target cache when the record item to be filtered is the second record value.

[0074] Optionally, any of the memory segments includes at least two memory units, and the first acquisition module is specifically used to: acquire the number of memory access hits of each memory unit included in each of the memory segments in the target cache; The device also includes: A selection module, configured to select, for any of the memory segments, at least one sampling memory unit from the memory units included in the memory segment; A determination module is used to determine that the memory access hit count of the memory segment is not higher than the preset number threshold when the proportion of the number of sampling memory units in the memory segment whose memory access hit count is higher than the preset number threshold is not greater than the preset proportion threshold.

[0075] Optionally, the device further comprises: The clearing module is used to clear inconsistent state data in the target cache before the first updating module updates the record item of the memory segment to the first record value in the preset filtering record table.

[0076] Optionally, the clearing module is specifically used to: For any cache block in the target cache, when the next-level data is inconsistent with the current data stored in the cache block, the current data stored in the cache block is written to the storage address corresponding to the cache block in the next-level storage, and the valid bit of the cache block is updated to invalid; the next-level data is the data stored at the storage address corresponding to the cache block in the next-level storage.

[0077] In summary, the request processing device provided by the embodiment of the present invention obtains the number of memory access hits of each memory segment in the target cache during the current program execution process. For any of the memory segments, when the number of memory access hits of the memory segment is not higher than the preset number threshold, the record item of the memory segment is updated to the first record value in the preset filtering record table; the filtering record table contains multiple record items, and one record item corresponds to one memory segment. Receive a memory access request, and obtain the record item corresponding to the memory access address of the memory access request in the filtering record table as the record item to be filtered. When the record item to be filtered is the first record value, send the memory access request to the next level storage of the target cache. In this way, the embodiment of the present invention can filter the memory access request based on the actual utilization of each memory segment by setting the filtering record table and setting the value of the record item based on the number of memory access hits of the memory segment, bypass part of the memory access to the next level storage without passing through the target cache, greatly alleviate the tension of cache resources, and improve cache utilization efficiency. And through the filtering record table, the filtering strategy can be adjusted according to the actual memory access situation, which can be applied to the memory access requirements of various different applications. At the same time, the cache optimization management method provided by the embodiment of the present invention does not need to modify the cache architecture design itself, and only needs to be implemented through filtering the record table, so the implementation cost is relatively low.

[0078] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0079] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0080] Regarding the request processing device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0081] An embodiment of the present invention further provides an electronic device, comprising: a processor and a memory for storing instructions executable by the processor, wherein the processor is configured to execute the above-mentioned request processing method.

[0082] Reference Figure 5 , is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 5 As shown, the electronic device includes: a processor, a memory, a communication interface and a communication bus, and the processor, the memory and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the request processing method of the aforementioned embodiment.

[0083] It should be noted that the electronic devices in the embodiments of the present application include mobile electronic devices and non-mobile electronic devices.

[0084] The processor may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmble Gate Array) or other editable devices, transistor logic devices, hardware components or any combination thereof. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0085] The communication bus may include a path for transmitting information between the memory and the communication interface. The communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 The fact that only one line is used in the diagram does not mean that there is only one bus or only one type of bus.

[0086] The memory may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable ReadOnly), a CD-ROM (Compact Disa Read Only), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0087] The embodiment of the present invention also provides a non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device (server or terminal), the processor can execute Figure 1 The request processing method shown.

[0088] The present invention also provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute Figure 1 The request processing method shown.

[0089] An embodiment of the present application also provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned request processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0090] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0091] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0092] It should be understood by those skilled in the art that the embodiments of the present invention can be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0093] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0094] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a predictable manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0095] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0096] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0097] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0098] It should be noted that the various data-related processes in the embodiments of the present application are all carried out in compliance with the corresponding data protection laws and policies of the country where the device is located, and with the authorization given by the owner of the corresponding device.

[0099] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0100] The request processing method, device, electronic device and readable storage medium provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A request processing method, characterized in that: The method comprises: During the current program execution, the number of memory access hits of each memory segment in the target cache is obtained; For any of the memory segments, when the number of memory access hits of the memory segment is not higher than a preset number threshold, updating the record item of the memory segment in a preset filtering record table to a first record value; the filtering record table includes a plurality of record items, one record item corresponding to one memory segment; Receiving a memory access request, and obtaining a record item corresponding to a memory access address of the memory access request in the filtering record table as a record item to be filtered; When the record item to be filtered is the first record value, the memory access request is sent to a next-level storage of the target cache.

2. The method according to claim 1, characterized in that The method further comprises: When the number of memory access hits of the memory segment is higher than the preset number threshold, the record item of the memory segment in the filter record table is updated to a second record value.

3. The method according to claim 2, characterized in that The method further comprises: When the record item to be filtered is the second record value, the memory access request is sent to the target cache.

4. The method according to claim 1, characterized in that: Any of the memory segments includes at least two memory units, and obtaining the number of memory access hits of each memory segment in the target cache includes: Obtaining the number of memory access hits of each memory unit included in each memory segment in the target cache; The method further comprises: For any of the memory segments, selecting at least one sampling memory unit from the memory units included in the memory segment; When the proportion of the number of sampled memory units in the memory segment whose memory access hit times are higher than the preset times threshold is not greater than the preset proportion threshold, it is determined that the memory access hit times of the memory segment is not higher than the preset times threshold.

5. The method according to claim 1, characterized in that Before updating the record item of the memory segment to the first record value in the preset filter record table, the method further includes: The inconsistent state data in the target cache is cleared.

6. The method according to claim 5, characterized in that The clearing of inconsistent data in the target cache includes: For any cache block in the target cache, when the next-level data is inconsistent with the current data stored in the cache block, the current data stored in the cache block is written to the storage address corresponding to the cache block in the next-level storage, and the valid bit of the cache block is updated to invalid; the next-level data is the data stored at the storage address corresponding to the cache block in the next-level storage.

7. A request processing device, characterized in that: The device comprises: A first acquisition module is used to acquire the number of memory access hits of each memory segment in the target cache during the execution of the current program; A first updating module is used for updating, for any of the memory segments, a record item of the memory segment to a first record value in a preset filtering record table when the number of memory access hits of the memory segment is not higher than a preset number threshold; the filtering record table includes a plurality of record items, one record item corresponding to one memory segment; A second acquisition module, configured to receive a memory access request, and acquire a record item corresponding to a memory access address of the memory access request in the filtering record table as a record item to be filtered; The first sending module is used to send the memory access request to the next level storage of the target cache when the record item to be filtered is the first record value.

8. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-6 when executing a program stored in a memory.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program, characterized in that When the computer program is executed by a computer, the method according to any one of claims 1 to 6 is implemented.

Images