Cache mirror image cleaning method and device, electronic equipment and storage medium

Abstract

The application provides a cache image cleaning method, which comprises the following steps: traversing all computing nodes in an artificial intelligence platform and determining the node types of the computing nodes within a specified time set by a user; determining a cleaning threshold range of cache spaces in the computing nodes according to the node types and a node threshold relationship table set by the user; obtaining the space utilization of the cache spaces and determining the cache image types of all cache images in the cache spaces; and cleaning the cache images according to the cleaning threshold range, the space utilization and the cache image types. By determining different types of computing nodes and corresponding cleaning threshold ranges and different types of cache images, the cache images in the cache spaces of different types of computing nodes can be cleaned according to different cleaning threshold ranges, and the reasonable space utilization of the cache spaces can be ensured without affecting the deep learning of the AI platform.

Description

Technical Field

[0001] This invention relates to the field of image cleanup, and in particular to a method, apparatus, electronic device, and storage medium for cleaning up cached images. Background Technology

[0002] Artificial intelligence products support caching images to compute nodes to improve training efficiency. However, as the number of cached images on compute nodes increases, not all cached data is frequently used, thus occupying a certain amount of cache space and causing storage resource shortages. In some cases, training tasks may even fail due to insufficient cache space. Currently, some artificial intelligence platforms support manually deleting cached images from nodes. However, users do not constantly monitor the available cache space during platform use. Therefore, they may only manually clear the cache space and re-execute training after the training task fails and an error occurs. This not only wastes time but also results in a poor user experience. Summary of the Invention

[0003] Therefore, it is necessary to provide a cache image cleaning method, apparatus, electronic device, and storage medium that can automatically clean up infrequently used cache images in the cache space to ensure the success of training tasks, in order to address the above-mentioned technical problems.

[0004] Firstly, a cache image cleanup method is provided, the method comprising:

[0005] Within a specified time frame set by the user, traverse all computing nodes in the artificial intelligence platform and determine the node type of each computing node;

[0006] The cache space cleanup threshold range in the computing node is determined based on the node type and the node threshold relationship table set by the user.

[0007] Obtain the space utilization rate of the cache space and determine the cache image type of all cache images in the cache space;

[0008] The cache image is cleaned up according to the cleanup threshold range, the space utilization rate, and the cache image type.

[0009] In one embodiment, traversing all computing nodes in the AI ​​platform and determining the node type of the computing nodes includes:

[0010] Record the number of times the computing nodes are accessed during artificial intelligence learning;

[0011] When the number of accesses does not exceed the first access threshold set by the user, the node type is determined to be a low-level node;

[0012] When the number of accesses does not exceed the second access threshold set by the user, the node type is determined to be a mid-level node;

[0013] When the number of accesses exceeds the second access threshold, the node type is determined to be an advanced node.

[0014] In one embodiment, obtaining the space utilization rate of the cache space and determining the cache image type of all cache images in the cache space includes:

[0015] Record the number of times the cached image is retrieved and the most recent retrieval time during artificial intelligence learning;

[0016] Determine whether the number of times the request is made is higher than the threshold number of times the user sets.

[0017] If so, then the cache image type of the cache image is determined to be an advanced cache image;

[0018] If not, the cache image type is determined based on the most recent acquisition time and the acquisition time threshold set by the user.

[0019] In one embodiment, determining the cache image type based on the most recent acquisition time and the acquisition time threshold set by the user includes:

[0020] When the most recent acquisition time is not lower than the acquisition time threshold, the cache image type is determined to be a low-level cache image;

[0021] When the most recent acquisition time is lower than the acquisition time threshold, the cache image type is determined to be a medium-level cache image.

[0022] In one embodiment, the cleanup threshold range includes a first cleanup threshold range and a second cleanup threshold range, and the step of cleaning up the cache image according to the cleanup threshold range, the space utilization rate, and the cache image type includes:

[0023] When the space utilization rate falls within the first cleanup threshold range, the user is reminded to clean up the cache image.

[0024] When the space utilization rate falls within the second cleanup threshold range, the cache image is cleaned up according to the second cleanup threshold range, the space utilization rate, and the cache image type.

[0025] In one embodiment, cleaning the cache image according to the second cleanup threshold range, the space utilization rate, and the cache image type includes:

[0026] When the node type is a low-level node, the low-level cache image is cleared;

[0027] If, after cleaning the low-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the intermediate cache image.

[0028] When the node type is a mid-level node, clean up the low-level cache image and the mid-level cache image;

[0029] If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the high-level cache image.

[0030] When the node type is a high-level node, clean up the low-level cache image and the mid-level cache image;

[0031] If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, then the high-level cache image is cleaned.

[0032] In one embodiment, cleaning up the low-level cache image and the mid-level cache image includes:

[0033] Clean up the lowest-level cache image that was most recently acquired;

[0034] After each cleanup of the low-level cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range.

[0035] If not, then stop cleaning up the low-level cache image;

[0036] If so, continue cleaning the low-level cache image, and after cleaning the low-level cache image, clean the intermediate cache image that has been retrieved the fewest times in sequence;

[0037] After each cleanup of the intermediate cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range;

[0038] If not, then stop cleaning the intermediate cache image.

[0039] On the other hand, a cache image cleanup device is provided, the device comprising:

[0040] The node determination module is used to traverse all computing nodes in the artificial intelligence platform and determine the node type of the computing nodes within a specified time set by the user.

[0041] The range determination module is used to determine the range of cache space cleanup thresholds in the computing node based on the node type and the node threshold relationship table set by the user.

[0042] The image determination module is used to obtain the space utilization rate of the cache space and determine the cache image type of all cache images in the cache space;

[0043] The cleanup module is used to clean up the cache image based on the cleanup threshold range, the space utilization rate, and the cache image type.

[0044] In one embodiment, the node determination module traverses all computing nodes in the AI ​​platform and determines the node type of each computing node, including:

[0045] Record the number of times the computing nodes are accessed during artificial intelligence learning;

[0046] When the number of accesses does not exceed the first access threshold set by the user, the node type is determined to be a low-level node;

[0047] When the number of accesses does not exceed the second access threshold set by the user, the node type is determined to be a mid-level node;

[0048] When the number of accesses exceeds the second access threshold, the node type is determined to be an advanced node.

[0049] In one embodiment, the image determination module obtains the space utilization rate of the cache space and determines the cache image type of all cache images in the cache space, including:

[0050] Record the number of times the cached image is retrieved and the most recent retrieval time during artificial intelligence learning;

[0051] Determine whether the number of times the request is made is higher than the threshold number of times the user sets.

[0052] If so, then the cache image type of the cache image is determined to be an advanced cache image;

[0053] If not, the cache image type is determined based on the most recent acquisition time and the acquisition time threshold set by the user.

[0054] In one embodiment, the image determination module determines the cache image type based on the most recent acquisition time and the user-set acquisition time threshold, including:

[0055] When the most recent acquisition time is not lower than the acquisition time threshold, the cache image type is determined to be a low-level cache image;

[0056] When the most recent acquisition time is lower than the acquisition time threshold, the cache image type is determined to be a medium-level cache image.

[0057] In one embodiment, the cleanup threshold range includes a first cleanup threshold range and a second cleanup threshold range. The cleanup module cleans the cache image according to the cleanup threshold range, the space utilization rate, and the cache image type, including:

[0058] When the space utilization rate falls within the first cleanup threshold range, the user is reminded to clean up the cache image.

[0059] When the space utilization rate falls within the second cleanup threshold range, the cache image is cleaned up according to the second cleanup threshold range, the space utilization rate, and the cache image type.

[0060] In one embodiment, the cleanup module cleans the cache image according to the second cleanup threshold range, the space utilization rate, and the cache image type, including:

[0061] When the node type is a low-level node, the low-level cache image is cleared;

[0062] If, after cleaning the low-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the intermediate cache image.

[0063] When the node type is a mid-level node, clean up the low-level cache image and the mid-level cache image;

[0064] If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the high-level cache image.

[0065] When the node type is a high-level node, clean up the low-level cache image and the mid-level cache image;

[0066] If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, then the high-level cache image is cleaned.

[0067] In one embodiment, the cleanup module cleans up the low-level cache image and the mid-level cache image by:

[0068] Clean up the lowest-level cache image that was most recently acquired;

[0069] After each cleanup of the low-level cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range.

[0070] If not, then stop cleaning up the low-level cache image;

[0071] If so, continue cleaning the low-level cache image, and after cleaning the low-level cache image, clean the intermediate cache image that has been retrieved the fewest times in sequence;

[0072] After each cleanup of the intermediate cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range;

[0073] If not, then stop cleaning the intermediate cache image.

[0074] In another aspect, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0075] Within a specified time frame set by the user, traverse all computing nodes in the artificial intelligence platform and determine the node type of each computing node;

[0076] The cache space cleanup threshold range in the computing node is determined based on the node type and the node threshold relationship table set by the user.

[0077] Obtain the space utilization rate of the cache space and determine the cache image type of all cache images in the cache space;

[0078] The cache image is cleaned up according to the cleanup threshold range, the space utilization rate, and the cache image type.

[0079] In one embodiment, the processor performs the following steps when executing the computer program:

[0080] The process of traversing all computing nodes in the AI ​​platform and determining the node type of each computing node includes:

[0081] Record the number of times the computing nodes are accessed during artificial intelligence learning;

[0082] When the number of accesses does not exceed the first access threshold set by the user, the node type is determined to be a low-level node;

[0083] When the number of accesses does not exceed the second access threshold set by the user, the node type is determined to be a mid-level node;

[0084] When the number of accesses exceeds the second access threshold, the node type is determined to be an advanced node.

[0085] In one embodiment, the processor performs the following steps when executing the computer program:

[0086] The step of obtaining the space utilization rate of the cache space and determining the cache image type of all cache images in the cache space includes:

[0087] Record the number of times the cached image is retrieved and the most recent retrieval time during artificial intelligence learning;

[0088] Determine whether the number of times the request is made is higher than the threshold number of times the user sets.

[0089] If so, then the cache image type of the cache image is determined to be an advanced cache image;

[0090] If not, the cache image type is determined based on the most recent acquisition time and the acquisition time threshold set by the user.

[0091] In one embodiment, the processor performs the following steps when executing the computer program:

[0092] Determining the cache image type based on the most recent acquisition time and the acquisition time threshold set by the user includes:

[0093] When the most recent acquisition time is not lower than the acquisition time threshold, the cache image type is determined to be a low-level cache image;

[0094] When the most recent acquisition time is lower than the acquisition time threshold, the cache image type is determined to be a medium-level cache image.

[0095] In one embodiment, the processor performs the following steps when executing the computer program:

[0096] The cleanup threshold range includes a first cleanup threshold range and a second cleanup threshold range. Cleaning the cache image based on the cleanup threshold range, the space utilization rate, and the cache image type includes:

[0097] When the space utilization rate falls within the first cleanup threshold range, the user is reminded to clean up the cache image.

[0098] When the space utilization rate falls within the second cleanup threshold range, the cache image is cleaned up according to the second cleanup threshold range, the space utilization rate, and the cache image type.

[0099] In one embodiment, the processor performs the following steps when executing the computer program:

[0100] The step of cleaning the cache image according to the second cleanup threshold range, the space utilization rate, and the cache image type includes:

[0101] When the node type is a low-level node, the low-level cache image is cleared;

[0102] If, after cleaning the low-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the intermediate cache image.

[0103] When the node type is a mid-level node, clean up the low-level cache image and the mid-level cache image;

[0104] If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the high-level cache image.

[0105] When the node type is a high-level node, clean up the low-level cache image and the mid-level cache image;

[0106] If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, then the high-level cache image is cleaned.

[0107] In one embodiment, the processor performs the following steps when executing the computer program:

[0108] The cleaning of the low-level cache image and the mid-level cache image includes:

[0109] Clean up the lowest-level cache image that was most recently acquired;

[0110] After each cleanup of the low-level cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range.

[0111] If not, then stop cleaning up the low-level cache image;

[0112] If so, continue cleaning the low-level cache image, and after cleaning the low-level cache image, clean the intermediate cache image that has been retrieved the fewest times in sequence;

[0113] After each cleanup of the intermediate cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range;

[0114] If not, then stop cleaning the intermediate cache image.

[0115] In another aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0116] Within a specified time frame set by the user, traverse all computing nodes in the artificial intelligence platform and determine the node type of each computing node;

[0117] The cache space cleanup threshold range in the computing node is determined based on the node type and the node threshold relationship table set by the user.

[0118] Obtain the space utilization rate of the cache space and determine the cache image type of all cache images in the cache space;

[0119] The cache image is cleaned up according to the cleanup threshold range, the space utilization rate, and the cache image type.

[0120] In one embodiment, the computer program performs the following steps when executed by a processor:

[0121] The process of traversing all computing nodes in the AI ​​platform and determining the node type of each computing node includes:

[0122] Record the number of times the computing nodes are accessed during artificial intelligence learning;

[0123] When the number of accesses does not exceed the first access threshold set by the user, the node type is determined to be a low-level node;

[0124] When the number of accesses does not exceed the second access threshold set by the user, the node type is determined to be a mid-level node;

[0125] When the number of accesses exceeds the second access threshold, the node type is determined to be an advanced node.

[0126] In one embodiment, the computer program performs the following steps when executed by a processor:

[0127] The step of obtaining the space utilization rate of the cache space and determining the cache image type of all cache images in the cache space includes:

[0128] Record the number of times the cached image is retrieved and the most recent retrieval time during artificial intelligence learning;

[0129] Determine whether the number of times the request is made is higher than the threshold number of times the user sets.

[0130] If so, then the cache image type of the cache image is determined to be an advanced cache image;

[0131] If not, the cache image type is determined based on the most recent acquisition time and the acquisition time threshold set by the user.

[0132] In one embodiment, the computer program performs the following steps when executed by a processor:

[0133] Determining the cache image type based on the most recent acquisition time and the acquisition time threshold set by the user includes:

[0134] When the most recent acquisition time is not lower than the acquisition time threshold, the cache image type is determined to be a low-level cache image;

[0135] When the most recent acquisition time is lower than the acquisition time threshold, the cache image type is determined to be a medium-level cache image.

[0136] In one embodiment, the computer program performs the following steps when executed by a processor:

[0137] The cleanup threshold range includes a first cleanup threshold range and a second cleanup threshold range. Cleaning the cache image based on the cleanup threshold range, the space utilization rate, and the cache image type includes:

[0138] When the space utilization rate falls within the first cleanup threshold range, the user is reminded to clean up the cache image.

[0139] When the space utilization rate falls within the second cleanup threshold range, the cache image is cleaned up according to the second cleanup threshold range, the space utilization rate, and the cache image type.

[0140] In one embodiment, the computer program performs the following steps when executed by a processor:

[0141] The step of cleaning the cache image according to the second cleanup threshold range, the space utilization rate, and the cache image type includes:

[0142] When the node type is a low-level node, the low-level cache image is cleared;

[0143] If, after cleaning the low-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the intermediate cache image.

[0144] When the node type is a mid-level node, clean up the low-level cache image and the mid-level cache image;

[0145] If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the high-level cache image.

[0146] When the node type is a high-level node, clean up the low-level cache image and the mid-level cache image;

[0147] If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, then the high-level cache image is cleaned.

[0148] In one embodiment, the computer program performs the following steps when executed by a processor:

[0149] The cleaning of the low-level cache image and the mid-level cache image includes:

[0150] Clean up the lowest-level cache image that was most recently acquired;

[0151] After each cleanup of the low-level cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range.

[0152] If not, then stop cleaning up the low-level cache image;

[0153] If so, continue cleaning the low-level cache image, and after cleaning the low-level cache image, clean the intermediate cache image that has been retrieved the fewest times in sequence;

[0154] After each cleanup of the intermediate cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range;

[0155] If not, then stop cleaning the intermediate cache image.

[0156] Within a user-defined timeframe, all computing nodes in the AI ​​platform are traversed, and their node types are determined. A cache cleanup threshold range for each computing node is determined based on its node type and a user-defined node threshold relationship table. The cache space utilization rate is obtained, and the cache image type of all cache images within that cache space is determined. The cache images are then cleaned according to the cleanup threshold range, the space utilization rate, and the cache image type. By identifying different types of computing nodes and cache images, the system automatically cleans up cache images of the same type in the cache spaces of different types of computing nodes when the space utilization rate falls within a second cleanup threshold range, ensuring reasonable cache space utilization without affecting the AI ​​platform's deep learning. Attached Figure Description

[0157] Figure 1 A flowchart illustrating the cache image cleanup method;

[0158] Figure 2 A schematic diagram illustrating the steps of the cache image cleanup method;

[0159] Figure 3 This is a schematic diagram of the cache image cleanup device;

[0160] Figure 4 This is an internal structural diagram of a computer device in an embodiment of the present invention. Detailed Implementation

[0161] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0162] The cache image cleanup method provided in this application can be applied to, for example... Figure 1 The flowchart illustrating the cache image cleanup method shows that different types of compute nodes represent different levels of importance. Higher-level nodes have higher importance, and when the AI ​​platform accesses a compute node, the corresponding cache space requires more free cache space, resulting in lower space utilization.

[0163] In one embodiment, such as Figure 2 As shown, the present invention provides a cache image cleanup method, the method comprising:

[0164] S201. Within the specified time set by the user, traverse all computing nodes in the artificial intelligence platform and determine the node type of the computing nodes;

[0165] S202. Determine the cache space cleanup threshold range in the computing node according to the node type and the node threshold relationship table set by the user;

[0166] S203. Obtain the space utilization rate of the cache space and determine the cache image type of all cache images in the cache space;

[0167] S204. Clean up the cache image according to the cleanup threshold range, the space utilization rate, and the cache image type.

[0168] Specifically, the system first determines the node type of each compute node in the platform, then determines the cache space cleanup threshold range corresponding to the node based on the node type. When the cache space utilization reaches the first cleanup threshold range, the user is prompted to clean up the cache image, and other devices are prohibited from using the cache space before the user cleans up the cache image. When the cache space utilization reaches the second cleanup threshold range, cache images of different cache image types are automatically cleaned up according to different node types.

[0169] In one embodiment, traversing all computing nodes in the AI ​​platform and determining the node type of the computing nodes includes:

[0170] Record the number of times the computing nodes are accessed during artificial intelligence learning;

[0171] When the number of accesses does not exceed the first access threshold set by the user, the node type is determined to be a low-level node;

[0172] When the number of accesses does not exceed the second access threshold set by the user, the node type is determined to be a mid-level node;

[0173] When the number of accesses exceeds the second access threshold, the node type is determined to be an advanced node.

[0174] Specifically, the system records the number of times the AI ​​(Artificial Intelligence) platform accesses each computing node during deep learning. The user-set access threshold can be 10 times, 100 times, or a percentage. For example, if the AI ​​platform accesses all computing nodes a total of 1000 times, and a particular computing node is accessed 10 times, then that computing node's percentage (importance) is 1%. If the user-set first access threshold is 2%, then that computing node is determined to be a low-level node; or if the user-set first access threshold is 5 times and the second access threshold is 100 times, then that computing node is determined to be a mid-level node. When a computing node is accessed by the AI ​​platform 150 times (more than 100 times), then that computing node is determined to be a high-level node; or if the user-set second access threshold is 8%, and the computing node is accessed 100 times (accounting for 10% of all accesses), then that computing node is also determined to be a high-level node.

[0175] In one embodiment, obtaining the space utilization rate of the cache space and determining the cache image type of all cache images in the cache space includes:

[0176] Record the number of times the cached image is retrieved and the most recent retrieval time during artificial intelligence learning;

[0177] Determine whether the number of times the request is made is higher than the threshold number of times the user sets.

[0178] If so, then the cache image type of the cache image is determined to be an advanced cache image;

[0179] If not, the cache image type is determined based on the most recent acquisition time and the acquisition time threshold set by the user.

[0180] Specifically, the system records the number of times and the time when the AI ​​platform accesses the cache space for deep learning to retrieve a cache image. For example, if the AI ​​platform retrieves a certain cache image three times, on January 5, 2022, January 7, 2022, and January 20, 2022, and the user-set threshold for the number of retrievals is 2 (the actual number of retrievals is higher than the threshold), then the cache image is determined to be a high-level cache image. If the user-set threshold for the number of retrievals is 3, then it is necessary to determine whether the cache image is a medium-level or low-level cache image.

[0181] In one embodiment, determining the cache image type based on the most recent acquisition time and the acquisition time threshold set by the user includes:

[0182] When the most recent acquisition time is not lower than the acquisition time threshold, the cache image type is determined to be a low-level cache image;

[0183] When the most recent acquisition time is lower than the acquisition time threshold, the cache image type is determined to be a medium-level cache image.

[0184] Specifically, as mentioned above, since the actual number of retrievals (3 times) is not higher than the retrieval threshold (3 times), the cache image type needs to be further determined based on the most recent retrieval time and the retrieval time threshold. For example, if the AI ​​platform is currently working on January 23, 2022, the most recent retrieval time of the cache image can be calculated as (3 days ago). If it is lower than the user-set retrieval time threshold (5 days), then the cache image is determined to be a medium-level cache image. Furthermore, the cache image type can change. For instance, if the AI ​​platform is working on January 25, 2022, and the most recent access to the cache image is still on January 20, 2022 (the most recent retrieval time is not lower than the retrieval time threshold), then the cache image type changes from a medium-level cache image to a low-level cache image. Of course, the time unit for the most recent retrieval time and the retrieval time threshold can be days, or it can be accurate to hours, minutes, etc.

[0185] In one embodiment, the cleanup threshold range includes a first cleanup threshold range and a second cleanup threshold range, and the step of cleaning up the cache image according to the cleanup threshold range, the space utilization rate, and the cache image type includes:

[0186] When the space utilization rate falls within the first cleanup threshold range, the user is reminded to clean up the cache image.

[0187] When the space utilization rate falls within the second cleanup threshold range, the cache image is cleaned up according to the second cleanup threshold range, the space utilization rate, and the cache image type.

[0188] Specifically, different node types have corresponding first and second cleanup threshold ranges. For example, the first cleanup threshold range for low-level nodes is 60%-79%, and the second cleanup threshold range is 80%-100%. If the space utilization rate of the low-level node is 65% (within the first cleanup threshold range), the user is prompted to clean up the cache image and other devices are prohibited from using the cache space of the low-level node. For example, the first cleanup threshold range for high-level nodes is 50%-60%, and the second cleanup threshold range is 61%-100%. If the space utilization rate of the high-level node is 67%, the cache data needs to be automatically cleaned up, and other devices are prohibited from using the cache space of the high-level node until the cleanup of the cache image is completed.

[0189] In one embodiment, cleaning the cache image according to the second cleanup threshold range, the space utilization rate, and the cache image type includes:

[0190] When the node type is a low-level node, the low-level cache image is cleared;

[0191] If, after cleaning the low-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the intermediate cache image.

[0192] When the node type is a mid-level node, clean up the low-level cache image and the mid-level cache image;

[0193] If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the high-level cache image.

[0194] When the node type is a high-level node, clean up the low-level cache image and the mid-level cache image;

[0195] If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, then the high-level cache image is cleaned.

[0196] Specifically, if the space utilization of low-level nodes is 90%, which falls within the second cleanup threshold range of 80%-100%, automatic cache image cleanup is required. Low-level cache images are cleaned first. Once all low-level cache images have been cleaned, regardless of whether the space utilization falls within the first cleanup threshold range of 60%-79% or the second cleanup threshold range of 80%-100%, the user is prompted to continue cleaning cache images. If the space utilization of mid-level nodes is 75%, which falls within the second cleanup threshold range of 70%-100%, low-level cache images are cleaned first, followed by mid-level cache images. Once all mid-level cache images have been cleaned, regardless of whether the space utilization falls within the first cleanup threshold range of 60%-69% or the second cleanup threshold range of 70%-100%, the user is prompted to continue cleaning cache images. If the space utilization of the high-level node is 67% at this time, which is within the second cleanup threshold range of 61%-100%, the low-level cache image will be cleaned first, followed by the mid-level cache image. If the space utilization is still within the second cleanup threshold range of 61%-100% after all mid-level cache images have been cleaned, the high-level cache image with the fewest accesses will be cleaned in turn.

[0197] In one embodiment, cleaning up the low-level cache image and the mid-level cache image includes:

[0198] Clean up the lowest-level cache image that was most recently acquired;

[0199] After each cleanup of the low-level cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range.

[0200] If not, then stop cleaning up the low-level cache image;

[0201] If so, continue cleaning the low-level cache image, and after cleaning the low-level cache image, clean the intermediate cache image that has been retrieved the fewest times in sequence;

[0202] After each cleanup of the intermediate cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range;

[0203] If not, then stop cleaning the intermediate cache image.

[0204] Specifically, the process of cleaning up low-level cache images is the same regardless of whether it's a low-level node, a mid-level node, or a high-level node. It involves sequentially cleaning up the low-level cache image in the corresponding cache space of the node that was most recently retrieved. For example, if a low-level node has two low-level cache images, A and B, and the AI ​​platform retrieved low-level cache image A 3 days ago and 5 days ago, while only retrieving low-level cache image B 2 days ago, then A will be cleaned first. After cleaning A, it's determined whether the space utilization rate is still within the second cleanup threshold range of 80%-100%. If it is, then B will continue to be cleaned. If the space utilization rate is within the first cleanup threshold range of 60%-79% after cleaning A, then cleanup stops and the user is reminded to manually clean up the cache images. For example, a certain intermediate node's cache space contains two low-level cache images, A and B, and two intermediate cache images, C and D. The AI ​​platform retrieves intermediate cache image C 10 times and intermediate cache image D 6 times. The cleanup order is A → B → D → C. After each low-level or intermediate cache image is cleaned, it is determined whether the space utilization rate is still within the second cleanup threshold range of 70%-100%. If the space utilization rate is within the first cleanup threshold range of 60%-69% after cleaning B, the cleanup of D and C is stopped. If the space utilization rate is still within the second cleanup threshold range of 70%-100% after cleaning C, the user is prompted to manually clean up the high-level cache image.

[0205] The proposed solution has the following beneficial effects:

[0206] 1) By identifying different types of computing nodes and cache images, the corresponding types of cache images can be automatically cleaned up in the cache space corresponding to different computing nodes, ensuring reasonable space utilization of the cache space without affecting the deep learning of the AI ​​platform;

[0207] 2) By setting a first and second cleanup threshold range, the system can remind users to manually clean up cache images when space utilization is low, and automatically clean up cache images according to cleanup rules when space utilization is high.

[0208] 3) When reminding users to manually clear the cache image and automatically clear the cache image, other devices should be prevented from accessing the corresponding cache space to avoid affecting the clearing process.

[0209] It should be understood that, although Figure 2 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are executed, and they can be performed in other orders. Furthermore, Figure 2At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0210] In one embodiment, such as Figure 3 As shown, a cache image cleanup device is provided, the device comprising:

[0211] The node determination module 301 is used to traverse all computing nodes in the artificial intelligence platform and determine the node type of the computing nodes within a specified time set by the user.

[0212] The range determination module 302 is used to determine the range of cache space cleanup thresholds in the computing node based on the node type and the node threshold relationship table set by the user.

[0213] The image determination module 303 is used to obtain the space utilization rate of the cache space and determine the cache image type of all cache images in the cache space;

[0214] The cleaning module 304 is used to clean up the cache image according to the cleaning threshold range, the space utilization rate and the cache image type.

[0215] In one embodiment, the node determination module traverses all computing nodes in the AI ​​platform and determines the node type of each computing node, including:

[0216] Record the number of times the computing nodes are accessed during artificial intelligence learning;

[0217] When the number of accesses does not exceed the first access threshold set by the user, the node type is determined to be a low-level node;

[0218] When the number of accesses does not exceed the second access threshold set by the user, the node type is determined to be a mid-level node;

[0219] When the number of accesses exceeds the second access threshold, the node type is determined to be an advanced node.

[0220] In one embodiment, the image determination module obtains the space utilization rate of the cache space and determines the cache image type of all cache images in the cache space, including:

[0221] Record the number of times the cached image is retrieved and the most recent retrieval time during artificial intelligence learning;

[0222] Determine whether the number of times the request is made is higher than the threshold number of times the user sets.

[0223] If so, then the cache image type of the cache image is determined to be an advanced cache image;

[0224] If not, the cache image type is determined based on the most recent acquisition time and the acquisition time threshold set by the user.

[0225] In one embodiment, the image determination module determines the cache image type based on the most recent acquisition time and the user-set acquisition time threshold, including:

[0226] When the most recent acquisition time is not lower than the acquisition time threshold, the cache image type is determined to be a low-level cache image;

[0227] When the most recent acquisition time is lower than the acquisition time threshold, the cache image type is determined to be a medium-level cache image.

[0228] In one embodiment, the cleanup threshold range includes a first cleanup threshold range and a second cleanup threshold range. The cleanup module cleans the cache image according to the cleanup threshold range, the space utilization rate, and the cache image type, including:

[0229] When the space utilization rate falls within the first cleanup threshold range, the user is reminded to clean up the cache image.

[0230] When the space utilization rate falls within the second cleanup threshold range, the cache image is cleaned up according to the second cleanup threshold range, the space utilization rate, and the cache image type.

[0231] In one embodiment, the cleanup module cleans the cache image according to the second cleanup threshold range, the space utilization rate, and the cache image type, including:

[0232] When the node type is a low-level node, the low-level cache image is cleared;

[0233] If, after cleaning the low-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the intermediate cache image.

[0234] When the node type is a mid-level node, clean up the low-level cache image and the mid-level cache image;

[0235] If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the high-level cache image.

[0236] When the node type is a high-level node, clean up the low-level cache image and the mid-level cache image;

[0237] If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, then the high-level cache image is cleaned.

[0238] In one embodiment, the cleanup module cleans up the low-level cache image and the mid-level cache image by:

[0239] Clean up the lowest-level cache image that was most recently acquired;

[0240] After each cleanup of the low-level cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range.

[0241] If not, then stop cleaning up the low-level cache image;

[0242] If so, continue cleaning the low-level cache image, and after cleaning the low-level cache image, clean the intermediate cache image that has been retrieved the fewest times in sequence;

[0243] After each cleanup of the intermediate cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range;

[0244] If not, then stop cleaning the intermediate cache image.

[0245] Specific limitations regarding the cache image cleanup device can be found in the limitations of the cache image cleanup method described above, and will not be repeated here. Each module in the aforementioned cache image cleanup device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0246] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 4As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements an alarm information processing method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0247] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0248] In one embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0249] Within a specified time frame set by the user, traverse all computing nodes in the artificial intelligence platform and determine the node type of each computing node;

[0250] The cache space cleanup threshold range in the computing node is determined based on the node type and the node threshold relationship table set by the user.

[0251] Obtain the space utilization rate of the cache space and determine the cache image type of all cache images in the cache space;

[0252] The cache image is cleaned up according to the cleanup threshold range, the space utilization rate, and the cache image type.

[0253] In one embodiment, the processor performs the following steps when executing the computer program:

[0254] The process of traversing all computing nodes in the AI ​​platform and determining the node type of each computing node includes:

[0255] Record the number of times the computing nodes are accessed during artificial intelligence learning;

[0256] When the number of accesses does not exceed the first access threshold set by the user, the node type is determined to be a low-level node;

[0257] When the number of accesses does not exceed the second access threshold set by the user, the node type is determined to be a mid-level node;

[0258] When the number of accesses exceeds the second access threshold, the node type is determined to be an advanced node.

[0259] In one embodiment, the processor performs the following steps when executing the computer program:

[0260] The step of obtaining the space utilization rate of the cache space and determining the cache image type of all cache images in the cache space includes:

[0261] Record the number of times the cached image is retrieved and the most recent retrieval time during artificial intelligence learning;

[0262] Determine whether the number of times the request is made is higher than the threshold number of times the user sets.

[0263] If so, then the cache image type of the cache image is determined to be an advanced cache image;

[0264] If not, the cache image type is determined based on the most recent acquisition time and the acquisition time threshold set by the user.

[0265] In one embodiment, the processor performs the following steps when executing the computer program:

[0266] Determining the cache image type based on the most recent acquisition time and the acquisition time threshold set by the user includes:

[0267] When the most recent acquisition time is not lower than the acquisition time threshold, the cache image type is determined to be a low-level cache image;

[0268] When the most recent acquisition time is lower than the acquisition time threshold, the cache image type is determined to be a medium-level cache image.

[0269] In one embodiment, the processor performs the following steps when executing the computer program:

[0270] The cleanup threshold range includes a first cleanup threshold range and a second cleanup threshold range. Cleaning the cache image based on the cleanup threshold range, the space utilization rate, and the cache image type includes:

[0271] When the space utilization rate falls within the first cleanup threshold range, the user is reminded to clean up the cache image.

[0272] When the space utilization rate falls within the second cleanup threshold range, the cache image is cleaned up according to the second cleanup threshold range, the space utilization rate, and the cache image type.

[0273] In one embodiment, the processor performs the following steps when executing the computer program:

[0274] The step of cleaning the cache image according to the second cleanup threshold range, the space utilization rate, and the cache image type includes:

[0275] When the node type is a low-level node, the low-level cache image is cleared;

[0276] If, after cleaning the low-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the intermediate cache image.

[0277] When the node type is a mid-level node, clean up the low-level cache image and the mid-level cache image;

[0278] If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the high-level cache image.

[0279] When the node type is a high-level node, clean up the low-level cache image and the mid-level cache image;

[0280] If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, then the high-level cache image is cleaned.

[0281] In one embodiment, the processor performs the following steps when executing the computer program:

[0282] The cleaning of the low-level cache image and the mid-level cache image includes:

[0283] Clean up the lowest-level cache image that was most recently acquired;

[0284] After each cleanup of the low-level cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range.

[0285] If not, then stop cleaning up the low-level cache image;

[0286] If so, continue cleaning the low-level cache image, and after cleaning the low-level cache image, clean the intermediate cache image that has been retrieved the fewest times in sequence;

[0287] After each cleanup of the intermediate cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range;

[0288] If not, then stop cleaning the intermediate cache image.

[0289] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0290] Within a specified time frame set by the user, traverse all computing nodes in the artificial intelligence platform and determine the node type of each computing node;

[0291] The cache space cleanup threshold range in the computing node is determined based on the node type and the node threshold relationship table set by the user.

[0292] Obtain the space utilization rate of the cache space and determine the cache image type of all cache images in the cache space;

[0293] The cache image is cleaned up according to the cleanup threshold range, the space utilization rate, and the cache image type.

[0294] In one embodiment, the computer program performs the following steps when executed by a processor:

[0295] The process of traversing all computing nodes in the AI ​​platform and determining the node type of each computing node includes:

[0296] Record the number of times the computing nodes are accessed during artificial intelligence learning;

[0297] When the number of accesses does not exceed the first access threshold set by the user, the node type is determined to be a low-level node;

[0298] When the number of accesses does not exceed the second access threshold set by the user, the node type is determined to be a mid-level node;

[0299] When the number of accesses exceeds the second access threshold, the node type is determined to be an advanced node.

[0300] In one embodiment, the computer program performs the following steps when executed by a processor:

[0301] The step of obtaining the space utilization rate of the cache space and determining the cache image type of all cache images in the cache space includes:

[0302] Record the number of times the cached image is retrieved and the most recent retrieval time during artificial intelligence learning;

[0303] Determine whether the number of times the request is made is higher than the threshold number of times the user sets.

[0304] If so, then the cache image type of the cache image is determined to be an advanced cache image;

[0305] If not, the cache image type is determined based on the most recent acquisition time and the acquisition time threshold set by the user.

[0306] In one embodiment, the computer program performs the following steps when executed by a processor:

[0307] Determining the cache image type based on the most recent acquisition time and the acquisition time threshold set by the user includes:

[0308] When the most recent acquisition time is not lower than the acquisition time threshold, the cache image type is determined to be a low-level cache image;

[0309] When the most recent acquisition time is lower than the acquisition time threshold, the cache image type is determined to be a medium-level cache image.

[0310] In one embodiment, the computer program performs the following steps when executed by a processor:

[0311] The cleanup threshold range includes a first cleanup threshold range and a second cleanup threshold range. Cleaning the cache image based on the cleanup threshold range, the space utilization rate, and the cache image type includes:

[0312] When the space utilization rate falls within the first cleanup threshold range, the user is reminded to clean up the cache image.

[0313] When the space utilization rate falls within the second cleanup threshold range, the cache image is cleaned up according to the second cleanup threshold range, the space utilization rate, and the cache image type.

[0314] In one embodiment, the computer program performs the following steps when executed by a processor:

[0315] The step of cleaning the cache image according to the second cleanup threshold range, the space utilization rate, and the cache image type includes:

[0316] When the node type is a low-level node, the low-level cache image is cleared;

[0317] If, after cleaning the low-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the intermediate cache image.

[0318] When the node type is a mid-level node, clean up the low-level cache image and the mid-level cache image;

[0319] If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the high-level cache image.

[0320] When the node type is a high-level node, clean up the low-level cache image and the mid-level cache image;

[0321] If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, then the high-level cache image is cleaned.

[0322] In one embodiment, the computer program performs the following steps when executed by a processor:

[0323] The cleaning of the low-level cache image and the mid-level cache image includes:

[0324] Clean up the lowest-level cache image that was most recently acquired;

[0325] After each cleanup of the low-level cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range.

[0326] If not, then stop cleaning up the low-level cache image;

[0327] If so, continue cleaning the low-level cache image, and after cleaning the low-level cache image, clean the intermediate cache image that has been retrieved the fewest times in sequence;

[0328] After each cleanup of the intermediate cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range;

[0329] If not, then stop cleaning the intermediate cache image.

[0330] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0331] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0332] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A cache image cleanup method, characterized in that, The method includes: Within a specified time frame set by the user, traverse all computing nodes in the artificial intelligence platform and determine the node type of each computing node; The cache space cleanup threshold range in the computing node is determined based on the node type and the node threshold relationship table set by the user. Obtain the space utilization rate of the cache space and determine the cache image type of all cache images in the cache space; Cleaning the cache image according to the cleanup threshold range, the space utilization rate, and the cache image type includes: When the space utilization rate falls within the first cleanup threshold range, the user is reminded to clean up the cache image. When the space utilization rate falls within the second cleanup threshold range, the cache image is cleaned up according to the second cleanup threshold range, the space utilization rate, and the cache image type. Specifically, when the cache space utilization rate reaches the second cleanup threshold range, cache images of different cache image types are automatically cleaned up according to different node types; when reminding users to clean up cache images and when automatically cleaning up cache images, other devices are prohibited from accessing the corresponding cache space; The step of cleaning the cache image according to the second cleanup threshold range, the space utilization rate, and the cache image type includes: When the node type is a low-level node, clean up the low-level cache image; If, after cleaning the low-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the intermediate cache image. When the node type is a mid-level node, clean up the low-level cache image and the mid-level cache image; If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, the user is reminded to clean the high-level cache image. When the node type is a high-level node, clean up the low-level cache image and the mid-level cache image; If, after cleaning the low-level cache image and the mid-level cache image, the space utilization rate is still within the second cleaning threshold range, then the high-level cache image is cleaned.

2. The method according to claim 1, characterized in that, The process of traversing all computing nodes in the artificial intelligence platform and determining the node type of each computing node includes: Record the number of times the computing nodes are accessed during artificial intelligence learning; When the number of accesses does not exceed the first access threshold set by the user, the node type is determined to be a low-level node; When the number of accesses does not exceed the second access threshold set by the user, the node type is determined to be a mid-level node; when the number of accesses exceeds the second access threshold, the node type is determined to be a high-level node.

3. The method according to claim 1, characterized in that, The step of obtaining the space utilization rate of the cache space and determining the cache image type of all cache images in the cache space includes: Record the number of times the cached image is retrieved and the most recent retrieval time during artificial intelligence learning; Determine whether the number of acquisitions exceeds the acquisition number threshold set by the user; If so, then the cache image type of the cache image is determined to be an advanced cache image; If not, the cache image type is determined based on the most recent acquisition time and the acquisition time threshold set by the user.

4. The method according to claim 3, characterized in that, Determining the cache image type based on the most recent acquisition time and the acquisition time threshold set by the user includes: When the most recent acquisition time is not lower than the acquisition time threshold, the cache image type is determined to be a low-level cache image; When the most recent acquisition time is lower than the acquisition time threshold, the cache image type is determined to be a medium-level cache image.

5. The method according to claim 1, characterized in that, The cleaning of the low-level cache image and the mid-level cache image includes: Clean up the lowest-level cache images that were most recently retrieved; After each cleanup of the low-level cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range. If not, then stop cleaning up the low-level cache image; If so, continue cleaning the low-level cache image and after cleaning the low-level cache image, clean the intermediate cache image that was retrieved the fewest times in sequence; After each cleanup of the intermediate cache image, it is checked whether the space utilization rate falls within the second cleanup threshold range. If not, then stop cleaning the intermediate cache image.

6. A cache image cleanup apparatus for the method of any one of claims 1-5, the apparatus comprising: The node determination module is used to traverse all computing nodes in the artificial intelligence platform and determine the node type of the computing nodes within a specified time set by the user. The range determination module is used to determine the range of cache space cleanup thresholds in the computing node based on the node type and the node threshold relationship table set by the user. The image determination module is used to obtain the space utilization rate of the cache space and determine the cache image type of all cache images in the cache space; The cleanup module is used to clean up the cache image based on the cleanup threshold range, the space utilization rate, and the cache image type.

7. An electronic device, characterized in that, include: One or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the method as described in any one of claims 1-5.

8. A computer storage medium, characterized in that, It stores a computer program thereon, wherein the program, when executed by a processor, implements the method as described in any one of claims 1-5.

Images