Data cache replacement system and method based on reinforcement learning
By adopting a data cache replacement system based on reinforcement learning in industrial inkjet printing systems, the problem of insufficient adaptability of cache replacement strategies in the prior art and inability to cope with real-time changing data requirements is solved, and efficient cache management and system performance improvement is achieved.
Patent Information
- Application Number
- CN202411719430.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-28
- Publication Date
- 2025-04-15
AI Technical Summary
The existing cache replacement strategy is not adaptable in the field of industrial inkjet printing and cannot effectively respond to the data reading modes and real-time changes in the data requirements of multiple nozzles, resulting in inefficient cache management and poor system performance.
A data cache replacement system based on reinforcement learning is adopted, which includes a cache replacement policy module, a cache module, a cache status monitoring module and a nozzle control module. The data access mode of the nozzle is predicted and cache replacement decisions are made through a pre-trained reinforcement learning model.
The system can dynamically learn and adapt to the data reading rules of different types of inkjet print heads, optimize cache replacement strategies, improve cache utilization and system transmission performance, support a parallel working environment for multiple nozzles, and significantly improve the overall working efficiency and data transmission stability of industrial inkjet printing systems.
Smart Images
Figure CN119821003A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of industrial inkjet printing, and in particular to a data cache replacement system and method based on reinforcement learning. Background Art
[0002] In recent years, industrial inkjet printing technology has been widely used in many industries, characterized by high precision, high speed and wide material adaptability. However, with the increasing complexity of industrial inkjet printing equipment and the increase in data demand, effectively managing and transmitting large amounts of printing data has become a technical challenge. In particular, due to the significant differences in data reading rules and physical structures of different types of print heads, existing cache replacement strategies are unable to cope with these complex dynamic data.
[0003] Traditional cache replacement strategies, such as Least Recently Used (LRU) and Least Frequently Used (LFU), are mainly based on static rules and fixed algorithm models, which are difficult to adapt to the specific needs of various types of sprinklers. These strategies have the following main problems:
[0004] First, the adaptability is insufficient. The traditional cache replacement strategy fails to dynamically adjust according to the different types of print heads, resulting in inefficient cache management when facing different print heads. Since the physical structure and data requirements of each print head are different, their data reading modes are also unique. Therefore, adopting a unified cache replacement strategy will reduce the utilization of cache space, thereby affecting the overall performance of the system.
[0005] Secondly, it is unable to cope with real-time changing data requirements. In the industrial inkjet printing process, data transmission has strict real-time requirements. The printing data stream is high-frequency and changes in real time, and the existing cache replacement strategy usually makes replacement decisions based on historical data and cannot effectively predict future cache access requirements. This may cause key data to fail to be cached in time or be replaced incorrectly.
[0006] In addition, there is a lack of targeted optimization. Most of the existing cache replacement algorithms are general algorithms and lack optimization for specific application scenarios. For high-real-time and high-precision application scenarios such as industrial inkjet printing, traditional cache management algorithms fail to fully learn and adapt to the data reading rules of different inkjet print heads. Therefore, when processing complex printing tasks, the system performance cannot reach the optimal state.
[0007] Finally, there is insufficient support for multi-nozzle parallel printing. Modern industrial inkjet printing systems usually involve multi-nozzle parallel operation. Different nozzles have different data requirements, and the workload may change at any time. Traditional cache replacement strategies perform poorly when dealing with multi-nozzle parallel operation. It is difficult to effectively cope with complex multi-threaded data scheduling, which easily leads to data transmission bottlenecks.
[0008] In summary, the existing cache replacement strategy has obvious shortcomings when facing the high real-time, high-precision and complex data reading requirements of industrial inkjet printing systems. Summary of the invention
[0009] To this end, an embodiment of the present invention provides a data cache replacement system and system based on reinforcement learning, which is used to solve the problem that the existing cache replacement strategy in the prior art is insufficiently adaptable in the field of industrial inkjet printing and cannot cope with real-time changing data requirements.
[0010] In order to solve the above problems, an embodiment of the present invention provides a data cache replacement system based on reinforcement learning, the system comprising:
[0011] A cache replacement strategy module is used to manage and optimize the set cache replacement strategy. The cache replacement strategy module uses a pre-trained reinforcement learning model to predict future cache demand based on the data access pattern of the sprinkler and make cache replacement decisions accordingly.
[0012] A cache module, used for storing data, wherein a cache replacement strategy of the cache module is determined by the cache replacement strategy module according to an output of a reinforcement learning model;
[0013] A cache status monitoring module, used to monitor the data status in the cache module in real time to ensure that the cache replacement strategy is effectively executed under different operating conditions;
[0014] The nozzle control module is used to interact with different types of nozzles, process the specific data reading requirements of the nozzles, and cooperate with the cache replacement strategy module to ensure that the data stored in the cache module can meet the real-time access requirements of the nozzles.
[0015] Preferably, the reinforcement learning model processes cache historical access data through an LSTM module, and uses an Attention mechanism to calculate the importance of current access data, and finally determines the cache replacement decision through a linear layer and a Softmax function.
[0016] Preferably, the LSTM module is responsible for receiving cache history access data, combining it with current cache data, and recursively updating the state to form an effective memory of cache access behavior.
[0017] Preferably, the Attention mechanism uses Query and Keys to calculate the relevance between current access data and historical access data, and assigns different weights to each historical record to indicate the relevance between current access data and historical access data.
[0018] Preferably, the system further comprises a model training module for training the reinforcement learning model using the reading data patterns generated by different nozzles in the actual printing process as training data to obtain a trained reinforcement learning model.
[0019] Preferably, the cache status monitoring module is used to monitor the data status in the cache module in real time, including the current cache usage, the access frequency of each cache data, and whether the data in the cache needs to be replaced.
[0020] Preferably, the system further comprises a performance evaluation module for evaluating cache hit rate, data throughput and cache replacement delay to determine the effectiveness of the cache replacement strategy and fine-tune the reinforcement learning model according to the evaluation results.
[0021] The embodiment of the present invention further provides a data cache replacement method based on reinforcement learning, which adopts the data cache replacement system based on reinforcement learning described above, and specifically includes:
[0022] Use the reading data patterns generated by different nozzles in the actual printing process as training data to train the reinforcement learning model to obtain a trained reinforcement learning model;
[0023] Deploy the trained reinforcement learning model to the cache replacement strategy module;
[0024] Initialize the cache and start caching the key data required by the sprinkler according to the set cache replacement strategy;
[0025] When the host computer issues a print task request, the cache replacement strategy module dynamically adjusts the data storage and replacement method according to the output of the reinforcement learning model;
[0026] Monitor the data status in the cache module in real time and dynamically adapt to the data access needs of the sprinkler;
[0027] Execute corresponding data reading operations according to cache hit conditions;
[0028] When the cache is full or needs to be replaced, a reinforcement learning model is used to analyze the access frequency of current data, the priority of tasks, and the possibility of future access to determine the cache replacement strategy;
[0029] Evaluate the effectiveness of the cache replacement strategy and adjust model parameters based on the evaluation results.
[0030] An embodiment of the present invention further provides an inkjet printing device, which includes the above-mentioned data cache replacement system based on reinforcement learning.
[0031] An embodiment of the present invention further provides a computer storage medium, which stores a computer software product. The computer software product includes a number of instructions for enabling a computer device to execute the above-mentioned data cache replacement method based on reinforcement learning.
[0032] It can be seen from the above technical solutions that the present invention has the following beneficial effects:
[0033] (1) Through the training of reinforcement learning models, the present invention is able to be compatible with the data reading modes of multiple print heads and dynamically adjust the cache replacement strategy according to the specific reading rules of different print heads. Compared with the traditional fixed replacement strategy, this adaptive cache management method significantly improves the flexibility of the system in handling complex and diverse printing tasks.
[0034] (2) Using the reinforcement learning model to make cache replacement decisions, the present invention reduces unnecessary replacement operations while efficiently utilizing cache space. This intelligent strategy significantly reduces the frequency and delay of cache replacement, ensuring that the system can quickly respond to data requests in high-concurrency industrial inkjet printing scenarios, thereby improving overall operating efficiency.
[0035] (3) By learning the historical access patterns of different print heads, the present invention can effectively predict the access rules of future data, thereby improving the cache hit rate. The high hit rate not only reduces the number of frequent data reads from the main memory, but also significantly improves the data processing speed, shortens the waiting time of printing tasks, and enhances the overall production efficiency of the printing device.
[0036] (4) The reinforcement learning model used in the present invention has high versatility and can adapt to a variety of inkjet printing nozzles with different physical structures. By adapting to a variety of nozzles, the present invention significantly enhances the compatibility of the system, ensures that the system can operate stably in various complex industrial environments, and reduces failures and delays caused by equipment differences.
[0037] (5) By implementing a reasonable cache replacement strategy, the present invention effectively avoids the waste of cache space and solves the problems of unnecessary data overflow and transmission bottleneck. In high-load data processing scenarios, the present invention can maintain a high cache utilization rate and prevent the system from experiencing performance degradation due to insufficient cache space during large-scale data transmission. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the implementation cases of the present invention or the technical solutions in the prior art, the following is a brief description of the drawings required for use in the embodiments. By referring to the drawings, the features and advantages of the present invention will be more clearly understood. The drawings are schematic and should not be understood as limiting the present invention in any way. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:
[0039] Figure 1 A block diagram of a data cache replacement system based on reinforcement learning provided in an embodiment;
[0040] Figure 2 This is a schematic diagram of the reinforcement learning model framework in the embodiment;
[0041] Figure 3 A flowchart of a data cache replacement method based on reinforcement learning provided in an embodiment;
[0042] Figure 4 It is a schematic diagram of the architecture of applying the reinforcement learning cache replacement strategy to the industrial inkjet printing system in the embodiment;
[0043] Figure 5 This is a schematic diagram of the reinforcement learning model training in the embodiment;
[0044] Figure 6 Schematic diagram of data throughput calculation results under different data sets in the embodiment. DETAILED DESCRIPTION
[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0046] Embodiment 1
[0047] Since there are many types of inkjet printing equipment, and the physical structure and data reading requirements of each nozzle are different, the traditional fixed cache replacement strategy is difficult to meet the personalized needs of various nozzles. Figure 1 As shown, an embodiment of the present invention proposes a data cache replacement system based on reinforcement learning, the system comprising:
[0048] The cache replacement strategy module is used to manage and optimize the set cache replacement strategy. The cache replacement strategy module uses a pre-trained reinforcement learning model to predict future cache demand based on the data access pattern of the sprinkler and make cache replacement decisions accordingly.
[0049] A cache module is used to store data. The cache replacement strategy of the cache module is determined by the cache replacement strategy module according to the output of the reinforcement learning model.
[0050] The cache status monitoring module is used to monitor the data status in the cache module in real time to ensure that the cache replacement strategy is effectively executed under different operating conditions;
[0051] The nozzle control module is used to interact with different types of nozzles, process the specific data reading requirements of the nozzles, and collaborate with the cache replacement strategy module to ensure that the data stored in the cache module can meet the real-time access requirements of the nozzles.
[0052] From the above technical solution, it can be seen that the present invention proposes a data cache replacement system based on reinforcement learning, which includes a cache replacement strategy module, a cache module, a cache status monitoring module and a nozzle control module, and predicts the data access mode of the nozzle through a pre-trained reinforcement learning model and makes a cache replacement decision. Its advantages are that it can dynamically learn and adapt to the data reading rules of different types of inkjet print heads, optimize the cache replacement strategy, improve the cache utilization efficiency and system transmission performance, support data management in a multi-nozzle parallel working environment, and effectively improve the overall work efficiency and data transmission stability of the industrial inkjet printing system.
[0053] Specifically, the core of the system of the present invention lies in its cache replacement strategy module based on a pre-trained reinforcement learning model. The cache replacement strategy module is used to manage and optimize the set cache replacement strategy. The cache replacement strategy uses a pre-trained reinforcement learning model to predict future cache requirements based on the data access pattern of the nozzle, and makes cache replacement decisions accordingly. The reinforcement learning model has deeply studied the access patterns of various types of nozzles, and predicts future cache requirements based on this, so as to make the optimal cache replacement decision. When the system is deployed, the reinforcement learning model has been fixed and the optimal cache strategy is adopted, aiming to reduce frequent access to the main memory, thereby improving the cache hit rate.
[0054] Furthermore, the reinforcement learning model first processes the cache history access data through the long short-term memory network (LSTM) module. Then, the importance of the current access data is calculated using the attention mechanism. Finally, the cache replacement decision is determined through the linear layer and the Softmax function.
[0055] Construct a network model. The overall reinforcement learning model framework diagram is as follows Figure 2 As shown:
[0056] LSTM module: This module is responsible for receiving the cache access history, including the instruction history from pc_t-H+1 to pc_t, and combining it with the current cache data mt, and recursively updating the state h_t to form an effective memory of the cache access behavior. In this model, the LSTM module is particularly good at processing long sequence data, especially when processing long-term access patterns in industrial inkjet printing, showing excellent performance.
[0057] Attention mechanism: The key part of the model is the Attention mechanism, which uses queries and keys to calculate the relevance of current access data and historical access data. The Attention layer assigns different weights g_w to each historical record by comparing the similarity between the current access request and the cached historical data. These weights reflect the relevance of the current access data and the historical access data. The more relevant the historical record is, the more important it is in the cache replacement decision.
[0058] Softmax and decision output: The calculated weights are processed through the linear layer and the Softmax function, and a probability distribution is output, indicating the probability of each cache block being replaced. This mechanism enables the system to dynamically adjust the cache strategy based on the access pattern without human intervention.
[0059] Model process:
[0060] Embedding of cache access history: The system processes past cache access history through a recurrent neural network (IndRNN) to form a temporal contextual understanding of historical access behavior.
[0061] Query generation for current access: The current cache access data is used as the query input, and the query vector Q is generated through embedded coding for comparison with the key in the cache state.
[0062] Calculate cache importance: The Attention mechanism calculates the importance (weight) of each historical access behavior based on the query vector Q and the key (Keys) K of the cache history.
[0063] Cache replacement decision: Calculate the final replacement probability through Softmax and select the appropriate cache block for replacement.
[0064] Specifically, the cache module is used to store data, and the cache replacement strategy of the cache module is determined by the cache replacement strategy module according to the output of the reinforcement learning model. The cache replacement strategy is optimized based on the data access mode of the sprinkler to ensure efficient data storage and access in different usage scenarios.
[0065] Specifically, the cache status monitoring module is used to monitor the data status in the cache module in real time to ensure that the cache replacement strategy is effectively executed under different operating conditions. This monitoring function is mainly used for system operation and maintenance and performance monitoring.
[0066] Furthermore, the cache status monitoring module is used to monitor the data status in the cache module in real time, including the current cache usage, the access frequency of each cache data, and whether the data in the cache needs to be replaced.
[0067] Specifically, the nozzle control module is used to interact with different types of nozzles, process specific data reading requirements of the nozzles, and collaborate with the cache replacement strategy module to ensure that the data stored in the cache module can meet the real-time access requirements of the nozzles.
[0068] In addition, the system of the present invention also includes a model training module, which is used to use the reading data patterns generated by different nozzles in the actual printing process as training data to train the reinforcement learning model to obtain a trained reinforcement learning model.
[0069] The system of the present invention also includes a performance evaluation module for evaluating cache hit rate, data throughput and cache replacement delay to determine the effect of the cache replacement strategy and fine-tune the reinforcement learning model based on the evaluation results.
[0070] The cache hit rate indicates the proportion of cache data that can be successfully hit when the system performs a cache query. The higher the hit rate, the more effectively the system can manage the cache and reduce unnecessary memory access. A high hit rate is the core indicator for evaluating the success of the cache replacement strategy, which directly affects the performance of the entire system.
[0071] Cache replacement latency refers to the time required to perform a replacement operation when the cache is full. This latency includes the entire process from when the cache block is identified as the least important block, triggering the replacement decision, to when the new data is written to the cache. In order to ensure the real-time performance of the system, the cache replacement latency needs to be as small as possible to ensure that the system can still respond quickly under high load and high concurrency.
[0072] Data throughput refers to the amount of data processed by the system in a given time. This indicator measures the system's parallel processing capabilities and the degree of optimization of the cache strategy. In industrial inkjet printing, the data reading speed and data stream size of different printheads may vary. The throughput of the cache system determines whether it can keep up with the high-speed data requirements of different printheads. Higher throughput means that the system can process more data streams in the same time, thereby improving overall efficiency.
[0073] By evaluating these three key performance indicators, the pros and cons of the cache replacement strategy can be accurately judged.
[0074] Through the above technical solution, the present invention realizes intelligent learning and adaptation to the data reading rules of different nozzles in industrial inkjet printing equipment, thereby significantly improving the flexibility and efficiency of cache management.
[0075] Embodiment 2
[0076] like Figure 3 As shown, the present invention provides a data cache replacement method based on reinforcement learning, which adopts the data cache replacement system based on reinforcement learning of the above embodiment 1, and specifically includes:
[0077] Use the reading data patterns generated by different nozzles in the actual printing process as training data to train the reinforcement learning model to obtain a trained reinforcement learning model;
[0078] Deploy the trained reinforcement learning model to the cache replacement strategy module;
[0079] Initialize the cache and start caching the key data required by the sprinkler according to the set cache replacement strategy;
[0080] When the host computer issues a print task request, the cache replacement strategy module dynamically adjusts the data storage and replacement method according to the output of the reinforcement learning model;
[0081] Monitor the data status in the cache module in real time and dynamically adapt to the data access needs of the sprinkler;
[0082] Execute corresponding data reading operations according to cache hit conditions;
[0083] When the cache is full or needs to be replaced, a reinforcement learning model is used to analyze the access frequency of current data, the priority of tasks, and the possibility of future access to determine the cache replacement strategy;
[0084] Evaluate the effectiveness of the cache replacement strategy and adjust model parameters based on the evaluation results.
[0085] The present invention relates to an architecture for applying a reinforcement learning cache replacement strategy to an industrial inkjet printing system, such as Figure 4 As shown in the figure. This architecture uses the host to centrally manage multiple caches through a cache replacement policy manager. Each cache is specially designed to store a specific type of sprinkler data. The system aims to optimize the allocation and replacement of cache resources by deploying a cache replacement policy that has been trained in advance through a reinforcement learning model. The specific steps include:
[0086] 1. Reinforcement learning model training:
[0087] Training data source: such as Figure 5 As shown in the figure, the data that the model training relies on comes from the reading data patterns generated by different nozzles in actual printing jobs.
[0088] Training process: Using reinforcement learning algorithm, through learning various nozzle access patterns, iterative optimization strategy is carried out until the optimal cache replacement strategy that adapts to various working scenarios is obtained.
[0089] 2. Model initialization phase:
[0090] Pre-trained model deployment: When the system starts, the trained reinforcement learning cache replacement model is deployed to the cache management module of the device, based on historical printing tasks and nozzle data access rules, to improve cache hit rate and reduce main memory access time.
[0091] Cache initialization: When the system is initialized, cache space is allocated and key data, including image data, control data, and printhead working instructions, begins to be cached according to the strategy.
[0092] 3. Data input and cache monitoring:
[0093] Print task request processing: When the host computer issues a print task, the data is sent to the cache module through the main memory. The cache replacement strategy module dynamically adjusts the data storage and replacement strategy based on the output of the reinforcement learning model, giving priority to retaining key data and removing infrequently used data.
[0094] Cache status monitoring: The cache replacement policy manager continuously monitors the cache status, including usage, data access frequency, and replacement requirements, to dynamically adapt to the data access requirements of the sprinkler.
[0095] 4. Cache hit and read operations:
[0096] Cache hit processing: If the required data exists in the cache module, it is directly returned to the inkjet printing execution module, reducing the main memory read delay and accelerating the task startup.
[0097] Cache miss processing: If the required data is not available in the cache module, new data is read from the main memory and the cache is updated. The reinforcement learning model determines the replacement strategy based on the current task and historical access records.
[0098] 5. Cache replacement strategy execution:
[0099] Application of reinforcement learning replacement strategy: When the cache is full or needs to be replaced, the model determines the replacement strategy based on access frequency, task priority, and future access probability.
[0100] Replacement decision execution: Based on model decisions, data with low priority or low re-access probability is replaced, and new data is loaded into the cache to ensure efficient execution of subsequent tasks.
[0101] 6. Task execution and cache synchronization:
[0102] Task data processing: During the execution of a printing task, the cache manager continuously provides the required data to the printhead to ensure the successful completion of the task. The reinforcement learning model adapts to the physical characteristics and task requirements of different printheads to improve data reading efficiency.
[0103] 7. Performance evaluation and adjustment:
[0104] Cache hit rate evaluation: Evaluate the effectiveness of the strategy based on indicators such as cache hit rate and read time during task execution.
[0105] Throughput and latency evaluation: Evaluate task throughput and overall data processing latency to determine the effectiveness of strategy optimization. After the reinforcement learning model is introduced, the system response time is shortened, and low latency and high throughput are maintained under high-concurrency tasks.
[0106] 8. Ending stage:
[0107] Task completion and cache cleaning: After the printing task is completed, clean up the data that is no longer needed to make room for subsequent tasks and avoid cache congestion affecting the execution of subsequent tasks.
[0108] Through the above-mentioned implementation steps, the present invention realizes the effective application of the reinforcement learning cache replacement strategy in the industrial inkjet printing system, which significantly improves the intelligence level of cache management and the overall performance of the system.
[0109] In order to further illustrate the advantages of the present invention, it is described below in conjunction with specific experiments.
[0110] In order to verify the performance of the reinforcement learning model under different print heads, cache hit rate tests of 6 different data sets were conducted. The test results are shown in the table, showing the cache hit rate comparison of Belady, LRU, Ours (the present invention) and Glider algorithms.
[0111] Table 1 below shows the hit rates of different cache replacement strategies under different data sets.
[0112] Table 1
[0113]
[0114] Table 2 below shows the replacement delays of different cache replacement strategies. The cache hit delay refers to the access delay when the task hits the cache, which is usually very low. The cache miss delay refers to the access delay when the cache misses and data needs to be read from the main memory. The cache replacement decision delay means that since the reinforcement learning model needs to make a cache replacement decision, there will be additional decision time (based on LRU), which may increase as the model complexity increases. The total delay represents the comprehensive performance of all these delays.
[0115] Table 2
[0116]
[0117] In order to evaluate the impact of the scheme on data throughput, the data throughput under different data sets is calculated, such as Figure 6 As shown. Data throughput (Mbps) = total data volume (bits) / total delay (seconds), and the unit of data throughput is Gbps. The total data volume refers to the total data size transmitted by the system within a specified time, usually in bits; the total delay refers to the time it takes to transmit this data, usually in seconds.
[0118] Embodiment 3
[0119] An embodiment of the present invention provides an inkjet printing device, which includes the data cache replacement system based on reinforcement learning of the above-mentioned embodiment one. The specific implementation method thereof can be found in the embodiment part of the data cache replacement system based on reinforcement learning mentioned above. In order to avoid redundancy, it will not be repeated here.
[0120] Embodiment 4
[0121] The embodiment of the present invention provides a computer storage medium, which stores a computer software product, and the computer software product includes several instructions for causing a computer device to execute the above-mentioned data cache replacement method based on reinforcement learning. The specific implementation method thereof can be seen in the embodiment of the data cache replacement method based on reinforcement learning above, and will not be repeated here to avoid redundancy.
[0122] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0123] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0124] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0125] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from them are still within the protection scope of the invention.
Claims
1. A data cache replacement system based on reinforcement learning, characterized in that: include: A cache replacement strategy module is used to manage and optimize the set cache replacement strategy. The cache replacement strategy module uses a pre-trained reinforcement learning model to predict future cache demand based on the data access pattern of the sprinkler and make cache replacement decisions accordingly. A cache module, used for storing data, wherein a cache replacement strategy of the cache module is determined by the cache replacement strategy module according to an output of a reinforcement learning model; A cache status monitoring module, used to monitor the data status in the cache module in real time to ensure that the cache replacement strategy is effectively executed under different operating conditions; The nozzle control module is used to interact with different types of nozzles, process the specific data reading requirements of the nozzles, and cooperate with the cache replacement strategy module to ensure that the data stored in the cache module can meet the real-time access requirements of the nozzles.
2. The data cache replacement system based on reinforcement learning according to claim 1, characterized in that: The reinforcement learning model processes cache historical access data through the LSTM module, and uses the Attention mechanism to calculate the importance of current access data, and finally determines the cache replacement decision through the linear layer and the Softmax function.
3. The data cache replacement system based on reinforcement learning according to claim 2, characterized in that: The LSTM module is responsible for receiving the cache history access data, combining it with the current cache data, and recursively updating the state to form an effective memory of the cache access behavior.
4. The data cache replacement system based on reinforcement learning according to claim 2, characterized in that: The Attention mechanism uses Query and Keys to calculate the relevance between current access data and historical access data, and assigns different weights to each historical record to indicate the relevance between current access data and historical access data.
5. The data cache replacement system based on reinforcement learning according to claim 1, characterized in that: The system also includes a model training module, which is used to use the reading data patterns generated by different nozzles in the actual printing process as training data to train the reinforcement learning model to obtain a trained reinforcement learning model.
6. The data cache replacement system based on reinforcement learning according to claim 1, characterized in that: The cache status monitoring module is used to monitor the data status in the cache module in real time, including the current cache usage rate, the access frequency of each cache data, and whether the data in the cache needs to be replaced.
7. The data cache replacement system based on reinforcement learning according to claim 1, characterized in that: The system also includes a performance evaluation module for evaluating cache hit rate, data throughput and cache replacement delay to determine the effectiveness of the cache replacement strategy and fine-tune the reinforcement learning model based on the evaluation results.
8. A data cache replacement method based on reinforcement learning, characterized in that: The method adopts the data cache replacement system based on reinforcement learning according to any one of claims 1 to 7, and specifically comprises: Use the reading data patterns generated by different nozzles in the actual printing process as training data to train the reinforcement learning model to obtain a trained reinforcement learning model; Deploy the trained reinforcement learning model to the cache replacement strategy module; Initialize the cache and start caching the key data required by the sprinkler according to the set cache replacement strategy; When the host computer issues a print task request, the cache replacement strategy module dynamically adjusts the data storage and replacement method according to the output of the reinforcement learning model; Monitor the data status in the cache module in real time and dynamically adapt to the data access needs of the sprinkler; Execute corresponding data reading operations according to cache hit conditions; When the cache is full or needs to be replaced, a reinforcement learning model is used to analyze the access frequency of current data, the priority of tasks, and the possibility of future access to determine the cache replacement strategy; Evaluate the effectiveness of the cache replacement strategy and adjust model parameters based on the evaluation results.
9. An inkjet printing device, characterized in that: The inkjet printing device comprises the data cache replacement system based on reinforcement learning as described in any one of claims 1 to 7.
10. A computer storage medium, characterized in that: The computer storage medium stores a computer software product, which includes a number of instructions for enabling a computer device to execute the data cache replacement method based on reinforcement learning as described in claim 8.