A redis-based insurance industry data intelligent cache optimization method and system

Abstract

The patent application discloses an insurance industry data intelligent cache optimization method and system based on Redis, and relates to the field of scientific and technological data, and comprises the following steps: S1: real-time collection and monitoring of original data by using a non-intrusive multi-source collection architecture; S2: hot spot prediction and weight calculation; S3: multi-level cache preloading execution; and S4: double-layer cache elimination. According to the patent application, the non-intrusive multi-source data collection architecture is used to acquire insurance business access metadata, a multi-dimensional time period template matching prediction model is used to calculate the final hotness of the cache key, and a weight grading preloading mechanism is used. Before the arrival of a business request, hot spot policies, claim settlement statistics and other business data are preloaded to Redis, full loading or incremental preheating is performed according to the data volume, large-volume business data is returned to the source in batches through a cursor page, the frequency of on-site database query during a business request is reduced, and the interface response performance in high-frequency and complex query scenarios is improved.

Description

Technical Field

[0001] This invention patent relates to the field of scientific and technological data, specifically a method and system for intelligent caching optimization of insurance industry data based on Redis. Background Technology

[0002] In insurance business systems, operations such as policy inquiry, claims statistics, customer profile loading, and rate calculation involve a large number of database join queries and aggregation calculations. As business volume increases, existing technologies have the following prominent problems: 1. In high-frequency business scenarios, the SQL execution time of existing direct-connection relational databases (such as Oracle and MySQL) increases exponentially with the expansion of data volume. The average response time of core query interfaces is slow, and some complex statistics even cause lag, which seriously affects the efficiency of counter and mobile terminal business development.

[0003] 2. During peak business periods, the number of concurrent users surges, quickly filling the database connection pool and causing connection timeouts or even service collapses.

[0004] 3. Existing caching solutions mostly use fixed expiration time (TTL) or LRU (Least Recently Used) eviction policies for data that has not been accessed for the longest time. These solutions cannot detect real-time changes in the popularity of insurance business, resulting in large fluctuations in cache hit rate and frequent cache breakdowns and cache avalanches.

[0005] 4. To cope with peak pressure, the existing system has over-configured database and caching hardware, resulting in insufficient resource utilization under normal load and still struggling to meet demand during peak periods, with hardware costs rising year by year. Summary of the Invention

[0006] The purpose of this invention is to provide a Redis-based intelligent caching optimization method and system for insurance industry data, in order to solve the problems mentioned in the background.

[0007] To address the aforementioned technical problems, this invention provides the following technical solution: a Redis-based intelligent data caching optimization method for the insurance industry, comprising the following steps: S1: Employs a non-intrusive multi-source acquisition architecture to collect and monitor raw data in real time; The system acquires access behavior metadata and combines the metadata with test traffic and duplicate reported noise to form the raw data. The raw data undergoes deduplication and light cleaning of the test traffic to obtain effective metadata, which is then written in batches to the time series database and Redis hash structure through pipelines. S2: Hotspot prediction and weight calculation; The system identifies the raw real-time heat value based on the time series database, and identifies the time period correction coefficient based on the historical feature vector in the historical time period template library. At the same time, it identifies the business impact factor by weighting. The system identifies the final heat value of each key by combining the raw real-time heat value, the time period correction coefficient and the business impact factor. Based on the final heat value of each key, the system maps the corresponding weight to execute the corresponding system action and serializes the data corresponding to each key into the heat_score_{Key} field in Redis. S21: Calculates the raw real-time heat value using a real-time heat calculation engine; Specifically, the metrics of all accessed data keys within a preset time period are read from the time-series database. The average and peak query rates per second are obtained by key aggregation. For historical high-frequency keys that have not been accessed within the preset time period, the heat value of the previous period is inherited with a decay factor. The formula for calculating the real-time raw heat value is as follows: ; The Real-time raw heat value; The : The current query rate per second in the real-time window; The : Average query rate per second for all keys in the current period; The penetration penalty is as follows: if a single key does not exist in Redis and frequently penetrates to the business database, the original value of the real-time heat index will be increased. S22: Use a time period template matcher to read historical feature vectors from the historical time period template library, calculate the cosine similarity between the historical feature vector and the current feature vector, filter out historical time periods with a cosine similarity greater than the cosine similarity filtering threshold, take the median of the number of visits to the filtered historical time periods, divide the median by the average number of visits to a single key in the past week, and obtain the time period correction coefficient. If the current time period is a preset business day, then the time period correction factor is multiplied by 1.5, where 1.5 is an acceleration factor; S23: Use a weighted fusion decision maker to read the business scenario priority cardinality associated with a single key from the data dictionary and calculate the business impact factor of the single key; The calculation formula for the business impact factor in S23 is as follows: ; The Business influencing factors; The Priority base for business scenarios; S24: The weighted fusion decision-maker calculates the final heat value based on the real-time heat value, the time period correction coefficient, and the business impact factor, and executes the corresponding system action according to the weight mapped to the final heat value. The formula for calculating the final heat value is as follows: ; The Final popularity score; The Time period correction factor; S25: Result writing and distribution; Each key's corresponding data is serialized and written into the heat_score_{Key} field in Redis. The key heat_score_{Key} is generated by concatenating the business key with a fixed prefix heat_score, and is consumed by the preload executor and the evictioner. The data corresponding to each key includes: final popularity value, weight level, suggested cache key lifespan, and preloading priority; S3: Multi-level cache preloading execution; The system periodically traverses the hot data in Redis, starts loading based on the final hot value corresponding to each key, performs full loading / incremental preheating based on a preset data volume threshold, and determines the Redis cluster writing method based on the final hot weight corresponding to each key, writing to the target shard in steps. S31: Use a preloading executor to periodically traverse the popularity data in Redis and start the preloading process based on the final popularity value corresponding to the single key; The conditions for starting the preloading process are as follows: If the final popularity value is ≥70, and the remaining suggested cache key in Redis does not exist or the lifetime of a single key is < the preset time, then preloading will be performed; If the final popularity value is ≥90, and the current time is ≤ the predicted peak time of the single-key business scenario, then preloading will be performed; S32: Perform full loading / incremental preheating based on a single key data volume to retrieve business data from the source; The execution conditions for full loading / incremental preheating are as follows: If the data size is less than 1MB, then perform a full load. The full loading process involves the preload executor initiating a single query to the database, reading all business data sets corresponding to a single key from the database at once, serializing all business data sets, generating a string cache key with an expiration time, and then writing the data to the Redis cluster at once using a Redis atomic write command. If the data volume is ≥1MB, then perform incremental preheating; The incremental preheating process involves the preload executor initiating a cursor pagination query to the database, reading the business data set corresponding to a single key from the database in batches, serializing the business data in batches, generating string cache keys with expiration times, and then writing the batches of data sequentially to the Redis cluster using Redis atomic write commands. S33: Determine the Redis cluster write method based on the final heat weight corresponding to a single key, and perform synchronous / asynchronous write to the target Redis cluster shard through CRC16 hash slot routing to the corresponding shard; If the final heat value is ≥90, then synchronous writing is performed, and the preload writing process is considered complete only after Redis confirms the AOF persistence flush. If the final heat value falls within the range of 45 to 89, then an asynchronous write method is used, without waiting for Redis's AOF persistence flush confirmation; S4: Double-level cache eviction; The system uses an eviction mechanism to distinguish between high-frequency and low-frequency cached data and performs a two-level progressive eviction process. The two-level progressive elimination includes a first-level elimination and a second-level elimination. The first-level elimination is based on the periodic wake-up feature of the preloaded executor. It calls the Redis progressive key traversal command to traverse all keys, and filters the candidate keys to be deleted in turn according to the business effective deadline, business status change time and cache write timestamp. It also calls the Redis synchronous delete key command in batches to delete the invalid cache. After deletion, the invalidation event is sent to the distributed message queue and reported to the monitoring. The secondary eviction process is triggered when the Redis memory usage reaches the startup threshold and automatically stops when the Redis memory usage drops to the shutdown threshold, wherein the startup threshold is greater than the shutdown threshold. The secondary elimination is based on the LRU-W weighted elimination algorithm to calculate the basic elimination value corresponding to each key, and combined with the business priority to obtain the comprehensive elimination weight. All cache keys to be eliminated are sorted in ascending order according to the comprehensive elimination weight, and cache deletion operations are performed sequentially. The formula for calculating the basic elimination value is as follows: ; The : Base elimination value, the aforementioned The value range of is [1~10]; The Current system time; The The last time the key was read; The Priority base for business scenarios; The Weighting level coefficient; The Business scenario acceleration factor; S5: Feedback adaptive optimization. The system collects user behavior signals to form a closed-loop feedback. It adopts a dual-track optimization of online parameter fine-tuning and offline relearning, and compares the cache hit rate before and after rollback. The collected user behavior signals include users immediately retrying, actively giving up, completing business conversion, staying for a long time, and users actively providing feedback. Differentiated penalties / gains are applied to the popularity calculation of a single key based on different behaviors. The penalties / gains are used to correct the true popularity corresponding to the single key. The online parameter fine-tuning is performed based on a preset time. The prediction model weights are adjusted using the mini-batch gradient descent method. The overall deviation between the predicted popularity of the prediction model and the actual popularity after penalty / gain correction is calculated. The smaller the deviation, the more accurate the prediction of the prediction model. The overall deviation is used as the loss basis of the mini-batch gradient descent method. The formula for calculating the deviation value is as follows: ; The Deviation value; The total number of samples refers to the number of cache keys that participated in the training. The i： Sample number; The : No. i The predictive model for each sample predicts the popularity. The : No. i The true popularity of each sample is the actual popularity value obtained based on historical real access statistics; The offline relearning is performed based on preset time nodes, and pulls all historical logs within a preset number of days, updates the historical time period template library, calibrates the popularity classification threshold, generates a pre-loaded whitelist, and releases it after gray-scale verification. The rollback involves comparing the cache hit rate before and after the prediction model weight adjustment. If the hit rate drops to the warning threshold, an automatic rollback is executed and an alarm is triggered.

[0008] A system based on Redis for optimizing intelligent data caching in the insurance industry includes a multi-source data acquisition layer, a hotspot prediction and weight decision layer, a multi-level cache execution layer, and a feedback adaptive tuning layer. The multi-source data acquisition layer adopts a non-intrusive multi-source acquisition architecture. It comprehensively collects cache access behavior metadata and business semantic data through three types of acquisition entry points: JDBC proxy interceptor, @CacheMonitor custom annotation tracking, and Redis slow query log. The JDBC proxy interceptor is deployed between the application layer and the database connection pool to collect SQL statements, execution time, number of returned rows, and data table information. The @CacheMonitor custom annotation relies on AOP aspect to collect business semantic information such as call source, user role, and business scenario encoding. The Redis slow query log is pulled and parsed periodically to identify cache miss keys and corresponding access frequencies, filling the data acquisition blind spot where the cache penetrates to the database. The hotspot prediction and weight decision layer includes a real-time heat calculation engine, a time period template matcher, a weight fusion decision-maker, and a result distribution unit. The real-time heat calculation engine reads the access indicators of the time series database, aggregates the keys to obtain the average and peak QPS, and calculates the original real-time heat value of each key by combining the heat decay mechanism of high-frequency keys without access history. The time period template matcher retrieves the feature vector of the historical time period template library, filters similar historical time periods by cosine similarity matching, calculates and generates the time period correction coefficient, and configures the acceleration factor for the preset business day to realize the dynamic correction of the time period heat. The weight fusion decision-maker reads the business scenario priority base, calculates the business impact factor, and merges the original real-time heat value, the time period correction coefficient, and the business impact factor to obtain the final heat value. Based on the final heat value, it completes multi-level heat classification and generates five cache management strategies: immediate preloading, timed preloading, regular on-demand refresh, passive loading, and no caching. It also matches the dynamic TTL expiration time with the preloading priority and finally serializes the heat parameters of each key and writes them into the heat_score_{Key} field of Redis. The multi-level cache execution layer consists of two core daemons: a preload executor and an eviction mechanism. These respectively implement intelligent hierarchical preloading and a two-layer progressive cache eviction. A built-in preload traffic control mechanism ensures stable operation of the database and cache cluster. The preload executor periodically traverses Redis hot data, determining the preload trigger condition based on the final hot value. It performs either full loading or incremental preloading differently depending on the data volume. Small data volumes use a single database query for full loading, while large data volumes use cursor-based paginated incremental preloading. Furthermore, the Redis cluster write strategy is differentiated based on the hotness level: high-hot keys are written synchronously and await AOF persistence flush confirmation, while medium-hot keys use an asynchronous write mechanism. The preloading process incorporates a flow control strategy, which limits the upper limit of concurrent threads, the number of SQL query rows, and dynamically adjusts the thread sleep duration to achieve adaptive speed reduction protection for database load. The eviction mechanism is used to distinguish between high-frequency and low-frequency cached data and implements a two-layer progressive eviction mechanism of first-level timed failure eviction and second-level memory pressure eviction. The first-level eviction relies on the periodic traversal capability to filter invalid caches and delete them in batches based on business timeliness and cache write time, and synchronously pushes failure events to the message queue to complete monitoring and reporting. The second-level eviction is triggered when the Redis memory usage exceeds the threshold. It calculates the basic eviction value of each key based on the LRU-W weighted eviction algorithm, integrates business priority to generate a comprehensive eviction weight, and sorts the low-frequency invalid caches in ascending order of weight to complete the cleanup of the cache. The feedback adaptive tuning layer relies on real user behavior signals to dynamically correct and continuously optimize model parameters. It adopts a dual-track optimization mechanism of online parameter fine-tuning and offline relearning, and is equipped with a hit rate rollback guarantee mechanism. By collecting multiple behavioral signals such as immediate retry, active abandonment, business conversion, long stay, and active feedback, it applies penalties or gain corrections to the real popularity of single keys based on differentiated behaviors to optimize the model training ground value. Online parameter fine-tuning is performed based on a preset cycle. It iteratively adjusts the model weights based on the mean squared error loss value using the mini-batch gradient descent method, and limits the weight adjustment range to ensure smooth model iteration. Offline relearning periodically pulls the full historical logs, updates the historical time period template library, calibrates the popularity classification threshold, and generates a pre-loaded whitelist. After gray-scale verification, the version iteration is completed. At the same time, the system monitors the cache hit rate in real time. If the hit rate is lower than the alarm threshold after model iteration, parameter rollback is automatically executed.

[0009] The beneficial effects achieved by this invention patent are as follows: By acquiring insurance business access metadata through a non-intrusive multi-source data collection architecture, and calculating the final popularity of cache keys based on a multi-dimensional time period template matching prediction model, coupled with a weighted hierarchical preloading mechanism, hot policies, claims statistics and other business data are preloaded into Redis before business requests arrive. The full load or incremental preheating is performed according to the data volume. For large-volume business data, cursor pagination is used to retrieve data from the source in batches, reducing the frequency of on-site database queries during business requests and improving the interface response performance in high-frequency and complex query scenarios.

[0010] By using the preload executor's built-in flow control mechanism, the upper limit of preload concurrent threads and the number of SQL query rows are limited. The thread sleep duration is dynamically adjusted according to the database load to avoid additional impact on the database during the preheating process. High-hot keys are preheated in advance according to the strategy. During peak business periods, a large number of requests directly read the Redis cache, reducing the number of concurrent requests directly hitting the database during peak periods, reducing the possibility of the database connection pool being exhausted instantly, alleviating the database concurrency pressure under peak business periods, and reducing the conditions for service avalanche.

[0011] By integrating real-time QPS, historical time period characteristics, and business priorities into a predictive model, the final popularity is calculated. The TTL and preloading strategy for each key are dynamically generated, abandoning the globally fixed expiration time. At the same time, a two-layer progressive eviction mechanism is used to evict expired business caches based on business timeliness. A weighted eviction algorithm is used, which incorporates business priority, weight level, and business acceleration factor into the eviction judgment. Unlike the native LRU, which only relies on access time for eviction, this approach retains high-priority business caches as much as possible. Combined with user behavior, a feedback loop is built, which enables the caching strategy to adapt to the dynamic changes in insurance business popularity, improve cache hit rate fluctuations, and reduce the probability of cache breakdown and cache avalanche.

[0012] By intelligently predicting hotspots, preloading in a tiered manner, and weighted eviction based on business awareness, the cache's ability to handle traffic from insurance business access is enhanced. By relying on algorithmic strategies to smooth out some peak traffic for business access, it is not necessary to simply rely on adding more hardware resources to combat peak traffic. This improves the utilization rate of software and hardware resources under normal operating conditions and reduces the cost waste caused by excessive hardware configuration. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating a Redis-based intelligent data caching optimization method for the insurance industry, as shown in this embodiment. Detailed Implementation

[0014] The present invention will be further described below with reference to the accompanying drawings.

[0015] Example 1

[0016] like Figure 1As shown in Example 1, a Redis-based intelligent data caching optimization method for the insurance industry is disclosed, including the following steps: S1: Employs a non-intrusive multi-source acquisition architecture to collect and monitor raw data in real time; Access behavior metadata is obtained through JDBC proxy interceptors, @CacheMonitor custom annotation tracking, and Redis slow query logs. The metadata, along with test traffic and duplicate reported noise, constitutes the raw data. The raw data undergoes deduplication and light cleaning of the test traffic to obtain effective metadata, which is then written in batches to a time series database and a Redis hash structure via pipeline for real-time reading by the hot data prediction model. At the same time, the historical effective metadata stored in the time series database is aggregated and statistically analyzed, and the statistical results of access characteristics for different time periods are written to the historical time period template library in MySQL. The JDBC proxy interceptor is located between the application layer and the database connection pool. It is used to collect SQL statements, execution time, number of returned rows, and table names. The @CacheMonitor custom annotation is used to collect business semantic information by AOP aspects, including call source, user role, and business scenario encoding. The sources of these calls include counter service, APP, mini-program, and API; The user roles include agents, underwriters, and customers; The business scenario codes include policy query (POLICY_QUERY) and claims statistics (CLAIM_STAT); The Redis slow query log is periodically pulled, and the missing keys and access frequencies in the slow log are parsed. The data blind spots caused by queries that missed the cache and those that penetrated into the database are also filled in. S2: Hotspot prediction and weight calculation; S21: Calculates the raw real-time heat value using a real-time heat calculation engine; Specifically, the metrics of all accessed data keys within a preset time period are read from the time-series database. The average and peak query rates per second are obtained by key aggregation. For historical high-frequency keys that have not been accessed within the preset time period, the heat value of the previous period is inherited with a decay factor. The formula for calculating the real-time raw heat value is as follows: ; The Real-time raw heat value; The : The current query rate per second in the real-time window; The : The average query rate per second for all keys within the current period; The penetration penalty is as follows: if the key does not exist in Redis and frequently penetrates to the business database, the original value of the real-time heat index will be increased. S22: Use a time period template matcher to read historical feature vectors from the historical time period template library, calculate the cosine similarity between the historical feature vector and the current feature vector, filter out historical time periods with a cosine similarity greater than the cosine similarity filtering threshold, take the median of the number of visits in the filtered historical time periods, divide the median by the average number of visits for the key in the past week, and obtain the time period correction coefficient T. If the current time period is a preset business day, then the time period correction coefficient T = T × 1.5, where 1.5 is an acceleration factor; Preferably, the historical feature vector adopts a 168-dimensional time period feature vector of 7 days × 24 hours; S23: Use the weighted fusion decision maker to read the priority cardinality of the business scenario associated with the key from the data dictionary, and calculate the business impact factor of the key; The formula for calculating the business impact factor is as follows: ; The Business influencing factors; The : Business scenario priority base, the The value range of is [1~10]; S24: The weighted fusion decision-maker calculates the final heat value based on the real-time heat value, the time period correction coefficient T, and the business impact factor, and executes the corresponding system action according to the weight mapped to the final heat value. The formula for calculating the final heat value is as follows: ; The The final popularity value, the The value range is [1~100]; If the final popularity value is ≥90, the key will be immediately preloaded and an independent memory shard will be allocated to it; If the final popularity value falls within 70~89, the key will complete preloading within the preset time period of the next cycle; If the final popularity value falls within the range of 45 to 69, the key will be retained in the Redis regular pool and refreshed as needed. If the final popularity value falls within the range of 20 to 44, then active preloading will not be performed on the key. The data will be loaded when the key is accessed, and the suggested cache time-to-live (TTL) of the key will be output. The suggested cache key's lifetime is derived from the weight level mapping, and is not a globally preset fixed expiration time. When the popularity level of the key changes, the suggested cache key's lifetime is updated accordingly. If the final popularity value is less than 20, the key will not be stored in the Redis cache, and business requests will directly query the database. S25: Result writing and distribution; Each key's corresponding data is serialized and written into the heat_score_{Key} field in Redis. The heat_score_{Key} is generated by concatenating the business key with a fixed prefix heat_score, and is consumed by the preload executor and the evictioner. The data corresponding to each key includes: final popularity value, weight level, suggested cache key lifespan, and preloading priority; S3: Multi-level cache preloading execution; S31: Use a preloading executor to periodically traverse the popularity data in Redis and start preloading based on the final popularity value corresponding to the key; The conditions for starting the preloading process are as follows: If the final popularity value is ≥70, and the lifetime of the suggested cache key is less than the preset time 'a' in Redis (or the time to live of the single key is not present), then preloading will be performed. Preferably, the preset time a is 120 seconds; If the final popularity value is ≥90, and the current time is ≤ the predicted peak time of the single-key business scenario, then preloading will be performed; The preset time b is 3 minutes; S32: Perform a full load / incremental preheating based on the data volume of this key to retrieve business data from the source; The execution conditions for full loading / incremental preheating are as follows: If the data size is less than 1MB, then perform a full load. The full loading process involves the preload executor initiating a single query to the database, reading all the business data sets corresponding to the key from the database at once, serializing all the business data sets, generating a string cache key with an expiration time, and then writing the data to the Redis cluster at once using a Redis atomic write command. If the data volume is ≥1MB, then perform incremental preheating; The incremental preheating process involves the preload executor initiating a cursor pagination query to the database, reading the set of business data corresponding to the key from the database in batches, serializing the business data in batches, generating a string cache key with an expiration time, and then writing the batches of data sequentially to the Redis cluster using a Redis atomic write command. S33: Determine the Redis cluster write method based on the final popularity weight corresponding to the key, and perform synchronous / asynchronous write to the target Redis cluster shard through CRC16 hash slot routing to the corresponding shard; If the final heat value is ≥90, then synchronous writing is performed, and the preload writing process is considered complete only after Redis confirms the AOF persistence flush. If the final heat value falls within the range of 45 to 89, then an asynchronous write method is used, without waiting for Redis's AOF persistence flush confirmation; Preferably, for the preloading executor, preloading flow control is performed: the upper limit of the number of concurrent threads in the preloading executor is set to 50% of the number of idle connections in the database connection pool, the SQL query statement corresponding to a single preloading by the preloading executor is configured with a LIMIT clause to limit the number of rows, and when the database CPU utilization is greater than a preset threshold, the preloading executor automatically increases the thread sleep time to achieve speed reduction. S4: Double-level cache eviction; A replacement mechanism is used to distinguish between high-frequency and low-frequency cached data, and a two-level progressive replacement is performed. The two-level progressive elimination includes a first-level elimination and a second-level elimination. The first-level elimination is based on the periodic wake-up feature of the preloaded executor. It calls the Redis progressive key traversal command to traverse all keys, and filters the candidate keys to be deleted according to the business effective deadline, business status change time and cache write timestamp. It also calls the Redis synchronous delete key command in batches to delete the invalid cache. After deletion, the invalidation event is sent to the distributed message queue Kafka and reported to the monitoring. The secondary eviction process is triggered when the Redis memory usage reaches the startup threshold and automatically stops when the Redis memory usage drops to the shutdown threshold, wherein the startup threshold is greater than the shutdown threshold. The secondary elimination is based on the LRU-W weighted elimination algorithm to calculate the basic elimination value corresponding to each key, and combined with the business priority to obtain the comprehensive elimination weight. All cache keys to be eliminated are sorted in ascending order according to the comprehensive elimination weight, and cache deletion operations are performed sequentially. The formula for calculating the basic elimination value is as follows: ; The : Base elimination value, the aforementioned The value range of is [1~10]; The Current system time; The The last read time of the key; The Priority base for business scenarios; The Weighting level coefficient; The Business scenario acceleration factor; S5: Feedback-adaptive tuning; User behavior signals are collected to form a closed-loop feedback, and online parameter fine-tuning and offline relearning are used for dual-track optimization. The cache hit rate is compared before and after rollback. The collected user behavior signals include users immediately retrying, actively giving up, completing business conversion, staying for a long time, and users actively providing feedback. Differentiated penalties / gains are applied to the popularity calculation of a single key based on different behaviors. The penalties / gains are used to correct the true popularity corresponding to a single key. The online parameter fine-tuning is performed based on a preset time. The prediction model weights are adjusted using the mini-batch gradient descent method. The overall deviation between the predicted popularity of the prediction model and the actual popularity after penalty / gain correction is calculated. The smaller the deviation, the more accurate the prediction of the prediction model. The overall deviation is used as the loss basis of the mini-batch gradient descent method. The formula for calculating the deviation value is as follows: ; The Deviation value; The : Total number of samples, where the total number of samples is the number of cache key samples that participated in the training; The i： Sample number; The : No. i The predictive model for each sample predicts the popularity. The : No. i The true popularity of each sample is the actual popularity value obtained based on historical real access statistics; Preferably, the adjustment range of the prediction model weights is limited to ±20%; The offline relearning is performed based on preset time nodes, and pulls all historical logs within a preset number of days, updates the historical time period template library, calibrates the popularity classification threshold, generates a pre-loaded whitelist, and releases it after gray-scale verification. The rollback involves comparing the cache hit rate before and after the prediction model weight adjustment. If the hit rate drops to the warning threshold, an automatic rollback is executed and an alarm is triggered.

[0017] Example 2 Example 2 discloses a system for running an intelligent data caching optimization method for the insurance industry based on Redis, including: a multi-source data acquisition layer, a hotspot prediction and weight decision layer, a multi-level caching execution layer, and a feedback adaptive tuning layer; The multi-source data acquisition layer adopts a non-intrusive multi-source acquisition architecture. It comprehensively collects cache access behavior metadata and business semantic data through three types of acquisition entry points: JDBC proxy interceptor, @CacheMonitor custom annotation tracking, and Redis slow query log. The JDBC proxy interceptor is deployed between the application layer and the database connection pool to collect SQL statements, execution time, number of returned rows, and data table information. The @CacheMonitor custom annotation relies on AOP aspect to collect business semantic information such as call source, user role, and business scenario encoding. The Redis slow query log is periodically pulled and parsed to identify cache miss keys and corresponding access frequencies, filling the data acquisition blind spots where the cache penetrates to the database. The hotspot prediction and weight decision layer includes a real-time heat calculation engine, a time period template matcher, a weight fusion decision-maker, and a result distribution unit. The real-time heat calculation engine reads the access indicators of the time series database, aggregates the keys to obtain the average and peak QPS, and calculates the real-time heat original value of each key by combining the heat decay mechanism of high-frequency keys without access history. The time period template matcher retrieves the feature vector of the historical time period template library, filters similar historical time periods by cosine similarity matching, calculates and generates the time period correction coefficient, and configures the acceleration factor for the preset business day to realize the dynamic correction of the time period heat. The weight fusion decision-maker reads the business scenario priority base, calculates the business impact factor, and merges the real-time heat original value, the time period correction coefficient, and the business impact factor to obtain the final heat value. Based on the final heat value, it completes multi-level heat classification and generates five cache management strategies: immediate preloading, timed preloading, regular on-demand refresh, passive loading, and no caching. It also matches the dynamic TTL expiration time with the preloading priority and finally serializes the heat parameters of each key and writes them into the heat_score_{Key} field of Redis. The multi-level cache execution layer consists of two core daemons: a preload executor and an eviction executor. These processes respectively implement intelligent hierarchical preloading and two-layer progressive cache eviction. The layer also incorporates a preload flow control mechanism to ensure the stable operation of the database and cache cluster. The preloading executor periodically traverses Redis hot data and determines the preloading trigger condition based on the final hot value. It performs full loading or incremental preloading differently for different data volumes. Small data volumes use a full loading method with a single database query, while large data volumes use an incremental preloading method with cursor pagination. At the same time, it distinguishes the Redis cluster write strategy based on the hotness level. High-hot keys are written synchronously and wait for AOF persistence flush confirmation, while medium-hot keys use an asynchronous write mechanism. The preloading process has a built-in traffic control strategy, which achieves adaptive speed reduction protection of database load by limiting the upper limit of concurrent threads, the number of SQL query rows, and dynamically adjusting the thread sleep time. The eviction mechanism is used to distinguish between high-frequency and low-frequency cached data. It implements a two-layer progressive eviction mechanism of first-level timed failure eviction and second-level memory pressure eviction. The first-level eviction relies on the periodic traversal capability to filter and delete invalid caches in batches based on business timeliness and cache write time. It also pushes failure events to the message queue to complete monitoring and reporting. The second-level eviction is triggered when the Redis memory usage exceeds the threshold. It calculates the basic eviction value of each key based on the LRU-W weighted eviction algorithm, integrates business priority to generate a comprehensive eviction weight, and sorts the low-frequency invalid caches in ascending order of weight to complete the cleanup of the cache. The feedback adaptive tuning layer relies on real user behavior signals to dynamically correct and continuously optimize model parameters. It adopts a dual-track optimization mechanism of online parameter fine-tuning and offline relearning, and is configured with a hit rate rollback guarantee mechanism. By collecting multiple behavioral signals such as immediate retry, active abandonment, business conversion, long stay, and active feedback, it applies penalties or gain corrections to the real popularity of single keys based on differentiated behaviors to optimize the model training ground value. Online parameter fine-tuning is performed according to a preset cycle. It iteratively adjusts the model weights based on the mean squared error loss value using the mini-batch gradient descent method, and limits the weight adjustment range to ensure smooth model iteration. Offline relearning periodically pulls the full historical logs, updates the historical time period template library, calibrates the popularity classification threshold, and generates a pre-loaded whitelist. After gray-scale verification, the version iteration is completed. At the same time, the system monitors the cache hit rate in real time. If the hit rate is lower than the alarm threshold after model iteration, the parameters are automatically rolled back to ensure the stability of the cache service.

[0018] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A Redis-based intelligent data caching optimization method for the insurance industry, characterized by: Includes the following steps: S1: Employs a non-intrusive multi-source acquisition architecture to collect and monitor raw data in real time; The system acquires access behavior metadata and combines the metadata with test traffic and duplicate reported noise to form the raw data. The raw data undergoes deduplication and light cleaning of the test traffic to obtain effective metadata, which is then written in batches to the time series database and Redis hash structure through pipelines. S2: Hotspot prediction and weight calculation; The system identifies the raw real-time heat value based on the time series database, and identifies the time period correction coefficient based on the historical feature vector in the historical time period template library. At the same time, it identifies the business impact factor by weighting. The system identifies the final heat value of each key by combining the raw real-time heat value, the time period correction coefficient and the business impact factor. Based on the final heat value of each key, the system maps the corresponding weight to execute the corresponding system action and serializes the data corresponding to each key into the heat_score_{Key} field in Redis. S3: Multi-level cache preloading execution; The system periodically traverses the hot data in Redis, starts loading based on the final hot value corresponding to each key, performs full loading / incremental preheating based on a preset data volume threshold, and determines the Redis cluster writing method based on the final hot weight corresponding to each key, writing to the target shard in steps. S4: Double-level cache eviction; The system uses an eviction mechanism to distinguish between high-frequency and low-frequency cached data and performs a two-level progressive eviction process. The two-level progressive elimination includes a first-level elimination and a second-level elimination. The first-level elimination is based on the periodic wake-up feature of the preloaded executor. It calls the Redis progressive key traversal command to traverse all keys, and filters the candidate keys to be deleted in turn according to the business effective deadline, business status change time and cache write timestamp. It also calls the Redis synchronous delete key command in batches to delete the invalid cache. After deletion, the invalidation event is sent to the distributed message queue and reported to the monitoring. The secondary eviction process is triggered when the Redis memory usage reaches the startup threshold and automatically stops when the Redis memory usage drops to the shutdown threshold, wherein the startup threshold is greater than the shutdown threshold. The secondary elimination is based on the LRU-W weighted elimination algorithm to calculate the basic elimination value corresponding to each key, and combined with the business priority to obtain the comprehensive elimination weight. All cache keys to be eliminated are sorted in ascending order according to the comprehensive elimination weight, and cache deletion operations are performed sequentially.

2. The intelligent caching optimization method for insurance industry data based on Redis according to claim 1, characterized in that: S2 includes the following sub-steps: S21: Calculates the raw real-time heat value using a real-time heat calculation engine; Specifically, the metrics of all accessed data keys within a preset time period are read from the time-series database. The average and peak query rates per second are obtained by key aggregation. For historical high-frequency keys that have not been accessed within the preset time period, the heat value of the previous period is inherited with a decay factor. S22: Use a time period template matcher to read historical feature vectors from the historical time period template library, calculate the cosine similarity between the historical feature vector and the current feature vector, filter out historical time periods with a cosine similarity greater than the cosine similarity filtering threshold, take the median of the number of visits to the filtered historical time periods, divide the median by the average number of visits to a single key in the past week, and obtain the time period correction coefficient. If the current time period is a preset business day, then the time period correction factor is multiplied by 1.5, where 1.5 is an acceleration factor; S23: Use a weighted fusion decision maker to read the business scenario priority cardinality associated with a single key from the data dictionary and calculate the business impact factor of the single key; S24: The weighted fusion decision-maker calculates the final heat value based on the real-time heat value, the time period correction coefficient, and the business impact factor, and executes the corresponding system action according to the weight mapped to the final heat value. S25: Result writing and distribution; Each key's corresponding data is serialized and written into the heat_score_{Key} field in Redis. The key heat_score_{Key} is generated by concatenating the business key with a fixed prefix heat_score, and is consumed by the preload executor and the evictioner. The data corresponding to each key includes: final popularity value, weight level, suggested cache key lifespan, and preloading priority.

3. The intelligent caching optimization method for insurance industry data based on Redis according to claim 1, characterized in that: S3 includes the following sub-steps: S31: Use a preloading executor to periodically traverse the popularity data in Redis and start the preloading process based on the final popularity value corresponding to the single key; The conditions for starting the preloading process are as follows: If the final popularity value is ≥70, and the remaining suggested cache key in Redis does not exist or the lifetime of a single key is < the preset time, then preloading will be performed; If the final popularity value is ≥90, and the current time is ≤ the predicted peak time of the single-key business scenario, then preloading will be performed; S32: Perform full loading / incremental preheating based on a single key data volume to retrieve business data from the source; The execution conditions for full loading / incremental preheating are as follows: If the data size is less than 1MB, then perform a full load. The full loading process involves the preload executor initiating a single query to the database, reading all business data sets corresponding to a single key from the database at once, serializing all business data sets, generating a string cache key with an expiration time, and then writing the data to the Redis cluster at once using a Redis atomic write command. If the data volume is ≥1MB, then perform incremental preheating; The incremental preheating process involves the preload executor initiating a cursor pagination query to the database, reading the business data set corresponding to a single key from the database in batches, serializing the business data in batches, generating string cache keys with expiration times, and then writing the batches of data sequentially to the Redis cluster using Redis atomic write commands. S33: Determine the Redis cluster write method based on the final heat weight corresponding to a single key, and perform synchronous / asynchronous write to the target Redis cluster shard through CRC16 hash slot routing to the corresponding shard; If the final heat value is ≥90, then synchronous writing is performed, and the preload writing process is considered complete only after Redis confirms the AOF persistence flush. If the final heat value is less than 90, then an asynchronous write method is used, without waiting for Redis's AOF persistence flush confirmation.

4. The intelligent caching optimization method for insurance industry data based on Redis according to claim 2, characterized in that: The calculation formula for the real-time raw heat value in S21 is as follows: ; The Real-time raw heat value; The : The current query rate per second in the real-time window; The : Average query rate per second for all keys in the current period; The penetration penalty is as follows: if a single key does not exist in Redis and frequently penetrates to the business database, the original value of the real-time heat index will be increased.

5. The intelligent caching optimization method for insurance industry data based on Redis according to claim 4, characterized in that: The calculation formula for the business impact factor in S23 is as follows: ; The Business influencing factors; The Priority base for business scenarios.

6. The intelligent caching optimization method for insurance industry data based on Redis according to claim 5, characterized in that: The formula for calculating the final heat value in S24 is as follows: ; The Final popularity score; The Time period correction factor.

7. The intelligent caching optimization method for insurance industry data based on Redis according to claim 6, characterized in that: The formula for calculating the basic elimination value in S4 is as follows: ; The : Base elimination value, the aforementioned The value range of is [1~10]; The Current system time; The The last time the key was read; The Priority base for business scenarios; The Weighting level coefficient; The Business scenario acceleration factor.

8. A Redis-based intelligent caching optimization method for insurance industry data according to any one of claims 1-7, characterized in that: It also includes step S5: feedback adaptive optimization, the system collects user behavior signals to form a closed-loop feedback, adopts online parameter fine-tuning and offline relearning dual-track optimization, and compares the cache hit rate before and after rollback; The collected user behavior signals include users immediately retrying, actively giving up, completing business conversion, staying for a long time, and users actively providing feedback. Differentiated penalties / gains are applied to the popularity calculation of a single key based on different behaviors. The penalties / gains are used to correct the true popularity corresponding to the single key. The online parameter fine-tuning is performed based on a preset time. The prediction model weights are adjusted using the mini-batch gradient descent method. The overall deviation between the predicted popularity of the prediction model and the actual popularity after penalty / gain correction is calculated. The smaller the deviation, the more accurate the prediction of the prediction model. The overall deviation is used as the loss basis of the mini-batch gradient descent method. The offline relearning is performed based on preset time nodes, and pulls all historical logs within a preset number of days, updates the historical time period template library, calibrates the popularity classification threshold, generates a pre-loaded whitelist, and releases it after gray-scale verification. The rollback involves comparing the cache hit rate before and after the prediction model weight adjustment. If the hit rate drops to the warning threshold, an automatic rollback is executed and an alarm is triggered.

9. The intelligent caching optimization method for insurance industry data based on Redis according to claim 8, characterized in that: The formula for calculating the deviation value in S5 is as follows: ; The Deviation value; The Total number of samples, where the total number of samples is the number of cache keys that participated in the training; The i： Sample number; The : No. i The predictive model for each sample predicts the popularity. The : No. i The true popularity of each sample is the actual popularity value obtained based on historical real visit statistics.

10. A system for implementing a Redis-based intelligent data caching optimization method for the insurance industry, characterized in that: It includes a multi-source data acquisition layer, a hotspot prediction and weight decision layer, a multi-level cache execution layer, and a feedback adaptive tuning layer; The multi-source data acquisition layer adopts a non-intrusive multi-source acquisition architecture. It comprehensively collects cache access behavior metadata and business semantic data through three types of acquisition entry points: JDBC proxy interceptor, @CacheMonitor custom annotation tracking, and Redis slow query log. The JDBC proxy interceptor is deployed between the application layer and the database connection pool to collect SQL statements, execution time, number of returned rows, and data table information. The @CacheMonitor custom annotation relies on AOP aspect to collect business semantic information such as call source, user role, and business scenario encoding. The Redis slow query log is pulled and parsed periodically to identify cache miss keys and corresponding access frequencies, filling the data acquisition blind spot where the cache penetrates to the database. The hotspot prediction and weight decision layer includes a real-time heat calculation engine, a time period template matcher, a weight fusion decision-maker, and a result distribution unit. The real-time heat calculation engine reads the access indicators of the time series database, aggregates the keys to obtain the average and peak QPS, and calculates the original real-time heat value of each key by combining the heat decay mechanism of high-frequency keys without access history. The time period template matcher retrieves the feature vector of the historical time period template library, filters similar historical time periods by cosine similarity matching, calculates and generates the time period correction coefficient, and configures the acceleration factor for the preset business day to realize the dynamic correction of the time period heat. The weight fusion decision-maker reads the business scenario priority base, calculates the business impact factor, and merges the original real-time heat value, the time period correction coefficient, and the business impact factor to obtain the final heat value. Based on the final heat value, it completes multi-level heat classification and generates five cache management strategies: immediate preloading, timed preloading, regular on-demand refresh, passive loading, and no caching. It also matches the dynamic TTL expiration time with the preloading priority and finally serializes the heat parameters of each key and writes them into the heat_score_{Key} field of Redis. The multi-level cache execution layer consists of two core daemons: a preload executor and an eviction mechanism. These respectively implement intelligent hierarchical preloading and a two-layer progressive cache eviction. A built-in preload traffic control mechanism ensures stable operation of the database and cache cluster. The preload executor periodically traverses Redis hot data, determining the preload trigger condition based on the final hot value. It performs either full loading or incremental preloading differently depending on the data volume. Small data volumes use a single database query for full loading, while large data volumes use cursor-based paginated incremental preloading. Furthermore, the Redis cluster write strategy is differentiated based on the hotness level: high-hot keys are written synchronously and await AOF persistence flush confirmation, while medium-hot keys use an asynchronous write mechanism. The preloading process incorporates a flow control strategy, which limits the upper limit of concurrent threads, the number of SQL query rows, and dynamically adjusts the thread sleep duration to achieve adaptive speed reduction protection for database load. The eviction mechanism is used to distinguish between high-frequency and low-frequency cached data and implements a two-layer progressive eviction mechanism of first-level timed failure eviction and second-level memory pressure eviction. The first-level eviction relies on the periodic traversal capability to filter invalid caches and delete them in batches based on business timeliness and cache write time, and synchronously pushes failure events to the message queue to complete monitoring and reporting. The second-level eviction is triggered when the Redis memory usage exceeds the threshold. It calculates the basic eviction value of each key based on the LRU-W weighted eviction algorithm, integrates business priority to generate a comprehensive eviction weight, and sorts the low-frequency invalid caches in ascending order of weight to complete the cleanup of the cache. The feedback adaptive tuning layer relies on real user behavior signals to dynamically correct and continuously optimize model parameters. It adopts a dual-track optimization mechanism of online parameter fine-tuning and offline relearning, and is equipped with a hit rate rollback guarantee mechanism. By collecting multiple behavioral signals such as immediate retry, active abandonment, business conversion, long stay, and active feedback, it applies penalties or gain corrections to the real popularity of single keys based on differentiated behaviors to optimize the model training ground value. Online parameter fine-tuning is performed based on a preset cycle. It iteratively adjusts the model weights based on the mean squared error loss value using the mini-batch gradient descent method, and limits the weight adjustment range to ensure smooth model iteration. Offline relearning periodically pulls the full historical logs, updates the historical time period template library, calibrates the popularity classification threshold, and generates a pre-loaded whitelist. After gray-scale verification, the version iteration is completed. At the same time, the system monitors the cache hit rate in real time. If the hit rate is lower than the alarm threshold after model iteration, parameter rollback is automatically executed.

Images