An operation and maintenance data processing method and system for an AI agent

Abstract

The application discloses an operation and maintenance data processing method and system for an AI intelligent agent, forms a heat data sequence through collecting access indexes of each shard of a storage cluster; calculates a heat value and a change degree to determine a target storage layer and compare the target storage layer with a current layer to obtain consistent and wrong layer candidates; constructs a hierarchical consistency index and a tolerance threshold to determine consistency; implements speed limiting, redirection and priority reduction isolation on the wrong layer shard, measures performance loss based on a historical baseline within an evaluation period, migrates and corrects when the performance loss is lower than a threshold, triggers an alarm and degrades when the performance loss exceeds the threshold; and outputs resource occupation and time delay after correction, gives resource configuration suggestions and hierarchical results. The method suppresses migration jitter, reduces tail time delay and cost, and improves throughput and stability.

Description

Technical Field

[0001] This invention relates to the field of distributed storage and operation and maintenance management technology, specifically a method and system for processing operation and maintenance data for AI intelligent agents. Background Technology

[0002] With the widespread application of large-model-driven AI agents in scenarios such as dialogue assistants, automated operation and maintenance, and workflow orchestration, the underlying storage needs to retain the agent's conversation history, memory state, and tool call results for a long time. This type of data is characterized by strong fluctuations and hotspot drift, and access is extremely uneven. Tiered storage is usually adopted as a trade-off between tail latency, throughput, and cost.

[0003] Existing operation and maintenance solutions mostly follow the traditional hot and cold layering strategy, perform coarse-grained statistics by node or database table, and rely on fixed thresholds to trigger migration. They lack cross-cycle tracking based on sharded global identifiers, making it difficult to characterize the session-level access trajectory of intelligent agents. This can easily lead to the splitting of the memory and state of the same intelligent agent or misjudgment of popularity. Furthermore, they only focus on the absolute value of popularity and ignore the trend of change, making it impossible to distinguish between short-term spikes and continuous high frequencies. When there is high concurrency of intelligent agents and frequent hot spot shifts, frequent migrations and storage jitter are likely to occur.

[0004] In multi-tiered storage clusters for AI agents, current technologies lack a global metric to quantify "tier consistency." Operations personnel struggle to promptly identify misaligned scenarios, such as high-frequency agents lingering in cold tiers or low-frequency historical sessions occupying expensive hot tiers, leading to reliance on experience for optimization. When correcting misalignments, common practices include batch migration or simple rate limiting, failing to consider agent task tail latency and throughput degradation as gating conditions. This easily increases P99 latency during peak business periods, and the lack of isolation control over misaligned shards during migration affects response stability. Traditional operational loops often remain at the level of "identifying hotspots—migrating data," failing to incorporate resource allocation feedback based on agent operational characteristics, making it difficult to promptly mitigate the risks and waste caused by long-term mismatches. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for processing operation and maintenance data for AI intelligent agents, comprising the following steps:

[0006] S1 collects access metrics for each data shard in the storage cluster over multiple operating cycles, records them in chronological order, and forms a hot data sequence associated with the global logical identifier of the data shard.

[0007] S2, based on the heat data sequence, obtain the heat value and heat change degree of each data segment, and determine the target storage layer of each data segment according to the heat threshold and cold threshold; compare the target storage layer with the current storage layer to obtain the consistent segment set and the misaligned layer candidate set;

[0008] S3: Calculate the hierarchical consistency index of the storage cluster based on the consistent shard set and the misaligned candidate set, and compare it with the second threshold; when the hierarchical consistency index is greater than the second threshold and the proportion of misaligned candidates is less than the tolerance threshold, determine that the storage cluster is hierarchically consistent and proceed to step S6; when the hierarchical consistency index is less than the second threshold or the proportion of misaligned candidates is greater than the tolerance threshold, determine that the storage cluster is hierarchically inconsistent and proceed to step S4.

[0009] S4. Based on the expected heat range of each storage layer, locate the misaligned fragment from the misaligned fragment candidate set and implement isolation control. Proceed to S5. The isolation control includes rate limiting access to the misaligned fragment, redirecting access requests to the misaligned fragment to the proxy or cache channel of the target storage layer, and lowering the access priority of the misaligned fragment.

[0010] S5. During the evaluation period, the performance loss is obtained based on the proportion of requests to isolated mis-layered shards, the increase in tail latency introduced by isolated mis-layered shards, and the decrease in throughput of the storage cluster. When the performance loss is less than the third threshold, consistency correction is performed and the process proceeds to step S6. When the performance loss is greater than the third threshold, an alarm is triggered.

[0011] S6, collect the resource usage and query latency after consistency correction, and output the consistency management results, which include resource configuration suggestions and data stratification results.

[0012] Furthermore, the access metrics of each data shard in the aforementioned data collection and storage cluster during multiple operating cycles are recorded in chronological order and form a heat data sequence associated with the global logical identifier of the data shard, including:

[0013] Access metrics for each data shard are obtained at a fixed statistical period. Access metrics include one or more of the following: access count, access frequency, throughput, hit rate, and response latency. The global logical identifier of the data shard is used as a unique identifier. Access metrics, along with runtime timestamps and current physical location metadata, are recorded together to form a heat data sequence in chronological order.

[0014] Furthermore, the process of obtaining the heat value and heat change rate of each data segment based on the heat data sequence, and determining the target storage layer of each data segment according to the heat threshold and cold threshold, includes:

[0015] Within a preset statistical window, the heat data sequence of each data shard is aggregated to obtain a heat value, and the trend within the same statistical window is measured to obtain the heat change rate. When the heat value is higher than the hot threshold, the target storage layer is determined as a hot layer; when the heat value is lower than the cold threshold, the target storage layer is determined as a cold layer. For data shards with heat values ​​between the hot and cold thresholds, hysteresis judgment and cooling time are used to suppress frequent migrations. When the heat value and heat change rate do not meet the entry conditions of any storage layer, the current storage layer is maintained and the data shard is marked as a shard to be reviewed, and reviewed in the next evaluation cycle.

[0016] Furthermore, the comparison of the target storage layer with the current storage layer to obtain a consistent sharding set and a misaligned sharding candidate set includes:

[0017] Read the current storage layer and the determined target storage layer for each data shard; when they match, the data shard is included in the consistent shard set; when they do not match, the data shard is included in the misaligned shard candidate set and the difference information between the current storage layer and the target storage layer and the determination confidence level are recorded.

[0018] Furthermore, the calculation of the hierarchical consistency index of the storage cluster based on the consistent shard set and the staggered candidate set includes:

[0019] Within the current evaluation period, an effective evaluation set is formed based on the consistent shard set and the misaligned candidate set. Consistency evaluation elements are collected for each data shard in the effective evaluation set. The evaluation elements include one or more of the following: consistent or inconsistent flag, request volume ratio, heat value, business importance level, judgment confidence level, and cooling time status. According to preset weights, the consistent shards and misaligned candidate shards are weighted and aggregated to obtain the hierarchical consistency index.

[0020] Furthermore, the step of locating misaligned fragments from the misaligned fragment candidate set and implementing isolation control based on the expected heat range of each storage layer includes:

[0021] The heat value of each data shard in the misaligned candidate set is matched with the expected heat range of its current storage layer. When the heat value of a data shard deviates from the expected heat range, it is identified as a misaligned shard and isolation control is implemented. When the misaligned shard completes migration or the heat value recovers to the expected heat range of the current storage layer, the isolation control is released. The heat value of a data shard deviating from the expected heat range means that the heat value of the data shard is not within the expected heat range.

[0022] Furthermore, the performance loss, calculated within the evaluation period based on the proportion of requests to isolated misaligned shards, the increase in tail latency introduced by isolated misaligned shards, and the decrease in throughput of the storage cluster, includes:

[0023] Using the historical stable state of the storage cluster as a baseline, the proportion of requests to isolated misaligned shards, the increase in tail latency relative to the baseline, and the decrease in throughput relative to the baseline are processed in a dimensionless manner during the evaluation period, and summarized according to preset weights to obtain the performance loss. When the performance loss exceeds the third threshold, an alarm is triggered and a degradation strategy is executed. The degradation strategy includes one or more of the following: reducing the migration rate, reducing the migration batch, limiting the migration time window, or enabling only read-only redirection.

[0024] Furthermore, the process of performing consistency correction when the performance loss is less than the third threshold includes:

[0025] Based on the determined target storage layer, perform controlled migration on the data shards in the mis-layer candidate set; after the migration is completed, update the metadata to record the new storage location of the data shards and delete the copy data in the original storage location.

[0026] Furthermore, the resource consumption and query latency after the aforementioned consistency correction are output as consistency management results, including resource configuration suggestions and data stratification results, including:

[0027] The system collects resource utilization metrics for hot and cold layers, including capacity utilization, input / output operations per second, and bandwidth utilization, as well as query latency metrics for each storage layer, including average query latency and target percentile latency. Based on these resource utilization metrics, the system generates resource configuration optimization suggestions. When resource utilization exceeds a preset upper limit or falls below a preset lower limit, the system proposes expansion or reduction suggestions and outputs data stratification results. The data stratification results include the global logical identifier of each data shard and its corresponding storage layer.

[0028] A data processing system for operation and maintenance of AI intelligent agents, using the aforementioned data processing method for operation and maintenance of AI intelligent agents, includes: a data acquisition module, a heat analysis and stratification module, a stratification consistency judgment module, a mis-stratification processing module, a performance evaluation module, a correction and alarm module, a result output module, and a data processing module.

[0029] The data acquisition module, heat analysis and stratification module, stratification consistency judgment module, mis-stratification processing module, performance evaluation module, correction and alarm module, and result output module are respectively connected to the data processing module;

[0030] The data acquisition module is used to access indicator collection and popularity data sequence records.

[0031] The aforementioned heat analysis and stratification module is used to obtain heat values ​​and heat change rates, determine the target storage layer, and read the current storage layer.

[0032] The aforementioned hierarchical consistency judgment module is used for calculating hierarchical consistency indicators, judging tolerance thresholds, and comparing thresholds.

[0033] The aforementioned staggered layer processing module is used for staggered layer segmentation positioning and isolation control;

[0034] The performance evaluation module is used for baseline setting, dimensionless index conversion, and performance loss generation.

[0035] The aforementioned correction and alarm module is used to perform consistency correction when the performance loss does not exceed the third threshold, and to issue an alarm and execute a degradation strategy when the performance loss exceeds the third threshold.

[0036] The result output module is used for collecting resource and latency indicators, generating resource configuration optimization suggestions, and outputting data hierarchical results.

[0037] The beneficial effects of this invention are:

[0038] Multi-cycle access collection based on sharded global logical identifiers forms a traceable heat data sequence, enabling refined heat profiles. Simultaneous aggregation yields heat values ​​and measures heat variability; combined with hot / cold thresholds, hysteresis, and cooling time, migration jitter and invalid migrations are significantly suppressed. By comparing the target layer with the current layer, a consistent / misaligned candidate set is constructed, recording differences and confidence levels to improve the interpretability of the judgment.

[0039] A hierarchical consistency metric is proposed, and a tolerance threshold is introduced to support global consistency judgment from individuals to the cluster. Isolation controls such as rate limiting, request redirection, and priority downgrading are implemented for mis-layered sharding to reduce tail latency impact before migration. Performance loss is assessed based on historical stable state; controlled correction is performed only if the loss is below a threshold, and alarms and degradation strategies are triggered if the loss exceeds the threshold to ensure business availability.

[0040] After correction, the system collects resource and latency data and outputs resource configuration suggestions and hierarchical results, forming an adaptive closed loop to reduce capacity / bandwidth costs and improve throughput and P99 stability. Attached Figure Description

[0041] Figure 1 This is a flowchart illustrating a method for processing operational data for AI agents. Detailed Implementation

[0042] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the following description.

[0043] The features and performance of the present invention will be further described in detail below with reference to embodiments. Example

[0044] A method for processing operation and maintenance data for AI intelligent agents includes the following steps:

[0045] S1 collects access metrics for each data shard in the storage cluster over multiple operating cycles, records them in chronological order, and forms a hot data sequence associated with the global logical identifier of the data shard.

[0046] S2, based on the heat data sequence, obtain the heat value and heat change degree of each data segment, and determine the target storage layer of each data segment according to the heat threshold and cold threshold; compare the target storage layer with the current storage layer to obtain the consistent segment set and the misaligned layer candidate set;

[0047] S3: Calculate the hierarchical consistency index of the storage cluster based on the consistent shard set and the misaligned candidate set, and compare it with the second threshold; when the hierarchical consistency index is greater than the second threshold and the proportion of misaligned candidates is less than the tolerance threshold, determine that the storage cluster is hierarchically consistent and proceed to step S6; when the hierarchical consistency index is less than the second threshold or the proportion of misaligned candidates is greater than the tolerance threshold, determine that the storage cluster is hierarchically inconsistent and proceed to step S4.

[0048] S4. Based on the expected heat range of each storage layer, locate the misaligned fragment from the misaligned fragment candidate set and implement isolation control. Proceed to S5. The isolation control includes rate limiting access to the misaligned fragment, redirecting access requests to the misaligned fragment to the proxy or cache channel of the target storage layer, and lowering the access priority of the misaligned fragment.

[0049] S5. During the evaluation period, the performance loss is obtained based on the proportion of requests to isolated mis-layered shards, the increase in tail latency introduced by isolated mis-layered shards, and the decrease in throughput of the storage cluster. When the performance loss is less than the third threshold, consistency correction is performed and the process proceeds to step S6. When the performance loss is greater than the third threshold, an alarm is triggered.

[0050] S6, collect the resource usage and query latency after consistency correction, and output the consistency management results, which include resource configuration suggestions and data stratification results.

[0051] The access metrics of each data shard in the aforementioned data collection and storage cluster over multiple operating cycles are recorded in chronological order and formed into a heat data sequence associated with the global logical identifier of the data shard, including:

[0052] Access metrics for each data shard are obtained at a fixed statistical period. Access metrics include one or more of the following: access count, access frequency, throughput, hit rate, and response latency. The global logical identifier of the data shard is used as a unique identifier. Access metrics, along with runtime timestamps and current physical location metadata, are recorded together to form a heat data sequence in chronological order.

[0053] The data shard global logical identifier is used to uniquely identify the data shard throughout the storage cluster's lifecycle and is independent of its physical location. The global logical identifier includes at least one of the following: business namespace identifier, partition rule identifier, and shard sequence number or key space range identifier. The mapping between the global logical identifier and the current physical location information of the data shard is maintained by the metadata service and remains unchanged during shard migration, replication, or snapshot recovery. Furthermore, the global logical identifier is not reused during its recycling period to avoid historical reference conflicts.

[0054] The process of obtaining the heat value and heat change rate of each data segment based on the heat data sequence, and determining the target storage layer of each data segment according to the heat threshold and cold threshold, includes:

[0055] Within a preset statistical window, the heat data sequence of each data shard is aggregated to obtain a heat value, and the trend within the same statistical window is measured to obtain the heat change rate. When the heat value is higher than the hot threshold, the target storage layer is determined as a hot layer; when the heat value is lower than the cold threshold, the target storage layer is determined as a cold layer. For data shards with heat values ​​between the hot and cold thresholds, hysteresis judgment and cooling time are used to suppress frequent migrations. When the heat value and heat change rate do not meet the entry conditions of any storage layer, the current storage layer is maintained and the data shard is marked as a shard to be reviewed, and reviewed in the next evaluation cycle.

[0056] The process of aggregating the popularity data sequence of each data segment within a preset statistical window to obtain a popularity value includes: cleaning the popularity data sequence to remove duplicate statistics and outliers; standardizing and dimensionless processing one or more access indicators among access counts, access frequency, throughput, hit rate, and response latency; aligning by sampling time and preserving or linearly interpolating missing sampling points; applying decreasing time weights to each sampling point within the window based on time proximity to highlight recent load changes, and then applying preset weights to each access indicator based on business priority and performing weighted aggregation to obtain the window aggregated popularity value; when the number of valid samples within the window is lower than a threshold, a low-traffic judgment rule is activated to generate a minimum popularity value; the statistical window is a fixed-length window or a scrolling window, and its length, sliding step, time weight, and indicator weight are preset values.

[0057] The method of measuring the trend within the same statistical window to obtain the degree of change in popularity includes: determining the monotonicity and trend direction of popularity values ​​arranged chronologically within the statistical window; determining the trend strength based on the duration and magnitude of continuous increases or decreases; assigning higher weight to changes near the end of the statistical window; and subtracting or reducing the weight of abnormal peaks or dips with short durations and large deviations before outputting the degree of change in popularity.

[0058] The process of comparing the target storage layer with the current storage layer to obtain a consistent sharding set and a misaligned sharding candidate set includes:

[0059] Read the current storage layer and the determined target storage layer for each data shard; when they match, the data shard is included in the consistent shard set; when they do not match, the data shard is included in the misaligned shard candidate set and the difference information between the current storage layer and the target storage layer and the determination confidence level are recorded.

[0060] The method for calculating the hierarchical consistency index of the storage cluster based on the consistent shard set and the staggered candidate set includes:

[0061] Within the current evaluation period, an effective evaluation set is formed based on the consistent shard set and the misaligned candidate set. Consistency evaluation elements are collected for each data shard in the effective evaluation set. The evaluation elements include one or more of the following: consistent or inconsistent flag, request volume ratio, heat value, business importance level, judgment confidence level, and cooling time status. According to preset weights, the consistent shards and misaligned candidate shards are weighted and aggregated to obtain the hierarchical consistency index.

[0062] Specifically, within the current evaluation period, an effective evaluation set is formed based on the consistent shard set and the misaligned candidate set. Data shards with substandard quality, insufficient valid samples, or marked as requiring review are removed, and the reasons for removal are recorded. Consistency evaluation elements are collected for each data shard in the effective evaluation set. These elements include one or more of the following: consistency or inconsistency flag, request volume percentage, popularity value, business importance level, judgment confidence level, and cooling-off time status. Consistent shards and misaligned candidate shards are weighted and aggregated according to preset weights. Shards with low judgment confidence, frequent migrations, or repeated triggering during the cooling-off period are penalized or downweighted to generate initial hierarchical consistency values. Time smoothing and debouncing processing are performed on the initial hierarchical consistency values, and a minimum stable duration and upper limit for jump amplitude are set to obtain the hierarchical consistency index.

[0063] Output a hierarchical consistency index and its accompanying statistical information, wherein the accompanying statistical information includes at least the number of valid samples, the proportion of candidates with misaligned hierarchies, and the average decision confidence level.

[0064] The method of locating misaligned fragments from the misaligned fragment candidate set and implementing isolation control based on the expected heat range of each storage layer includes:

[0065] The heat value of each data shard in the misaligned candidate set is matched with the expected heat range of its current storage layer. When the heat value of a data shard deviates from the expected heat range, it is identified as a misaligned shard and isolation control is implemented. When the misaligned shard completes migration or the heat value recovers to the expected heat range of the current storage layer, the isolation control is released. The heat value of a data shard deviating from the expected heat range means that the heat value of the data shard is not within the expected heat range.

[0066] The performance loss, calculated based on the proportion of requests to isolated misaligned shards, the increase in tail latency introduced by isolated misaligned shards, and the decrease in throughput of the storage cluster during the evaluation period, includes:

[0067] Using the historical stable state of the storage cluster as a baseline, the proportion of requests to isolated misaligned shards, the increase in tail latency relative to the baseline, and the decrease in throughput relative to the baseline are processed in a dimensionless manner during the evaluation period, and summarized according to preset weights to obtain the performance loss. When the performance loss exceeds the third threshold, an alarm is triggered and a degradation strategy is executed. The degradation strategy includes one or more of the following: reducing the migration rate, reducing the migration batch, limiting the migration time window, or enabling only read-only redirection.

[0068] Specifically, a baseline window for historical stable states is determined, wherein the baseline window satisfies the condition that business fluctuations do not exceed a preset threshold and no changes occur in capacity, topology, or key parameters; baseline tail latency and baseline throughput are obtained within the baseline window;

[0069] The percentage of requests, tail latency, and throughput observations within the evaluation period are obtained using the same statistical caliber as the baseline window.

[0070] Map the request volume percentage to a preset upper and lower limit and the business importance level; map the tail latency increase to a relative baseline and the service level target and set a saturation upper limit; map the throughput reduction to a relative baseline and set a dead zone and a saturation upper limit.

[0071] Boundary clipping and data quality labeling are performed on the three mapping results to obtain dimensionless scores within a unified range, and then weighted summation is performed according to their respective preset weights to obtain the performance loss.

[0072] The aforementioned process of performing consistency correction when the performance loss is less than the third threshold includes:

[0073] Based on the determined target storage layer, perform controlled migration on the data shards in the mis-layer candidate set; after the migration is completed, update the metadata to record the new storage location of the data shards and delete the copy data in the original storage location.

[0074] The resource consumption and query latency after consistency correction are analyzed, and the consistency management results are output, including resource configuration suggestions and data stratification results, including:

[0075] The system collects resource utilization metrics for hot and cold layers, including capacity utilization, input / output operations per second, and bandwidth utilization, as well as query latency metrics for each storage layer, including average query latency and target percentile latency. Based on these resource utilization metrics, the system generates resource configuration optimization suggestions. When resource utilization exceeds a preset upper limit or falls below a preset lower limit, the system proposes expansion or reduction suggestions and outputs data stratification results. The data stratification results include the global logical identifier of each data shard and its corresponding storage layer. Example

[0076] A data processing system for operation and maintenance of AI intelligent agents, using the aforementioned data processing method for operation and maintenance of AI intelligent agents, includes: a data acquisition module, a heat analysis and stratification module, a stratification consistency judgment module, a mis-stratification processing module, a performance evaluation module, a correction and alarm module, a result output module, and a data processing module.

[0077] The data acquisition module, heat analysis and stratification module, stratification consistency judgment module, mis-stratification processing module, performance evaluation module, correction and alarm module, and result output module are respectively connected to the data processing module;

[0078] The data acquisition module is used to access indicator collection and popularity data sequence records.

[0079] The aforementioned heat analysis and stratification module is used to obtain heat values ​​and heat change rates, determine the target storage layer, and read the current storage layer.

[0080] The aforementioned hierarchical consistency judgment module is used for calculating hierarchical consistency indicators, judging tolerance thresholds, and comparing thresholds.

[0081] The aforementioned staggered layer processing module is used for staggered layer segmentation positioning and isolation control;

[0082] The performance evaluation module is used for baseline setting, dimensionless index conversion, and performance loss generation.

[0083] The aforementioned correction and alarm module is used to perform consistency correction when the performance loss does not exceed the third threshold, and to issue an alarm and execute a degradation strategy when the performance loss exceeds the third threshold.

[0084] The result output module is used for collecting resource and latency indicators, generating resource configuration optimization suggestions, and outputting data hierarchical results.

[0085] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. An operation and maintenance data processing method for an AI agent, characterized in that, The method comprises the following steps: S1, collecting access indexes of each data shard of the storage cluster in multiple running periods, recording in time sequence and forming a heat data sequence associated with a global logical identifier of the data shard; S2, obtaining a heat value and a heat change degree of each data shard based on the heat data sequence, and determining a target storage layer of each data shard according to a hot threshold and a cold threshold; Comparing the target storage layer with the current storage layer to obtain a consistent shard set and a mislayer candidate set; S3, calculating a hierarchical consistency index of the storage cluster based on the consistent shard set and the mislayer candidate set, and comparing with a second threshold; When the hierarchical consistency index is greater than the second threshold and the mislayer candidate ratio is less than a tolerance threshold, it is determined that the hierarchical consistency of the storage cluster is consistent, and step S6 is entered; when the hierarchical consistency index is less than the second threshold or the mislayer candidate ratio is greater than the tolerance threshold, it is determined that the hierarchical consistency of the storage cluster is inconsistent, and step S4 is entered; S4, locating mislayer shards from the mislayer candidate set according to the expected heat interval of each storage layer and implementing isolation control, and entering S5, the isolation control including access speed limiting of the mislayer shards, redirecting access requests of the mislayer shards to a proxy or cache channel of the target storage layer, and downgrading access priority of the mislayer shards; S5, obtaining performance loss based on the request amount ratio of the isolated mislayer shards, the tail latency increase introduced by the isolated mislayer shards, and the throughput reduction of the storage cluster in the evaluation period; when the performance loss is less than a third threshold, performing consistency correction and entering step S6; when the performance loss is greater than the third threshold, triggering an alarm; S6, collecting resource occupation and query latency after consistency correction, and outputting a consistency management result, the consistency management result including resource configuration suggestions and data layering results.

2. The operation and maintenance data processing method for an AI agent according to claim 1, characterized in that, The method of collecting access indexes of each data shard of the storage cluster in multiple running periods, recording in time sequence and forming a heat data sequence associated with a global logical identifier of the data shard comprises: Access indexes of each data shard are obtained in a fixed statistical period, and the access indexes include one or more of access times, access frequency, throughput, hit rate and response time; the access indexes, the running period timestamp and the current physical location metadata are recorded together, and the heat data sequence is formed in time sequence with the global logical identifier of the data shard as the unique identifier.

3. The operation and maintenance data processing method for an AI agent according to claim 2, characterized in that, The method of obtaining a heat value and a heat change degree of each data shard based on the heat data sequence, and determining a target storage layer of each data shard according to a hot threshold and a cold threshold comprises: The heat data sequence of each data shard is aggregated in a preset statistical window to obtain a heat value, and the trend in the same statistical window is measured to obtain a heat change degree; when the heat value is higher than the hot threshold, the target storage layer is determined as a hot layer; when the heat value is lower than the cold threshold, the target storage layer is determined as a cold layer; for data shards with heat values between the hot threshold and the cold threshold, hysteresis judgment and cooling time are used to suppress frequent migration; when the heat value and the heat change degree do not meet the entry conditions of any storage layer, the current storage layer is maintained and the data shard is marked as a review shard, which is reviewed in the next evaluation period.

4. The operation and maintenance data processing method for an AI agent according to claim 3, characterized in that, The comparing the target storage layer with the current storage layer, obtaining a consistent shard set and a mislayer candidate set comprises: reading the current storage layer and the determined target storage layer of each data shard; when the two are consistent, the data shard is included in the consistent shard set; when the two are inconsistent, the data shard is included in the mislayer candidate set and the difference information and the determination confidence of the current storage layer and the target storage layer are recorded.

5. The operation and maintenance data processing method for an AI agent according to claim 3, characterized in that, The level consistency index of the storage cluster is calculated based on the consistent shard set and the mislayer candidate set, comprising: In the current evaluation period, based on the consistent shard set and the mislayer candidate set, an effective evaluation set is formed, and one or more of the consistency evaluation elements of each data shard in the effective evaluation set is collected, including consistent or inconsistent mark, request quantity proportion, hotness value, business importance level, determination confidence and cooling time state; according to the preset weight, the consistent shard and the mislayer candidate shard are respectively weighted and aggregated to obtain the level consistency index.

6. The operation and maintenance data processing method for an AI agent according to claim 1, characterized in that, The expected hotness interval of each storage layer is determined from the mislayer candidate set, and the mislayer shard is located and isolated, comprising: The hotness value of each data shard in the mislayer candidate set is matched with the expected hotness interval of its current storage layer, and when the hotness value of the data shard deviates from the expected hotness interval, the mislayer shard is determined and isolated; when the mislayer shard is completed migration or the hotness value is restored to the expected hotness interval of the current storage layer, the isolation control is released; the hotness value of the data shard deviates from the expected hotness interval, that is, the hotness value of the data shard is not in the expected hotness interval.

7. The operation and maintenance data processing method for an AI agent according to claim 1, characterized in that, In the evaluation period, the performance loss is obtained based on the request quantity proportion of the isolated mislayer shard, the increase of tail delay introduced by the isolated mislayer shard and the decrease amplitude of the throughput of the storage cluster, comprising: Taking the historical stable state of the storage cluster as the baseline, the request quantity proportion of the isolated mislayer shard in the evaluation period, the increase of tail delay relative to the baseline and the decrease amplitude of the throughput relative to the baseline are dimensionless processed, and the performance loss is obtained by summarizing according to the preset weight; when the performance loss exceeds the third threshold, the alarm is triggered and the degradation strategy is executed, the degradation strategy includes one or more of reducing the migration rate, reducing the migration batch, limiting the migration time window or enabling only read redirection.

8. The operation and maintenance data processing method for an AI agent according to claim 1, characterized in that, When the performance loss is less than the third threshold, the consistency correction is executed, comprising: According to the determined target storage layer, the data shard in the mislayer candidate set is executed controlled migration; after migration, the metadata is updated to record the new storage location of the data shard, and the copy data of the original storage location is deleted.

9. The operation and maintenance data processing method for an AI agent according to claim 1, characterized in that, The resource occupation and query delay after consistency correction are collected, and the consistency management result is output, including resource configuration suggestion and data layering result, comprising: The resource occupation indicators of the hot layer and the cold layer are collected, including capacity utilization, input / output times per second and bandwidth usage, and the query delay indicators of each storage layer, including average query delay and target percentile tail delay; resource configuration optimization suggestions are generated based on the resource occupation indicators, expansion or contraction suggestions are proposed when the resource usage level exceeds the preset upper limit or is lower than the preset lower limit, and data layering results are output; the data layering results include the global logical identifier of each data shard and the storage layer to which it belongs.

10. An operation and maintenance data processing system for an AI agent, applied to the operation and maintenance data processing method for an AI agent in any one of claims 1-9, characterized in that, Comprise: a data collection module, a heat analysis and layering module, a level consistency judgment module, a cross-layer processing module, a performance evaluation module, a correction and alarm module, a result output module, and a data processing module; The data collection module, heat analysis and layering module, level consistency judgment module, cross-layer processing module, performance evaluation module, correction and alarm module, result output module are respectively connected with the data processing module; Among them, the data collection module is used to access index collection and heat data sequence records; The heat analysis and layering module is used to obtain heat value and heat change degree, determine target storage layer and read current storage layer; The level consistency judgment module is used for level consistency index calculation, tolerance threshold judgment and threshold comparison; The cross-layer processing module is used for cross-layer shard positioning and isolation control; The performance evaluation module is used for baseline setting, index dimensionless and performance loss generation; The correction and alarm module is used to perform consistency correction when the performance loss does not exceed the third threshold, and issue an alarm and execute a degradation strategy when the performance loss exceeds the third threshold; The result output module is used for resource index and delay index collection, resource configuration optimization suggestion generation and data layering result output.

Images