A service fault intelligent diagnosis method based on multi-agent reinforcement learning

Abstract

The application discloses a kind of service fault intelligent diagnosis methods based on multi-agent reinforcement learning, it is related to reinforcement learning technical field, including: according to the service running data of target service, generate double-layer copy registration item;According to double-layer copy registration item, generate the residual page set of target service;According to residual page set and double-layer copy registration item, generate the page stripping organization agent and the migration write arrangement agent, and generate the residual page set after migration writing;The residual state division is carried out to the residual page after migration writing in residual page set after migration writing, to generate residual page classification result;According to residual page classification result, generate service fault diagnosis result;The application can improve the data integrity of subsequent fault analysis.

Description

Technical Field

[0001] This invention relates to the field of reinforcement learning technology, and in particular to a method for intelligent diagnosis of service faults based on multi-agent reinforcement learning. Background Technology

[0002] With the widespread adoption of cloud-native architectures, container orchestration platforms, and microservice technologies, a single business function is typically processed collaboratively by multiple service replicas. During operation, these service replicas continuously generate operational data such as monitoring metrics, logs, events, and service status. During service elastic scaling, service migration, instance rebuilding, rolling updates, and fault recovery, the relationship between the observed data of service replicas and the actual business load changes continuously. Some observed data, although no longer aligned with the current load, remains in the observation chain to continue participating in subsequent monitoring and fault analysis, while other observed data has migrated to the new load and continues to reflect the current business status. Therefore, historical observation information, current load information, and migration process information may coexist within the same target service. When a service anomaly occurs, fault information may be distributed across multiple service replicas and data from different operational stages. This necessitates analyzing not only the current load status but also the retention status and migration relationships of the observed data before and after migration to accurately pinpoint the location and attribution of the fault.

[0003] Existing service fault diagnosis methods typically use current observation data or current bearer data as the main analysis object. After service replicas are migrated, rebuilt, or bearer relationships change, fault location is often based solely on existing observation results, lacking continuous analysis of the correspondence between the observed data retention status and the actual bearer status. This leads to the mixing of data that still retains historical observation sources but has undergone bearer migration and data that has not yet completed bearer migration, causing fault representations to remain attached to historical service replicas or incorrectly attributed to the current service replica. Ultimately, this results in inaccurate service fault attribution, incomplete identification of fault propagation paths, and reduced reliability of fault diagnosis results, making it difficult to meet the actual needs of intelligent service fault diagnosis in complex microservice operating environments. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies that are difficult to meet the actual needs of intelligent service fault diagnosis in complex microservice operating environments, and to propose an intelligent service fault diagnosis method based on multi-agent reinforcement learning.

[0005] To address the problems existing in the prior art, the present invention adopts the following technical solution:

[0006] A service fault intelligent diagnosis method based on multi-agent reinforcement learning includes:

[0007] S1. Generate a two-level copy registration item based on the service operation data of the target service;

[0008] S2. Generate a set of residual pages for the target service based on the two-level replica registration entries;

[0009] S3. Based on the set of residual pages and the double-copy registration items, generate the page stripping organization agent and the rewrite orchestration agent, and generate the set of residual pages after rewrite.

[0010] S4. Divide the residual pages in the residual page set after rewriting into residual states to generate residual page classification results;

[0011] S5. Generate service fault diagnosis results based on the residual page classification results.

[0012] Compared with the prior art, the beneficial effects of the present invention are:

[0013] 1. This invention establishes a two-layer replica registration relationship between observation representative layer data and actual carrying layer data, continuously mapping observation data to actual service carrying relationships, and extracting data that has not entered the actual carrying layer to form residual pages. This ensures that the observation retention status before and after service replica migration is fully preserved, avoiding direct mixing of historical observation data and current carrying data, and improving the data integrity of subsequent fault analysis.

[0014] 2. This invention introduces a multi-agent reinforcement learning mechanism to perform organizational and rewrite arrangement processing on residual pages, so that the rewritten residual pages retain the observation source, carrying relationship and rewrite process information at the same time. Based on the residual nature of the observation, the residual state is divided, so that the data of different migration stages enter the corresponding residual state category, reducing the possibility of confusion in fault attribution during migration and improving the traceability of fault propagation relationship.

[0015] 3. This invention generates corresponding markers based on residual states and completes statistical processing. Then, it combines various residual states to form service fault diagnosis results. This allows the fault diagnosis results to not only reflect the location of the fault, but also to distinguish between different states where the fault is still in the historical observation relationship, has been migrated into the actual carrying relationship, or has completed the migration closure. This improves the accuracy of service fault attribution and location and the reliability of the diagnosis results in complex microservice operating environments. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0017] Figure 1This is a flowchart illustrating a service fault intelligent diagnosis method based on multi-agent reinforcement learning, as provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0019] This embodiment provides a service fault intelligent diagnosis method based on multi-agent reinforcement learning. (See also...) Figure 1 Specifically, including:

[0020] S1. Generate a two-level copy registration item based on the service operation data of the target service;

[0021] In an embodiment of the present invention, a two-level replica registry is generated based on the service operation data of the target service, including:

[0022] Obtain the service operation data of the target service; the service operation data includes replica identifiers, observation characterization data, and bearer status data;

[0023] Service operation data refers to the set of data continuously generated by the target service during its operation and that reflects the service's operational status. This set of data originates from the status information formed by the service replica during its operation and serves as the data basis for the subsequent generation of observation representative layer data and actual carrier layer data. Replica identifier refers to a data tag that uniquely corresponds to a service replica. Different service replicas correspond to different replica identifiers. Through the replica identifier, data generated by the same service replica at different operational stages can be correlated.

[0024] Specifically, the service management platform receives the name, namespace, deployment object name, or service object name of the target service as the target service location information. Based on the target service location information, it accesses the service runtime data storage location and retrieves the replica list, observation characterization data list, and bearer status data list associated with the target service from the service runtime data storage location. Each replica name recorded in the replica list is used as a replica identifier. The indicator data, alarm data, log data, and event data corresponding to the replica identifier in the observation characterization data list are included in the observation characterization data of the corresponding replica. The access status data, ready status data, request processing status data, and replica lifecycle data corresponding to the replica identifier in the bearer status data list are included in the bearer status data of the corresponding replica. The replica identifier, observation characterization data, and bearer status data are written into the same service runtime data table according to the target service affiliation, thus obtaining the service runtime data of the target service.

[0025] Based on the replica identifier, identify the service replica belonging to the target service from the service operation data;

[0026] Specifically, the replica identifier field in the service operation data table is used as the retrieval field, and the target service location information is used as the attribution field. Data records with the same attribution field in the service operation data table are extracted to obtain a candidate replica record set. For each data record in the candidate replica record set, the replica identifier, observation characterization data, and bearer status data are extracted. Data records with replica identifiers and belonging to the target service location information are used as service replica records of the target service. The replica identifier in the service replica record is used as the service replica identifier of the target service. The observation characterization data and bearer status data in the service replica record are retained under the corresponding service replica identifier to obtain the service replica belonging to the target service.

[0027] Observational representation data of service replicas are extracted from service operation data to generate observational representation layer data of service replicas;

[0028] Specifically, the service replicas identified in the service operation data are taken as the processing objects. The observation representation data records corresponding to the service replicas are located in the service operation data. The indicator records, alarm records, log records, and event records in the observation representation data records are organized according to their data sources. The organized indicator records, alarm records, log records, and event records are written into the observation data container corresponding to the same service replica. The data in the observation data container is arranged in the order of generation to obtain the observation representative layer data of the service replica.

[0029] Extract the service replica's bearer status data from the service operation data to generate the actual bearer layer data of the service replica;

[0030] Observational representation data refers to the operational data presented externally by the service replica in the observation system, which describes the current operational status of the service replica as recorded and displayed by the monitored system; bearer status data refers to the operational data generated during the actual processing of business requests undertaken by the service replica, which reflects the actual status of the business processing tasks currently undertaken by the service replica.

[0031] Specifically, the service replicas already identified in the service operation data are taken as the processing objects. The bearer status data record corresponding to the service replica is located in the service operation data. The access status record, ready status record, request processing record, and replica operation status record in the bearer status data record are organized according to the data source. The organized access status record, ready status record, request processing record, and replica operation status record are written into the bearer data container corresponding to the same service replica. The data in the bearer data container is arranged according to the generation order to obtain the actual bearer layer data of the service replica.

[0032] Establish a correspondence between the observed representative layer data and the actual carrier layer data, and generate a two-layer replica registration item.

[0033] A two-layer replica register refers to a data unit formed by establishing a correspondence between the observation representative layer data and the actual carrier layer data corresponding to the same service replica. It is used to record the correspondence between the observation layer and the actual carrier layer of the same service replica, and serves as the data basis for subsequent observation residual paging processing.

[0034] Specifically, the observation representative layer data and the actual carrier layer data corresponding to the same service replica are loaded into the same registration container. The storage locations for the observation representative layer data and the actual carrier layer data are set in the registration container. The observation representative layer data is written to the observation representative layer data storage location, and the actual carrier layer data is written to the actual carrier layer data storage location. The registration container is then associated with the corresponding service replica to obtain a two-layer replica registration item.

[0035] S2. Generate a set of residual pages for the target service based on the two-level replica registration entries;

[0036] In an embodiment of the present invention, a set of residual pages for the target service is generated based on the two-level replica registry entries, including:

[0037] For each two-layer replica registration item, based on the correspondence between the observation representative layer data and the actual carrier layer data, the portion of the observation representative layer data that has not entered the actual carrier layer data is determined as the residual observation data of the service replica;

[0038] Residual observation data refers to the data content in the observation representative layer data that has formed observation records but has not entered the actual carrier layer data. This data still maintains the observation correspondence with the original service copy and is used to represent the data part that has not been migrated between the observation information and the actual carrier information.

[0039] Specifically, each two-layer replica registration item is used as a processing unit. The observation representative layer data and the actual carrier layer data are retrieved from the two-layer replica registration item. Each observation record in the observation representative layer data is matched with the carrier record in the actual carrier layer data according to the registration relationship of the same service replica. The observation records that are not received by the corresponding carrier record are retained under the two-layer replica registration item. The retained observation records are written into the residual observation data container. The residual observation data container is associated with the corresponding two-layer replica registration item to obtain the residual observation data of the service replica.

[0040] Based on the residual observation data, generate residual pages for the service copy;

[0041] Residual pages refer to data carriers constructed based on residual observation data. Each residual page is used to centrally record residual observation data corresponding to the same service replica and maintain the association between residual observation data and the corresponding service replica.

[0042] Specifically, using the residual observation data of the service replica as input, the residual observation data corresponding to the same service replica is written into the same residual page container. The residual observation data storage location and the association location of the two-level replica registration item are set in the residual page container. The residual observation data is written into the residual observation data storage location, the corresponding two-level replica registration item is written into the two-level replica registration item association location, and the residual page container is marked as the residual page corresponding to the service replica, thus obtaining the residual page of the service replica.

[0043] Collect residual pages to generate a set of residual pages for the target service.

[0044] The residual page set refers to the data set formed by collecting the residual pages corresponding to all service replicas of the target service. It is used to uniformly store the residual pages of each service replica in the target service and to serve as the data basis for subsequent residual page migration and orchestration processing.

[0045] Specifically, using the residual pages corresponding to each service replica as input, a collection container corresponding to the target service is established. Each residual page is written into the collection container, and each residual page is associated with the double-replica registration item of the corresponding service replica. The collection container is recorded as the residual page set of the target service, thus obtaining the residual page set used for residual page migration orchestration.

[0046] S3. Based on the set of residual pages and the double-copy registration items, generate the page stripping organization agent and the rewrite orchestration agent, and generate the set of residual pages after rewrite.

[0047] In embodiments of the present invention, based on the residual page set and the two-level copy registration entries, a page stripping organization agent and a write-and-arrangement agent are generated, including:

[0048] Based on the residual page set and the two-level copy registration items, a multi-agent reinforcement learning unit is established;

[0049] A multi-agent reinforcement learning unit refers to a data processing unit used to carry out tasks collaboratively by multiple agents. This data processing unit takes a set of residual pages and a two-level copy register as input, and forms multiple cooperative agents by configuring different tasks, while maintaining the data transfer relationship between the agents.

[0050] Specifically, using the residual page set and the two-layer copy registration items as input data, a reinforcement learning environment is established in the data processing device. The residual pages in the residual page set are used as environment state data, the actual carrier layer data in the two-layer copy registration items are used as environment acceptance data, the organization and rewriting actions of the residual pages are used as the agent action space, the result of the residual page processing sequence formed after the residual pages are organized is used as the page peeling organization feedback, and the result of establishing a rewriting correspondence between the residual page processing sequence and the actual carrier layer data is used as the rewriting orchestration feedback. The environment state data, environment acceptance data, agent action space, page peeling organization feedback, and rewriting orchestration feedback are written into the same reinforcement learning environment to obtain a multi-agent reinforcement learning unit.

[0051] Configure page-peeling organization tasks and write-and-arrangement tasks in the multi-agent reinforcement learning unit;

[0052] The page stripping organization task refers to the task content of performing organization processing on the residual pages in the residual page set. It is used to specify the organization method, organization order and organization objects of the residual pages in the subsequent processing, and serves as the basis for generating the page stripping organization intelligent agent. The migration orchestration task refers to the task content of performing orchestration processing on the residual page migration process. It is used to specify the orchestration content, orchestration order and orchestration objects in the process of migrating residual pages to the actual carrying layer data, and serves as the basis for generating the migration orchestration intelligent agent.

[0053] Specifically, the set of residual pages is input into the reinforcement learning environment, enabling the page-peeling organization agent to perform arrangement, grouping, and output actions on the residual pages. The reinforcement learning environment returns page-peeling organization feedback to the page-peeling organization agent based on the formation result of the residual page processing sequence, allowing the page-peeling organization agent to update the arrangement, grouping, and output actions based on the page-peeling organization feedback. The updated residual page processing sequence is then input into the rewriting orchestration agent, enabling the rewriting orchestration agent to perform acceptance selection, rewriting correspondence, and rewriting output actions on the residual page processing sequence. The reinforcement learning environment returns rewriting orchestration feedback to the rewriting orchestration agent based on the formation result of the residual page set after rewriting, allowing the rewriting orchestration agent to update the acceptance selection, rewriting correspondence, and rewriting output actions based on the rewriting orchestration feedback, resulting in a multi-agent reinforcement learning unit configured with page-peeling organization and rewriting orchestration tasks.

[0054] Generate a page-peeling intelligent agent based on the page-peeling organization task;

[0055] A page-peeling organization agent refers to a data processing entity generated based on a page-peeling organization task. It is used to organize and process the residual pages in the residual page set according to the page-peeling organization task and output the residual page processing sequence.

[0056] Specifically, using the page-tearing organization task already configured in the multi-agent reinforcement learning unit as the basis for generation, a state receiver, action execution end, and feedback receiver are established in the reinforcement learning environment for the page-tearing organization agent. The set of residual pages is used as the data received by the state receiver. The residual page arrangement action, residual page grouping action, and residual page output action are written into the action execution end. The page-tearing organization feedback is written into the feedback receiver. After receiving the set of residual pages, the state receiver drives the action execution end to perform arrangement, grouping, and output of the residual pages. The feedback receiver receives the page-tearing organization feedback returned by the reinforcement learning environment and updates the residual page arrangement action, residual page grouping action, and residual page output action in the action execution end according to the page-tearing organization feedback, thus obtaining the page-tearing organization agent.

[0057] Based on the migration orchestration task, a migration orchestration agent is generated.

[0058] The rewrite orchestration agent refers to the data processing entity generated based on the rewrite orchestration task. It is used to perform rewrite orchestration processing on the residual page processing sequence according to the rewrite orchestration task and generate a set of residual pages after rewrite.

[0059] Specifically, based on the rewrite orchestration task already configured in the multi-agent reinforcement learning unit, a state receiver, an action execution end, and a feedback receiver are established in the reinforcement learning environment for the rewrite orchestration agent. The residual page processing sequence output by the page-peeling organization agent and the actual carrier layer data in the double-layer copy registration item are used as the data received by the state receiver. The acceptance selection action, rewrite correspondence action, and rewrite output action are written into the action execution end, and the rewrite orchestration feedback is written into the feedback receiver. After receiving the residual page processing sequence and the actual carrier layer data, the state receiver drives the action execution end to perform acceptance selection, rewrite correspondence, and rewrite output on the residual page processing sequence. The feedback receiver receives the rewrite orchestration feedback returned by the reinforcement learning environment and updates the acceptance selection action, rewrite correspondence action, and rewrite output action in the action execution end according to the rewrite orchestration feedback, thus obtaining the rewrite orchestration agent.

[0060] In an embodiment of the present invention, generating a set of residual pages after a write-over includes:

[0061] Input the set of residual pages into the page stripping organization agent to generate a residual page processing sequence;

[0062] The residual page processing sequence refers to the data sequence formed after the page stripping and organizing agent arranges, groups and organizes each residual page in the residual page set according to the page stripping and organizing task. This data sequence maintains the organizational relationship and processing order between each residual page and serves as the data basis for the rewrite orchestration agent to perform rewrite orchestration.

[0063] Specifically, taking the set of residual pages and the already generated page-peeling organization agent as input, the set of residual pages is sent to the state receiving end of the page-peeling organization agent, enabling the page-peeling organization agent to obtain each residual page in the set. The action execution end of the page-peeling organization agent then performs arrangement and grouping actions on each residual page. The arrangement result of the residual pages obtained from the arrangement action and the grouping result of the residual pages obtained from the grouping action are written to the output end of the page-peeling organization agent. The output end generates a residual page processing sequence, which maintains the organizational order and grouping relationship of each residual page in the set.

[0064] Based on the actual carrier layer data in the two-layer replica registration item, generate the actual carrier layer data set;

[0065] The actual carrier layer data set refers to the data set formed by the collection of actual carrier layer data extracted from each two-level replica registration item. This data set completely saves the actual carrier layer data corresponding to all service replicas of the target service and serves as the source of the carrier data corresponding to the residual page migration.

[0066] Specifically, taking the two-level replica registration item as input, the generated actual carrier layer data is obtained from each two-level replica registration item, the obtained actual carrier layer data is written into the same carrier set container, the actual carrier layer data is associated with the corresponding two-level replica registration item, the carrier set container is recorded as the actual carrier layer data set, and the actual carrier layer data set stores the actual carrier layer data in each two-level replica registration item.

[0067] The residual page processing sequence and the actual carrier layer data set are input into the rewrite orchestration agent to generate the rewrite residual page set.

[0068] The set of residual pages after migration refers to the set of data formed after the migration orchestration agent completes the migration orchestration based on the residual page processing sequence and the actual carrier layer data set. Each residual page in this set is associated with the corresponding actual carrier layer data and serves as the data basis for subsequent intelligent diagnosis of service faults.

[0069] Specifically, the residual page processing sequence, the actual carrier layer data set, and the generated rewrite orchestration agent are used as inputs. The residual page processing sequence is sent to the residual page input end of the rewrite orchestration agent, and the actual carrier layer data set is sent to the carrier data input end of the rewrite orchestration agent. The action execution end of the rewrite orchestration agent performs acceptance selection action and rewrite corresponding action on the residual pages in the residual page processing sequence. The residual pages corresponding to the completed rewrite are written to the output end of the rewrite orchestration agent, and the output end generates the rewrite residual page set.

[0070] S4. Divide the residual pages in the residual page set after rewriting into residual states to generate residual page classification results;

[0071] In an embodiment of the present invention, generating residual page classification results includes:

[0072] Based on the observation representative layer data and the actual carrying layer data of the service replica, define the observation representative residual;

[0073] Observation representative persistence refers to the data retention relationship between the observation representative layer data and the actual carrier layer data of the same service copy. When some data in the observation representative layer data does not complete the corresponding migration with the actual carrier layer data, this part of the data continues to maintain its original observation source and exists independently in the data after the migration. It is used to indicate the data state in which the observation information is still retained relative to the actual carrier information.

[0074] Specifically, each observation record in the observation representative layer data and the bearing record in the actual bearing layer data are configured to correspond to each other according to the two-level replica registration item of the same service replica. The observation records that have entered the actual bearing layer data are marked as bearing observation records, and the observation records that have not entered the actual bearing layer data are marked as residual observation records. The state in which the residual observation records exist independently relative to the actual bearing layer data is recorded as observation representative residue, and the observation representative residue is written into the two-level replica registration item of the corresponding service replica.

[0075] Based on the observed representative remnants, the residual states of the rewritten residual pages in the rewritten residual page set are divided into residual state categories, resulting in residual state classification results. Among them, the residual state classification results include historical representative residual categories, residual carrying relocation categories, and residual closure categories.

[0076] Post-migration residual pages refer to the residual pages after the migration and arrangement are completed. This data carrier preserves the correspondence of residual observation data after migration and maintains the migration association between residual observation data and actual carrier layer data. Residual state classification results refer to the data results formed after classifying all post-migration residual pages according to the residuality of observation representatives. This data result records the residual state category and correspondence of each post-migration residual page and serves as the data basis for generating residual page classification results. Historical representative residual category refers to the residual state category of post-migration residual pages that still retains the historical observation representative source and has not completed the actual carrier affiliation. It is used to indicate post-migration residual pages where the observation information is still in the historical representative relationship. Carrier migration residual category refers to the residual state category of post-migration residual pages that has established the actual carrier affiliation but still retains the historical observation representative source. It is used to indicate post-migration residual pages where the observation information has completed the carrier migration but the historical representative relationship has not been eliminated. Residual closure category refers to the residual state category of post-migration residual pages where the historical observation representative source and the actual carrier affiliation both maintain a complete correspondence. It is used to indicate post-migration residual pages where the observation representative residual process has been closed.

[0077] In embodiments of the present invention, the residual state division results are obtained, including:

[0078] Read each remnant page from the remnant page set;

[0079] Specifically, the page index records stored in the set of residual pages after the rewrite are obtained. Based on the page index records, the storage location of each residual page after the rewrite in the set of residual pages after the rewrite is located. The corresponding residual page after the rewrite is obtained from the storage location. The obtained residual page after the rewrite is sent to the residual state division processing flow, so that each residual page after the rewrite is treated as an independent processing object for residual state division.

[0080] Read the residual source record, residual carrier record, and residual migration record from the residual pages after the migration;

[0081] Specifically, the storage locations of the source record, the carrier record, and the migration record are located in the residual page after migration. The residual source record is obtained from the source record storage location, the residual carrier record is obtained from the carrier record storage location, and the residual migration record is obtained from the migration record storage location. The residual source record, the residual carrier record, and the residual migration record are written into the same residual state temporary storage container, so that the three types of records corresponding to the same residual page after migration maintain the same page ownership relationship.

[0082] Residual pages after rewriting that retain the observation representative layer data in the residual source records and whose residual carrier records do not form an actual carrier layer data association are classified into the historical representative residual category;

[0083] Specifically, the observation representative layer data retention field is searched in the residual source record, and the actual carrier layer data association field is searched in the residual carrier record. The rewritten residual pages that contain the observation representative layer data retention field but do not contain the actual carrier layer data association field are written into the historical representative residual class container. The page index record of the rewritten residual page in the rewritten residual page set is synchronously written into the historical representative residual class container, so that the historical representative residual class saves the corresponding rewritten residual page and its page index record.

[0084] It needs to be explained that because the observation information in the residual pages after the migration still retains the original observation representative layer's affiliation, but has not established a connection with the actual carrying layer data, the observation information only retains the historical observation representative source and has not yet migrated to the current actual carrying location along with the business carrying relationship. Its data is still in the initial retention stage of observation representative residualness. Therefore, this type of residual page after migration is uniformly classified into the historical representative residual category, which can separate and save the data that has not yet formed an actual carrying affiliation from the data that has already undergone carrying migration, providing a unified data foundation for subsequent residual mark extraction and residual status statistics.

[0085] Residual source records that retain the observation representative layer data and residual bearer records that form an association with the actual bearer layer data are classified as bearer migration residuals.

[0086] Specifically, since the residual pages after the migration have established a connection with the actual carrier layer data, it indicates that the observation information has completed the migration to the current actual carrier object. However, the residual source records still retain the original observation representative layer data, indicating that the historical observation representative relationship has not been completely eliminated. The observation information has two ownership states at the same time: historical source and current carrier. Therefore, this type of residual page after migration is independently classified into the carrier migration residual category. This can centrally save the data that has completed the carrier migration but still retains the historical observation source, so that the migration stage of the observation representative residual is fully preserved. This provides a unified data foundation for subsequent analysis of the state changes of the observation information from historical representative to actual carrier.

[0087] Specifically, the data retention content of the observation representative layer is obtained from the residual source record, and the data association content of the actual carrier layer is obtained from the residual carrier record. The rewritten residual page that simultaneously contains the data retention content of the observation representative layer and the data association content of the actual carrier layer is written into the carrier migration-in residual class container. The page position record of the rewritten residual page in the rewritten residual page set is written into the carrier migration-in residual class container, so that the carrier migration-in residual class saves the rewritten residual page that still retains the observation source and has formed a carrier association.

[0088] Residual pages that contain residual source records, residual carrier records, and residual migration records in the same residual page are classified as residual closure pages.

[0089] Specifically, residual pages containing residual source records, residual carrying records, and residual transfer records are categorized into residual closure pages because these residual pages not only preserve the historical observation representative source and the current actual carrying attribution, but also completely record the entire process of observation information migrating from the observation representative layer data to the actual carrying layer data. This indicates that the formation, migration, and carrying process of observation representative residuals has formed a complete data loop. At this point, the various records can correspond to each other and completely reflect the same transfer process. Therefore, they are uniformly categorized into residual closure pages, which can separate and preserve data that has completed the entire observation representative residual process from data that is still in the migration stage, providing a complete and continuous data source for subsequent residual marker extraction and residual status statistics.

[0090] Specifically, the data retention content of the observation representative layer, the associated data content of the actual carrier layer, and the content of the completed migration process are obtained from the same rewritten residual page. The rewritten residual page that simultaneously contains the data retention content of the observation representative layer, the associated data content of the actual carrier layer, and the content of the completed migration process is written into the residual closed class container. The page position record of the rewritten residual page in the rewritten residual page set is also written into the residual closed class container, so that the residual closed class can store the rewritten residual page with complete source relationship, carrier relationship, and migration relationship.

[0091] By combining historical representative residues, residues carrying migration, and residue closures, the residue state classification results are obtained.

[0092] Specifically, using historical representative residual containers, carrier-in residual containers, and residual closed containers as inputs, a residual state partitioning result container is established. Within this container, storage locations for historical representative residual containers, carrier-in residual containers, and residual closed containers are set. The residual pages and their positions after migration from the historical representative residual containers are written to the historical representative residual container storage location. Similarly, the residual pages and their positions after migration from the carrier-in residual containers are written to the carrier-in residual container storage location. Finally, the residual pages and their positions after migration from the residual closed containers are written to the residual closed container storage location, thus obtaining the residual state partitioning result.

[0093] Based on the residual state classification results and the corresponding residual pages after rewriting, residual page classification results are generated.

[0094] The residual page classification result refers to the data result formed after classifying each residual page after migration according to the residual status category. This data result preserves the correspondence between each residual status category and the corresponding residual page after migration, and serves as the data basis for subsequent residual mark extraction.

[0095] Specifically, using the residual state classification results and the set of residual pages after migration as input, the following steps are taken: From the residual state classification results, obtain the historical representative residual class, the carrier-immigrated residual class, and the residual closure class. From the historical representative residual class, obtain the residual pages after migration belonging to that class and their corresponding page position records. From the carrier-immigrated residual class, obtain the residual pages after migration belonging to that class and their corresponding page position records. From the residual closure class, obtain the residual pages after migration belonging to that class and their corresponding page position records. Establish a residual page classification result container. Within this container, set the storage locations for classification names, residual pages after migration, and page position records. Write the historical representative residual class, carrier-immigrated residual class, and residual closure class into the classification name storage location. Write the residual pages corresponding to each class into the residual page after migration storage location. Write the page position records corresponding to each class into the page position record storage location. Establish a class-based correspondence between the classification names, residual pages after migration, and page position records to obtain the residual page classification results.

[0096] S5. Generate service fault diagnosis results based on the residual page classification results.

[0097] In an embodiment of the present invention, a service fault diagnosis result is generated based on the residual page classification result, including:

[0098] The rewritten residual pages in the residual page classification results are read to obtain the residual page content;

[0099] The residual page content refers to all the data content saved in the residual page after migration. This data content includes the retained content of the observation representative layer data after migration, the associated content of the actual carrying layer data, and the data content formed during the migration process, and serves as the data source for residual marker extraction.

[0100] In the residual page classification results, locate the corresponding classification storage locations of historical representative residual class, carrier-immigrated residual class, and residual closure class. Obtain the rewritten residual pages stored in each classification storage location. Send the obtained rewritten residual pages into the content reading container. In the content reading container, obtain the residual source record, residual carrier record, residual rewriting record, and residual observation data stored in the rewritten residual pages. Merge the residual source record, residual carrier record, residual rewriting record, and residual observation data according to the page position record of the same rewritten residual page to obtain the residual page content.

[0101] Based on the observed representative remnants, historical representative markers, migration-bearing markers, and residual closure markers are extracted from the content of the residual pages.

[0102] The historical representative marker refers to the data identifier generated based on the observation representative layer data retained in the residual page content, used to indicate that the residual page still retains the historical observation representative source after the rewrite; the carrier migration marker refers to the data identifier generated based on the actual carrier layer data association formed in the residual page content, used to indicate that the residual page has established an actual carrier attribution relationship after the rewrite; the residual closure marker refers to the data identifier generated based on the simultaneous existence of observation representative layer data retention content, actual carrier layer data association content, and rewrite process data in the residual page content, used to indicate that the residual page has formed a complete observation source, carrier attribution, and rewrite correspondence relationship after the rewrite;

[0103] Using the residual page content as the processing object, residual source records, residual carrying records, and residual migration records are obtained from the residual page content. Historical representative markers are generated by corresponding the observation representative layer data retained in the residual source records, and carrying migration markers are generated by corresponding the actual carrying layer data saved in the residual carrying records. Residual closure markers are generated by corresponding the residual source records, residual carrying records, and residual migration records saved simultaneously in the same residual page content. Historical representative markers, carrying migration markers, and residual closure markers are saved under the corresponding residual page content.

[0104] The quantities of historical representative markers, carrier migration markers, and residual closure markers are registered separately to generate residual statistical records;

[0105] Residual statistical records refer to the data records formed after the quantity registration of historical representative markers, carrier migration markers, and residual closure markers is completed. These data records are used to save the quantity information corresponding to various markers and serve as the data basis for generating service fault diagnosis results.

[0106] Specifically, a marker quantity registration table is established, and historical representative marker registration positions, carrier migration marker registration positions, and residual closure marker registration positions are set in the marker quantity registration table. Historical representative markers generated in the residual page content are written to the historical representative marker registration positions, carrier migration markers generated in the residual page content are written to the carrier migration marker registration positions, and residual closure markers generated in the residual page content are written to the residual closure marker registration positions. The number of markers in the historical representative marker registration positions, the number of markers in the carrier migration marker registration positions, and the number of markers in the residual closure marker registration positions are counted separately, and the counting results are written to the same statistical record container to generate residual statistical records.

[0107] Based on residual statistics, service fault diagnosis results are generated.

[0108] Service fault diagnosis results refer to the target service fault analysis results generated based on residual statistical records. These results are used to characterize the fault attribution of each service copy in the target service and the corresponding fault content.

[0109] Specifically, the number of historical representative markers, the number of carrier migration markers, and the number of residual closure markers are obtained from the residual statistical records. The number of historical representative markers is taken as the historical observation residual quantity of the target service, the number of carrier migration markers is taken as the carrier migration quantity of the target service, and the number of residual closure markers is taken as the residual closure quantity of the target service. The historical observation residual quantity, carrier migration quantity, and residual closure quantity are aggregated within the same service to form the residual state structure of the target service. The data content corresponding to the historical observation residual quantity in the residual state structure is written into the observation residual fault item in the fault diagnosis result. The data content corresponding to the carrier migration quantity in the residual state structure is written into the carrier migration fault item in the fault diagnosis result. The data content corresponding to the residual closure quantity in the residual state structure is written into the residual closure fault item in the fault diagnosis result. The observation residual fault item is marked as the fault representation is still attached to the historical observation representative copy. The carrier migration fault item is marked as the fault representation has migrated into the actual carrier layer but still retains the historical observation source. The residual closure fault item is marked as the fault representation has completed the closure of the observation source, actual carrier, and migration process. The observation residual fault item, carrier migration fault item, and residual closure fault item are merged to generate the service fault diagnosis result of the target service.

[0110] It should be noted that in this scheme, reinforcement learning is reflected in the continuous learning process of the page-picking organization agent and the rewrite orchestration agent. The multi-agent reinforcement learning unit constructs a learning environment with the residual page set and the two-layer copy registration items. Each agent executes corresponding actions according to the environment state and continuously updates its action strategy based on the environment feedback, so that the residual page organization results and rewrite orchestration results are gradually optimized, and finally a rewrite residual page set that better matches the correspondence between the observed representative layer data and the actual carrying layer data is obtained, thereby improving the accuracy of service fault diagnosis.

[0111] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A service fault intelligent diagnosis method based on multi-agent reinforcement learning, characterized in that, Includes the following steps: S1. Generate a two-level copy registration item based on the service operation data of the target service; S2. Generate a set of residual pages for the target service based on the two-level replica registration entries; S3. Based on the set of residual pages and the double-copy registration items, generate the page stripping organization agent and the rewrite orchestration agent, and generate the set of residual pages after rewrite. S4. Divide the residual pages in the residual page set after rewriting into residual states to generate residual page classification results; S5. Generate service fault diagnosis results based on the residual page classification results.

2. The intelligent service fault diagnosis method based on multi-agent reinforcement learning according to claim 1, characterized in that, Based on the service operation data of the target service, generate a two-level replica registry entry, including: Obtain the service operation data of the target service; the service operation data includes replica identifiers, observation characterization data, and bearer status data; Based on the replica identifier, identify the service replica belonging to the target service from the service operation data; Observational representation data of service replicas are extracted from service operation data to generate observational representation layer data of service replicas; Extract the service replica's bearer status data from the service operation data to generate the actual bearer layer data of the service replica; Establish a correspondence between the observed representative layer data and the actual carrier layer data to generate a two-layer replica registration item.

3. The intelligent service fault diagnosis method based on multi-agent reinforcement learning according to claim 2, characterized in that, Based on the two-level replica registry entries, generate a set of residual pages for the target service, including: For each two-layer replica registration item, based on the correspondence between the observation representative layer data and the actual carrier layer data, the portion of the observation representative layer data that has not entered the actual carrier layer data is determined as the residual observation data of the service replica; Based on the residual observation data, generate residual pages for the service copy; Collect residual pages to generate a set of residual pages for the target service.

4. The intelligent service fault diagnosis method based on multi-agent reinforcement learning according to claim 1, characterized in that, Based on the residual page set and the two-level copy registration entries, a pagination organization agent and a rewrite orchestration agent are generated, including: Based on the set of residual pages and the two-level copy registration items, a multi-agent reinforcement learning unit is established; Configure page-peeling organization tasks and write-and-arrangement tasks in the multi-agent reinforcement learning unit; Generate a page-peeling intelligent agent based on the page-peeling organization task; Based on the migration orchestration task, a migration orchestration agent is generated.

5. The intelligent service fault diagnosis method based on multi-agent reinforcement learning according to claim 1, characterized in that, Generate a set of residual pages after the write-over, including: Input the set of residual pages into the page stripping organization agent to generate a residual page processing sequence; Based on the actual carrier layer data in the two-layer replica registration item, generate the actual carrier layer data set; The residual page processing sequence and the actual carrier layer data set are input into the rewrite orchestration agent to generate the rewrite residual page set.

6. The intelligent service fault diagnosis method based on multi-agent reinforcement learning according to claim 1, characterized in that, Generate residual page classification results, including: Based on the observation representative layer data and the actual carrying layer data of the service replica, define the observation representative residual; Based on the observed representative remnants, the residual states of the rewritten residual pages in the rewritten residual page set are divided into residual state categories, resulting in residual state classification results. Among them, the residual state classification results include historical representative residual categories, residual carrying relocation categories, and residual closure categories. Based on the residual state classification results and the corresponding residual pages after rewriting, residual page classification results are generated.

7. The intelligent service fault diagnosis method based on multi-agent reinforcement learning according to claim 6, characterized in that, Based on the residual page classification results, service fault diagnosis results are generated, including: The rewritten residual pages in the residual page classification results are read to obtain the residual page content; Based on the observed representative remnants, historical representative markers, migration-bearing markers, and residual closure markers are extracted from the content of the residual pages. The quantities of historical representative markers, carrier migration markers, and residual closure markers are registered separately to generate residual statistical records; Based on residual statistics, service fault diagnosis results are generated.

8. The intelligent service fault diagnosis method based on multi-agent reinforcement learning according to claim 6, characterized in that, The residual state division results are obtained, including: Read each remnant page from the remnant page set; Read the residual source record, residual carrier record, and residual migration record from the residual pages after the migration; Residual pages after rewriting that retain the observation representative layer data in the residual source records and whose residual carrier records do not form an actual carrier layer data association are classified into the historical representative residual category; Residual source records that retain the observation representative layer data and residual bearer records that form an association with the actual bearer layer data are classified as bearer migration residuals. Residual pages that contain residual source records, residual carrier records, and residual migration records in the same residual page are classified as residual closure pages. By combining historical representative residues, residues carrying migration, and residue closures, the results of residue state classification are obtained.

Images