A change management system

Abstract

The embodiment of the application provides a change management system, and relates to the technical field of operation and maintenance, and the system comprises: a central dispatcher, which extracts a change object, a change type and a change expectation in an operation and maintenance change requirement input by a user; queries associated information of the change object from a configuration management asset library, queries historical change information of historical changes of the change type from a change experience library, and obtains change specification information; a change design agent, which generates a target change scheme based on the change specification information; a change audit agent, which detects whether the target change scheme passes target audit items corresponding to a business level of the change object, and generates a modification suggestion; a change design agent, which updates the target change scheme based on the modification suggestion; and a change execution agent, which executes change operation steps in the target change scheme on a business instance under the change object. In this way, the efficiency of the change can be improved.

Description

Technical Field

[0001] This application relates to the field of operation and maintenance technology, and in particular to a change management system. Background Technology

[0002] Changes to information technology (IT) systems are the leading cause of service disruptions. Statistics show that most major failures are directly related to these changes.

[0003] Users submit operational change requests, which are typically described in free text or verbally, often with incomplete fields and ambiguous wording. Operations engineers need to repeatedly communicate and confirm the objects of the change, the expected changes, and the scope of impact before they can begin designing a solution, resulting in low efficiency in the change process. Summary of the Invention

[0004] The purpose of this application is to provide a change management system to improve the efficiency of changes. The specific technical solution is as follows:

[0005] This application provides a change management system, the system comprising:

[0006] The central scheduler is used to obtain the operation and maintenance change requirements for the target business system input by the user, and extract the change object, change type and change expectation from the operation and maintenance change requirements; query the association information of the change object from the configuration management asset library, and query the historical change information of the change type from the change experience library, and combine the queried information with the extracted information to obtain change specification information;

[0007] A design change agent is used to generate a target change plan based on the change specification information;

[0008] The change review intelligent agent is used to detect whether the current target change plan passes the target review item corresponding to the business level of the change object based on the change specification information, and generate modification suggestions for the review items that fail the review; wherein, the target review item includes at least one of the following: whether the change object has an architecture guarantee to eliminate single point of failure risk, whether the current target change plan has a basis for secure execution, and whether it has recovery capability when an anomaly occurs in the execution of the current target change plan;

[0009] The design change agent is also used to update the current target change plan based on the modification suggestions;

[0010] The change execution agent is used to execute the change operation steps in the target change plan that passes the target approval item on the business instances under the change object.

[0011] Optionally, the change design agent is specifically used to query the solution templates used in the historical changes of the change type from the change experience base, and generate the target change solution by combining the solution templates, the change context and the change specification information;

[0012] The change context represents the actual environment of the target business system at present.

[0013] Optionally, the target change plan may also include: rollback steps corresponding to the change operation steps, indicator verification steps for key indicators after the change operation steps, and a gray-scale mechanism used when executing the change operation steps.

[0014] The change execution agent is specifically used to execute the indicator verification step after completing the change operation steps for any batch of business instances according to the grayscale mechanism; if the verification is successful, execute the change operation steps for the next batch of business instances; and if the verification fails, execute the rollback operation steps for the batch of business instances.

[0015] Optionally, the system further includes: a change risk control intelligent agent, used to detect whether the global status indicator of the target business system has reached a first circuit breaker threshold and whether the local status indicator associated with the change object has reached a second circuit breaker threshold when the change execution intelligent agent executes the target change plan, based on the target change plan, and to synchronize the detection results with the change execution intelligent agent; wherein, the local status indicator is determined based on the key indicator;

[0016] The change execution agent is also used to stop executing the change operation steps when the global status indicator reaches the first circuit breaker threshold, and to execute the rollback operation steps for the current batch of service instances and execute the change operation steps for the next batch of service instances when the local status indicator reaches the second circuit breaker threshold.

[0017] Optionally, the modified risk control intelligent agent is also used to increase the sampling frequency of the detected indicators and issue an alarm when the detected indicators reach an abnormal value but do not reach the circuit breaker threshold.

[0018] Optionally, the system further includes:

[0019] An experience-accumulating intelligent agent is used to acquire verification data for the target change plan, verification data for the review results of the change review intelligent agent, verification data for the execution results of the change execution intelligent agent, and verification data for the test results of the change risk control intelligent agent.

[0020] Experience entries are generated based on the obtained verification data and provided to users for review; each experience entry includes: the agent that needs optimization, the content to be optimized, the type of change targeted, and the data source of the experience entry;

[0021] The knowledge bases of the change design agent, change review agent, change execution agent, and change risk control agent are updated using the approved experience entries.

[0022] Optionally, the historical change information includes: the historical success rate of the historical changes; the target change plan also includes: the predicted success rate of the target change plan;

[0023] The change review intelligent agent is also used to dynamically adjust the target review items corresponding to the current target change plan based on the predicted success rate.

[0024] The modified risk control agent is also used to dynamically adjust the sampling frequency, corresponding outliers, and circuit breaker thresholds of the detected indicators based on the prediction success rate.

[0025] Optionally, the historical change information includes: risk points existing in the historical changes and corresponding risk handling templates; the key indicators are determined based on the risk points existing in the historical changes;

[0026] The target change plan also includes: a pre-check step that needs to be performed before the change operation step, and the target risk points; the pre-check step and the target risk points are determined based on the risk points existing in the historical changes and the corresponding risk handling templates;

[0027] The target audit items also include: risk point audit items;

[0028] The change review agent is also used to detect whether the current target change plan includes risk management operations for the target risk point, so as to determine whether the current target change plan passes the risk point review item.

[0029] Optionally, the target change plan may also include: a post-verification step;

[0030] The change execution agent is also used to perform a post-verification step after the change operation steps have been completed for all batches of business instances; if the verification is successful, the change is determined to be successful; otherwise, the change is determined to be unsuccessful.

[0031] Optionally, the central scheduler is specifically used to combine the queried information with the extracted information, and to perform semantic ambiguity detection on the combination result;

[0032] If semantic ambiguity is determined, in response to the user's confirmation instruction, the semantics indicated by the confirmation instruction are determined, and change specification information is obtained.

[0033] Beneficial effects of the embodiments in this application:

[0034] The change management system provided in this application includes: a central scheduler, used to acquire user-inputted operation and maintenance change requirements for a target business system, and extract the change object, change type, and change expectation from the operation and maintenance change requirements; query the associated information of the change object from the configuration management asset library, and query the historical change information of the change type from the change experience library, and combine the queried information with the extracted information to obtain change specification information; a change design intelligent agent, used to generate a target change plan based on the change specification information; a change review intelligent agent, used to detect whether the current target change plan passes the target review item corresponding to the business level of the change object based on the change specification information, and generate modification suggestions for the review items that fail the review; wherein, the target review item includes at least one of the following: whether the change object has an architecture guarantee to eliminate single point of failure, whether the current target change plan has a basis for secure execution, and whether it has recovery capability when an anomaly occurs during the execution of the current target change plan; the change design intelligent agent is also used to update the current target change plan based on the modification suggestions; and a change execution intelligent agent, used to execute the change operation steps in the target change plan that has passed the target review item on the business instances under the change object.

[0035] Based on the change management system provided in this application embodiment, the system can automatically parse users' operation and maintenance change requirements, generate change plans based on these requirements, and execute the change operation steps within the change plans, thereby improving change efficiency. Furthermore, the central scheduler can supplement the extracted information based on historical change information, and the change design agent can improve the change plan based on the review results of the change review agent, thus further enhancing the reliability of the changes.

[0036] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0038] Figure 1 This application provides a schematic diagram of the structure of a change management system.

[0039] Figure 2 A schematic diagram of the state machine of a central scheduler provided in an embodiment of this application;

[0040] Figure 3 This application provides a schematic diagram of a change processing procedure as illustrated in an embodiment.

[0041] Figure 4 This is a schematic diagram of the architecture of a change management system provided in an embodiment of this application. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0043] First, the technical names involved in the embodiments of this application will be introduced.

[0044] An intelligent agent is a proxy capable of perceiving its environment and taking actions to achieve specific goals. It can be software, hardware, or a system, possessing autonomy, adaptability, and interactivity. By perceiving changes in the environment (e.g., through sensors or data input), an intelligent agent makes judgments and decisions based on its learned knowledge and algorithms, and then executes actions to influence the environment or achieve predetermined goals. For example, an autonomous decision-making and execution unit built on a large language model, oriented towards a specific task, possesses data understanding, intent recognition, task planning, and result feedback capabilities.

[0045] Large Language Model (LLM): refers to a pre-trained artificial intelligence model with a large number of parameters and strong generalization and reasoning capabilities, used to support the cognitive and decision-making processes of intelligent agents.

[0046] Retrieval Augmented Generation (RAG) is a generative artificial intelligence (AI) approach for large language models that enables generative AI applications to output more accurate answers by linking large language models to external resources.

[0047] Mixed Change Specification (Spec): Refers to the operational change requirements described by users in natural language.

[0048] Configuration Management Database (CMDB): Records structured basic information of all operation and maintenance assets, covering all categories of change objects: business services, middleware components, database instances, and various cloud infrastructure resources. It stores the static attributes and related topology relationships of assets, providing underlying asset data support for standardized collection, solution design, and production readiness review.

[0049] To improve the efficiency of IT system changes, this application provides a change management system, including: a central scheduler, used to acquire user-inputted operational change requirements for a target business system, and extract the change object, change type, and change expectation from the operational change requirements; query the associated information of the change object from the configuration management asset library, and query the historical change information of the change type from the change experience library, and combine the queried information with the extracted information to obtain change specification information; a change design intelligent agent, used to generate a target change plan based on the change specification information; a change review intelligent agent, used to detect whether the current target change plan passes the target review items corresponding to the business level of the change object based on the change specification information, and generate modification suggestions for the review items that fail the review; wherein, the target review items include at least one of the following: whether the change object has an architecture guarantee to eliminate single point of failure, whether the current target change plan has a basis for secure execution, and whether it has recovery capabilities when an anomaly occurs during the execution of the current target change plan; the change design intelligent agent is also used to update the current target change plan based on the modification suggestions; and a change execution intelligent agent, used to execute the change operation steps in the target change plan that has passed the target review items on the business instances under the change object.

[0050] Based on the change management system provided in this application, the system can automatically parse users' operation and maintenance change requirements, generate change plans based on these requirements, and execute the change operation steps within the change plans, thereby improving change efficiency. Furthermore, the central scheduler can supplement the extracted information based on historical change information, and the change design agent can improve the change plan based on the review results of the change review agent, thus further enhancing the reliability of the changes.

[0051] Furthermore, the change management system provided in this application adopts a Spec-driven change standardization mechanism, which significantly reduces the time for manual communication and confirmation of change requirements, reducing information preparation time from hours to minutes, thus laying the foundation for high-quality output of intelligent agents in the future.

[0052] See Figure 1The change management system provided in this application includes: a central scheduler 101, used to obtain user input of operation and maintenance change requirements for the target business system, and extract the change object, change type and change expectation from the operation and maintenance change requirements; query the associated information of the change object from the configuration management asset library, and query the historical change information of the change type from the change experience library, and combine the queried information and the extracted information to obtain change specification information.

[0053] The design change agent 102 is used to generate the target change plan based on the change specification information.

[0054] The change review intelligent agent 103 is used to detect whether the current target change plan has passed the target review item corresponding to the business level of the change object based on the change specification information, and to generate modification suggestions for the review items that have not passed the review.

[0055] The target review items include at least one of the following: whether the object to be changed has architectural safeguards to eliminate single points of risk, whether the current target change plan has a basis for secure execution, and whether it has the ability to recover if an anomaly occurs when executing the current target change plan.

[0056] The design agent 102 is also used to update the current target change plan based on the modification suggestions.

[0057] Change execution agent 104 is used to execute the change operation steps in the target change plan that has passed the target approval item on the business instance under the change object.

[0058] In one embodiment, a central scheduler is communicatively connected to each intelligent agent in this application. The intelligent agents interact with each other through the central scheduler. For example, the central scheduler can be a server node. The intelligent agents can be distributed across different hardware, or they can be deployed on the same hardware.

[0059] The target business system is the current object of operation and maintenance; that is, the user currently needs to make changes to objects within the target business system. For example, the target business system could be a business system built on a cloud platform.

[0060] In one embodiment, the central scheduler can provide a change management interface where users can input maintenance change requests in natural language.

[0061] In one embodiment, after receiving an operational change request, the central scheduler can invoke a large language model to parse the request and extract structured fields (i.e., change object, change type, and change expectation). The change object refers to the object in the target business system that the user currently needs to modify; this object can be a service, component, database, or infrastructure. The change type indicates the type of change required to the object; for example, it could be a release change, configuration change, database change, or architecture change. The change expectation represents the state that the operational change request expects the object to reach.

[0062] For example, if a user inputs an operation and maintenance change request as "increase the maximum number of DB connections in the order-service from 50 to 150", then through parsing, we can determine that: the object of the change is order-service, the change type is configuration change, and the expected change is to adjust the maximum number of connections in the connection pool from 50 to 150.

[0063] The associated information of the object being changed may include at least one of the following: technology stack, version information, deployment topology, and upstream and downstream dependencies.

[0064] A change experience library can record information from historical changes made to a target business system. For example, historical change information can include at least one of the following: historical success rate, risk points (also known as typical risk points) that existed in historical changes, and corresponding risk handling templates.

[0065] Correspondingly, by combining the associated information of the changed object with the historical change information of the previous changes, the information extracted from the operation and maintenance change requirements can be supplemented, thereby improving the information content of the change specification information.

[0066] In one embodiment, the central scheduler can also query the change window rule base to determine the allowed change time window (which may be called the standard release window), i.e., a fixed period of time preset by the system that allows routine service releases, configuration adjustments, database changes, and cloud architecture transformations. Accordingly, if the operation and maintenance change request includes a change time, the central scheduler can determine whether the change time is within the standard release window to determine the final change time. If the operation and maintenance change request does not include a change time, the central scheduler can determine the final change time based on the standard release window.

[0067] In one embodiment, a central scheduler combines the queried and extracted information and performs semantic ambiguity detection on the combination result. If semantic ambiguity is determined, in response to the user's confirmation command, the scheduler determines the semantics indicated by the confirmation command and obtains the change specification information. This further improves the accuracy of the change specification information, thereby enhancing the reliability of the change.

[0068] The central scheduler combines the queried and extracted information; that is, it uses the queried information to complete the extracted information. After completion, a large language model is used to perform semantic ambiguity detection on the completed information (i.e., the combined result). Accordingly, in the case of semantic ambiguity, the large language model can also determine all possible semantic understandings and display them.

[0069] For example, the completed information includes "restart order-service". However, order-service deploys multiple Pods (container groups). Therefore, the possible semantic interpretations are a hot restart of a single Pod, a rolling restart of all Pods, or a cold restart of the entire cluster. In this case, the central scheduler can display all possible semantic interpretations, and the user can then choose a restart method from them.

[0070] In one embodiment, the operational change request input by the user also includes the potential scope of impact (e.g., a list of services the changed object directly depends on) and the level of urgency (e.g., planned / urgent / major fault repair). Correspondingly, the central scheduler can also extract this information to obtain change specification information. This allows the subsequent change design agent to refer to this information to generate the target change plan.

[0071] After receiving the change specification information, the central scheduler can send the change specification information to the change design agent. Based on the change specification information, the change design agent generates a target change plan; that is, the generated target change plan conforms to the change specification information.

[0072] In one embodiment, the change design agent queries a change experience database for historical change templates (which can be referred to as target change templates) used for the change type. It then combines these templates with the change context and change specification information to generate a target change plan. The change context represents the actual environment of the current target business system. This further improves the effectiveness of the target change plan and enhances the reliability of the change process.

[0073] The change experience library records solution templates used in historical changes. For example, the overall change logic of the change solutions used in historical changes can be extracted to obtain solution templates. For example, the change context includes: the current system load of the target business system.

[0074] It is understandable that for the same type of change, there may be multiple solution templates in the change experience library.

[0075] For example, the design change agent can select the template with the highest matching degree with the change specification information from a variety of templates and use it as the target template.

[0076] For example, the change design agent can determine the template quality of multiple solution templates based on quality evaluation dimensions, and then select the template with the highest template quality as the target solution template. Quality evaluation dimensions can include at least one of the following: risk level, scope of business impact, fault recovery time, and implementation cost. Regarding risk level, templates with the highest historical success rate among similar changes, the fewest triggers of circuit breakers / failures, complete coverage of all typical risk points, and comprehensive pre-check steps are of higher quality. Regarding scope of business impact, templates with low business latency, errors, and traffic fluctuations during the change process are of higher quality. Regarding recovery time, templates that include complete rollback operation steps, whose estimated rollback time meets the business's Recovery Time Objective (RTO), and allow for batch rollback of changes without manual intervention in case of anomalies are of higher quality. Regarding implementation cost, templates with streamlined change operation steps and no redundant operations are of higher quality.

[0077] By combining the solution template, change context, and change specification information, a target change plan is generated. In other words, the solution template is adjusted based on the change context and change specification information to generate the target change plan. The change context ensures that the target change plan is effectively applicable to this change. For example, if the current system load is high, the number of update instances per batch is automatically reduced, and the observation window is extended.

[0078] After receiving the change specification information, the central scheduler can also synchronize the change specification information to the change review agent, and the change design agent can also synchronize the generated target change plan to the change review agent. Accordingly, the change review agent can review the target change plan to obtain the review result of whether it passes the corresponding review item (i.e., the target review item).

[0079] In one embodiment, the change review agent determines the corresponding review item based on the business level (referred to as the target business level) to which the change object belongs in the change specification information. For example, business levels are divided into: core business, secondary core business, and non-core business.

[0080] If the target business level is core business, the target audit items include: whether the object to be changed has architectural guarantees to eliminate single point of risk (called high availability architecture capability audit item), whether the current target change plan has a basis for secure execution (called changeability capability audit item), and whether it has recovery capability when an anomaly occurs during the execution of the current target change plan (called fast recovery capability audit item).

[0081] If the target business is classified as a secondary core business, the target audit items include: high availability architecture capability audit items and fast recovery capability audit items. In this case, the change audit agent can also detect changeable capability audit items, but this will not affect the audit results; it will only be used as a reference and suggestion.

[0082] If the target business level is non-core business, the target audit items include: quick recovery capability audit items. In this case, the change audit agent can also detect high availability architecture capability audit items and changeable capability audit items, but this will not affect the audit results; it will only be used as a reference and suggestion.

[0083] For example, the audit items for high availability architecture capabilities include at least one of the following: whether it has a multi-active / master-slave architecture, whether there is a traffic degradation path during changes (i.e., whether it supports circuit breaking, rate limiting or degradation switching), and whether the dependent downstream services have timeout protection and retry mechanisms.

[0084] The change capability review items include at least one of the following: whether it has the ability to release in batches (e.g., whether it supports batch release by business layer or traffic ratio), whether the changed content supports hot update, whether the database change is a compatibility change (e.g., backward compatibility, no table locking), and whether the change operation steps can be executed step by step.

[0085] The audit items for quick recovery capability include at least one of the following: whether the rollback operation steps are complete and effective, whether key configurations are automatically backed up with snapshots before changes, and whether there is a one-click rollback automation script and the rollback duration meets the business's RTO requirements.

[0086] Based on the above-mentioned target audit items, the automated production readiness audit using three capability dimensions significantly improves audit efficiency and avoids omissions and inconsistencies caused by inconsistent standards in manual audits. The audit conclusions are interpretable and auditable.

[0087] Understandably, if the target review item includes multiple items, then if the target change plan passes each review item, the change review agent will determine that the target change plan passes the corresponding target review item.

[0088] In one embodiment, the change review agent outputs a review report containing the results of each test. If all items pass, the review is approved, indicating that the target change plan is ready for production. Subsequently, the central scheduler can activate the change execution agent to execute the approved target change plan. If any item fails the review, the review is rejected, and the reasons for rejection and modification suggestions are returned to the change design agent. Accordingly, the change design agent can update the current target change plan based on the modification suggestions to obtain a new target change plan.

[0089] Understandably, after the design change agent generates a new target change plan, the change review agent can review the new target change plan again, and so on, until the latest generated target change plan passes the review.

[0090] The change execution agent executes the change operation steps in the target change plan that have passed the target approval item on the business instances under the change object.

[0091] For example, if the object being changed is the order microservice (order-service), then the business instances under the changed object can be the Pod instances providing that service. If the object being changed is a Redis caching component, then the business instances under the changed object are the master and slave node instances of each shard in the cache cluster. If the object being changed is a database, then the business instances under the changed object are the running instances of the database master and slave databases.

[0092] In one embodiment, the target change scheme further includes: a rollback operation step corresponding to the change operation step, an indicator verification step for key indicators after the change operation step, and a canary mechanism used when executing the change operation step; a change execution agent, specifically used to execute the indicator verification step after executing the change operation step for any batch of business instances according to the canary mechanism; if the verification passes, execute the change operation step for the next batch of business instances, and if the verification fails, execute the rollback operation step for the batch of business instances. This improves the stability of the target business system during the change process.

[0093] Key metrics are those that are associated with changes in operational procedures; that is, changes in operational procedures will affect the values ​​of key metrics.

[0094] The indicator verification step involves verifying whether the key indicators (KPIs) have reached their expected values ​​within an observation window. In other words, the target change plan records the key indicators, the duration of the observation window, and the expected values. Furthermore, considering the volatility of business operations, the target change plan can also record acceptable ranges. Correspondingly, if the KPI value falls within the acceptable range of the expected value, the KPI is considered to have reached its expected value. It is understandable that there can be multiple key indicators, and for each key indicator, the target change plan records the corresponding observation window duration and expected value.

[0095] For example, if the change involves an order microservice, the key metric is the average response time of the service interface, with an expected value of ≤80ms (milliseconds), an acceptable range of 80ms to 120ms, and an observation window of 5 minutes. If the change involves a database, the key metric is the query rate per second (QPS) of a single database table, with an expected value of ≥2000 queries / second, an acceptable range of 1800 to 2400 queries / second, and an observation window of 3 minutes.

[0096] That is, after performing change operation steps on a batch of business instances, the change execution agent continuously collects the values ​​of key indicators. When all key indicators in the observation window are within the acceptable range, the change operation steps are deemed to have passed the verification and can be performed on the next batch of business instances. If there are key indicators in the observation window that are not within the acceptable range, the change operation steps are deemed to have failed the verification and a rollback process is executed.

[0097] In one embodiment, business instances can be divided into batches according to business layers. For example, business instances can be divided into core / secondary core / non-core, and the corresponding canary mechanism can be: non-core business instances are the first batch, secondary core business instances are the second batch, and core business instances are executed last, and the observation window duration of core business instances is longer than that of non-core business instances.

[0098] In one embodiment, the target change scheme also records the rollback determination of each change operation step, such as whether it can be automatically rolled back, requires manual rollback, or cannot be rolled back. It is understood that the target change scheme records corresponding rollback operation steps for change operation steps that can be automatically rolled back and those that require manual rollback.

[0099] In one embodiment, the target change plan also records the estimated impact range of each change operation step and the estimated rollback duration of the corresponding rollback operation step. Accordingly, the change review agent can determine whether the rollback duration meets the business's RTO requirements based on the estimated rollback duration, thus determining whether the target change plan passes the fast recovery capability review. The change review agent can also dynamically adjust the review intensity based on the estimated impact range. For example, if the estimated impact range is large, the review intensity is increased; if the estimated impact range is small, the review intensity is decreased.

[0100] In one embodiment, the change management system further includes: a change risk control intelligent agent, used to detect whether the global status indicators of the target business system have reached a first circuit breaker threshold and whether the local status indicators associated with the change object have reached a second circuit breaker threshold when the change execution intelligent agent executes the target change plan, and to synchronize the detection results with the change execution intelligent agent; wherein, the local status indicators are determined based on key indicators; the change execution intelligent agent is also used to stop executing the change operation steps when the global status indicators reach the first circuit breaker threshold, and to execute rollback operation steps for the current batch of business instances when the local status indicators reach the second circuit breaker threshold, and to execute change operation steps for the next batch of business instances. Thus, through the real-time two-layer inspection (global detection and local detection) of the change risk control intelligent agent, global and local impacts are immediately perceived, and circuit breakers can be quickly triggered in case of anomalies, significantly reducing the risk of change operations escalating into failures.

[0101] In one embodiment, after the target change plan passes the corresponding target review item, the central scheduler can activate the change execution agent and the change risk control agent in parallel. Accordingly, while the change execution agent executes the target change plan, the change risk control agent can detect global and local status indicators.

[0102] For example, for global status metrics, monitor the overall core business metrics dashboard (e.g., site-wide success rate, core link P99 latency, core business QPS). The purpose is to detect whether changes trigger unexpected global ripple effects (e.g., a traffic switch for change A unexpectedly affects business B). Correspondingly, if a global status metric reaches the circuit breaker threshold (i.e., the first circuit breaker threshold), it indicates an anomaly in the global status metric, triggering a "global circuit breaker" signal. In this case, all ongoing change operations are suspended, an alert is pushed, and manual evaluation is pending.

[0103] Local status indicators can be specific indicators for the changed object and its dependent upstream and downstream nodes. These specific indicators can be determined based on the aforementioned key indicators. For example, some key indicators can be selected as specific indicators, or other indicators can be added to the key indicators to obtain specific indicators. The purpose is to verify whether the actual effect of each batch of gray-scale changes meets expectations. Correspondingly, if a local status indicator reaches the circuit breaker threshold (i.e., the second circuit breaker threshold), it indicates that the local status indicator is abnormal, triggering a "local circuit breaker" signal. In this case, the execution of change operation steps for the current batch of business instances is suspended, and the corresponding rollback operation steps are executed. Then, the change operation steps can be executed for the next batch of business instances. That is, it does not affect other batches.

[0104] In one embodiment, the change risk control agent and the change execution agent synchronize data through a shared state channel, enabling the change execution agent to stop the change / rollback operation steps when the state indicator triggers the circuit breaker.

[0105] In one embodiment, the risk control agent is modified to increase the sampling frequency of the detected indicators and issue an alarm when the detected indicators reach an abnormal value but do not reach the circuit breaker threshold. This improves the timeliness of risk control management and ensures the stability of the target business system.

[0106] For the detected indicators (including global and local state indicators), the risk control agent changes the target change plan and sets corresponding outlier values ​​and circuit breaker thresholds according to the change plan document. When an indicator reaches an outlier value but does not trigger the circuit breaker threshold, an early warning signal is sent, and the detection sampling frequency is increased.

[0107] In one embodiment, the change risk control agent can also provide specific risk blocking suggestions when a circuit breaker is triggered, based on the dependencies between change operation steps.

[0108] In one embodiment, the change management system further includes: a Change Experience Agent (CEA), used to acquire verification data for the target change plan (referred to as first verification data), verification data for the review results of the change review agent (referred to as second verification data), verification data for the execution results of the change execution agent (referred to as third verification data), and verification data for the detection results of the change risk control agent (referred to as fourth verification data); generating experience entries based on the acquired verification data and providing them to users for review; wherein, an experience entry includes: the agent to be optimized, the optimization content, the type of change targeted, and the data source of the experience entry; updating the knowledge bases of the change design agent, change review agent, change execution agent, and change risk control agent using the approved experience entries. This enables the change management system to have the ability to continuously and autonomously evolve. With the accumulation of change experience, change plan recommendations become more accurate, review standards become more dynamically calibrated, risk control alarms become more sensitive, and the success rate of the same type of change becomes higher, realizing an operation and maintenance AI system that gets better with use.

[0109] In the embodiments of this application, after the change task is closed, that is, after the change execution agent finishes executing the target change plan (including all executions or only partial batches), the experience accumulation agent collects the original materials in the following dimensions: the plan design stage, the review stage, the execution stage, and the risk control stage.

[0110] The original materials from the design phase, i.e., the first verification data, can be manually verified by users to obtain this first verification data. For example, the first verification data includes: which steps in the target change plan had significant actual deviations, and which design decisions were subsequently proven effective / ineffective.

[0111] The original materials from the review phase, i.e., the second verification data, can be obtained by users manually verifying the review results of the change review agent. For example, the second verification data includes: which review items passed but still pose risks during implementation, and which review items failed but had no actual impact.

[0112] The raw materials for the execution phase, namely, the third verification data. Users can manually verify the execution results of the change execution agent to obtain the third verification data. For example, the third verification data includes: the deviation between the actual observation window and the preset window (i.e., the observation window recorded in the target change plan) of each batch of grayscale, and the actual time consumed by each step in the target change plan.

[0113] The raw data for the risk control phase, namely the fourth verification data, can be obtained by users manually verifying the detection results of the risk control agent. For example, the fourth verification data includes: which status indicators issued the first warning and which global indicators were noise in this change.

[0114] Furthermore, the experience-accumulating agent invokes a large language model to perform causal reasoning and experience abstraction on the aforementioned verification data, generating experience entries. For example, each experience entry includes: the target scenario (including the type of change and the business layer to which the changed object belongs), experience content (i.e., optimization content, including rules / precautions / recommended practices), source evidence (i.e., data source, such as specific data supporting this change), and suggested update target (i.e., the agent that needs optimization, and the type of knowledge that the agent needs to optimize).

[0115] Once experience entries are obtained, they can be displayed to users for evaluation. Users can determine the credibility of each experience entry, whether it requires further review, the reviewer responsible for review, and the subsequent processing method.

[0116] For example, after user evaluation, the evaluation result is:

[0117] Experience item 1: Grade A, credibility ≥ 0.85, no modification to existing knowledge items, no review required, automatic application, notification to the operations and maintenance manager, officially effective after 7 days without objection.

[0118] Experience Item 2: Level B, credibility of 0.65-0.85, or requires modification of existing knowledge items, requires review by the domain technical leader, and requires the submission of an experience item review form. You can then choose to accept / reject / accept after modification. The review period is 3 working days.

[0119] Experience item 3: Level C, credibility <0.65, or involves changes to the review standards / prompts, requires joint review by the operations architect and business leader, and subsequent comparison of the standards / prompts before and after the changes, and formally takes effect after clear authorization.

[0120] Subsequently, after further review, the experience-accumulating agent can obtain the approved experience items and distribute them to four different agents, driving the evolution of the entire change management system:

[0121] Level 1: Drive the evolution of the change design agent. For the next similar change, refer to this case through RAG search, automatically avoid known risks, and realize the update of the change case knowledge base.

[0122] Level 2: Drive the evolution of the change review agent, upgrade overly lenient review items to blocking items, and downgrade or refine overly strict review items to achieve updates to production readiness review standards.

[0123] Level 3: Drive the evolution of risk control intelligence, automatically incorporate proven effective early warning indicators, reduce the weight of risk control indicators proven to be noise, and realize the update of detection indicator configuration.

[0124] Level 4: Drive the evolution of reasoning ability of each agent by adding constraints or few-shot examples to the prompts to improve the reasoning bias of each agent in specific scenarios (e.g., requiring approval from a specific level of user before it can take effect), and realize the prompts of the agents.

[0125] In one embodiment, historical change information includes: the historical success rate of historical changes; the target change plan further includes: the predicted success rate of the target change plan; the change review agent is further used to dynamically adjust the target review items corresponding to the current target change plan based on the predicted success rate; the change risk control agent is further used to dynamically adjust the sampling frequency of the measured indicators, the corresponding outliers, and the circuit breaker threshold based on the predicted success rate. In this way, dynamic adjustment of change plan review and dynamic adjustment of risk control are achieved, improving the intelligence of change management.

[0126] In this embodiment of the application, the success rate of historical changes (i.e., historical success rate) is recorded in the change specification information. The change design agent can also combine the historical success rate to predict the success rate of the target change plan, obtain the predicted success rate, and record it in the target change plan.

[0127] Correspondingly, if the predicted success rate is lower than the first threshold, the change review agent can directly reject the current target change proposal. If the predicted success rate is not lower than the first threshold and not lower than the second threshold, the change review agent can add review items to increase the review intensity. If the predicted success rate is not lower than the second threshold, the change review agent can reduce the review items to reduce the review intensity.

[0128] If the prediction success rate is high, the risk control agent can be modified to set a lower sampling frequency, as well as strict outlier and circuit breaker thresholds; conversely, if the prediction success rate is low, the risk control agent can be modified to set a higher sampling frequency, as well as lenient outlier and circuit breaker thresholds.

[0129] In one embodiment, historical change information includes: risk points existing in historical changes and corresponding risk handling templates; key indicators are determined based on the risk points existing in historical changes (referred to as historical risk points); the target change plan further includes: pre-check steps to be performed before the change operation steps, and the existing target risk points; the pre-check steps and target risk points are determined based on the risk points existing in historical changes and the corresponding risk handling templates; the target review items further include: risk point review items; the change review intelligent agent is also used to detect whether the current target change plan contains risk handling operations for the target risk points, so as to determine whether the current target change plan passes the risk point review items. Thus, through pre-check steps, the stability of the target business system during the change process is improved.

[0130] In this embodiment, after obtaining historical risk points and corresponding risk handling templates, the change design agent determines the key indicators that need to be verified in the target change plan based on the historical risk points. In other words, historical risk points affect key indicators.

[0131] In addition, the design change agent can also determine the detection steps (i.e., pre-check steps) that need to be performed before the change operation steps based on the risk handling templates corresponding to historical risk points. In other words, the pre-check steps can check whether there are historical risk points and, if so, eliminate the risk points through the pre-check steps.

[0132] Furthermore, the change design agent can also generate risk points (i.e. target risk points) in the target change plan by combining historical risk points. Subsequently, the change review agent detects whether the target change plan includes risk handling operations for the target risk points in order to determine whether the target change plan passes the risk point review item.

[0133] For example, the pre-check steps include: dependency health checks, capacity reserve checks, and conflict change detection. Dependency health checks automatically verify whether the upstream and downstream dependent services, middleware, databases, and third-party interfaces of the object being changed are functioning correctly, preventing a chain reaction of crashes caused by dependency failures compounded by the change itself. Capacity reserve checks check the remaining resources and traffic capacity of the cluster to determine whether the target business system can handle the new version expansion and traffic fluctuations, preventing resource exhaustion and service avalanche after the change. Conflict change detection determines whether there are operational conflicts, resource preemption conflicts, or mutually exclusive data structure modifications between the change plans of multiple objects, prohibiting the simultaneous execution of changes to multiple conflicting objects.

[0134] In one embodiment, if conflicting changes are detected, the change review agent can provide execution suggestions, namely, the order of changes among multiple change objects. Then, the central scheduler can execute the change processing for each change object sequentially according to the determined change order.

[0135] In one embodiment, the target change scheme further includes: a post-verification step; a change execution agent, which is further configured to execute the post-verification step after performing the change operation steps on all batches of business instances; if the verification passes, the change is determined to be successful; otherwise, the change is determined to be unsuccessful. This provides a global fallback verification to validate the validity of the change and reduce the probability of missed change failures.

[0136] The post-verification step can verify the values ​​of some indicators to determine whether these indicators have reached the expected values; if they have reached the expected values, it indicates that the change was successful; otherwise, the change was unsuccessful.

[0137] For example, if the object being changed is an order microservice, the post-verification step can verify the end-to-end success rate of the entire order chain, as well as the total amount of order messages accumulated across the entire cluster.

[0138] In one embodiment, the central scheduler maintains and modifies the task state machine, see [link to relevant documentation]. Figure 2 The state machine includes the following state nodes: change request reception (i.e., obtaining user-inputted operation and maintenance change requirements), specification collection (i.e., generating change specification information), solution design, user confirmation, under review, concurrent execution and risk control, rollback of the current batch, change closure, experience accumulation, experience review and storage, and manual intervention.

[0139] If the user confirms the plan is approved, it enters the review process; if the review fails, it is rejected, and the plan design process restarts. During execution, if a partial circuit breaker is triggered, only the current batch is rolled back; if a global circuit breaker is triggered, manual intervention is required. After approval, plan execution and risk control are handled concurrently.

[0140] Correspondingly, the core state transition constraint rules include: If specifications are not fully collected, entry into solution design is prohibited; if the solution has not been confirmed by the user, entry into production-ready review is prohibited; if any blocking item fails the review, the process returns to the change design agent for correction; after the review is approved, the change execution agent and the change risk control agent are simultaneously activated by the central scheduler and run concurrently; if any batch verification failure triggers a circuit breaker, only the affected batches are rolled back, without needing to return to the solution design stage; if a global circuit breaker is triggered in the risk control stage, all change operations are suspended, awaiting manual intervention; if a state timeout or unrecoverable error occurs in any stage, escalation to manual processing is triggered.

[0141] Example 1: Taking the operational change requirement of "adjusting the maximum number of database connection pools in the production environment order-service from 50 to 150" as an example, this application provides a detailed explanation of the implementation process of the embodiment.

[0142] Step 1: Spec-Driven Change Specification Collection. The user inputs an operational change request: "Increase the maximum number of DB connections in the order-service from 50 to 150, and deploy it tonight." The specification collection module extracts the following fields: Change Object = order-service, Change Type = Configuration Change, Change Expectation = Maximum number of connections in the connection pool adjusted from 50 to 150. Completion: Querying the CMDB, order-service belongs to the core business line, deployed on 8 Pods, and depends on order-db (MySQL master database); querying the change window rule base, the standard release window is 22:00-06:00 daily. The change management system generates a clarification question: Does "deploy tonight" for the change time mean execution within tonight's standard window? After user confirmation, a standardized change specification document (i.e., change specification information) is generated, marked as "Core Business - Configuration Change - Planned."

[0143] Step 2: Change Plan Design. The change design agent retrieves recommended solutions for connection pool configuration changes from the change experience library and generates a change plan: The canary deployment strategy involves first hot-updating the configuration on two non-core traffic Pods (without restarting), observing for 15 minutes, and then expanding to all eight Pods. Each rollback operation is clearly defined (dynamically restoring environment variables to their original values). Verification points are connection pool wait time P99 and HTTP 503 error rate. The user confirms the plan.

[0144] Step 3: Production Readiness Review. The change review agent determines that the change pertains to core business operations and performs a rigorous review. High Availability Architecture Capability Review Item: order-service has multi-Pod deployment and load balancing; review result: passed. Changeability Review Item: Supports dynamic adjustment of connection pool parameters at runtime; review result: passed. Fast Recovery Capability Review Item: Rollback operations only require updating environment variables and then a rolling restart; review result: passed. Review Report: All three blocking items passed; the change plan is approved.

[0145] Step 4: Concurrent collaboration between the change execution agent and the change risk control agent. The central scheduler synchronously activates both the change execution agent and the change risk control agent. The change execution agent executes the change in a phased, gradual, and incremental manner according to business needs. The change risk control agent monitors global status metrics (overall order success rate, API gateway latency) and local status metrics (order-service connection pool wait time, active_connections trend) in real time. After the first batch of two Pods completes execution and the metrics are observed and verified to be normal, the second batch is expanded to all eight Pods.

[0146] During execution, the risk control agent detected a sudden spike in CPU usage on the order-db server and a slight increase in site-wide order latency (P99), which did not reach the circuit breaker threshold, triggering an alert. The risk control alert was flagged and the operations team was notified. Ten minutes later, the relevant metrics returned to normal, and the risk control alert was lifted. The entire batch of changes was executed successfully, and the effect was verified, confirming a significant decrease in connection pool waiting time and normal order-service response.

[0147] Step 5: Experience Accumulation and Agent Evolution. After the change is closed, the experience accumulation agent collects materials from the entire change process: no deviations in the solution design stage; no omissions of risk points in the review stage; data accumulation in the observation window during the execution stage: the actual stabilization period of the first batch of connection pool parameters adjusted from 50 to 80 was about 8 minutes (shorter than the preset 15 minutes); risk control stage: the instantaneous increase in order-db CPU is a known effect and a harmless temporary phenomenon.

[0148] Generate experience entries and determine their confidence levels: Experience entry A (confidence level 0.92): "The first batch of hot updates for order-service connection pool configuration changes had an actual stabilization period of approximately 8 minutes. It is recommended that the initial observation window for subsequent similar changes be adjusted to 10 minutes." Recommended update target: Change scheme template library. Classified as Level B.

[0149] Experience item B (credibility 0.95): "Scaling up the order-service connection pool will cause a momentary increase of about 5% in order-db CPU usage, which will stabilize within about 10 minutes. This is a normal phenomenon and not a risk signal." Recommended update target: Risk control detection indicator configuration (mark this as a non-risk transient effect). Classified as Level A (automatically effective).

[0150] Experience A was submitted to the database team's technical lead for review and approved two days later. The observation window for the corresponding connection pool change template in the solution design knowledge base was updated from 15 minutes to 10 minutes. Experience B was automatically added to the risk control knowledge base. The next time a similar change occurs, the change risk control agent will no longer trigger an alert due to a sudden increase in order-db CPU usage; instead, the change design agent will recommend an initial observation window of 10 minutes.

[0151] Example 2: Conflict Detection Capabilities of the Change Management System in Concurrent Change Scenarios. One night, during a window, two change requests are received simultaneously: Change A: release of the order-service version; Change B: optimization of the order-db index. During the review phase, the change review agent detects an overlap between the impact domain of Change A (order-service cluster + order-db dependency) and the impact domain of Change B (order-db master database), triggering a conflict alert. The recommended strategy of the change management system is: execute Change B (database change, higher risk) first, and then initiate the execution process of Change A after the order-db metrics stabilize (approximately 15-20 minutes). After the user confirms this execution strategy, the central scheduler automatically orchestrates the execution sequence, executing Change B first and closing the loop before initiating the specification collection and process for Change A.

[0152] See Figure 3 The change management system receives natural language input from users (operational change requests). The central scheduler then collects specifications, including field completion and ambiguity detection, to obtain change specification information. The change design agent generates a change plan based on this specification information. The change review agent reviews the change plan based on the specification information; if the review fails, it can be rejected, and modification suggestions are provided. The change design agent updates the change plan according to the modification suggestions. If the change plan passes review, execution and risk control occur concurrently; that is, the central scheduler simultaneously activates the change execution agent and the change risk control agent to monitor status indicators while executing the change plan. Subsequently, after the change loop is closed, the experience accumulation agent generates experience entries, driving the evolution of each agent.

[0153] See Figure 4 The business layer includes: change plan generation, change plan review, change plan execution, and change review (i.e., generating experience items based on verification data).

[0154] Architecture layer: Multi-agent collaborative architecture, including: central scheduler, change design agent, change review agent, change execution agent, change risk control agent, and experience accumulation agent.

[0155] The infrastructure and data layer includes: a large language model, a change experience library, a configuration management asset library, a change control platform, a change automation system, and detection tools.

[0156] The change management platform is the infrastructure that hosts the aforementioned intelligent agents. Through the application programming interface (API) provided by the change automation system, data from the configuration management asset repository and change experience repository can be obtained. Monitoring tools (such as Grafana) are used to obtain real-time values ​​for various metrics.

[0157] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0158] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0159] The various embodiments in this specification are described in a related manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0160] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A change management system, characterized in that, The system includes: The central scheduler is used to obtain the operation and maintenance change requirements for the target business system input by the user, and extract the change object, change type and change expectation from the operation and maintenance change requirements; query the association information of the change object from the configuration management asset library, and query the historical change information of the change type from the change experience library, and combine the queried information with the extracted information to obtain change specification information; A design change agent is used to generate a target change plan based on the change specification information; The change review intelligent agent is used to detect whether the current target change plan passes the target review item corresponding to the business level of the change object based on the change specification information, and generate modification suggestions for the review items that fail the review; wherein, the target review item includes at least one of the following: whether the change object has an architecture guarantee to eliminate single point of failure risk, whether the current target change plan has a basis for secure execution, and whether it has recovery capability when an anomaly occurs in the execution of the current target change plan; The design change agent is also used to update the current target change plan based on the modification suggestions; The change execution agent is used to execute the change operation steps in the target change plan that passes the target approval item on the business instances under the change object.

2. The system according to claim 1, characterized in that, The change design agent is specifically used to query the historical change templates used for the change type from the change experience library, and generate the target change plan by combining the plan templates, change context and change specification information. The change context represents the current real environment of the target business system.

3. The system according to claim 1, characterized in that, The target change plan also includes: the rollback operation steps corresponding to the change operation steps, the indicator verification steps for key indicators after the change operation steps, and the gray-scale mechanism used when executing the change operation steps. The change execution agent is specifically used to execute the indicator verification step after completing the change operation steps for any batch of business instances according to the grayscale mechanism; if the verification is successful, execute the change operation steps for the next batch of business instances; and if the verification fails, execute the rollback operation steps for the batch of business instances.

4. The system according to claim 3, characterized in that, The system further includes: a change risk control intelligent agent, used to detect whether the global status indicator of the target business system has reached a first circuit breaker threshold and whether the local status indicator associated with the change object has reached a second circuit breaker threshold when the change execution intelligent agent executes the target change plan, based on the target change plan, and to synchronize the detection results with the change execution intelligent agent; wherein, the local status indicator is determined based on the key indicator; The change execution agent is also used to stop executing the change operation steps when the global status indicator reaches the first circuit breaker threshold, and to execute the rollback operation steps for the current batch of service instances and execute the change operation steps for the next batch of service instances when the local status indicator reaches the second circuit breaker threshold.

5. The system according to claim 4, characterized in that, The modified risk control intelligent agent is also used to increase the sampling frequency of the detected indicators and issue an alarm when the detected indicators reach an abnormal value but do not reach the circuit breaker threshold.

6. The system according to claim 4, characterized in that, The system also includes: An experience-accumulating intelligent agent is used to acquire verification data for the target change plan, verification data for the review results of the change review intelligent agent, verification data for the execution results of the change execution intelligent agent, and verification data for the detection results of the change risk control intelligent agent. Experience entries are generated based on the obtained verification data and provided to users for review; each experience entry includes: the agent that needs optimization, the content to be optimized, the type of change targeted, and the data source of the experience entry; The knowledge bases of the change design agent, change review agent, change execution agent, and change risk control agent are updated using the approved experience entries.

7. The system according to claim 4, characterized in that, The historical change information includes: the historical success rate of the historical changes; the target change plan also includes: the predicted success rate of the target change plan; The change review intelligent agent is also used to dynamically adjust the target review items corresponding to the current target change plan based on the predicted success rate. The modified risk control agent is also used to dynamically adjust the sampling frequency, corresponding outliers, and circuit breaker thresholds of the detected indicators based on the prediction success rate.

8. The system according to claim 3, characterized in that, The historical change information includes: the risk points existing in the historical changes and the corresponding risk handling templates; the key indicators are determined based on the risk points existing in the historical changes; The target change plan also includes: a pre-check step that needs to be performed before the change operation step, and the target risk points; the pre-check step and the target risk points are determined based on the risk points existing in the historical changes and the corresponding risk handling templates; The target audit items also include: risk point audit items; The change review agent is also used to detect whether the current target change plan includes risk management operations for the target risk point, so as to determine whether the current target change plan passes the risk point review item.

9. The system according to claim 3, characterized in that, The target change plan also includes: a post-verification step; The change execution agent is also used to perform a post-verification step after the change operation steps have been completed for all batches of business instances; if the verification is successful, the change is determined to be successful; otherwise, the change is determined to be unsuccessful.

10. The system according to claim 1, characterized in that, The central scheduler is specifically used to combine the queried information with the extracted information, and to perform semantic ambiguity detection on the combination result. If semantic ambiguity is determined, in response to the user's confirmation instruction, the semantics indicated by the confirmation instruction are determined, and change specification information is obtained.

Images