Network operation and maintenance command decision method and device based on double-path feedback, medium

Abstract

The application discloses a network operation and maintenance command decision method and device based on double-path feedback, and a medium, the method comprising: obtaining a current decision context comprising a natural language operation and maintenance request, a current SONiC device observation state and a historical interaction record; encoding the natural language operation and maintenance request into a retrieval query, and obtaining operation memory data from a knowledge base according to the retrieval query; generating a candidate command set through a first large language model based on the current decision context and the operation memory data; for each candidate command, predicting a corresponding predicted execution result through a second large language model in combination with the current decision context; obtaining a task benefit and an operation risk corresponding to each candidate command according to the predicted execution result, and screening a final command based on the balance relationship between the two; executing the final command in a SONiC environment, and generating a task success marker according to an actual execution result.

Description

Technical Field

[0001] This invention belongs to the field of network automated operation and maintenance technology, and particularly relates to a network operation and maintenance command decision-making method, device, and medium based on dual-path feedback. Background Technology

[0002] As network equipment scales up and network architectures become increasingly complex, network operation and maintenance tasks are gradually evolving towards automation and intelligence. In open network operating systems and programmable network device environments, operations such as network interface queries, routing protocol status checks, network configuration modifications, and fault diagnosis typically rely on command-line interfaces. Because the operating status of network devices is dynamically changing, network operation and maintenance commands not only need to meet syntactic specifications but also need to match the current device status, network objects, and historical interaction processes. Otherwise, command execution may fail, operation and maintenance tasks may not be completed, or even unexpected network status changes may occur.

[0003] In recent years, with the development of large language model technology, intelligent operation and maintenance agents based on large language models have been gradually applied to network command generation scenarios. Existing intelligent operation and maintenance systems typically generate corresponding commands or configuration operations based on the natural language operation and maintenance requirements input by users, thereby reducing the difficulty of manually writing complex commands and improving network operation and maintenance efficiency.

[0004] However, existing network operation and maintenance command decision-making methods based on large language models still have certain shortcomings. On the one hand, existing methods typically treat the network operation and maintenance command generation process as a text generation task, mainly relying on language model output probabilities, text similarity, or preset rules to determine the target command, lacking sufficient consideration of the actual execution effect of the command under the current device state. Therefore, even if the generated command is grammatically correct, it may fail to complete the target task due to mismatches in device environment, operation object, or task stage, and may even generate potential operational risks.

[0005] On the other hand, existing intelligent operation and maintenance methods typically focus on the correctness of command generation, configuration generation, or the final configuration result, while rarely utilizing the execution feedback of commands in the actual network environment to continuously optimize subsequent decision-making processes. When the preferred command generated by the agent fails to execute, the reasons for the failure may include: the candidate command generation stage did not cover feasible commands that can complete the task, or although there are feasible commands in the candidate command set, the command selection and sorting mechanism failed to accurately identify their execution effects, leading to the incorrect selection of commands with low success rates or high risks.

[0006] Furthermore, existing technologies typically lack effective mechanisms for accumulating experience from failed execution cases, making it impossible to structurally utilize erroneous commands, device feedback information, and successful recovery solutions from historical execution processes. This means that intelligent operation and maintenance systems may repeatedly generate the same errors when faced with similar tasks, making it difficult to achieve continuous learning and performance improvement.

[0007] Therefore, existing network operation and maintenance command decision-making methods have the following technical problems: (1) The inability to fully combine the current network device status and historical execution information to effectively predict the actual execution result of the candidate command results in the case of a high probability of execution failure of the generated command; (2) The lack of a mechanism for evaluating the execution effect and risk analysis of multiple candidate commands makes it difficult to balance the task benefits and operational risks, resulting in insufficient accuracy in selecting the preferred command; (3) The inability to effectively utilize actual execution feedback to optimize subsequent command generation and command selection processes results in the intelligent operation and maintenance system lacking continuous learning capabilities and making it difficult to accumulate network operation and maintenance experience as it runs.

[0008] Therefore, there is an urgent need to provide a method that can combine actual execution feedback to collaboratively optimize the network operation and maintenance command generation and command selection process, so as to improve the command execution success rate, security and continuous improvement capability in automated network operation and maintenance tasks. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides a network operation and maintenance command decision-making method, device, and medium based on dual-path feedback.

[0010] In a first aspect, embodiments of the present invention provide a network operation and maintenance command decision-making method based on dual-path feedback, the method comprising: Obtain the current decision context; the current decision context includes: natural language operation and maintenance request, current observation status of SONiC device, and historical interaction records; The natural language operation and maintenance request is encoded into a retrieval query, and a number of operation memory data are retrieved from the knowledge base based on the retrieval query; In response to the current decision context and operational memory data, the first language model outputs a set of candidate commands; For each candidate command in the candidate command set, the prediction execution result of each candidate command under the current observation state of the SONiC device is predicted by combining the current decision context and the second largest language model; Based on the predicted execution results, obtain the task benefits and operational risks corresponding to each candidate command; based on the balance between task benefits and operational risks, select the final command from the candidate command set; The final command is executed in the SONiC environment to obtain the actual execution result; and a task success marker is generated based on the current decision context and the actual execution result.

[0011] In a second aspect, embodiments of the present invention provide an electronic device, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to execute the network operation and maintenance command decision method based on dual-path feedback described above.

[0012] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described network operation and maintenance command decision method based on dual-path feedback.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention provides a network operation and maintenance command decision-making method based on dual-path feedback. First, by acquiring the current decision context, including natural language operation and maintenance requests, the current observation status of the SONiC device, and historical interaction records, the command decision-making process can comprehensively consider user operation and maintenance needs, real-time device status, and historical operation information, thus avoiding the problem of command mismatch with the actual network environment caused by generating commands based solely on text semantics in the prior art.

[0014] Secondly, this invention encodes natural language operation and maintenance requests into retrieval queries, retrieves operation memory data from the knowledge base based on the retrieval queries, and then generates a candidate command set by the first language model in combination with the operation memory data and the current decision context. This enables the system to use historical operation and maintenance experience to assist the current command generation process and increase the probability that the candidate command set contains executable commands.

[0015] Furthermore, for each candidate command in the candidate command set, the present invention uses a second language model combined with the current device status to predict the corresponding prediction execution result. Based on the prediction execution result corresponding to the candidate command, the task benefits and operational risks are obtained, and the final command is selected based on the balance between task benefits and operational risks. This enables the reduction of potential operational risks while meeting the operation and maintenance task objectives, thereby improving the security and reliability of automated network operation and maintenance processes.

[0016] Finally, the present invention executes the final command obtained through filtering in the SONiC environment and generates a task success marker based on the actual execution result, so that the execution feedback during the actual operation process can be effectively recorded, providing a data foundation for subsequent command decision optimization and continuous learning, thereby improving the adaptability of the network operation and maintenance intelligence agent during long-term operation. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of a network operation and maintenance command decision-making method based on dual-path feedback provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a network operation and maintenance command decision-making method based on dual-path feedback provided in an embodiment of the present invention; Figure 3 A schematic diagram of dual-path feedback provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0021] like Figure 1 and Figure 2 As shown, this embodiment of the invention provides a network operation and maintenance command decision-making method based on dual-path feedback, the method comprising: Step S1: Obtain the current decision context; the current decision context includes: natural language operation and maintenance request, current observation status of SONiC device, and historical interaction records.

[0022] Specifically, in the t-th round of interaction, the current decision context is denoted as... ,in, Indicates the first The user submits a natural language operation and maintenance request. Indicates the first The current observation status of the SONiC device in the wheel interaction, This represents the historical interaction records up to round t. SONiC device observation status. Accessed via SSH (Secure Shell). Historical interaction log. This includes at least the natural language operation and maintenance requests submitted by users in previous rounds, the actual executed commands, and the results returned by the SONiC device.

[0023] Step S2: Encode the natural language operation and maintenance request into a retrieval query, and retrieve some operation memory data from the knowledge base based on the retrieval query.

[0024] Among them, several operational memory data are retrieved from the knowledge base based on the search query. The process includes: ; In the formula, This represents a text embedding function. Represents the similarity function. This represents the retrieval key for the i-th operation's memory data. Indicates according to the first Natural language operation and maintenance requests in round-robin interaction The constructed search query key, R represents the amount of operation memory data, m i This represents the data to be stored in the i-th operation.

[0025] In one implementation, the text embedding function Using the open-source Sentence-BERT model, and Convert to a fixed-length vector. This example calculates the similarity between the search query key and the search keys of each memory entry, and selects the R memory entries with the highest similarity to form a vector. This is used as a prompt word in the first major language model; if no memory entry that meets the preset similarity criteria is found, Empty.

[0026] Step S3: In response to the current decision context and operational memory data, the first large language model outputs a set of candidate commands.

[0027] Specifically, the current decision context and operational memory data Generate a set of candidate commands, denoted as ={ }.in, This represents the k-th candidate command in the t-th round of interaction. The first major language model outputs the result as a structured list. Each candidate operation is a complete and distinct one, and each candidate command is either an independently executable SONiC command or a sequence of short commands arranged in execution order. In this example, .

[0028] Step S4: For each candidate command in the candidate command set, combine the current decision context and use the second language model to predict the predicted execution result of each candidate command under the current observation state of the SONiC device.

[0029] Specifically, the expression is as follows: In the formula, Indicate candidate command Predicted execution consequences under the current observation conditions of the SONiC device. This represents the parameters of the second largest language model in round t. The predicted execution consequences include at least the expected command output, expected error messages, and expected state changes, and can be further categorized into consequence types such as successful execution, syntax error, unsupported command, non-existent operation object, or incomplete task.

[0030] The fine-tuning process of the second largest language model includes: Training data is selected from the historical operation and maintenance records of SONiC devices; each training data entry includes the decision context when executing a command, the candidate command actually executed, and the actual execution result returned by the SONiC device. The second language model is supervised and fine-tuned by taking the decision context and the candidate commands actually executed as inputs and the actual execution results as outputs, so that it learns the correspondence between the observation state of the SONiC device, the candidate commands and the actual execution results.

[0031] In this example, the open-source Qwen3-0.6B model is selected as the second largest language model, and the AdamW optimizer is used with an initial learning rate set to... The maximum input length was set to 2048 words, and supervised training was performed for 3 rounds. The model parameters obtained after training were used as the initial parameters of the consequence predictor. It is used to predict the execution results of candidate commands.

[0032] Step S5: Based on the predicted execution results, obtain the task benefits and operational risks corresponding to each candidate command; based on the balance between task benefits and operational risks, select the final command from the candidate command set.

[0033] Specifically, the predicted execution result of each candidate command under the current observation state of the SONiC device is input into the first feedforward neural network and the second feedforward neural network, respectively, to obtain the task benefit and operational risk corresponding to each candidate command; the expression is as follows: in, This indicates the task reward corresponding to the candidate command. This indicates the operational risk associated with the candidate command. This represents the first feedforward neural network. This represents the second feedforward neural network.

[0034] In one implementation, the intermediate layer dimension is set to 128, the activation function is the ReLU function, and the output layer uses the Sigmoid function. A larger output from the first feedforward neural network indicates that the predicted execution result of the candidate command is more conducive to completing the current operational task; a larger output from the second feedforward neural network indicates that the candidate command is more likely to cause execution failure, non-target state changes, or service interruption. The first and second feedforward neural networks are trained using 300 historical execution records containing manually labeled task benefit and operational risk labels, and the network parameters are updated based on the difference between the predicted score and the corresponding labeled score. During session-level low-rank adaptation, the parameters of the first and second feedforward neural networks remain unchanged. The manually labeled task benefit and operational risk labels are normalized to between 0 and 1, and the parameters of the two evaluation networks are updated during training based on the mean squared error between the network output score and the corresponding manually labeled score.

[0035] The process of selecting the final command from the candidate command set based on the balance between task benefits and operational risks includes: ; In the formula, This represents the final command in the t-th round of interaction. Represents the set of candidate commands. This represents the k-th candidate command in the candidate command set during the t-th round of interaction. Indicate candidate command Corresponding task rewards Indicate candidate command The corresponding operational risk, where λ represents the risk penalty coefficient.

[0036] Step S6: Execute the final command in the SONiC environment to obtain the actual execution result; and generate a task success marker based on the current decision context and the actual execution result.

[0037] Specifically, the final order The command is submitted to the SONiC environment for execution via SSH (Secure Shell protocol), and the SONiC environment returns the actual execution result. Based on the current decision context and actual implementation consequences Successful task generation flag .

[0038] in, This represents the final command in round t. The actual feedback generated by the SONiC device after execution includes normal command output, error messages, and observable changes in device status; This represents the final command in round t. The execution was marked as successful. When the final command... When the SONiC device reaches the target state corresponding to the current task after execution, ;otherwise, For status query tasks, determine whether the actual execution result includes the target object and status information requested by the user; for configuration or fault recovery tasks, determine the final command... After execution, re-query the relevant device status to determine if the query results meet the user-specified target status. If the command returns an error, target information is missing, or the status after execution does not meet the task requirements, the success flag is set to 0.

[0039] Furthermore, the method also includes: Construct the objective function, with the following expression: ; In the formula, π represents the strategy, and T represents the total number of interaction rounds. This represents the average result of the successful execution of the final command in round t under strategy π. The objective function is decomposed into: ; In the formula, when the candidate command set If the current task contains at least one feasible command that can complete the current task, ;otherwise, ; This indicates the probability that the candidate command set generated by the largest language model contains feasible commands, thereby optimizing the largest language model; This represents the probability of selecting a feasible command as the final command and successfully executing it, given that the candidate command set already contains feasible commands. This optimizes the second large language model, the first feedforward neural network, and the second feedforward neural network.

[0040] Furthermore, such as Figure 3As shown, the method further includes: An execution record is created by combining the decision context of the current interaction round, the final command, the actual execution result, and the task success marker. , This represents the set of candidate command indices actually executed in this round; The execution records are extracted into structured operation memory data and stored in a knowledge base.

[0041] In this example, the open-source Qwen3-0.6B model is used to process the execution records through preset prompt words, extract the operation and maintenance intent, failed commands, device error clues, failure reasons, successful recovery commands and their applicable conditions, and generate structured memory entries including memory type, operation and maintenance intent, applicable conditions and candidate generation guidance.

[0042] For example, when a failed command differs from a successful command only in the command keyword, parameter position, or scope of the operation, the memory entry records the correct command form and the incorrect forms to be avoided. New entries are written to the operation memory at preset update intervals; for content that is identical or highly similar to existing entries, its applicable conditions and execution evidence are merged to avoid duplicate storage.

[0043] Furthermore, the update process of the second largest language model includes: The low-rank parameters of the second largest language model are updated as follows: ; ; In the formula, Let be the model weights in the t-th round of interaction. The base weights for freezing, and All of these are trainable low-rank parameters. This represents the weight relative to the frozen base weight in the t-th round of interaction. The resulting low-rank weight increment; The decision context and the final command are used as inputs to the second-largest language model. The cross-entropy between the predicted execution result and the actual execution result corresponding to the final command is used as the loss function to update the low-rank parameters. and This yields the updated parameters of the second largest language model; The next round of interaction will be based on the updated parameters of the second largest language model.

[0044] In summary, this invention provides a network operation and maintenance command decision-making method based on dual-path feedback. By decoupling the candidate command generation process and the candidate command selection process, and utilizing actual execution feedback to optimize the candidate command generation path and the candidate command selection path respectively, the accuracy and reliability of network operation and maintenance command decisions are improved. This aspect constructs an execution feedback-driven dual-path continuous learning framework. On the one hand, it uses execution records to extract operational experience and update the operational memory on the candidate generation side, increasing the probability that the candidate command set contains executable commands. On the other hand, it uses execution results to update the execution consequence prediction model, enabling the system to more accurately predict the execution effects of different candidate commands. Furthermore, it ranks the candidates based on task benefits and operational risks, increasing the probability of successful execution of the final selected command.

[0045] Accordingly, this application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; and when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the network operation and maintenance command decision method based on dual-path feedback as described above. Figure 4 The diagram shown illustrates a hardware structure of any data processing-capable device for implementing the network operation and maintenance command decision-making method based on dual-path feedback, as provided in this embodiment of the invention. (Except for...) Figure 4 In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0046] Accordingly, this application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the network operation and maintenance command decision-making method based on dual-path feedback as described above. The computer-readable storage medium can be an internal storage unit of any data processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0047] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A network operation and maintenance command decision-making method based on dual-path feedback, characterized in that, The method includes: Obtain the current decision context; the current decision context includes: natural language operation and maintenance request, current observation status of SONiC device, and historical interaction records; The natural language operation and maintenance request is encoded into a retrieval query, and a number of operation memory data are retrieved from the knowledge base based on the retrieval query; In response to the current decision context and operational memory data, the first language model outputs a set of candidate commands; For each candidate command in the candidate command set, the prediction execution result of each candidate command under the current observation state of the SONiC device is predicted by combining the current decision context and the second largest language model; Based on the predicted execution results, obtain the task benefits and operational risks corresponding to each candidate command; based on the balance between task benefits and operational risks, select the final command from the candidate command set; The final command is executed in the SONiC environment to obtain the actual execution result; and a task success marker is generated based on the current decision context and the actual execution result.

2. The network operation and maintenance command decision-making method based on dual-path feedback according to claim 1, characterized in that, The process of retrieving operational memory data from a knowledge base based on a search query includes: ； In the formula, This represents a text embedding function. Represents the similarity function. This represents the retrieval key for the i-th operation's memory data. Indicates according to the first Natural language operation and maintenance requests in round-robin interaction The constructed search query key, R represents the amount of operation memory data, m i This represents the data to be stored in the i-th operation.

3. The network operation and maintenance command decision-making method based on dual-path feedback according to claim 1, characterized in that, The fine-tuning process of the second major language model includes: Training data is selected from the historical operation and maintenance records of SONiC devices; each training data entry includes the decision context when executing a command, the candidate command actually executed, and the actual execution result returned by the SONiC device. The second language model is supervised and fine-tuned by taking the decision context and the candidate commands actually executed as inputs and the actual execution results as outputs, so that it learns the correspondence between the observation state of the SONiC device, the candidate commands and the actual execution results.

4. The network operation and maintenance command decision-making method based on dual-path feedback according to claim 1, characterized in that, The process of obtaining the task benefits and operational risks corresponding to each candidate command based on the predicted execution results includes: The prediction execution result of each candidate command under the current observation state of the SONiC device is input into the first feedforward neural network and the second feedforward neural network respectively to obtain the task benefit and operational risk corresponding to each candidate command.

5. The network operation and maintenance command decision-making method based on dual-path feedback according to claim 1, characterized in that, The process of selecting the final command from the candidate command set, based on the balance between task benefits and operational risks, includes: ； In the formula, This represents the final command in the t-th round of interaction. Represents the set of candidate commands. This represents the k-th candidate command in the candidate command set during the t-th round of interaction. Indicate candidate command Corresponding task rewards Indicate candidate command The corresponding operational risk, where λ represents the risk penalty coefficient.

6. The network operation and maintenance command decision-making method based on dual-path feedback according to claim 1, characterized in that, The method further includes: The decision context, final command, actual execution result, and task success marker of the current interaction round are combined into an execution record; The execution records are extracted into structured operation memory data and stored in a knowledge base.

7. The network operation and maintenance command decision-making method based on dual-path feedback according to claim 1 or 6, characterized in that, The update process of the second largest language model includes The low-rank parameters of the second largest language model are updated as follows: ； ； In the formula, Let be the model weights in the t-th round of interaction. The base weights for freezing, and All of these are trainable low-rank parameters. This represents the weight relative to the frozen base weight in the t-th round of interaction. The resulting low-rank weight increment; The decision context and the final command are used as inputs to the second-largest language model. The cross-entropy between the predicted execution result and the actual execution result corresponding to the final command is used as the loss function to update the low-rank parameters. and This yields the updated parameters of the second largest language model; The next round of interaction will be based on the updated parameters of the second largest language model.

8. The network operation and maintenance command decision-making method based on dual-path feedback according to claim 1, characterized in that, The method further includes: Construct the objective function, with the following expression: ； In the formula, π represents the strategy, and T represents the total number of interaction rounds. This represents the average result of the successful execution of the final command in round t under strategy π. The objective function is decomposed into: ； In the formula, when the candidate command set If the current task contains at least one feasible command that can complete the current task, ;otherwise, ; This indicates the probability that the candidate command set generated by the largest language model contains feasible commands; This represents the probability of selecting a feasible command as the final command and executing it successfully, given that the candidate command set already contains feasible commands.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to execute the network operation and maintenance command decision method based on dual-path feedback as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the network operation and maintenance command decision method based on dual-path feedback as described in any one of claims 1-7.

Images