Operation and maintenance data processing method, system, electronic device and storage medium

Abstract

The present application discloses an operation and maintenance data processing method, system, electronic device and storage medium. It relates to the field of cloud computing technology; wherein, the method comprises: obtaining a configuration change instruction, the configuration change instruction is used to change the configuration information of the operation and maintenance data processing node to be changed, the configuration change instruction includes part of the data in the data stream corresponding to the current operation and maintenance operation, and the configuration change instruction is transmitted along with the data stream in each operation and maintenance processing node in the operation and maintenance operation end; when the current operation and maintenance data processing node meets the configuration change conditions, the configuration information is changed based on the configuration change instruction. The embodiment of the present application can solve the technical problem of low efficiency of the configuration corresponding to the detection object and the network cluster by manually changing the detection object in the related technology.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to an operation and maintenance data processing method, system, electronic device and storage medium. Background Art

[0002] In the operation and maintenance scenarios of large-scale cloud network environments, a large number of time series indicator calculation jobs need to be run in real time to detect massive devices, clusters, and virtualized instance objects in real time. Cloud computing scenarios are usually accompanied by frequent online and offline devices and the creation and destruction of network instance objects. For the constantly changing detection objects and network clusters, traditional time series indicator calculation jobs require manual changes to the configurations of the detection objects and network clusters, such as pointing the data source (Source) to the new network cluster and detection object. This change operation method not only consumes a lot of manpower, but also requires stopping the operation and maintenance operations first, making configuration changes, and then restarting the operation and maintenance operation process. This not only leads to low efficiency of configuration changes, but also affects the continuity of the operation and maintenance operations of the time series indicator calculation and the stability of the operation and maintenance system. Summary of the invention

[0003] In view of the above problems, the present application provides an operation and maintenance data processing method, device, electronic device and storage medium to at least solve the technical problem of low configuration efficiency of manually changing detection objects and network clusters in related technologies.

[0004] According to a first aspect of an embodiment of the present application, a method for processing operation and maintenance data is provided, the method comprising: obtaining a configuration change instruction, the configuration change instruction being used to change configuration information of an operation and maintenance data processing node whose configuration is to be changed, the configuration change instruction comprising part of the data in a data stream corresponding to a current operation and maintenance job, the configuration change instruction being transmitted along with the data stream among each operation and maintenance processing node in an operation and maintenance job end; and performing a configuration information change based on the configuration change instruction when the current operation and maintenance data processing node meets configuration change conditions.

[0005] According to the second aspect of an embodiment of the present application, there is provided an operation and maintenance data processing method, which is applied to a configuration control end, and the method comprises: in response to a change in configuration information of an operation and maintenance data processing node associated with the configuration control end, generating a configuration change instruction according to the changed configuration information; sending the configuration change instruction to a data source node of an operation and maintenance operation end, and the configuration change instruction is used to be embedded as a preset control field in a data stream corresponding to the operation and maintenance operation, so that the operation and maintenance data processing node changes the configuration information according to the configuration change instruction.

[0006] According to a third aspect of an embodiment of the present application, an operation and maintenance data processing system is provided, which is configured with a management and control end and an operation and maintenance operation end; The configuration control terminal is used to generate a configuration change instruction according to the changed configuration information in response to a change in the configuration information of the operation and maintenance data processing node associated with the configuration control terminal; the configuration change instruction is used to change the configuration information of the operation and maintenance data processing node associated with the configuration control terminal; and the configuration change instruction is sent to the operation and maintenance operation terminal; The operation and maintenance operation end is used to receive the configuration change instruction; add the configuration change instruction to the data stream corresponding to the current operation and maintenance operation, and pass the data stream carrying the configuration change instruction to the operation and maintenance data processing node in the current operation and maintenance operation; control the operation and maintenance data processing node in the current operation and maintenance operation to change the configuration information based on the configuration change instruction carried in the data stream.

[0007] According to a fourth aspect of an embodiment of the present application, there is provided an operation and maintenance data processing system, comprising a configuration control center and a stream processing engine, wherein the stream processing engine comprises a data source node, an operator node and a data output node; The configuration control center is used to obtain configuration information of the detected object from the configuration management database; the detected object includes at least one of an operation and maintenance data processing node, a cloud network device, and a virtual instance associated with the configuration control center; based on changes in the configuration information of the detected object, a configuration change instruction is generated according to the changed configuration information; and the configuration change instruction is sent to the data source node; The data source node is configured to receive the configuration change instruction; add the configuration change instruction to the data stream corresponding to the current operation and maintenance job; change the data source configuration information based on the configuration change instruction when the configuration change conditions are met; and transmit the data stream carrying the configuration change instruction to the operator node; The operator node is used to change the data source configuration information based on the configuration change instruction when the configuration change condition is met; The data output node is used to change the data source configuration information based on the configuration change instruction when the configuration change conditions are met, and send the changed configuration results of the operation and maintenance data processing nodes that meet the configuration change conditions to the configuration control center.

[0008] According to the fifth aspect of the embodiments of the present application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor runs the computer program to implement the operation and maintenance data processing method in the above embodiments.

[0009] According to a sixth aspect of the embodiments of the present application, a computer-readable storage medium is further provided, in which a computer program is stored, wherein the computer program is configured to execute the detection method in the above embodiment when running.

[0010] According to a seventh aspect of the embodiments of the present application, a computer program product includes a computer program, and the computer program is executed by a processor to perform the operation and maintenance data processing method in the above embodiments.

[0011] In the embodiment of the present application, a configuration change instruction is obtained, and the configuration change instruction is used to change the configuration information of the operation and maintenance data processing node whose configuration is to be changed. The configuration change instruction includes part of the data in the data stream corresponding to the current operation and maintenance operation, and the configuration change instruction is transmitted along with the data stream in each operation and maintenance processing node in the operation and maintenance operation end; When the current operation and maintenance data processing node meets the configuration change conditions, a method for changing the configuration information based on the configuration change instruction; compared with the method in the prior art of manually stopping the operation and maintenance operation, performing the configuration change operation, and then restarting the operation and maintenance operation process, the application embeds the configuration change instruction into the data stream of the operation and maintenance operation, so that the operation and maintenance data processing node can change the configuration information based on the configuration change instruction carried in the data stream without stopping the operation and maintenance operation. Therefore, it not only improves the continuity and stability of the operation of the operation and maintenance system, but also improves the efficiency of the configuration change of the operation and maintenance system. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Moreover, the same reference numerals are used throughout the drawings to represent the same components. In the drawings: Figure 1 is a schematic diagram of an application environment of an optional operation and maintenance data processing method according to an embodiment of the present application; Figure 2 is a flowchart of an optional operation and maintenance data processing method according to an embodiment of the present application; Figure 3 is a flowchart of another optional operation and maintenance data processing method according to an embodiment of the present application; Figure 4 is a schematic diagram of the architecture of an optional operation and maintenance data processing method according to an embodiment of the present application; Figure 5 is a flowchart of another optional operation and maintenance data processing method according to an embodiment of the present application; Figure 6is a flowchart of another optional operation and maintenance data processing method according to an embodiment of the present application; Figure 7 is a flowchart of another optional operation and maintenance data processing method according to an embodiment of the present application; Figure 8 is a schematic structural diagram of an optional operation and maintenance data processing device according to an embodiment of the present application; Fig. 9 is a schematic structural diagram of another optional operation and maintenance data processing device according to an embodiment of the present application; Fig.10 is a structural schematic diagram of an electronic device provided in an embodiment of the present application; Fig.11 is a schematic diagram of the structure of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0014] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0015] This application explains the technical terms that may be involved as follows: Cloud network: refers to the network architecture and operation and maintenance methods based on cloud computing technology, which can provide flexible and elastic network services. Time series indicators: refers to indicators used to describe time series data, including but not limited to timestamps, time intervals, time series values, etc. In the cloud network field, time series indicators are usually used to monitor the operating performance and status of online devices and instances. Dynamic configuration: refers to dynamically adjusting system parameters and configuration items as needed during system operation to optimize system performance. Spark: is a unified analytical engine for large-scale data processing.

[0016] Flink: is an open source stream processing framework, mainly used for big data processing and real-time data analysis. It can process large amounts of data streams and supports event-driven computing models. It can be used in a variety of application scenarios such as real-time data processing and machine learning.

[0017] Source: Data source node, usually used to describe the source of data in the field of real-time computing, that is, the input of the computing program, such as the collected online original traffic logs, configuration information of devices and control programs, etc. Sink: A data output node. In the field of real-time computing, it is usually used to describe the destination of data, that is, the output of the computing program, such as a data warehouse, database, cloud storage, etc. Operator: Operator node, which is usually used to describe the computing unit of data in the field of real-time computing, that is, the operator. For example, it can perform computing operations such as summing the data, finding the maximum value MAX, finding the minimum value MIN, and calculating the average AVG. It can also be the calculation of percentiles such as P90 and P50 that are unique to the network field. Pipeline: In the field of real-time computing, it is usually represented as a data processing flow, which includes the complete data processing process from Source, Operator to Sink and other nodes. CMDB: Configuration Management Database is a database used to store and manage information technology equipment and their relationships. It is usually used for information technology service management and configuration management processes.

[0018] As an optional implementation, the above operation and maintenance data processing method can be applied to, but not limited to, Figure 1In the application environment shown. The application environment may include, but is not limited to: a configuration control terminal 102, a network 108, and an operation and maintenance operation terminal 110. The above-mentioned configuration control terminal 102 includes a processor 104 and a memory 106. The processor 104 is used to generate a configuration change instruction based on the configuration information of the operation and maintenance data processing node associated with the configuration control terminal 102. The configuration information of the changed configuration information is changed; the memory 106 is used to detect the configuration information and configuration change instructions corresponding to the object, and the detection object includes each operation and maintenance data processing node of the operation and maintenance operation terminal. In addition, the operation and maintenance operation terminal 110 includes a database 112 and a processing engine 114, and the database 112 is used to store configuration change instructions. The processing engine 114 is used to add the configuration change instruction to the data stream corresponding to the current operation and maintenance operation, and pass the data stream carrying the configuration change instruction to the operation and maintenance data processing node in the current operation and maintenance operation; control the operation and maintenance data processing node in the current operation and maintenance operation, and change the configuration information based on the configuration change instruction carried in the data stream.

[0019] The specific process is as follows: the configuration control terminal 102 executes steps S102-S104, and based on the change in the configuration information of the operation and maintenance data processing node associated with the configuration control terminal 102, generates a configuration change instruction according to the changed configuration information; and sends the configuration change instruction to the operation and maintenance operation terminal 110. The operation and maintenance operation terminal 110 executes steps S106-S110, adds the configuration change instruction to the data stream corresponding to the current operation and maintenance operation, and passes the data stream carrying the configuration change instruction to the operation and maintenance data processing node in the current operation and maintenance operation; controls the operation and maintenance data processing node in the current operation and maintenance operation to change the configuration information based on the configuration change instruction carried in the data stream. Send the configuration change result to the configuration control terminal 102.

[0020] Optionally, the configuration control terminal 102 can be, for example, a virtual machine that can provide elastic computing services (ECS), a virtual server or other types of cloud computing nodes, and the network can include but is not limited to: local area network, metropolitan area network, virtual private network (VPN) and software defined network (SDN). The operation and maintenance operation terminal 110 can be a single server based on Flink or Spark architecture, or a server cluster based on Flink or Spark architecture composed of multiple servers, or a cloud server, and the cloud server includes but is not limited to a private cloud server or a public cloud server. The above is only an example, and no limitation is made in this embodiment.

[0021] As an optional implementation, Figure 2 As shown, the embodiment of the present application provides an operation and maintenance data processing method, which is applied to the operation and maintenance operation end, and includes the following steps: S202, obtaining a configuration change instruction, wherein the configuration change instruction is used to change the configuration information of the operation and maintenance data processing node whose configuration is to be changed, and the configuration change instruction includes part of the data in the data stream corresponding to the current operation and maintenance job, and the configuration change instruction is transmitted along with the data stream in each operation and maintenance processing node in the operation and maintenance job end.

[0022] Specifically, combined Figure 4 As shown, the execution subject in the embodiment of the present application can be any node in the operation and maintenance operation end. The operation and maintenance operation end receives the configuration change instruction issued by the configuration control end. The configuration control end is used to subscribe to the status or configuration change notification of the associated object from the configuration management database (CMDB). The above-mentioned associated object can be, for example, the operation and maintenance operation end. The above-mentioned operation and maintenance operation end can be at least one of the cloud network equipment, cluster or virtual instance objects; in one example, the above-mentioned associated object can be, for example, one or more operation and maintenance data processing nodes in the stream processing engine (Flink or Spark). The above-mentioned operation and maintenance operation end is used to perform time series indicator calculation operations, such as being responsible for receiving, cleaning and calculating the data collected and reported by one or more components of the cloud network (such as traffic, status, logs, etc.). Changes in the above-mentioned configuration information can be, for example, changes in the storage location of the data source, changes in the authentication and authorization credentials for reading data, changes in the network interface address when the operation and maintenance operation end communicates with an external system or service, changes in the field Schema in the operation process of the operation and maintenance operation end, etc.

[0023] In one example, it is assumed that the operation and maintenance data processing nodes associated with the configuration control end include: node A, node B and node C, node D and node E. The above-mentioned operation and maintenance operation end includes three operation and maintenance data processing nodes: node A, node B and node C; when the configuration information of at least one of nodes A, node B and node C changes, such as when the data source for node A to obtain operation and maintenance data changes, the data source node of the operation and maintenance operation end will receive a configuration change instruction from the configuration control end, which is used to update or modify the configuration information of the data source of node A.

[0024] The data stream involved in the embodiment of the present application can be a channel for transmitting data and instructions in the operation and maintenance system, and the data in the channel includes, for example, log information and detection data, etc. When the operation and maintenance operation end receives the configuration change instruction, the received configuration change instruction is integrated into the data stream corresponding to the current operation and maintenance operation, for example, these instructions are encapsulated into a format that can be processed by the data stream (such as encapsulated into a specific command field), and added to the data stream corresponding to the current operation and maintenance operation. After the addition is completed, the configuration change instruction will be transmitted in each operation and maintenance processing node in the operation and maintenance operation end according to the data stream.

[0025] S204: When the current operation and maintenance data processing node meets the configuration change condition, the configuration information is changed based on the configuration change instruction.

[0026] Specifically, in an embodiment of the present application, when the operation and maintenance data processing node receives a data stream carrying a configuration change instruction, it first extracts the configuration change instruction from the data stream, then parses the configuration change instruction, and converts the configuration change instruction into an operational configuration parameter. The operation and maintenance data processing node checks its own status to confirm whether it meets the configuration change conditions (for example, it is currently in operation and has sufficient computing resources, or the configuration parameters in the configuration change instruction include the configuration information of the current data source node). When the current operation and maintenance data processing node meets the configuration change conditions, it updates its configuration information according to the configuration change instruction. In one example, the configuration information in the above-mentioned configuration change information can be written into the memory of the operation and maintenance data processing node, so that the operation and maintenance data processing node can read the configuration directly from the memory instead of obtaining it from a remote configuration center or storage system, which can greatly reduce network delays and I / O operation time.

[0027] In one example, it is assumed that the above-mentioned operation and maintenance operation end includes three operation and maintenance data processing nodes: node A, node B and node C; when the data source for node B to obtain operation and maintenance data changes, the operation and maintenance operation end transmits the configuration change instruction for node B to node B through the data stream, and node B changes the configuration information based on the configuration change instruction carried in the data stream.

[0028] In the embodiment of the present application, a configuration change instruction is obtained, and the configuration change instruction is used to change the configuration information of the operation and maintenance data processing node whose configuration is to be changed. The configuration change instruction includes part of the data in the data stream corresponding to the current operation and maintenance operation, and the configuration change instruction is transmitted along with the data stream in each operation and maintenance processing node in the operation and maintenance operation end; When the current operation and maintenance data processing node meets the configuration change conditions, a method for changing the configuration information based on the configuration change instruction; compared with the method in the prior art of manually stopping the operation and maintenance operation, performing the configuration change operation, and then restarting the operation and maintenance operation process, the application embeds the configuration change instruction into the data stream of the operation and maintenance operation, so that the operation and maintenance data processing node can change the configuration information based on the configuration change instruction carried in the data stream without stopping the operation and maintenance operation. Therefore, it not only improves the continuity and stability of the operation of the operation and maintenance system, but also improves the efficiency of the configuration change of the operation and maintenance system.

[0029] As an optional implementation, combined with Figure 4 As shown, the operation and maintenance data processing node in the operation and maintenance operation end includes a data source node, an operator node and a data output node; in the case where the current operation and maintenance data processing node is a data source node of the operation and maintenance operation end, as shown in Figure 3 As shown, the above operation and maintenance data processing method includes the following steps: S302, when the configuration information of any operation and maintenance data processing node in the operation and maintenance operation end changes, receiving the configuration change instruction sent by the configuration control end.

[0030] S304: When the current operation and maintenance data processing node meets the configuration change condition, the configuration information is changed based on the configuration change instruction.

[0031] S306: Add the acquired configuration change instruction to the data stream corresponding to the current operation and maintenance job, and transmit the data stream carrying the configuration change instruction to the operator node at the operation and maintenance job end.

[0032] Specifically, in the embodiment of the present application, the above-mentioned operation and maintenance operation end may be, for example, an operation and maintenance operation server based on the Flink architecture. The number of data source nodes (Source), operator nodes (Operator) and data output nodes (Sink) in the above-mentioned operation and maintenance operation end may be one or more.

[0033] The configuration control end stores the configuration information of the detection objects associated with it. When the configuration information of any operation and maintenance data processing node in the operation and maintenance operation end changes, the configuration control end obtains the configuration change information of the changed dimension data processing node from the configuration management database, and then generates a configuration change instruction based on the configuration change information and sends it to the data source node of the operation and maintenance operation end.

[0034] Among them, the data source node is the starting point of the stream processing job in the operation and maintenance job end, and is responsible for receiving configuration change instructions issued by the configuration management and control end. When the data source node receives the configuration change instructions, it embeds these instructions into the preset control field in the data stream. The data source node passes the above configuration change instructions to the operator node in the stream processing job along with the data stream. In addition, when the data source node detects that it meets the configuration change conditions, it changes its own configuration information according to the configuration change instructions.

[0035] In one example, it is assumed that the above-mentioned operation and maintenance work end includes three operation and maintenance data processing nodes: node A (data source node), node B (operator node) and node C (data output node), and the data flow of the operation and maintenance processing task among the three is: node A→node B→node C; when the node A meets the configuration change conditions, node A changes the configuration information based on the change instruction.

[0036] The above technical solution of the present application realizes dynamic configuration changes during data flow processing without stopping or restarting the entire operation and maintenance data processing flow, which significantly improves the flexibility and response speed of the operation and maintenance system.

[0037] In one or more embodiments, adding the acquired configuration change instruction to the data stream corresponding to the current operation and maintenance job includes: The data source node assembles the acquired configuration change instruction into a character sequence in a target format, and then embeds the character sequence as a preset control field into the data stream corresponding to the current operation and maintenance job.

[0038] Specifically, in the embodiment of the present application, the character sequence in the above target format can be, for example, JSON, XML or a custom string format. The data source node assembles all relevant information of the configuration change instruction (such as parameter name, parameter value, change time, etc.) into a character sequence, and then embeds the assembled character sequence as a value into the control field of the data stream. In this way, the configuration change instruction can be used as part of the data stream and flow throughout the operation and maintenance operation, making configuration management more flexible and automated, greatly reducing the intervention of human operations, and improving the response speed and reliability of the operation and maintenance system.

[0039] In one or more embodiments, the operation and maintenance data processing method further includes: The data source node adds the configuration change result to the result field corresponding to the data source node, where the result field is a data field embedded in the data stream corresponding to the current operation and maintenance job; The data source node transmits the data stream carrying the configuration change instruction and the result field to the operator node.

[0040] Specifically, in an embodiment of the present application, after the above-mentioned data source node completes the configuration update according to the configuration change instruction, a configuration change result will be generated. This result may include a status of successful or failed change, or any relevant error information or confirmation information. The configuration change result is then added to the data field in the data stream to achieve this. This field is called the result field. After the data source node adds the configuration change result to the result field, it updates the data stream, and the data stream contains new information about the result field; in this way, the data stream not only carries the original data, but also carries the configuration change instruction about the configuration change, as well as feedback information after the data source node configuration is changed. The updated data stream flows and is transmitted in the stream processing job, carrying the configuration change instruction and the result field. The above technical solution of the present application not only improves the traceability of configuration changes by adding configuration change results in the data stream, but also enhances the system's fault diagnosis and self-adaptation capabilities, further improving the overall system performance and operation and maintenance efficiency.

[0041] As an optional implementation, Figure 5 As shown, the embodiment of the present application provides an operation and maintenance data processing method, comprising the following steps: S502, when the current operation and maintenance data processing node is the data source node of the operation and maintenance operation end, when the configuration information of any operation and maintenance data processing node in the operation and maintenance operation end changes, the data source node receives the configuration change instruction sent by the configuration control end.

[0042] S504: When the configuration parameters in the configuration change instruction include the configuration information of the data source node, it is determined that the data source node meets the configuration change condition, and the data source node changes the configuration information based on the configuration change instruction.

[0043] To ensure that each data source node, operator node, and data output node can access the latest configuration information and perform necessary configuration updates based on this information, each of the above data source nodes, operator nodes, and data output nodes will parse the data stream and then extract the configuration change instructions from the data stream. Each node will determine whether it meets the configuration change conditions based on the configuration parameters in the configuration change instructions.

[0044] In the embodiment of the present application, for example, the data source node will check the configuration parameters to determine whether there is an update related to itself, such as determining that the node ID, role or function-related identifier, etc. in the configuration change instruction are related to itself, and then determining that it meets the configuration change conditions, and updating its own configuration according to the parameters corresponding to itself in the configuration parameters. For example, when the configuration parameters in the configuration change instruction include the configuration information of the data source node, it is determined that the data source node meets the configuration change conditions, and then the data source node can change the configuration information based on the configuration change instruction.

[0045] S506, the data source node adds the acquired configuration change instruction to the data stream corresponding to the current operation and maintenance job, and transmits the data stream carrying the configuration change instruction to the operator node at the operation and maintenance job end.

[0046] The above-mentioned technical solution of the present application can enable the operation and maintenance system to quickly adapt to environmental changes and service requirements by dynamically receiving and processing configuration change instructions without manual intervention, thereby improving the flexibility and scalability of the operation and maintenance system.

[0047] The above steps S502 and S506 have been explained in the previous text and will not be repeated here.

[0048] In one or more embodiments, the operation and maintenance data processing method further includes: When the operation and maintenance operation end meets the change rollback condition, the data source node receives the change rollback instruction sent by the configuration control end; the change rollback condition is used to indicate that there is an operation and maintenance data processing node whose change failed, or there is an operation and maintenance data processing node whose change timed out, on the operation and maintenance operation end; The data source node restores the data source node to a state before the configuration information is changed based on the change rollback instruction; The data source node adds the change rollback instruction to the data stream corresponding to the current operation and maintenance job, and transmits the data stream carrying the change rollback instruction to the operator node.

[0049] Specifically, in the implementation of this application, when all target processing nodes (operation and maintenance data processing nodes where configuration changes occur) have successfully completed the configuration changes, the configuration control end determines that the change process of the entire operation and maintenance operation end is in a successful state, and then generates an end change instruction, which indicates that the configuration change process has been successfully completed. The end change instruction will be sent to all relevant nodes in the operation and maintenance operation through the data source node as a confirmation of the successful change.

[0050] If it is found in the configuration change result that any target processing node configuration fails, or a target processing node fails to complete the configuration change within the predetermined time (timeout), the configuration management and control end determines that the operation and maintenance job end is in a change failure state. When a configuration failure or timeout is detected, the configuration management and control end will generate a change rollback instruction and send it to the data source node. The data source node adds the change rollback instruction to the data stream corresponding to the current operation and maintenance job, and passes the data stream carrying the change rollback instruction to the operator node. After receiving the change rollback instruction, the operator node continues to pass the change rollback instruction to the data output node through the data stream. The data source node will perform a rollback operation based on the change rollback instruction to restore the data source node to the state before the configuration information was changed.

[0051] In one or more embodiments, when the current operation and maintenance data processing node is an operator node of an operation and maintenance operation end, the operation and maintenance data processing method includes: the operator node obtains a configuration change instruction from a data stream transmitted by a data source node of the operation and maintenance operation end; The method also includes: the operator node transmits the data stream carrying the configuration change instruction to the data output node of the operation and maintenance operation end.

[0052] Specifically, in the embodiment of the present application, the operator node is a node that performs specific operations (such as data filtering, data conversion, and data aggregation) in the stream processing job in the operation and maintenance operation end. When the data stream carrying the configuration change instruction reaches the operator node, the operator node detects that it meets the configuration change conditions and changes its own configuration information according to the configuration change instruction; and the operator node will continue to pass these instructions to the data output node, while executing the data processing task it carries.

[0053] In one or more embodiments, when the current operation and maintenance data processing node meets the configuration change condition, changing the configuration information based on the configuration change instruction includes: In a case where the configuration parameters in the configuration change instruction include the configuration information of the operator node, the operator node determines that the operator node meets the configuration change condition, and the operator node changes the configuration information based on the configuration change instruction.

[0054] In the embodiment of the present application, after the operator node obtains the configuration change instruction from the data stream, it will check the configuration parameters to determine whether there is an update related to itself, such as determining that the node ID, role or function-related identifier in the configuration change instruction is related to itself, and then determining that it meets the configuration change conditions, and updating its own configuration according to the parameters corresponding to itself in the configuration parameters. For example, when the configuration parameters in the configuration change instruction include the configuration information of the operator node, it is determined that the operator node meets the configuration change conditions, and then the operator node can change the configuration information based on the configuration change instruction.

[0055] In one or more embodiments, the operation and maintenance data processing method further includes: The operator node adds the configuration change result to the result field corresponding to the operator node, where the result field is a data field embedded in the data stream corresponding to the current operation and maintenance job; The operator node transmits the data stream carrying the configuration change instruction and the result field to the data output node.

[0056] After the operator node completes the configuration update according to the configuration change instruction, it will generate a configuration change result, which may include the status of successful or failed change, or any related error information or confirmation information. The operator node then adds the configuration change result to the data field in the data stream to implement it. This field is the result field. After the operator node adds the configuration change result to the result field, it updates the data stream, which contains the new information of the result field. In this way, the data stream not only carries the original data, but also carries the configuration change instruction about the configuration change, as well as the feedback information after the configuration change of the data source node and the operator node. The updated data stream flows in the stream processing job, carrying the configuration change instruction and the result field. The above technical solution of the present application not only improves the traceability of configuration changes by adding configuration change results in the data stream, but also enhances the system's fault diagnosis and self-adaptation capabilities, further improving the overall system performance and operation and maintenance efficiency.

[0057] In one or more embodiments, the operation and maintenance data processing method further includes: The operator node obtains the change rollback instruction from the data stream transmitted by the data source node of the operation and maintenance operation end; The operator node restores the operator node to a state before the configuration information is changed based on the change rollback instruction; The operator node transmits the data stream carrying the change rollback instruction to the data output node of the operation and maintenance operation end.

[0058] Specifically, in an embodiment of the present application, if it is found in the configuration change result that any target processing node configuration fails, or a target processing node fails to complete the configuration change within a predetermined time (timeout), the configuration management and control end determines that the operation and maintenance operation end is in a change failure state. When a configuration failure or timeout is detected, the configuration management and control end will generate a change rollback instruction and send it to the data source node. The data source node adds the change rollback instruction to the data stream corresponding to the current operation and maintenance operation, and passes the data stream carrying the change rollback instruction to the operator node. After receiving the change rollback instruction, the operator node continues to pass the change rollback instruction to the data output node through the data stream. The operator node will perform a rollback operation based on the change rollback instruction to restore the operator node to the state before the configuration information was changed.

[0059] In one or more embodiments, when the current operation and maintenance data processing node is a data output node of the operation and maintenance operation end, the operation and maintenance data processing method includes: the data output node obtains a configuration change instruction from a data stream transmitted by an operator node of the operation and maintenance operation end; The data output node adds the configuration change result to the result field corresponding to the data output node, and the result field is a data field embedded in the data stream corresponding to the current operation and maintenance job.

[0060] Specifically, in an embodiment of the present application, the data output node is the end point of the stream processing job in the operation and maintenance job end, and is responsible for outputting the processed data to an external system (such as a database, file system, or message queue). When the data stream carrying the configuration change instruction arrives at the data output node, when the data output node detects that it meets the configuration change conditions, it changes its own configuration information according to the configuration change instruction. After the above-mentioned data output node completes the configuration update according to the configuration change instruction, a configuration change result will be generated. This result may include a status of successful or failed change, or any related error information or confirmation information. Afterwards, the data output node adds the configuration change result to the data field in the data stream to achieve this, and the data field is the result field.

[0061] In one or more embodiments, when the current operation and maintenance data processing node meets the configuration change condition, changing the configuration information based on the configuration change instruction includes: In the case where the configuration parameters in the configuration change instruction include the configuration information of the data output node, the data output node determines that the data output node meets the configuration change condition, and the data output node changes the configuration information based on the configuration change instruction.

[0062] In the embodiment of the present application, after the data output node obtains the configuration change instruction from the data stream, it will check the configuration parameters to determine whether there is an update related to itself, such as determining that the node ID, role or function-related identifier in the configuration change instruction is related to itself, and then determining that it meets the configuration change conditions, and updating its own configuration according to the parameters corresponding to itself in the configuration parameters. For example, when the configuration parameters in the configuration change instruction include the configuration information of the data output node, it is determined that the data output node meets the configuration change conditions, and then the data output node can change the configuration information based on the configuration change instruction.

[0063] In one or more embodiments, the operation and maintenance data processing method further includes: The data output node obtains the configuration change result of each operation and maintenance processing node from the result field corresponding to each operation and maintenance processing node; The data output node summarizes the configuration change results obtained and sends the change summary results to the configuration management and control end.

[0064] Specifically, Figure 4 As shown, in an embodiment of the present application, the data output node obtains the configuration change result of each operation and maintenance processing node from the result field corresponding to each operation and maintenance processing node. The above-mentioned configuration change result includes the status information of the configuration change of the operation and maintenance data processing node, including success, failure or timeout status. In one example, for example, after the configuration change of an operation and maintenance processing node is successful, the value corresponding to the result field corresponding to the operation and maintenance processing node can be 1. If the configuration change of the operation and maintenance processing node fails, the value corresponding to the result field corresponding to the operation and maintenance processing node can be 0. If the configuration change of the operation and maintenance processing node is not performed, the value corresponding to the result field corresponding to the operation and maintenance processing node is empty.

[0065] The data output node summarizes the collected configuration change results and then sends the summary results to the configuration control end. After receiving the configuration change results of each target processing node, the configuration control end will analyze these results to determine whether the overall configuration change of the operation and maintenance operation end is successful. The configuration control end generates corresponding feedback instructions based on the configuration change results and then sends the feedback instructions to the data source node of the operation and maintenance processing end.

[0066] The above-mentioned technical solution of the present application can realize closed-loop control of the change operation by feeding back the configuration change results to the configuration control end, thereby ensuring the accuracy and effectiveness of the configuration change; through the automated configuration change result processing and feedback process, it can also reduce manual intervention and improve operation and maintenance efficiency.

[0067] In one or more embodiments, after the configuration information is changed based on the configuration change instruction, the method further includes: The data output node obtains the change rollback instruction from the data stream transmitted by the operator node of the operation and maintenance operation end; The data output node restores the data output node to a state before the configuration information is changed based on the change rollback instruction.

[0068] Specifically, in an embodiment of the present application, if it is found in the configuration change result that any target processing node configuration fails, or a target processing node fails to complete the configuration change within a predetermined time (timeout), the configuration management and control end determines that the operation and maintenance operation end is in a change failure state. In the event of a configuration failure or timeout, the configuration management and control end will generate a change rollback instruction and send it to the data source node. The data source node will add the change rollback instruction to the data stream corresponding to the current operation and maintenance operation, and pass the data stream carrying the change rollback instruction to the operator node. After receiving the change rollback instruction, the operator node will continue to pass the change rollback instruction to the data output node through the data stream. The data output node will perform a rollback operation based on the change rollback instruction to restore the data output node to the state before the configuration information was changed.

[0069] As an optional implementation, Figure 6 As shown, the embodiment of the present application provides an operation and maintenance data processing method, which is applied to the configuration control end, including the following steps: S602: In response to a change in configuration information of an operation and maintenance data processing node associated with the configuration control terminal, a configuration change instruction is generated according to the changed configuration information.

[0070] S604, sending the configuration change instruction to the data source node of the operation and maintenance operation end, wherein the configuration change instruction is used to be embedded into the data stream corresponding to the operation and maintenance operation as a preset control field, so that the operation and maintenance data processing node changes the configuration information according to the configuration change instruction.

[0071] Specifically, in an embodiment of the present application, the above-mentioned configuration control end is used to subscribe to the status or configuration change notification of the associated object from the configuration management database, and when the configuration information of the associated object changes, a configuration change instruction is generated according to the changed configuration information. In one example, the above-mentioned associated object can be, for example, one or more operation and maintenance data processing nodes in a stream processing engine (Flink or Spark). The above-mentioned operation and maintenance operation end is used to perform time series indicator calculation operations, such as being responsible for receiving, cleaning and calculating the data collected and reported by one or more components of the cloud network (such as traffic, status, logs, etc.). Changes in the above-mentioned configuration information may, for example, be changes in the storage location of the data source, changes in the authentication and authorization credentials for reading data, changes in the network interface address when the operation and maintenance operation end communicates with an external system or service, changes in the field Schema in the operation process of the operation and maintenance operation end, etc.

[0072] In one example, it is assumed that the operation and maintenance data processing nodes associated with the configuration control end include: Node A, Node B and Node C, Node D and Node E. The above-mentioned operation and maintenance operation end includes three operation and maintenance data processing nodes: Node A, Node B and Node C; when the configuration information of at least one of Node A, Node B and Node C changes, such as when the data source for obtaining operation and maintenance data by Node A changes, the configuration control end generates a corresponding configuration change instruction, and then sends the configuration change instruction to the operation and maintenance operation end, and the operation and maintenance data processing node updates or modifies the configuration information of the data source of Node A according to the instruction.

[0073] The above-mentioned technical solution of the present application improves the efficiency and accuracy of operation and maintenance data configuration management and enhances the flexibility, adaptability and reliability of the system by automating the generation and issuance of configuration change instructions.

[0074] As an optional implementation, Figure 7 As shown, the embodiment of the present application provides an operation and maintenance data processing method, which is applied to the configuration control end, including the following steps: S702, subscribing to a subscription message of configuration information change of a detected object from a configuration management database; the detected object includes at least one of an operation and maintenance data processing node, a cloud network device, and a virtual instance associated with the configuration control center; S704: When the subscription message is received, obtain configuration information of the detected object from the configuration management database.

[0075] Specifically, in an embodiment of the present application, the configuration control end subscribes to a subscription message of a configuration information change of a detected object from a configuration management database; the detected object includes at least one of an operation and maintenance data processing node, a cloud network device, and a virtual instance associated with the configuration control center; upon receiving the subscription message, the configuration control end obtains the configuration information of the detected object from the configuration management database.

[0076] S706: In response to a change in configuration information of an operation and maintenance data processing node associated with the configuration control terminal, a configuration change instruction is generated according to the changed configuration information.

[0077] S708, sending the configuration change instruction to the data source node of the operation and maintenance operation end, the configuration change instruction is used to be embedded in the data stream corresponding to the operation and maintenance operation as a preset control field, so that the operation and maintenance data processing node changes the configuration information according to the configuration change instruction.

[0078] The above steps S706 and S708 have been explained in the previous text and will not be repeated here.

[0079] In one or more embodiments, the above operation and maintenance data processing method further includes: receiving a configuration change result of a target processing node among the operation and maintenance data processing nodes sent by the operation and maintenance operation end, wherein the target processing node is the operation and maintenance data processing node where the configuration change occurs; Based on the configuration change result indicating that there is an operation and maintenance data processing node with a failed configuration, or an operation and maintenance data processing node with a change timeout, a change rollback instruction is sent to the data source node of the operation and maintenance operation end, so that the target processing node performs a change rollback operation based on the change rollback instruction.

[0080] Specifically, in the embodiments of the present application, Figure 4 As shown, the data output node obtains the configuration change results of each target processing node (i.e., the operation and maintenance data processing node where the configuration change occurs) from the result field in the data stream. The above configuration change results include the status information of the configuration change of the operation and maintenance data processing node, including success, failure, or timeout. The data output node integrates the collected configuration change results and sends them to the configuration control end. After receiving the configuration change results of each target processing node, the configuration control end will analyze these results to determine whether the overall configuration change of the operation and maintenance operation end is successful. Based on the configuration change results, the configuration control end will generate corresponding feedback instructions and then send the feedback instructions to the operation and maintenance processing end.

[0081] When all target processing nodes (operation and maintenance data processing nodes where configuration changes occur) have successfully completed the configuration changes, the configuration control end determines that the change process of the entire operation and maintenance operation end is in a successful state, and then generates an end change instruction to indicate that the configuration change process has been successfully completed.

[0082] If any target processing node fails to be configured in the configuration change result, or a target processing node fails to complete the configuration change within the scheduled time (timeout), the configuration control end determines that the operation and maintenance operation end is in a change failure state. When a configuration failure or timeout is detected, the configuration control end generates a change rollback instruction and sends it to the data source node, which is then sent to all target processing nodes through the data source node, instructing the target processing node to start the rollback operation and restore the target processing node to the state before the configuration information was changed.

[0083] The above-mentioned technical solution of the present application can ensure that in the event of configuration change failure or timeout, the system can be quickly restored to a stable state through a rollback operation by sending the end change instruction and the change rollback instruction to the operation and maintenance operation end, thereby improving the stability and reliability of the system.

[0084] As an optional implementation method, the embodiment of the present application provides an operation and maintenance data system, including a configuration control end and an operation and maintenance operation end; The configuration control terminal is used to generate a configuration change instruction according to the changed configuration information in response to a change in the configuration information of the operation and maintenance data processing node associated with the configuration control terminal; the configuration change instruction is used to change the configuration information of the operation and maintenance data processing node associated with the configuration control terminal; and the configuration change instruction is sent to the operation and maintenance operation terminal; The operation and maintenance operation end is used to receive the configuration change instruction; add the configuration change instruction to the data stream corresponding to the current operation and maintenance operation, and pass the data stream carrying the configuration change instruction to the operation and maintenance data processing node in the current operation and maintenance operation; control the operation and maintenance data processing node in the current operation and maintenance operation to change the configuration information based on the configuration change instruction carried in the data stream.

[0085] As an optional implementation, an embodiment of the present application provides an operation and maintenance data system, including a configuration control center and a stream processing engine, wherein the stream processing engine includes a data source node, an operator node, and a data output node; The configuration control center is used to obtain configuration information of the detected object from the configuration management database; the detected object includes at least one of an operation and maintenance data processing node, a cloud network device, and a virtual instance associated with the configuration control center; based on changes in the configuration information of the detected object, a configuration change instruction is generated according to the changed configuration information; and the configuration change instruction is sent to the data source node; The data source node is configured to receive the configuration change instruction; add the configuration change instruction to the data stream corresponding to the current operation and maintenance job; change the data source configuration information based on the configuration change instruction when the configuration change conditions are met; and transmit the data stream carrying the configuration change instruction to the operator node; The operator node is used to change the data source configuration information based on the configuration change instruction when the configuration change condition is met; The data output node is used to change the data source configuration information based on the configuration change instruction when the configuration change conditions are met, and send the changed configuration results of the operation and maintenance data processing nodes that meet the configuration change conditions to the configuration control center.

[0086] In one or more embodiments, the configuration information of the detected object includes at least one of the following: change information of cloud network devices and virtual instances, configuration information of data source nodes, computing logic of operator nodes, and configuration information of data output nodes.

[0087] In the related technology, the network timing indicator calculation job based on the Flink architecture usually sets the detection data objects and data sources before the job is deployed. When the job is started, modifying the configuration information requires stopping the current job, which will cause the data processing of the calculation job to be interrupted.

[0088] In order to solve the above technical problems, based on the above embodiments, combined with Figure 4 As shown, in an application embodiment, the above operation and maintenance data processing method includes: S1. First, the configuration control center subscribes to the change information of the cloud network online equipment and virtual instance object list from the CMDB. The configuration control center can support the dynamic perception of the creation and destruction of hundreds of millions of monitoring objects. For example, when a cloud network product is enabled and configured and deployed in a new region, the configuration control center will obtain the detailed configuration information of each component of the cloud network in this new region from the CMDB. This information can be used to update and create a new time series indicator calculation and monitoring link deployment for this new region. S2. The configuration control center service generates configuration change execution commands for time series indicator calculation jobs that need to be changed or newly issued based on the changes in the perceived CMDB, and sends them to the corresponding time series indicator Flink calculation jobs. This application uses a custom connector to transform the data source node (Source) on the Flink side (operation and maintenance job side). On the premise of having the basic function of data ingestion, it also has the ability to listen to the change commands issued by the configuration control center. If the configuration change involves the Source node, the Flink job will modify its configuration.

[0089] S3. The entire data processing logic and process (data flow) is described in the form of Structured Query Language (SQL). The Source node will assemble the received commands into a custom string field and embed it into the Flink data flow. The command string is placed in the Flink SQL for data processing as an internally reserved control field and passed to the downstream of the data flow, including but not limited to each operator node (Operator) and output node (Sink). At the same time, the Source node will also assemble the configuration change execution results of its own node into a result field and place it in Flink SQL to pass it to the downstream nodes of the data flow.

[0090] S4. This application transforms the data output node (Sink) by using a custom connector. In addition to the basic function of data output, the data output node also has the ability to send feedback on the change results of commands to the configuration control center service. The Sink node will summarize the change results of each node in a configuration change command job, and then feed them back to the configuration control center service. After receiving the result of the change execution, the configuration control center service will choose to end the change or initiate a command to roll back the change based on the feedback. The process of passing and executing the rollback change command is consistent with the above steps S1-S4.

[0091] This application uses data coloring to pass the configuration change command of the operation and maintenance job in the computing job along the data flow of the network timing indicator calculation, ensuring the consistency of the configuration change command in the entire Flink computing job, and avoiding the complex control of the direct connection and command transmission between the configuration control center and all computing nodes, and can coordinate configuration among multiple nodes in distributed deployment. Finally, the configuration change is smoothly executed among multiple nodes of the Flink job to achieve fast and lossless configuration changes, significantly improving the operation and maintenance efficiency.

[0092] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0093] According to another aspect of the embodiment of the present application, there is also provided an operation and maintenance data processing device for implementing the above-mentioned operation and maintenance data processing method, which is applied to the operation and maintenance operation end, such as Figure 8 As shown, the operation and maintenance data processing device includes: An acquisition unit 802 is used to acquire a configuration change instruction, wherein the configuration change instruction is used to change the configuration information of the operation and maintenance data processing node whose configuration is to be changed, and the configuration change instruction includes part of the data in the data stream corresponding to the current operation and maintenance operation, and the configuration change instruction is transmitted along with the data stream in each operation and maintenance processing node in the operation and maintenance operation end; The changing unit 804 is used to change the configuration information based on the configuration change instruction when the current operation and maintenance data processing node meets the configuration change condition.

[0094] In the embodiment of the present application, a configuration change instruction is obtained, and the configuration change instruction is used to change the configuration information of the operation and maintenance data processing node whose configuration is to be changed. The configuration change instruction includes part of the data in the data stream corresponding to the current operation and maintenance operation, and the configuration change instruction is transmitted along with the data stream in each operation and maintenance processing node in the operation and maintenance operation end; When the current operation and maintenance data processing node meets the configuration change conditions, a method for changing the configuration information based on the configuration change instruction; compared with the method in the prior art of manually stopping the operation and maintenance operation, performing the configuration change operation, and then restarting the operation and maintenance operation process, the application embeds the configuration change instruction into the data stream of the operation and maintenance operation, so that the operation and maintenance data processing node can change the configuration information based on the configuration change instruction carried in the data stream without stopping the operation and maintenance operation. Therefore, it not only improves the continuity and stability of the operation of the operation and maintenance system, but also improves the efficiency of the configuration change of the operation and maintenance system.

[0095] In one or more embodiments, when the current operation and maintenance data processing node is a data source node of the operation and maintenance operation end, the acquisition unit 802 includes: a receiving module, configured to receive the configuration change instruction sent by the configuration control end when the configuration information of any operation and maintenance data processing node in the operation and maintenance operation end changes; The operation and maintenance data processing device also includes: a first adding and transmitting unit, which is used to add the acquired configuration change instruction to the data stream corresponding to the current operation and maintenance operation, and transmit the data stream carrying the configuration change instruction to the operator node of the operation and maintenance operation end.

[0096] In one or more embodiments, the first adding transfer unit comprises: The first assembly and embedding module is used to assemble the acquired configuration change instruction into a character sequence in a target format, and embed the character sequence as a preset control field into a data stream corresponding to the current operation and maintenance job.

[0097] In one or more embodiments, the operation and maintenance data processing device further includes: A first embedding unit, configured to add a configuration change result to a result field corresponding to the data source node, wherein the result field is a data field embedded in a data stream corresponding to a current operation and maintenance job; The first transmission unit is used to transmit the data stream carrying the configuration change instruction and the result field to the operator node.

[0098] In one or more embodiments, the changing unit 804 includes: The first change module is used to determine that the data source node meets the configuration change condition when the configuration parameters in the configuration change instruction include the configuration information of the data source node, and change the configuration information based on the configuration change instruction.

[0099] In one or more embodiments, the operation and maintenance data processing device further includes: A receiving unit, configured to receive a change rollback instruction sent by the configuration control end when the operation and maintenance operation end meets a change rollback condition; the change rollback condition is used to indicate that there is an operation and maintenance data processing node whose change failed or there is an operation and maintenance data processing node whose change timed out on the operation and maintenance operation end; A first restoring unit, configured to restore the data source node to a state before the configuration information is changed based on a change rollback instruction; An adding and transmitting unit is used to add the change rollback instruction to the data stream corresponding to the current operation and maintenance job, and transmit the data stream carrying the change rollback instruction to the operator node.

[0100] In one or more embodiments, when the current operation and maintenance data processing node is an operator node of the operation and maintenance operation end, the acquisition unit 802 includes: a first acquisition module, configured to acquire a configuration change instruction from a data stream transmitted by a data source node of the operation and maintenance operation end; The operation and maintenance data processing device further includes: a second transmission unit, configured to transmit the data stream carrying the configuration change instruction to a data output node of the operation and maintenance operation end.

[0101] In one or more embodiments, when the current operation and maintenance data processing node meets the configuration change condition, the changing unit 804 includes: The second change module is used to determine that the operator node meets the configuration change condition when the configuration parameters in the configuration change instruction include the configuration information of the operator node, and change the configuration information based on the configuration change instruction.

[0102] In one or more embodiments, the operation and maintenance data processing device further includes: A first adding unit, configured to add the configuration change result to a result field corresponding to the operator node, wherein the result field is a data field embedded in a data stream corresponding to the current operation and maintenance job; The second transmission unit is used to transmit the data stream carrying the configuration change instruction and the result field to the data output node.

[0103] In one or more embodiments, the operation and maintenance data processing device further includes: A first rollback instruction acquisition unit is used to acquire a change rollback instruction from a data stream transmitted by a data source node of the operation and maintenance operation end; A second restoring unit, configured to restore the operator node to a state before the configuration information is changed based on the change rollback instruction; The third transmission unit is used to transmit the data stream carrying the change rollback instruction to the data output node of the operation and maintenance operation end.

[0104] In one or more embodiments, when the current operation and maintenance data processing node is a data output node of the operation and maintenance operation end, the acquisition unit 802 includes: acquiring a configuration change instruction from a data stream transmitted by an operator node of the operation and maintenance operation end; The second adding unit is used to add the configuration change result to the result field corresponding to the data output node, and the result field is a data field embedded in the data flow corresponding to the current operation and maintenance job.

[0105] In one or more embodiments, the changing unit 804 includes: The third changing module is used to determine that the data output node meets the configuration change condition when the configuration parameters in the configuration change instruction include the configuration information of the data output node, and change the configuration information based on the configuration change instruction.

[0106] In one or more embodiments, the operation and maintenance data processing device further includes: A configuration result obtaining unit, used to obtain the configuration change result of each operation and maintenance processing node from the result field corresponding to each operation and maintenance processing node; The summary unit is used to summarize the obtained configuration change results and send the change summary results to the configuration management and control end.

[0107] In one or more embodiments, the operation and maintenance data processing device further includes: A second rollback instruction acquisition unit is used to acquire a change rollback instruction from a data stream transmitted by an operator node of the operation and maintenance operation end; The third restoring unit is used to restore the data output node to a state before the configuration information is changed based on the change rollback instruction.

[0108] According to another aspect of the embodiment of the present application, there is also provided an operation and maintenance data processing device for implementing the above-mentioned operation and maintenance data processing method, which is applied to the configuration control end, such as Fig. 9 As shown, the operation and maintenance data processing device includes: A generating unit 902 is used to generate a configuration change instruction according to the changed configuration information in response to a change in the configuration information of the operation and maintenance data processing node associated with the configuration control terminal; The sending unit 904 is used to send the configuration change instruction to the data source node of the operation and maintenance operation end. The configuration change instruction is used to be embedded in the data stream corresponding to the operation and maintenance operation as a preset control field, so that the operation and maintenance data processing node changes the configuration information according to the configuration change instruction.

[0109] In one or more embodiments, the operation and maintenance data processing device further includes: A subscription unit, configured to subscribe to a subscription message of a configuration information change of a detected object from a configuration management database; the detected object includes at least one of an operation and maintenance data processing node, a cloud network device, and a virtual instance associated with the configuration control center; The acquisition unit is used to acquire the configuration information of the detected object from the configuration management database when the subscription message is received.

[0110] In one or more embodiments, the operation and maintenance data processing device further includes: A receiving unit, configured to receive a configuration change result of a target processing node among the operation and maintenance data processing nodes sent by the operation and maintenance operation end, wherein the target processing node is the operation and maintenance data processing node where the configuration change occurs; A sending unit is used to send a change rollback instruction to the data source node of the operation and maintenance operation end based on the configuration change result indicating that there is an operation and maintenance data processing node with a configuration failure, or an operation and maintenance data processing node with a change timeout, so that the target processing node performs a change rollback operation based on the change rollback instruction.

[0111] According to another aspect of the embodiment of the present application, an electronic device for implementing the above-mentioned operation and maintenance data processing method is also provided. The electronic device may be Figure 1 The operation and maintenance end shown in FIG. This embodiment is described by taking the electronic device as the operation and maintenance end as an example. Fig.10 As shown, the electronic device includes a memory 1002 and a processor 1004. The memory 1002 stores a computer program, and the processor 1004 is configured to execute the steps in any of the above method embodiments through the computer program.

[0112] Optionally, in this embodiment, the electronic device may be at least one network device among a plurality of network devices of a computer network.

[0113] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program: S11, obtaining a configuration change instruction, wherein the configuration change instruction is used to change the configuration information of the operation and maintenance data processing node whose configuration is to be changed, and the configuration change instruction includes part of the data in the data stream corresponding to the current operation and maintenance operation, and the configuration change instruction is transmitted along with the data stream in each operation and maintenance processing node in the operation and maintenance operation end; S12: When the current operation and maintenance data processing node meets the configuration change conditions, the configuration information is changed based on the configuration change instruction.

[0114] Among them, the memory 1002 can be used to store software programs and modules, such as the program instructions / modules corresponding to the operation and maintenance data processing method and device in the embodiment of the present application, and the processor 1004 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002, that is, realizing the above-mentioned operation and maintenance data processing method. The memory 1002 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1002 may further include a memory remotely arranged relative to the processor 1004, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. Among them, the memory 1002 can be specifically used, but not limited to, for storing configuration information and configuration change instructions corresponding to the detection object.

[0115] As an example, Fig.10 As shown, the memory 1002 may include but is not limited to the acquisition unit 802 and the change unit 804 in the operation and maintenance data processing device. In addition, it may also include but is not limited to other module units in the operation and maintenance data processing device, which will not be repeated in this example.

[0116] Optionally, the transmission device 1006 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 1006 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 1006 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0117] In addition, the electronic device mentioned above further includes: a connection bus 1008, which is used to connect various module components in the electronic device mentioned above.

[0118] According to another aspect of the embodiment of the present application, an electronic device for implementing the above-mentioned operation and maintenance data processing method is also provided. The electronic device may be Figure 1 The configuration control terminal shown. This embodiment takes the electronic device as an example for description. Fig.11 As shown, the electronic device includes a memory 1102 and a processor 1104. The memory 1102 stores a computer program, and the processor 1104 is configured to execute the steps in any of the above method embodiments through the computer program.

[0119] Optionally, in this embodiment, the electronic device may be at least one network device among a plurality of network devices of a computer network.

[0120] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program: S21, in response to a change in configuration information of an operation and maintenance data processing node associated with the configuration control terminal, generating a configuration change instruction according to the changed configuration information; S22, sending the configuration change instruction to the data source node of the operation and maintenance operation end, the configuration change instruction is used to be embedded in the data stream corresponding to the operation and maintenance operation as a preset control field, so that the operation and maintenance data processing node changes the configuration information according to the configuration change instruction.

[0121] Among them, the memory 1102 can be used to store software programs and modules, such as the program instructions / modules corresponding to the operation and maintenance data processing method and device in the embodiment of the present application, and the processor 1104 executes various functional applications and data processing by running the software programs and modules stored in the memory 1102, that is, realizing the above-mentioned operation and maintenance data processing method. The memory 1102 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1102 may further include a memory remotely arranged relative to the processor 1104, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. Among them, the memory 1102 can be specifically used, but not limited to, for storing configuration information and configuration change instructions corresponding to the detection object.

[0122] As an example, Fig.11 As shown, the memory 1102 may include, but is not limited to, the generating unit 902 and the sending unit 904 in the operation and maintenance data processing device. In addition, it may also include, but is not limited to, other module units in the operation and maintenance data processing device, which will not be described in detail in this example.

[0123] Optionally, the transmission device 1106 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 1106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 1106 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0124] In addition, the electronic device mentioned above further includes: a connection bus 1108, which is used to connect various module components in the electronic device mentioned above.

[0125] In other embodiments, the electronic device may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. The nodes may form a peer-to-peer (P2P) network, and any form of computing device, such as a server, terminal, or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.

[0126] In one or more embodiments, the present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned operation and maintenance data processing method. Wherein, the computer program is configured to execute the steps of any of the above-mentioned method embodiments when running.

[0127] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps: S11, obtaining a configuration change instruction, wherein the configuration change instruction is used to change the configuration information of the operation and maintenance data processing node whose configuration is to be changed, and the configuration change instruction includes part of the data in the data stream corresponding to the current operation and maintenance operation, and the configuration change instruction is transmitted along with the data stream in each operation and maintenance processing node in the operation and maintenance operation end; S12, when the current operation and maintenance data processing node meets the configuration change conditions, the configuration information is changed based on the configuration change instruction. Or a computer program that performs the following steps: S21, in response to a change in configuration information of an operation and maintenance data processing node associated with the configuration control terminal, generating a configuration change instruction according to the changed configuration information; S22, sending the configuration change instruction to the data source node of the operation and maintenance operation end, the configuration change instruction is used to be embedded in the data stream corresponding to the operation and maintenance operation as a preset control field, so that the operation and maintenance data processing node changes the configuration information according to the configuration change instruction.

[0128] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0129] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0130] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to execute all or part of the steps of the methods of various embodiments of the present invention.

[0131] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0132] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0135] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

[0136] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

Claims

1. A method for processing operation and maintenance data, characterized in that: The method comprises: Obtaining a configuration change instruction, wherein the configuration change instruction is used to change the configuration information of the operation and maintenance data processing node whose configuration is to be changed, the configuration change instruction includes part of the data in the data stream corresponding to the current operation and maintenance operation, and the configuration change instruction is transmitted along with the data stream in each operation and maintenance processing node in the operation and maintenance operation end; When the current operation and maintenance data processing node meets the configuration change conditions, the configuration information is changed based on the configuration change instruction.

2. The method according to claim 1, characterized in that In the case where the current operation and maintenance data processing node is a data source node of the operation and maintenance operation end, the obtaining of the configuration change instruction includes: When the configuration information of any operation and maintenance data processing node in the operation and maintenance operation end changes, receiving the configuration change instruction sent by the configuration control end; The method further includes: adding the acquired configuration change instruction to the data stream corresponding to the current operation and maintenance job, and transmitting the data stream carrying the configuration change instruction to the operator node at the operation and maintenance job end.

3. The method according to claim 2, characterized in that The step of adding the acquired configuration change instruction to the data stream corresponding to the current operation and maintenance job includes: The acquired configuration change instruction is assembled into a character sequence in a target format, and the character sequence is embedded as a preset control field into a data stream corresponding to the current operation and maintenance job.

4. The method according to claim 2, characterized in that: The method further comprises: Add the configuration change result to the result field corresponding to the data source node, where the result field is a data field embedded in the data stream corresponding to the current operation and maintenance job; The data stream carrying the configuration change instruction and the result field is delivered to the operator node.

5. The method according to any one of claims 2 to 4, characterized in that: When the current operation and maintenance data processing node meets the configuration change condition, changing the configuration information based on the configuration change instruction includes: In a case where the configuration parameters in the configuration change instruction include configuration information of the data source node, it is determined that the data source node meets the configuration change condition, and the configuration information is changed based on the configuration change instruction.

6. The method according to any one of claims 2 to 4, characterized in that: The method further comprises: When the operation and maintenance operation end meets the change rollback condition, receiving the change rollback instruction sent by the configuration control end; the change rollback condition is used to indicate that there is an operation and maintenance data processing node whose change failed, or there is an operation and maintenance data processing node whose change timed out, on the operation and maintenance operation end; Restoring the data source node to a state before the configuration information was changed based on the change rollback instruction; The change rollback instruction is added to the data stream corresponding to the current operation and maintenance job, and the data stream carrying the change rollback instruction is transmitted to the operator node.

7. The method according to claim 1, characterized in that In the case where the current operation and maintenance data processing node is an operator node of the operation and maintenance operation end, the obtaining of the configuration change instruction includes: obtaining the configuration change instruction from the data stream transmitted by the data source node of the operation and maintenance operation end; The method further includes: transmitting the data stream carrying the configuration change instruction to a data output node of the operation and maintenance operation end.

8. The method according to claim 7, characterized in that When the current operation and maintenance data processing node meets the configuration change condition, changing the configuration information based on the configuration change instruction includes: In a case where the configuration parameters in the configuration change instruction include the configuration information of the operator node, it is determined that the operator node meets the configuration change condition, and the configuration information is changed based on the configuration change instruction.

9. The method according to claim 7 or 8, characterized in that: The method further comprises: Add the configuration change result to the result field corresponding to the operator node, where the result field is a data field embedded in the data stream corresponding to the current operation and maintenance job; The data stream carrying the configuration change instruction and the result field is transmitted to the data output node.

10. The method according to claim 7 or 8, characterized in that: The method further comprises: Obtaining a change rollback instruction from a data stream transmitted by a data source node of the operation and maintenance operation end; Restoring the operator node to a state before the configuration information is changed based on the change rollback instruction; The data stream carrying the change rollback instruction is transmitted to the data output node of the operation and maintenance operation end.

11. The method according to claim 1, characterized in that: In the case where the current operation and maintenance data processing node is a data output node of the operation and maintenance operation end, the obtaining of the configuration change instruction includes: obtaining the configuration change instruction from the data stream transmitted by the operator node of the operation and maintenance operation end; The configuration change result is added to the result field corresponding to the data output node, where the result field is a data field embedded in the data stream corresponding to the current operation and maintenance job.

12. The method according to claim 11, characterized in that When the current operation and maintenance data processing node meets the configuration change condition, changing the configuration information based on the configuration change instruction includes: In a case where the configuration parameters in the configuration change instruction include configuration information of the data output node, it is determined that the data output node meets the configuration change condition, and the configuration information is changed based on the configuration change instruction.

13. The method according to claim 11 or 12, characterized in that: The method further comprises: Obtaining the configuration change result of each operation and maintenance processing node from the result field corresponding to each operation and maintenance processing node; Summarize the obtained configuration change results and send the change summary results to the configuration management end.

14. The method according to claim 11 or 12, characterized in that: After the configuration information is changed based on the configuration change instruction, the method further includes: Obtaining a change rollback instruction from a data stream transmitted by an operator node of the operation and maintenance operation end; The data output node is restored to a state before the configuration information is changed based on the change rollback instruction.

15. A method for processing operation and maintenance data, characterized in that: Applied to the configuration control end, the method includes: In response to a change in configuration information of an operation and maintenance data processing node associated with the configuration control terminal, generating a configuration change instruction according to the changed configuration information; The configuration change instruction is sent to the data source node of the operation and maintenance operation end. The configuration change instruction is used to be embedded in the data stream corresponding to the operation and maintenance operation as a preset control field, so that the operation and maintenance data processing node changes the configuration information according to the configuration change instruction.

16. The method according to claim 15, characterized in that Before generating the configuration change instruction according to the changed configuration information, the method further includes: Subscribe to a subscription message of configuration information change of a detected object from a configuration management database; the detected object includes at least one of an operation and maintenance data processing node, a cloud network device, and a virtual instance associated with the configuration control center; When the subscription message is received, the configuration information of the detected object is obtained from the configuration management database.

17. The method according to claim 15 or 16, characterized in that The method further comprises: receiving a configuration change result of a target processing node among the operation and maintenance data processing nodes sent by the operation and maintenance operation end, wherein the target processing node is the operation and maintenance data processing node where the configuration change occurs; Based on the configuration change result indicating that there is an operation and maintenance data processing node with a failed configuration, or an operation and maintenance data processing node with a change timeout, a change rollback instruction is sent to the data source node of the operation and maintenance operation end, so that the target processing node performs a change rollback operation based on the change rollback instruction.

18. An operation and maintenance data processing system, characterized in that: Including configuration control end and operation and maintenance end; The configuration control terminal is used to generate a configuration change instruction according to the changed configuration information in response to a change in the configuration information of the operation and maintenance data processing node associated with the configuration control terminal; the configuration change instruction is used to change the configuration information of the operation and maintenance data processing node associated with the configuration control terminal; and the configuration change instruction is sent to the operation and maintenance operation terminal; The operation and maintenance operation end is used to receive the configuration change instruction; add the configuration change instruction to the data stream corresponding to the current operation and maintenance operation, and transmit the data stream carrying the configuration change instruction to the operation and maintenance data processing node in the current operation and maintenance operation; The operation and maintenance data processing node in the current operation and maintenance job is controlled to change configuration information based on the configuration change instruction carried in the data stream.

19. An operation and maintenance data processing system, characterized in that: It includes configuring a control center and a stream processing engine, wherein the stream processing engine includes a data source node, an operator node, and a data output node; The configuration control center is used to obtain configuration information of the detected object from the configuration management database; the detected object includes at least one of an operation and maintenance data processing node, a cloud network device, and a virtual instance associated with the configuration control center; based on changes in the configuration information of the detected object, a configuration change instruction is generated according to the changed configuration information; and the configuration change instruction is sent to the data source node; The data source node is used to receive the configuration change instruction; and add the configuration change instruction to the data stream corresponding to the current operation and maintenance job; If the configuration change conditions are met, change the data source configuration information based on the configuration change instruction; transmit the data stream carrying the configuration change instruction to the operator node; The operator node is used to change the data source configuration information based on the configuration change instruction when the configuration change condition is met; The data output node is used to change the data source configuration information based on the configuration change instruction when the configuration change conditions are met, and send the changed configuration results of the operation and maintenance data processing nodes that meet the configuration change conditions to the configuration control center.

20. The operation and maintenance data processing system according to claim 19, characterized in that: The configuration information of the detected object includes at least one of the following: change information of cloud network devices and virtual instances, configuration information of data source nodes, calculation logic of operator nodes, and configuration information of data output nodes.

21. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The processor runs the computer program to implement the method according to any one of claims 1 to 14 or 15 to 17.

22. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method according to any one of claims 1 to 14 or 15 to 17.

23. A computer program product comprising a computer program, characterized in that The computer program is executed by a processor to implement the method of any one of claims 1 to 14 or 15 to 17.

Images