System and method for analyzing point resource cutover influence based on graph database

Abstract

The invention discloses a point resource cutover influence analysis system and method based on a graph database, and the method comprises the steps: carrying out the cutover of point resources occupied in a transmission process through a data monitoring module, a data pushing module, a data consumption module, a data module conversion module, a cutover capability module, a graph library algorithm module, a cutover topology module and a cutover falling graph module; on the basis of the characteristics of a graph database, point resources serve as different entity nodes, the relation between the point resources serves as an edge, meanwhile, service systems influenced by point resource cutover are quickly queried through a graph database query algorithm, and services influenced by cutover and associated resources are subjected to topological presentation, so that the method is more efficient and efficient. Therefore, a detailed cutover scheme and an emergency plan can be made for an operator, and the cutover can be quickly and smoothly completed.

Description

Technical Field

[0001] The present invention belongs to the technical field of network cutover impact analysis, and particularly relates to a system and method for analyzing the impact of point resource cutover based on a graph database. Background Art

[0002] In order to ensure the stable operation of network lines, operations such as regular maintenance and servicing of the existing network, improvement of line performance, replacement of equipment, line capacity expansion, and adjustment of the existing network operation mode are required. Therefore, cutover adjustments need to be made to point resources such as sites, computer rooms, equipment, chassis, channels, boards, and ports, and line resources such as channels, systems, and optical cables used in the transmission process. The cutover will directly affect the services carried on them. Therefore, before the cutover, it is necessary to know which services are affected and the interruption methods of the services, and then formulate a detailed cutover plan and fallback plan to minimize the cutover impact.

[0003] Based on the bearing relationship between point resources and services, for the station-computer room-equipment-chassis-board-channel-port used in the transmission process, service circuits are carried on the ports, and at the same time, the circuits can also occupy channels. The resource occupation hierarchical relationship is complex, and the prior art cannot clearly, comprehensively, and timely query all service information affected by the cutover of a point resource. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a system and method for analyzing the impact of point resource cutover based on a graph database in view of the above-mentioned deficiencies of the prior art.

[0005] To achieve the above technical purpose, the technical solution adopted by the present invention is as follows:

[0006] A system for analyzing the impact of point resource cutover based on a graph database includes a data monitoring module, a data pushing module, a data consuming module, a data model conversion module, a cutover capability module, a graph database algorithm module, a cutover topology module, and a cutover mapping module. Specifically:

[0007] The data monitoring module is used to monitor the change log of resource data in the relational database table after the cutover of the occupied point resources during the transmission process, and collect the log information of each data change;

[0008] The data pushing module is used to push the collected log information to different partitions in batches through the message storage middleware kafka for real-time consumption by different consumers;

[0009] The data consuming module is used to initiate the process of message storage in the database after the information pushed by kafka is received by the kafka message monitoring program, which is a pre-step of the model conversion program;

[0010] The data model conversion module is used to perform data model conversion according to the field mapping relationship between the relational database and the graph database of each type of resource data, taking the point resources as different entity nodes and the relationships between the point resources as edges and storing them in the graph database;

[0011] The cutover capacity module is used to query the resource data relationships in the graph database based on the fixed - ends algorithm of the graph database in the graph library algorithm module, and obtain the affected service types, service information, and service scopes;

[0012] The graph library algorithm module is used to adopt the fixed - ends algorithm of the graph database, combine the relationships between the point resources occupied during the transmission process, query the bearing relationships upward according to different resource types until the final bearing circuits affected by the cutover of the point resources are found, and at the same time obtain the intermediate affected circuits;

[0013] The cutover topology module is used to present the cutover - affected topology based on the data obtained from the query, visually display the association relationships between resources and the data of the affected service scopes;

[0014] The cutover mapping module is used to perform dot mapping on the gis map based on the data obtained from the query, and present the positions of each type of resource, as well as the directions and impacts of the resources on the map.

[0015] To optimize the above - mentioned technical solutions, the specific measures also include:

[0016] The above - mentioned resource data includes point resources, service data, and bearing relationship data.

[0017] The above - mentioned log information is collected using the wal2json collection plugin.

[0018] When setting up the process of storing the initiated messages into the database, multiple consumer groups are set to correspond to the data operation logs generated by different data sources. Each data source corresponds to a kafka topic. There are multiple partitions under one topic. Each topic corresponds to a consumer group. Each consumer group consists of multiple consumers. Each consumer consumes the data of different partitions to process a large number of incoming data change logs.

[0019] When performing the above - mentioned data model conversion, if the edge data comes in but the node data has not been pushed yet, tags are added to the incoming data, its status is set to pending, and then it is processed by regular polling.

[0020] The relationships between the above - mentioned transmission resources include inclusion, composition, division, and bearing.

[0021] The above - mentioned fixed - ends algorithm of the graph library is used to query the data that meets the head and tail ends and whose intermediate relationships are not disconnected.

[0022] During the execution of the above-mentioned gallery fixed-end algorithm, if the fixed node at the head end is the cut-over point resource type and the fixed node at the tail end is the port type under the device, the query syntax is as follows:

[0023] match p=(n:Site{gid:xxx})-[:Rack|Frame|Board|Slot*1..100]-(m:Port) return p

[0024] Among them, "[[:Rack|Frame|Board|Slot*1..100]]" are the passed hierarchical nodes. This syntax returns all port data under this site. Site {gid:xxx} represents a certain fixed site, gid is the unique identifier of the site, Rack|Frame|Board|Slot are the data types associated with the site, and 1..100 is within 100 levels;

[0025] If the fixed port type at the head end and the affected circuit type at the tail end are fixed, the query syntax is as follows:

[0026] match p=(n:Port{gid:xxx})-[:WDM System|SDH System|TMUX Time Slot System|Transmission Circuit*1..100]-(m:Transmission Circuit) return p

[0027] Among them, "[[:WDM System|SDH System|TMUX Time Slot System*1..100]]" are the relationships carried by different system types. No matter how many levels they carry each other, this syntax returns all affected circuits. Port {gid:xxx} represents a certain fixed port, gid is the unique identifier of the port, WDM System|SDH System|TMUX Time Slot System|Transmission Circuit are the data types associated with the port, and 1..100 is within 100 levels.

[0028] A method for analyzing the impact of point resource cut-over based on a graph database includes:

[0029] Monitoring the change log of resource data in the relational database table after the cut-over of the occupied point resources during the transmission process, and collecting the log information of each data change;

[0030] Pushing the collected log information to different partitions in batches through the message storage middleware kafka for different consumers to consume in real time;

[0031] After the information pushed by kafka is received by the kafka message monitoring program in the pre-step of the analog conversion program, initiate the process of message storage into the database;

[0032] Perform data model conversion according to the field mapping relationship between the relational database and the graph database of each resource data. Treat the point resources as different entity nodes, and store the relationships between the point resources as edges in the graph database;

[0033] Based on the fixed-end algorithm of the graph database in the graph library algorithm module, query the resource data relationship of the graph database to obtain the affected business types, business information, and business scope;

[0034] Adopt the fixed-end algorithm of the graph database, combine the relationships between the point resources occupied during the transmission process, and query the bearing relationship upwards according to different resource types until the final bearing circuit affected by the cutover of the point resources is found, and at the same time obtain the intermediate affected circuits;

[0035] Present the cutover impact topology based on the data obtained from the query, and intuitively display the association relationships between resources and the data of the affected business scope;

[0036] Based on the data obtained from the query, mark and plot on the GIS map to present the location of each resource, as well as the direction and impact of the resources on the map.

[0037] The present invention has the following beneficial effects:

[0038] The present invention performs cutover on the point resources occupied during the transmission process. Based on the characteristics of the graph database, the point resources are regarded as different entity nodes, and the relationships between the point resources are regarded as edges. At the same time, through the graph library query algorithm, it quickly queries which business systems are affected by the cutover of the point resources, and presents the topology of the affected services and associated resources, thereby providing guarantee for the operator to formulate a detailed cutover plan and emergency plan, and ensuring the rapid and smooth completion of the cutover.

[0039] The present invention is based on the graph database as the resource storage object, utilizes the fixed-end algorithm of the graph database, high query performance, and graph computing advantages, quickly queries and obtains all affected circuit information, solves the problems of difficult traversal of multi-layer relationships of end-to-end resources and complex data bearing relationships, improves query efficiency and accuracy, ensures the perfection of the cutover plan, improves the cutover efficiency, and reduces the potential risks brought by the cutover.

[0040] Regardless of the number of levels, the fixed-end algorithm of the graph database of the present invention will query out the data that meets the conditions at both ends and the intermediate relationships are not disconnected. Compared with the original relational database (pgsql or mysql), there is no need to query layer by layer or loop multiple times, and the performance consumption of the database is extremely small, and the query speed is fast.

[0041] The present invention comprehensively presents the relationship topology. Meanwhile, based on the GIS map, resource information and service information are displayed by dotting and connecting on the map, which can visually present the impact on the service network topology and the dotting and connecting on the GIS map can more intuitively observe the location of resource distribution, the routes of affected circuits, the routing directions and scopes of affected service routes, ensuring the perfection of the cutover plan, improving the cutover efficiency, reducing the potential risks brought by the cutover, and thus enhancing customer satisfaction.

[0042] The present invention can be applied to industries such as the communication industry, the Internet industry, and the financial industry, and scenarios such as network upgrade, equipment replacement, configuration change, and service migration.

[0043] The present invention can minimize the cutover impact on customers, enhance customer satisfaction, and contribute to increasing the market share. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is the system structure diagram of the present invention;

[0045] Figure 2 is the relationship between devices and hardware;

[0046] Figure 3 is the resource hosting relationship;

[0047] Figure 4 is the cutover impact topology diagram. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0049] Although the steps in the present invention are numbered, they are not used to limit the order of the steps. Unless the order of the steps is clearly stated or the execution of a certain step requires other steps as a basis, the relative order of the steps can be adjusted. It can be understood that the term "and / or" used herein involves and covers any and all possible combinations of one or more of the associated listed items.

[0050] A system for analyzing the impact of dot resource cutover based on a graph database according to the present invention, as Figure 1 shown, includes a data monitoring module, a data pushing module, a data consuming module, a data model conversion module, a cutover capability module, a graph library algorithm module, a cutover topology module, and a cutover mapping module. Specifically:

[0051] Data Monitoring Module: Specify which major categories of resource data (point resources, business data, bearer relationship data, etc.) need to be transferred from the relational database to the graph database in real time. By monitoring the change logs of relational database tables and using the wal2json collection plugin, collect the log information of each data change.

[0052] Data Push Module: According to the change logs of the collected resource data, push them to different partitions through the message storage middleware kafka. When pushing, set to push in batches, and configure to send no more than a fixed number of messages together and wait for a time less than a fixed delay time. This purpose is to ensure the real-time and accurate disorder-free of data and provide guarantee for subsequent model conversion. Because during the data model conversion operation, there will be millions of data modification logs, which are pushed to different partitions for different consumers to consume in real time, improving the real-time consumption volume and the efficiency of data model conversion.

[0053] Data Consumption Module: When the pushed kafka message is received by the kafka message monitoring program, which is a pre-step of the model conversion program, the program will start a separate process of message storage, ensuring that the consumption message and the storage logic are in different spaces, guaranteeing the isolation of transactions and improving the stability of message storage. At the same time, set multiple consumer groups, and each consumer group is assigned multiple consumer users to process the large amount of message data pushed. Specifically, multiple consumer groups correspond to the data operation logs generated by different data sources. Each data source corresponds to a kafka topic. There are multiple partitions under one topic. Each topic corresponds to a consumer group. Each consumer group consists of multiple consumers. Each consumer consumes the data of different partitions to process the large amount of data change logs pushed.

[0054] Data Model Conversion Module: According to the initialized configuration data and the field mapping relationship between the relational database and the graph database of each resource, to ensure the correctness of data order. In case there are situations where edge data comes in but node data has not been pushed yet, etc., label the incoming data and set the status to pending, and then process it by timed polling. This operation is mainly to ensure the accuracy and integrity of data.

[0055] For example, the relationship between the port and the circuit is pushed through kafka first. But when the program processes the model conversion, it is found that neither the port nor the circuit exists in the graph database, so this relationship data cannot be model-converted. At this time, the message will be set to the pending status. The program processes these pending data every 10 minutes (the time can be adjusted). If the port and the circuit also come in through messages and are model-converted within 10 minutes, then this pending relationship data will also be model-converted into the graph database, and the status will be changed to processed.

[0056] Cutover Capability Module: It provides a capability query service. Based on different incoming point resource information, it can query the resource data relationships in the graph database through the graph database algorithm module, obtain the affected service types, service information, and service scopes, and return the results after data processing.

[0057] Graph Database Algorithm Module: Here, the fixed two - end algorithm of the graph database (using depth - first search) is cited. Combining the relationships of inclusion, composition, division, and bearing among transmission resources, according to different resource types, it queries the bearing relationships upwards (without restricting the query level) until the final bearing circuit is found, and it also includes the intermediate affected circuits.

[0058] Fixed Two - End Algorithm of the Graph Database: Regardless of the number of levels, it will query out the data as long as the two ends meet the conditions and the intermediate relationships are not disconnected. Compared with the original relational database (pgsql or mysql), which requires layer - by - layer query or multiple loop queries, the performance consumption of the database is extremely large and the query speed is slow.

[0059] 1. The fixed node at the head end is the point resource type of the cutover, and the fixed node at the tail end is the port type under the device.

[0060] The query syntax is as follows:

[0061] match p=(n:Site{gid:xxx})-[:Rack|Frame|Board|Slot*1..100]-(m:Port)return p

[0062] The middle “[:Rack|Frame|Board|Slot*1..100]” is the passed - through hierarchical node. This syntax returns all port data under the site. Site{gid:xxx} represents a certain fixed site, gid is the unique identifier of the site, Rack|Frame|Board|Slot are the data types associated with the site, and 1..100 means within 100 levels.

[0063] 2. The fixed node at the head end is the port type, and the fixed node at the tail end is the affected circuit type.

[0064] The query syntax is as follows:

[0065] match p=(n:Port{gid:xxx})-[:WDM System|SDH System|TMUX Time Slot System|Transmission Circuit*1..100]-(m:Transmission Circuit)return p

[0066] In the middle, "[Wavelength Division Multiplexing (WDM) System|Synchronous Digital Hierarchy (SDH) System|TMUX Time Slot System*1..100]" represents the relationship carried by different system types passed through. Regardless of the number of levels of mutual carrying, all affected circuits will be returned. The port {gid:xxx} represents a certain fixed port, where gid is the unique identifier of the port. Wavelength Division Multiplexing (WDM) System|Synchronous Digital Hierarchy (SDH) System|TMUX Time Slot System|Transmission Circuit is the data type associated with the port, and 1..100 represents within 100 levels.

[0067] As Figure 2 shown: A site contains multiple machine rooms, a machine room contains multiple devices, a device contains multiple boards, a board contains multiple ports, and the ports are associated with both ends of Circuit 1 (this Circuit 1 is the affected circuit). Circuit 1 can be divided into time slots, and the time slots can carry circuits to obtain Affected Circuit 2. This cycle continues. Cutting a device directly affects the circuits associated with all ports under the device. And through the division of time slots by the circuit and the carrying of circuits by the time slots, more upper-layer service circuits will be affected.

[0068] If the port is associated with Wavelength Division Multiplexing (WDM) System resources, the Wavelength Division Multiplexing (WDM) System divides wavelength links, and the wavelength links carry Affected Circuit 1. Circuit 1 can be divided into time slots, and the time slots can carry circuits to obtain Affected Circuit 2. This cycle continues. Cutting a device directly affects the circuits associated with all ports under the device. And through the division of time slots by the circuit and the carrying of circuits by the time slots, more upper-layer service circuits will be affected.

[0069] If the port is associated with Synchronous Digital Hierarchy (SDH) System resources, port-associated TMUX systems, etc., refer to Figure 3 the process.

[0070] Cutting Topology Module: Present the cutting impact topology based on the data obtained after calculation according to the above capabilities. As Figure 4 shown, so as to more intuitively observe the association relationship between resources and the data of the affected service scope.

[0071] Cutting Map Module: Based on the data returned by the capabilities, make dot mapping on the returned data map based on the GIS map and display the layers, so as to more clearly observe the location of each resource on the map, as well as the trend and impact of the resources based on the map.

[0072] A method for analyzing the impact of point resource cutting based on a graph database according to the present invention includes:

[0073] Listening to the change logs of resource data in the relational database table after cutting the occupied point resources during the transmission process, and collecting the log information of each data change;

[0074] Pushing the collected log information to different partitions in batches through the message storage middleware kafka for real-time consumption by different consumers;

[0075] After the information pushed by Kafka is received by the Kafka message monitoring program, which is the pre-step of the analog conversion program, the process of message storage is initiated.

[0076] Data analog conversion is performed according to the field mapping relationship between the relational database and the graph database of each type of resource data. The point resources are used as different entity nodes, and the relationships between the point resources are stored as edges in the graph database.

[0077] Based on the fixed-end algorithm of the graph database in the graph library algorithm module, the resource data relationships in the graph database are queried to obtain the affected service types, service information, and service scopes.

[0078] The fixed-end algorithm of the graph database is adopted, combined with the relationships between the point resources occupied during the transmission process. According to different resource types, the bearing relationships are queried upwards until the final bearing circuit affected by the cutover of the point resources is found, and the intermediate affected circuits are also obtained.

[0079] According to the data obtained by the query, the presentation of the cutover impact topology is carried out to intuitively display the association relationships between resources and the data of the affected service scopes.

[0080] According to the data obtained by the query, dot mapping is performed based on the GIS map to present the positions of each type of resource, as well as the directions and impacts of the resources on the map.

[0081] The glossary of terms in this article is as follows:

[0082] Graph database: GRAPHIC DATABASE, which uses points and edges as basic storage units and has the characteristics of efficient storage and query. In this patent, it is mainly used to store point resource information and relationship data.

[0083] Cutover: cutover, which refers to operations such as network capacity expansion and upgrade on the network running in the live network. In this patent, the impacts brought by the cutover of point resources are mainly analyzed and queried.

[0084] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention. Any reference signs in the claims should not be regarded as limiting the claimed rights.

[0085] In addition, it should be understood that although this specification is described in terms of embodiments, not every embodiment contains only one independent technical solution. This narrative manner of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A point resource cutover impact analysis system based on a graph database, characterized in that: It includes data monitoring module, data push module, data consumption module, data conversion module, cutover capability module, image library algorithm module, cutover topology module and cutover map module. Specifically: The data monitoring module is used to monitor the change log of resource data in the relational database table after the occupied point resources are cutover during the transmission process, and collect the log information of each data change; The data push module is used to push the collected log information to different partitions in batches through the message storage middleware Kafka so that different consumers can consume it in real time; The data consumption module is used to initiate the process of storing messages after the information pushed by Kafka is received by the Kafka message monitoring program, which is the pre-step of the model conversion program; The data model conversion module is used to perform data model conversion based on the field mapping relationship between the relational database and the graph database for each resource data, and store the point resources as different entity nodes and the relationship between the point resources as edges in the graph database; The cutover capability module is used to query the resource data relationship of the graph database based on the fixed-end algorithm of the graph database of the graph library algorithm module to obtain the affected business type, business information, and business scope; The graph library algorithm module is used to use the graph database fixed-end algorithm, combined with the relationship between the point resources occupied during the transmission process, to query the upper layer for the bearer relationship according to different resource types, until the final bearer circuit affected by the point resource cutover is found, and the affected circuit in the middle is obtained at the same time; The cutover topology module is used to present the topology affected by the cutover based on the data obtained from the query, and intuitively display the relationship between resources and the affected business scope data; The cutover and mapping module is used to map the data obtained from the query based on the GIS map, and present the location of each resource, its direction and impact on the map.

2. According to the graph database-based point resource cutover impact analysis system of claim 1, it is characterized in that: The resource data includes point resources, business data, and bearer relationship data.

3. According to the graph database-based point resource cutover impact analysis system of claim 1, it is characterized in that: The log information is collected using the wal2json collection plug-in.

4. According to the graph database-based point resource cutover impact analysis system of claim 1, it is characterized in that: When initiating the process of message storage, multiple consumer groups are set to correspond to the data operation logs generated by different data sources. Each data source corresponds to a Kafka topic. There are multiple partitions under a topic. Each topic corresponds to a consumer group. Each consumer group consists of multiple consumers. Each consumer consumes different partition data to process the large amount of data change logs pushed.

5. According to the graph database-based point resource cutover impact analysis system of claim 1, it is characterized in that: When performing data model conversion, if edge data comes in but node data has not been pushed, the incoming data is labeled, its status is set to pending, and then polled and processed at a regular interval.

6. The point resource cutover impact analysis system based on graph database according to claim 1 is characterized in that: The relationship between the transmission resources includes inclusion, composition, division, and bearing.

7. The point resource cutover impact analysis system based on graph database according to claim 1 is characterized in that: The graph database fixed-end algorithm is used to query data that meets the head end and the tail end and whose intermediate relationships are continuously connected.

8. The point resource cutover impact analysis system based on graph database according to claim 1 is characterized in that: During the execution of the algorithm for fixing both ends of the graph database, if the head-end fixed node is a cutover point resource type and the end-end fixed node is a port type under the device, the query syntax is as follows: match p=(n:site{gid:xxx})-[:rack|frame|board|slot*1..100]-(m:port)return p Among them, "[: rack|frame|board|slot*1..100]" is the passed level node. This syntax returns all port data under the site. Site {gid:xxx} represents a fixed site. gid is the unique identifier of the site. Rack|frame|board|slot is the data type associated with the site. 1..100 is within 100 levels. If the port type of the head node is fixed and the affected circuit type is fixed at the end, the query syntax is as follows: match p=(n:port{gid:xxx})-[:Wavelength Division System|SDH System|TMUX Time Slot System|Transmission Circuit*1..100]-(m:Transmission Circuit)return p Among them, "[:Wavelength Division System|SDH System|TMUX Time Slot System*1..100]" is the relationship carried by different system types. This syntax returns all affected circuits regardless of how many levels of mutual carrying there are. Port {gid:xxx} indicates a fixed port, gid is the unique identifier of the port, wavelength division system|SDH system|TMUX time slot system|transmission circuit is the data type associated with the port, and 1..100 is within 100 levels.

9. A method for analyzing the impact of point resource cutover based on a graph database, characterized in that: include: Monitor the change log of resource data in the relational database table after the occupied point resources are cut over during the transmission process, and collect the log information of each data change; The collected log information is pushed to different partitions in batches through the message storage middleware Kafka so that different consumers can consume it in real time. After the information pushed by Kafka is received by the Kafka message monitoring program, which is the precursor of the model conversion program, the process of storing the message in the database is initiated; Data modeling is performed based on the field mapping relationship between the relational database and the graph database for each resource data. Point resources are treated as different entity nodes, and the relationships between point resources are stored as edges in the graph database. The graph database fixed-end algorithm based on the graph library algorithm module queries the resource data relationship of the graph database to obtain the affected business type, business information, and business scope; The graph database is used to fix the two ends of the algorithm. Combined with the relationship between the point resources occupied during the transmission process, the upper layer is queried for the bearer relationship according to different resource types until the final bearer circuit affected by the point resource cutover is found, and the affected circuits in the middle are obtained at the same time. The cutover-affected topology is presented based on the query data, and the relationship between resources and the affected business scope data are intuitively displayed; According to the data obtained from the query, the map is marked and mapped based on the GIS map to present the location of each resource as well as the direction and impact of the resource.

Images