Circuit prediction and activation method, device, equipment, medium and program product based on graphic database

Abstract

The present invention provides a circuit prediction and opening method, device, equipment, medium and program product based on a graphic database, and relates to the field of information technology technology. The present invention uses the original urban bureau stations with fixed ends as the starting and ending sites for circuit prediction and opening, and expands them to use the starting city as the starting point to predict all circuits that customers can open, thereby reducing the limitations of circuit prediction and opening, and providing more resource-saving circuit ranges for selection. Compared with the existing use of relational databases to occupy system resources, medium resources, and identification data resources for circuit opening, the query speed of the present invention is greatly improved compared with the previous ones, and resource consumption is greatly reduced. Compared with the existing query logic, which requires the development of a customized program to process separately, the present invention can support the configuration of screening conditions, without the need to add separate logic processing, and reduce the subsequent investment in manpower and material resources.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a circuit prediction and activation method, device, equipment, medium and program product based on a graphic database. Background Art

[0002] With the rapid development of the Internet, the types and scale of data on the Internet are growing at an alarming rate, and the application scenarios of non-relational database storage are becoming more and more social and life-oriented. With the gradual application of big data in various industries, massive data and complex related hierarchical queries have encountered unprecedented challenges. This is especially true in business scenarios involving multi-layer related queries, such as querying the occupied system resources, physical resources, and the carrying relationship between links related to circuits.

[0003] Generally, relational databases are usually used to store and process relational data. However, due to the inherent disadvantages of relational databases, they are unable to efficiently and quickly query multi-level relational data, which affects the business use of multi-level queries. Summary of the invention

[0004] Purpose of the invention: To propose a circuit prediction and activation method, device, equipment, medium and program product based on a graphic database to solve the above-mentioned problems existing in the prior art.

[0005] In the first aspect, a circuit prediction and activation method based on a graph database is proposed, and the steps are as follows:

[0006] S1. Circuits, devices, and ports are taken as nodes, where nodes represent entity data; relationships between entity data are taken as edges; and the nodes and edges are combined to form a graph database;

[0007] S2, real-time monitoring of the data table transferred from the database server to the image database, and collection of data change logs;

[0008] S3. Push the data change log collected in step S2 to different partitions of the topic through Kafka messages;

[0009] S4, matching the mapping fields of the data table transferred to the image database in step S2, and obtaining a mapping relationship between each entity data and the attribute structure to be transferred to the image library;

[0010] S5. Set up multiple consumer groups, assign multiple consumers to each consumer group to consume the messages pushed by Kafka in step S3;

[0011] S6, according to the mapping relationship output in step S4, processing the pushed message module and transferring it into the graphic database;

[0012] S7, setting the priority of each node, calculating the optimal path, executing the circuit prediction and activation process based on the optimal path, and obtaining complete routing information;

[0013] S8, presenting the complete route information on a map page by dotting and connecting lines.

[0014] In a further embodiment of the first aspect, the data change log in step S2 includes a new record, a deletion record, and a modification record for each piece of data.

[0015] In a further embodiment of the first aspect, the process of calculating the optimal path in step S7 includes:

[0016] Calculate and assign the length value of the system resources of the bearer link, and calculate the specific length between the two devices based on the longitude and latitude of the attribute information of the device information at both ends of the link;

[0017] If the device lacks longitude and latitude information and the length value cannot be calculated, the system resource is labeled with the lowest priority and will not be prioritized when calculating the optimal path.

[0018] In a further embodiment of the first aspect, step S4 also includes setting the following filtering conditions to filter the data transferred to the graphic database: configuring the main tag value and other tag values; checking whether the data source of the device is a backbone library or a cloud library; checking the link information associated with the device; and querying the system resource conditions for circuit filtering.

[0019] In a further embodiment of the first aspect, the circuit prediction and activation method also includes: according to the filtering conditions configured in step S4, when querying the gallery, the configured filtering conditions are used as the hierarchical query conditions of the gallery, and according to the circuit carrying relationship, the type of node and the type of relationship are restricted, and all available system resources are queried as an available resource pool.

[0020] In a further embodiment of the first aspect, the circuit prediction and activation method also includes: based on the available resource pool, starting from the starting point, adopting a greedy algorithm strategy, each time traversing the adjacent node of the vertex that is closest to the starting point and has not been visited, until expanding to the end point, and calculating all available resources.

[0021] In a further embodiment of the first aspect, step S7 further includes: interacting with the province to obtain routing information from the provincial bureau to the customer side; splicing the provincial routing information with the backbone side routing information according to predetermined rules to obtain complete routing information.

[0022] In a further embodiment of the first aspect, the splicing rules include: connecting the Z end of the starting province with the A end of the backbone side, and connecting the Z end of the backbone side with the A end of the terminating province according to the mapping relationship between the bureau stations on the backbone side and the bureau stations within the province; identifying the occupied port names; and identifying the occupied time slots.

[0023] In a second aspect, a circuit prediction opening device is provided, the device comprising:

[0024] A source data acquisition unit, used for real-time monitoring of the data table transferred from the database server to the image database, and collecting data change logs;

[0025] A data push unit, used to push the data change log collected by the source data collection unit to different partitions of the topic through a Kafka message;

[0026] A data configuration unit is used to match the mapping fields of the data table to be transferred to the image database, and obtain the mapping relationship between each entity data and the attribute structure to be transferred to the image library;

[0027] The data receiving unit is used to set up multiple consumer groups. Each consumer group is assigned multiple consumers to consume the messages pushed by Kafka.

[0028] A data model conversion unit, used for processing the pushed message model into the graphic database according to the mapping relationship output by the data configuration unit;

[0029] A data processing unit, used to set the priority of each node, calculate the optimal path, execute the circuit prediction and activation process based on the optimal path, and obtain complete routing information;

[0030] The data presentation unit is used to present the complete route information on a map page by dotting and connecting lines.

[0031] In a further embodiment of the second aspect, the apparatus further comprises:

[0032] The multi-threaded warehousing unit uses a multi-threaded approach to simultaneously process data for modular transfer and warehousing, thus avoiding message backlogs.

[0033] The data extraction and filtering unit is used to use the configured filtering conditions as the hierarchical query conditions of the library according to the filtering conditions configured by the data configuration unit when querying the library, and to query all available system resources according to the circuit carrying relationship, limiting the type of nodes and the type of relationships;

[0034] A data algorithm unit is used to calculate all available resources based on all available system resources queried by the data extraction and filtering unit as an available resource pool;

[0035] The intra-province routing unit is used to interact with the province and obtain the routing information from the provincial office station to the customer side;

[0036] The data splicing unit is used to splice the intra-provincial routing information with the backbone side routing information according to predetermined rules.

[0037] In a third aspect, an electronic device is proposed, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the circuit prediction activation method as described in the first aspect is implemented.

[0038] In a fourth aspect, a computer-readable storage medium is proposed, wherein at least one executable instruction is stored in the storage medium. When the executable instruction is executed on an electronic device, the electronic device executes the circuit prediction activation method as described in the first aspect.

[0039] In a fifth aspect, a computer program product is proposed. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the circuit prediction activation method as described in the first aspect.

[0040] The present invention has the following beneficial effects: the present invention expands the original urban bureau stations with fixed two ends as the starting and ending sites for predicting the opening of circuits to use the starting city as the starting point to predict all circuits that can be opened by customers, thereby reducing the limitations of predicting the opening of circuits and providing more resource-saving circuit ranges for selection.

[0041] Compared with the existing system resources, medium resources, and identification data resources required for circuit opening using a relational database, the query speed of the present invention is greatly improved compared with before, and resource consumption is greatly reduced.

[0042] Compared with the existing query logic, which requires the development of a customized program to process, the present invention can support the configuration of screening conditions without adding separate logic processing, thereby reducing the subsequent investment in manpower and material resources.

[0043] The present invention calculates and splices the routing data within the province with the backbone segment, presents the complete routing information between two points, and can directly calculate the information of 1A / 2A / 3A circuits. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flowchart of a circuit prediction and activation method based on a graphic database in one embodiment of the present invention.

[0045] Figure 2 It is a schematic diagram of the structure of a circuit prediction and activation device based on a graphic database in one embodiment of the present invention. Implementation

[0046] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features known in the art are not described. Example

[0047] The applicant has found that with the gradual application of big data in various industries, massive data and complex query levels have encountered unprecedented challenges. Especially in business scenarios involving multi-layered query, such as querying the occupied system resources, physical resources and the bearing relationship between links related to circuits, relational databases are generally used to store and process relational data. However, due to the inherent disadvantages of relational databases, they cannot efficiently query multi-layered relational data quickly, which affects the business use of multi-layered query.

[0048] Currently, relevant technicians have adopted special graph databases to store big data, using circuits, system resources, physical resources, etc. as nodes and the relationships between them as edges in the graph database. By adding labels, directions and attributes to nodes and relationships, it is the best way to store, query and analyze highly interconnected data.

[0049] To this end, an embodiment of the present invention proposes a circuit prediction and activation method based on a graph database to solve the inherent disadvantages of relational databases, which make it impossible to query related resources in the process of circuit prediction and activation, and to perform efficient storage and efficient query. The present invention solves the efficiency problem of multi-level interconnected data processing by using a special storage method, and the storage space is halved compared to the relational database. Taking the most common Cypher graph query language as an example, the amount of code for complex association queries is greatly reduced compared to SQL, and the query speed is greatly improved.

[0050] In this embodiment, we first need to build a graph database: a graph is a collection of nodes and edges, where nodes represent entity data, i.e., circuits, devices, ports, etc., and edges represent relationships between entity data, i.e., circuits terminate at devices, optical cables contain optical cable segments, etc. These entities and relationships are stored on the graph, enabling fast responses to complex association queries and more intuitive visualization.

[0051] The steps of the circuit prediction and activation method based on the graphic database proposed in this embodiment are shown in Figure 1 :

[0052] S1. Circuits, devices, and ports are taken as nodes, and nodes represent entity data; the relationships between entity data are taken as edges; and the nodes and edges are combined to form a graph database.

[0053] S2. Real-time monitoring of the data table transferred from the database server to the image database, and collection of data change logs.

[0054] S3. Push the data change log collected in step S2 to different partitions of the topic through Kafka messages.

[0055] S4. Match the mapping fields of the data table modeled into the image database in step S2 to obtain the mapping relationship between each entity data and the attribute structure to be modeled into the gallery. Set the following filtering conditions to filter the data modeled into the graphic database: configure the main tag value and other tag values; check whether the data source of the device is the backbone library or cloud library; check the link information associated with the device; query the system resource conditions for circuit filtering. According to the filtering conditions configured in step S4, when querying the gallery, the configured filtering conditions are used as the hierarchical query conditions of the gallery, and according to the circuit carrying relationship, the types of nodes and relationships are restricted, and all available system resources are queried as an available resource pool.

[0056] S5. Set up multiple consumer groups, assign multiple consumers to each consumer group to consume the messages pushed by Kafka in step S3.

[0057] S6. According to the mapping relationship outputted in step S4, the pushed message module is processed and transferred into the graphic database.

[0058] S7. Set the priority of each node, calculate the optimal path, execute the circuit prediction and activation process based on the optimal path, and obtain complete routing information; calculate and assign the length value of the system resources that carry the link, and calculate the specific length between the two devices based on the longitude and latitude attribute information of the device information at both ends of the link; if the device lacks longitude and latitude information and the length value cannot be calculated, then the system resource is labeled with the lowest priority and will not be given priority when calculating the optimal path.

[0059] S8, presenting the complete route information on a map page by dotting and connecting lines. Example

[0060] This embodiment provides a circuit prediction and activation device based on a graphic database, the device comprising:

[0061] Source data collection unit: Use the wal2json collection plug-in to monitor the data tables that need to be transferred from postgresql (database server) to the gallery, and collect the monitored logs (changes in each data such as addition, deletion, and modification).

[0062] Data push unit: pushes the collected data change logs to different partitions of the topic through Kafka messages.

[0063] Data configuration unit: Through the page configuration function, the data that needs to be transferred from postgresql (database server) to the library is matched with mapping fields, the main tag value and other tag values ​​are configured, the data source of the device is the backbone library / cloud library, etc., the link information associated with the device, and the system resource conditions for query circuit filtering.

[0064] Data receiving unit: To avoid backlog of Kafka messages, the Java program sets up multiple consumer groups, and each consumer group is assigned multiple consumers to consume the messages pushed by Kafka.

[0065] Data model conversion unit: According to the relationship that has been configured by the data configuration unit, that is, the mapping relationship between each entity data of postgresql and the attribute structure to be modeled into the library, the pushed messages are processed and modeled into the library.

[0066] Multi-threaded warehousing unit: Use multi-threading to simultaneously process data for modular transfer and warehousing, avoiding message backlogs that cause high memory usage and program jams.

[0067] Data processing unit: Circuit prediction and opening involve the calculation of the shortest path or the optimal path. The link exists as a node. It is necessary to first calculate and assign the length value of the system resources that carry the link. According to the attribute information of the longitude and latitude of the device information at both ends of the link, the specific length (KM) between the two devices is calculated. If the device lacks longitude and latitude information and the length value cannot be calculated, then the system resource is labeled with the lowest priority and will not be given priority when calculating the optimal path.

[0068] Data extraction and filtering unit: According to the filtering conditions configured by the data configuration unit, when querying the library, the configured filtering conditions are used as the hierarchical query conditions of the library, and according to the circuit carrying relationship, the types of nodes and relationships are restricted to query all available system resources.

[0069] Data algorithm unit: Based on all available system resources queried by the data extraction and filtering unit as the available resource pool, according to the Dijkstra algorithm (the shortest path in a weighted graph), starting from the starting point, using the greedy algorithm strategy, each time traversing to the adjacent node of the vertex closest to the starting point and not visited, until expanding to the end point, calculate all available resources.

[0070] Provincial routing unit: interacts with the province to obtain routing information from the provincial office to the customer side.

[0071] Data splicing unit: splices the intra-provincial routing information with the backbone routing information according to the following rules:

[0072] 1) According to the mapping relationship between the bureau stations on the backbone side and the bureau stations within the province, the Z end of the starting province is connected to the A end of the backbone side, and the Z end of the backbone side is connected to the A end of the terminating province.

[0073] 2) Use 〖〗 to mark the occupied port name.

[0074] 3) Use "" to mark the occupied time slot.

[0075] Example: Shanghai Telecom "Shanghai Telecom Building Yokohama (Denchuan 7#) Optical Cable / F77&78" Shanghai Yokohama "South Coast 16-wavelength DWDM System / dλ14" Hangzhou Wulin Bureau "South Coast 16-wavelength DWDM System / dλ14" Ningbo Jiebei Hub Building "South Coast 16-wavelength DWDM System / dλ14" Wenzhou South Station Bureau "South Coast 16-wavelength DWDM System / dλ14" Fuzhou May Day.

[0076] Data presentation unit: The complete route information calculated by the gallery program is presented on the map page by dot-connecting. Example

[0077] The following embodiment proposes an electronic device, which includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the circuit prediction opening method described in Embodiments 1 to 2 is implemented. Specifically, the above-mentioned processor may include a central processing unit (CPU), or a specific integrated circuit (Application Specific Integrated Circuit, ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application. The memory may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory is a non-volatile solid-state memory.

[0078] The memory may include a read-only memory (ROM), a random access memory (RAM), a disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of the present disclosure. The processor implements the circuit prediction opening method in the above-mentioned embodiment by reading and executing the computer program instructions stored in the memory.

[0079] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes in form and details may be made without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A circuit prediction and activation method based on a graphic database, characterized in that: include: S1. Circuits, devices, and ports are taken as nodes, where nodes represent entity data; relationships between entity data are taken as edges; and the nodes and edges are combined to form a graph database; S2, real-time monitoring of the data table transferred from the database server to the graphic database, and collection of data change logs; S3. Push the data change log collected in step S2 to different partitions of the topic through Kafka messages; S4, matching the mapping fields of the data table transferred to the graphic database in step S2, and obtaining the mapping relationship between each entity data and the attribute structure to be transferred to the graphic library; setting the following screening conditions to screen the data transferred to the graphic database: Configure the main tag value and other tag values; Check whether the data source of the device is the backbone library or cloud library; Check the link information associated with the device; Query the system resource conditions for circuit filtering; According to the configured filtering conditions, when querying the library, the configured filtering conditions are used as the hierarchical query conditions of the library, and according to the circuit carrying relationship, the types of nodes and the types of relationships are restricted, and all available system resources are queried as an available resource pool; S5. Set up multiple consumer groups, assign multiple consumers to each consumer group to consume the messages pushed by Kafka in step S3; S6, according to the mapping relationship output in step S4, processing the pushed message module and transferring it into the graphic database; S7, setting the priority of each node, calculating the optimal path, executing the circuit prediction and activation process based on the optimal path, and obtaining complete routing information; The calculating of the optimal path specifically includes: Calculate and assign the length value of the system resources of the bearer link, and calculate the specific length between the two devices based on the longitude and latitude of the attribute information of the device information at both ends of the link; If the device lacks longitude and latitude information and the length value cannot be calculated, the system resource will be labeled with the lowest priority and will not be prioritized when calculating the optimal path. The obtaining of complete routing information specifically includes: Interact with the province to obtain routing information from the provincial office to the customer side; The intra-provincial routing information and the backbone-side routing information are spliced ​​according to predetermined rules to obtain complete routing information; The splicing according to the predetermined rules specifically includes: connecting the Z end of the starting province with the A end of the backbone side, and connecting the Z end of the backbone side with the A end of the terminating province according to the mapping relationship between the bureau station on the backbone side and the bureau station in the province; Identify the occupied port name; Identify the occupied time slots; S8, presenting the complete route information on a map page by dotting and connecting lines.

2. The circuit prediction and activation method according to claim 1, characterized in that: The data change log in step S2 includes a new record, a deletion record, and a modification record for each piece of data.

3. The circuit prediction and activation method according to claim 1, characterized in that: Also includes: According to the available resource pool, starting from the starting point, a greedy algorithm strategy is adopted to traverse the adjacent nodes of the vertices that are closest to the starting point and have not been visited each time until the end point is expanded to calculate all available resources.

4. A circuit prediction and activation device, which can implement the circuit prediction and activation method according to any one of claims 1 to 3, characterized in that: The device comprises: A source data acquisition unit, used for real-time monitoring of the data table transferred from the database server to the graphic database, and collecting data change logs; A data push unit, used to push the data change log collected by the source data collection unit to different partitions of the topic through a Kafka message; A data configuration unit is used to match the mapping fields of the data table to be transferred to the graphic database, and obtain the mapping relationship between each entity data and the attribute structure to be transferred to the graphic library; The data receiving unit is used to set up multiple consumer groups. Each consumer group is assigned multiple consumers to consume the messages pushed by Kafka. A data model conversion unit, used for processing the pushed message model into the graphic database according to the mapping relationship output by the data configuration unit; A data processing unit, used to set the priority of each node, calculate the optimal path, execute the circuit prediction and activation process based on the optimal path, and obtain complete routing information; The data presentation unit is used to present the complete route information on a map page by dotting and connecting lines.

5. The circuit prediction and opening device according to claim 4, characterized in that: The device also includes: The multi-threaded warehousing unit uses a multi-threaded approach to simultaneously process data for modular transfer and warehousing, thus avoiding message backlogs. The data extraction and filtering unit is used to use the configured filtering conditions as the hierarchical query conditions of the library according to the filtering conditions configured by the data configuration unit when querying the library, and to query all available system resources according to the circuit carrying relationship, limiting the type of nodes and the type of relationships; A data algorithm unit is used to calculate all available resources based on all available system resources queried by the data extraction and filtering unit as an available resource pool; The intra-province routing unit is used to interact with the province and obtain the routing information from the provincial office station to the customer side; The data splicing unit is used to splice the intra-provincial routing information with the backbone side routing information according to predetermined rules.

6. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the circuit prediction activation method according to any one of claims 1 to 3 is implemented.

7. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction. When the executable instruction is executed on the electronic device, the electronic device executes the circuit prediction and activation method according to any one of claims 1 to 3.

8. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the circuit prediction and activation method according to any one of claims 1 to 3.

Images