Circuit prediction connection method and apparatus based on graphic database, and device, medium and program product

Abstract

The present invention relates to the technical field of information technology. Provided are a circuit prediction connection method and apparatus based on a graphic database, and a device, a medium and a program product. The present invention expands originally fixed urban local stations at two ends serving as a start station and an end station for circuit prediction connection into a start city serving as a start to predict all circuits that a customer can connect to, thereby reducing the limitations of circuit prediction connection, and providing more resource efficient circuit ranges for selection. Compared with existing methods in which relational databases are used to identify data resources for system resources and medium resources which need to be occupied for circuit connection, the present invention involves a query speed that is greatly faster than the original one, thereby significantly reducing the resource consumption. Compared with the existing query logic which requires the separate development of customized programs for processing, the present invention can support the configuration of screening conditions without the need for additional separate logical processing, thereby reducing the investment of subsequent manpower and material resources.

Description

Circuit prediction and activation method, device, equipment, medium and program product based on graphic database Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a circuit prediction and activation method, device, equipment, medium and program product based on a graphic database. Background Art

[0002] With the rapid development of the internet, the variety and volume of data on the internet are growing at an astonishing rate, marking the advent of the big data era. The application scenarios of non-relational database storage are becoming increasingly social and practical. As big data is increasingly applied across various industries, querying massive amounts of data and complex hierarchical relationships presents unprecedented challenges. This is particularly true in business scenarios involving multi-layered queries, such as those involving circuit-related queries involving occupied system resources, physical resources, and the carrying relationships between links.

[0003] Generally, relational databases are usually used to store and process relational data. However, due to the inherent disadvantages of relational databases, they are unable to efficiently and quickly query multi-level relational data, which affects the business use of multi-level queries.

[0004] Summary of the Invention

[0005] Purpose of the invention: To propose a circuit prediction and activation method, device, equipment, medium and program product based on a graphic database to solve the above-mentioned problems existing in the prior art.

[0006] First, a circuit activation prediction method based on a graph database is proposed, with the following steps:

[0007] S1. Circuits, devices, and ports are considered nodes, where nodes represent entity data; relationships between entity data are considered edges; and the nodes and edges are combined to form a graph database.

[0008] S2. Real-time monitoring of the data table transferred from the database server to the image database, and collection of data change logs;

[0009] S3. Push the data change log collected in step S2 to different partitions of the topic through Kafka messages;

[0010] S4, matching mapping fields of the data table transferred to the image database in step S2 to obtain a mapping relationship between each entity data and the attribute structure to be transferred to the image library;

[0011] S5. Set up multiple consumer groups, assign multiple consumers to each consumer group to consume the messages pushed by Kafka in step S3;

[0012] S6. Process the pushed message module and transfer it into the graphic database according to the mapping relationship output in step S4;

[0013] S7. Setting the priority of each node, calculating the optimal path, and executing the circuit prediction and activation process based on the optimal path to obtain complete routing information;

[0014] S8. The complete routing information is presented on a map page by dotting and connecting lines.

[0015] In a further embodiment of the first aspect, the data change log in step S2 includes a new record, a deletion record, and a modification record for each piece of data.

[0016] In a further embodiment of the first aspect, the process of calculating the optimal path in step S7 includes:

[0017] Calculate and assign the length value of the system resources of the bearer link. Calculate the specific length between the two devices based on the longitude and latitude attribute information of the devices at both ends of the link.

[0018] If the device lacks longitude and latitude information and the length value cannot be calculated, the system resource is labeled with the lowest priority and will not be prioritized when calculating the optimal path.

[0019] In a further embodiment of the first aspect, step S4 also includes setting the following filtering conditions to filter the data transferred to the graphic database: configuring the main tag value and other tag values; checking whether the data source of the device is the backbone library or cloud library; checking the link information associated with the device; and querying the system resource conditions for circuit filtering.

[0020] In a further embodiment of the first aspect, the circuit prediction and activation method also includes: according to the filtering conditions configured in step S4, when querying the gallery, the configured filtering conditions are used as the hierarchical query conditions of the gallery, and according to the circuit carrying relationship, the type of node and the type of relationship are restricted, and all available system resources are queried as an available resource pool.

[0021] In a further embodiment of the first aspect, the circuit prediction activation method also includes: starting from the starting point according to the available resource pool, adopting a greedy algorithm strategy, traversing to the adjacent node of the vertex closest to the starting point and not visited each time until expanding to the end point, and calculating all available resources.

[0022] In a further embodiment of the first aspect, step S7 further includes: interacting with the province to obtain routing information from the provincial bureau to the customer side; splicing the provincial routing information with the backbone side routing information according to predetermined rules to obtain complete routing information.

[0023] In a further embodiment of the first aspect, the splicing rules include: connecting the Z end of the starting province with the A end of the backbone side, and connecting the Z end of the backbone side with the A end of the terminating province according to the mapping relationship between the bureau station on the backbone side and the bureau station within the province; identifying the occupied port name; and identifying the occupied time slot.

[0024] In a second aspect, a circuit prediction and activation device is provided, the device comprising:

[0025] A source data acquisition unit, configured to monitor in real time the data tables transferred from the database server to the image database and collect data change logs;

[0026] A data push unit is used to push the data change log collected by the source data collection unit to different partitions of the topic through Kafka messages;

[0027] A data configuration unit is used to match mapping fields of the data table to be transferred to the image database, and obtain a mapping relationship between each entity data and the attribute structure to be transferred to the image library;

[0028] The data receiving unit is used to set up multiple consumer groups. Each consumer group is assigned multiple consumers to consume the messages pushed by Kafka.

[0029] a data module conversion unit, configured to process the pushed message module and convert it into the graphic database according to the mapping relationship output by the data configuration unit;

[0030] A data processing unit, configured to set the priority of each node, calculate the optimal path, execute the circuit prediction and activation process based on the optimal path, and obtain complete routing information;

[0031] The data presentation unit is used to present the complete routing information on a map page by dotting and connecting lines.

[0032] In a further embodiment of the second aspect, the apparatus further comprises:

[0033] The multi-threaded warehousing unit uses a multi-threaded approach to simultaneously process data for modular transfer and warehousing, thus avoiding message backlogs.

[0034] The data extraction and filtering unit is used to use the filtering conditions configured by the data configuration unit as the hierarchical query conditions of the library when querying the library, and to limit the types of nodes and relationships based on the circuit carrying relationship, and query all available system resources;

[0035] A data algorithm unit is used to calculate all available resources based on all available system resources queried by the data extraction and filtering unit as an available resource pool;

[0036] The provincial routing unit is used to interact with the province and obtain routing information from the provincial office station to the customer side;

[0037] The data splicing unit is used to splice the intra-provincial routing information and the backbone side routing information according to predetermined rules.

[0038] In a third aspect, an electronic device is proposed, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the circuit prediction activation method as described in the first aspect is implemented.

[0039] In a fourth aspect, a computer-readable storage medium is proposed, wherein the storage medium stores at least one executable instruction. When the executable instruction is executed on an electronic device, the electronic device executes the circuit prediction activation method as described in the first aspect.

[0040] In a fifth aspect, a computer program product is proposed. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the circuit prediction activation method as described in the first aspect.

[0041] The present invention has the following beneficial effects: the present invention expands the original urban bureau stations with fixed ends as the starting and ending sites for circuit prediction and activation to use the starting city as the starting point to predict all circuits that can be activated by customers, reducing the limitations of circuit prediction and activation and providing more resource-saving circuit ranges for selection.

[0042] Compared with the existing relational database that occupies system resources, medium resources, and identification data resources for circuit opening, the query speed of the present invention is greatly improved, and resource consumption is greatly reduced.

[0043] Compared with the existing query logic, which requires the development of a customized program to process, the present invention can support the configuration of screening conditions, eliminating the need to add separate logical processing and reducing subsequent investment in manpower and material resources.

[0044] The present invention calculates and splices the routing data within the province with the backbone segment to present the complete routing information between two points. At the same time, it can directly calculate the information of 1A / 2A / 3A circuits. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] FIG1 is a flow chart of a circuit prediction and activation method based on a graph database in one embodiment of the present invention.

[0046] FIG2 is a schematic diagram of the structure of a circuit prediction and activation device based on a graphic database in one embodiment of the present invention. DETAILED DESCRIPTION

[0047] In the following description, numerous specific details are provided to provide a more thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without one or more of these details. In other instances, certain technical features well known in the art have not been described to avoid confusion with the present invention.

[0048] Example 1:

[0049] The applicant's research has found that with the increasing application of big data in various industries, queries involving massive amounts of data and complex hierarchical relationships are facing unprecedented challenges. This is particularly true in business scenarios involving multi-layered queries, such as those related to circuit-related occupied system resources, physical resources, and the carrying relationships between links. Relational databases are typically used to store and process relational data. However, due to the inherent shortcomings of relational databases, they are unable to efficiently and quickly query multi-layered relational data, which has a significant impact on the business use of multi-layered queries.

[0050] Currently, relevant technicians use special graph databases to store big data, using circuits, system resources, physical resources, etc. as nodes and the relationships between them as edges in the graph database. By adding labels, directions and attributes to nodes and relationships, it is the best way to store, query and analyze highly interconnected data.

[0051] To this end, embodiments of the present invention propose a graph-based circuit activation method, addressing the inherent limitations of relational databases, which hinder efficient storage and querying of relevant resources during the circuit activation process. This method addresses the efficiency issues associated with multi-level interconnected data processing by utilizing a unique storage method. This method reduces the storage space required by relational databases by half, and, using the most common Cypher graph query language as an example, significantly reduces the amount of code required for complex relational queries compared to SQL, significantly improving query speed.

[0052] In this example, we first need to build a graph database: a graph is a collection of nodes and edges. Nodes represent entity data, such as circuits, devices, and ports; edges represent relationships between entity data, such as a circuit terminating at a device, or an optical cable containing segments. These entities and relationships are stored in the graph, enabling rapid responses to complex relational queries and more intuitive visualization.

[0053] The steps of the circuit prediction and activation method based on the graph database proposed in this embodiment are shown in FIG1 :

[0054] S1. Circuits, devices, and ports are regarded as nodes, where nodes represent entity data; relationships between entity data are regarded as edges; and the nodes and edges are combined to form a graph database.

[0055] S2. Real-time monitoring of the data table transferred from the database server to the image database, and collection of data change logs.

[0056] S3. Push the data change log collected in step S2 to different partitions of the topic through Kafka messages.

[0057] S4. Match the mapping fields of the data table transferred to the image database in step S2 to obtain a mapping relationship between each entity data and the attribute structure to be transferred to the image library. Set the following filtering conditions to filter the data transferred to the image database: configure the main tag value and other tag values; check whether the device's data source is a backbone library or cloud library; check the link information associated with the device; and query the system resource conditions for circuit filtering. Based on the filtering conditions configured in step S4, when querying the image library, the configured filtering conditions are used as the hierarchical query conditions of the image library. Based on the circuit carrying relationship, the node type and relationship type are restricted, and all available system resources are queried to form an available resource pool.

[0058] S5. Set up multiple consumer groups, assign multiple consumers to each consumer group to consume the messages pushed by Kafka in step S3.

[0059] S6. According to the mapping relationship output in step S4, the pushed message module is processed and transferred into the graphic database.

[0060] S7. Set the priority of each node, calculate the optimal path, execute the circuit prediction and activation process based on the optimal path, and obtain complete routing information; calculate and assign the length value of the system resource carrying the link, and calculate the specific length between the two devices based on the longitude and latitude attribute information of the device information at both ends of the link; if the device lacks longitude and latitude information and the length value cannot be calculated, then the system resource is marked with the lowest priority and will not be given priority when calculating the optimal path.

[0061] S8. The complete routing information is presented on a map page by dotting and connecting lines.

[0062] Example 2:

[0063] This embodiment provides a circuit prediction and activation device based on a graph database, the device comprising:

[0064] Source data collection unit: Use the wal2json collection plug-in to monitor the data tables that need to be transferred from postgresql (database server) to the library, and collect the monitoring logs (changes in each data such as addition, deletion, and modification).

[0065] Data push unit: pushes the collected data change logs to different partitions of the topic through Kafka messages.

[0066] Data configuration unit: Through the page configuration function, the data that needs to be transferred from postgresql (database server) to the library is matched with mapping fields, the main tag value and other tag values ​​are configured, the data source of the device is the backbone library / cloud library, etc., the link information associated with the device, the system resource conditions for query circuit filtering, etc.

[0067] Data receiving unit: To avoid Kafka message backlog, the Java program sets up multiple consumer groups, and each consumer group is assigned multiple consumers to consume the messages pushed by Kafka.

[0068] Data model conversion unit: Based on the relationship configured by the data configuration unit, that is, the mapping relationship between each entity data of PostgreSQL and the attribute structure to be modeled into the library, the pushed messages are processed and modeled into the library.

[0069] Multi-threaded warehousing unit: uses multi-threading to simultaneously process data for modular warehousing, avoiding message backlogs that cause high memory usage and program freezes.

[0070] Data Processing Unit: Because circuit prediction and activation involve calculating the shortest or optimal path, links exist as nodes. The system resources carrying the link must first be assigned a length value. Based on the longitude and latitude attribute information of the devices at both ends of the link, the specific distance (in kilometers) between the two devices is calculated. If a device lacks longitude and latitude information, the length cannot be calculated, and the system resource is labeled as the lowest priority and will not be prioritized when calculating the optimal path.

[0071] Data extraction and filtering unit: According to the filtering conditions configured by the data configuration unit, when querying the library, the configured filtering conditions are used as the hierarchical query conditions of the library, and according to the circuit carrying relationship, the type of node and the type of relationship are restricted to query all available system resources.

[0072] Data algorithm unit: Based on all available system resources queried by the data extraction and filtering unit as the available resource pool, according to the Dijkstra algorithm (the shortest path in a weighted graph), starting from the starting point, using the greedy algorithm strategy, each time traversing to the adjacent node of the vertex closest to the starting point and not visited, until expanding to the end point, calculate all available resources.

[0073] Intra-provincial routing unit: interacts with the province to obtain routing information from the provincial office station to the customer side.

[0074] Data splicing unit: Splices the provincial routing information with the backbone routing information according to the following rules:

[0075] 1) According to the mapping relationship between the bureau stations on the backbone side and the bureau stations within the province, connect the Z end of the starting province to the A end of the backbone side, and connect the Z end of the backbone side to the A end of the terminating province.

[0076] 2) Use 〖〗 to mark the occupied port name.

[0077] 3) Use "" to mark the occupied time slot.

[0078] For example: Shanghai Telecom "Shanghai Telecom Building Yokohama (Denchuan 7#) Optical Cable / F77&78", Shanghai Yokohama "South Coast 16-wavelength DWDM System / dλ14", Hangzhou Wulin Bureau "South Coast 16-wavelength DWDM System / dλ14", Ningbo Jiebei Hub Building "South Coast 16-wavelength DWDM System / dλ14", Wenzhou South Railway Station Bureau "South Coast 16-wavelength DWDM System / dλ14", Fuzhou May Day.

[0079] Data presentation unit: The complete routing information calculated by the gallery program is presented on the map page by dot-connecting.

[0080] Example 3:

[0081] The following embodiment proposes an electronic device, which includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the circuit prediction activation method described in Examples 1 to 2. Specifically, the above-mentioned processor may include a central processing unit (CPU), or a specific integrated circuit (Application Specific Integrated Circuit, ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application. The memory may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory may include removable or non-removable (or fixed) media. Where appropriate, the memory may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory is a non-volatile solid-state memory.

[0082] The memory may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of the present disclosure. The processor implements the circuit prediction activation method in the above-mentioned embodiment by reading and executing the computer program instructions stored in the memory.

[0083] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes may be made to it in form and detail without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A circuit prediction and activation method based on a graphic database, characterized in that: include: S1. Circuits, devices, and ports are taken as nodes, where nodes represent entity data; relationships between entity data are taken as edges; and the nodes and edges are combined to form a graph database; S2, real-time monitoring of the data table transferred from the database server to the image database, and collection of data change logs; S3. Push the data change log collected in step S2 to different partitions of the topic through Kafka messages; S4, matching the mapping fields of the data table transferred to the image database in step S2, and obtaining a mapping relationship between each entity data and the attribute structure to be transferred to the image library; S5. Set up multiple consumer groups, assign multiple consumers to each consumer group to consume the messages pushed by Kafka in step S3; S6, according to the mapping relationship output in step S4, processing the pushed message module and transferring it into the graphic database; S7, setting the priority of each node, calculating the optimal path, executing the circuit prediction and activation process based on the optimal path, and obtaining complete routing information; S8, presenting the complete route information on a map page by dotting and connecting lines.

2. The circuit prediction and activation method according to claim 1, characterized in that: The data change log in step S2 includes a new record, a deletion record, and a modification record for each piece of data.

3. The circuit prediction and activation method according to claim 1, characterized in that: The process of calculating the optimal path in step S7 includes: Calculate and assign the length value of the system resources of the bearer link, and calculate the specific length between the two devices based on the longitude and latitude of the attribute information of the device information at both ends of the link; If the device lacks longitude and latitude information and the length value cannot be calculated, the system resource is labeled with the lowest priority and will not be prioritized when calculating the optimal path.

4. The circuit prediction and activation method according to claim 1, characterized in that: Step S4 also includes setting the following screening conditions to screen the data transferred to the graphic database: Configure the primary tag value and other tag values; Check whether the data source of the device is the backbone library or cloud library; Check the link information associated with the device; Query the system resource conditions for circuit filtering.

5. The circuit prediction and activation method according to claim 4, characterized in that: Also includes: According to the filtering conditions configured in step S4, when querying the gallery, the configured filtering conditions are used as the hierarchical query conditions of the gallery, and according to the circuit carrying relationship, the type of node and the type of relationship are restricted, and all available system resources are queried as an available resource pool.

6. The circuit prediction and activation method according to claim 5, characterized in that: Also includes: According to the available resource pool, starting from the starting point, a greedy algorithm strategy is adopted to traverse the adjacent nodes of the vertices that are closest to the starting point and have not been visited each time until the end point is expanded to calculate all available resources.

7. The circuit prediction and activation method according to claim 1, characterized in that: Step S7 further comprises: Interact with the province to obtain routing information from the provincial office to the customer side; The intra-provincial routing information and the backbone side routing information are spliced ​​according to predetermined rules to obtain complete routing information.

8. The circuit prediction and activation method according to claim 7, characterized in that: The splicing rules include: According to the mapping relationship between the office stations on the backbone side and the office stations in the province, the Z end of the starting province is connected to the A end of the backbone side, and the Z end of the backbone side is connected to the A end of the terminating province; Identify the occupied port name; Identifies the occupied time slots.

9. A circuit prediction and opening device, characterized in that: The device comprises: A source data acquisition unit, used for real-time monitoring of the data table transferred from the database server to the image database, and collecting data change logs; A data push unit, used to push the data change log collected by the source data collection unit to different partitions of the topic through a Kafka message; A data configuration unit is used to match the mapping fields of the data table to be transferred to the image database, and obtain the mapping relationship between each entity data and the attribute structure to be transferred to the image library; The data receiving unit is used to set up multiple consumer groups. Each consumer group is assigned multiple consumers to consume the messages pushed by Kafka. A data model conversion unit, used for processing the pushed message model into the graphic database according to the mapping relationship output by the data configuration unit; A data processing unit, used to set the priority of each node, calculate the optimal path, execute the circuit prediction and activation process based on the optimal path, and obtain complete routing information; The data presentation unit is used to present the complete route information on a map page by dotting and connecting lines.

10. The circuit prediction and activation device according to claim 9, characterized in that: The device also includes: The multi-threaded warehousing unit uses a multi-threaded approach to simultaneously process data for modular transfer and warehousing, thus avoiding message backlogs. The data extraction and filtering unit is used to use the configured filtering conditions as the hierarchical query conditions of the library when querying the library according to the filtering conditions configured by the data configuration unit, and to limit the types of nodes and the types of relationships according to the circuit carrying relationship. Type, query all available system resources; A data algorithm unit is used to calculate all available resources based on all available system resources queried by the data extraction and filtering unit as an available resource pool; The intra-province routing unit is used to interact with the province and obtain the routing information from the provincial office station to the customer side; The data splicing unit is used to splice the intra-provincial routing information with the backbone side routing information according to predetermined rules.

11. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the circuit prediction activation method according to any one of claims 1 to 8 is implemented.

12. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction. When the executable instruction is executed on the electronic device, the electronic device executes the circuit prediction and activation method according to any one of claims 1 to 8.

13. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the circuit prediction and activation method according to any one of claims 1 to 8.

Images