Network element equipment information acquisition model conversion synchronization method and system

By adopting a new digital twin solution in network element equipment information collection, real-time/quasi-real-time acquisition and data comparison are achieved, the problems of long acquisition cycle and data accuracy are solved, data quality and availability are improved, and data is ensured in all links. transparency and visualization.

CN117807164BActive Publication Date: 2024-11-15CHINA TELECOM DIGITAL INTELLIGENCE TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202311852474.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-29
Publication Date
2024-11-15
Estimated Expiration
2043-12-29

AI Technical Summary

Technical Problem

In the prior art, network element equipment information collection has problems such as long collection and comparison periods, data cannot be guaranteed to be error-free, the division of labor between the equity and procurement control is unclear, and a large amount of invalid garbage data cannot be cleaned in time.

Method used

Adopt a new digital twin solution, through real-time acquisition/quasi-real-time acquisition, dynamic and real-time update of network configuration data is realized, and data comparison is performed using network element configuration twin database to ensure data error-free collection and accuracy, clear division of labor, and realization of Transparency and visualization of the entire data process.

Benefits of technology

It realizes dynamic and real-time updates of network configuration data, ensures data error-free collection, improves data quality and availability, and ensures the transparency and visualization of data in all links to a great extent.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117807164B_ABST
    Figure CN117807164B_ABST
Patent Text Reader

Abstract

The present invention provides a network element device information collection and analog conversion synchronization method and system, which relates to the field of data synchronization technology. The method includes: accessing a network element configuration buffer database, collecting configuration information data of the current stage from the network element configuration buffer database, pre-processing the configuration information data and storing it in a network element configuration twin database; reading first incremental log data from the network element configuration twin database, analog-converting the incremental log data to obtain second incremental log data, and storing it in the network element configuration analog conversion database; performing data processing on the second incremental log data and transferring it to a unified resource model database; accessing the network element configuration twin database and the unified resource model database to the application layer, configuring a collection task template at the application layer, configuring the collection indicators of the corresponding network element device, and completing the collection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data synchronization, and in particular to a method and system for synchronizing network element equipment information acquisition and analog conversion. Background Art

[0002] Cloud network digital twin is the core cornerstone of the new generation of cloud network operation system. Based on the unified data model of telecom cloud network resources, it realizes the data integration and association of network resources (network operation data + resource data) across professions, regions and levels, and gradually realizes the visualization and management of all cloud network resources, one-stop scheduling of two-level resources, and intensive management to improve the efficiency of cloud network business operation. It provides basic data support and services for business scenarios such as precision marketing, precision investment, speed increase and fee reduction, network optimization and data transfer, base station energy saving, mobile asset subdivision, and tower assessment.

[0003] The network element configuration twin database mainly stores network element configuration information, and converts resource-related data into the network element configuration conversion database according to the unified model mapping. Resource data includes equipment, hardware, links, ports, codes, and other resources and relationship data that can be collected and converted. Alarm, performance, log and other network operation data are still collected and shared and distributed using the original collection channels.

[0004] The current acquisition and control connection solution that uses scheduled collection, AUDK labeling for comparison between production and intermediate libraries, and deletion based on the number of collections has problems such as long collection and comparison cycles, the inability to guarantee error-free collection of collected data, unclear division of labor between resource sharing and acquisition and control boundaries, and the inability to clean up a large amount of invalid garbage data in a timely manner. Summary of the invention

[0005] Purpose of the invention: To propose a network element equipment information acquisition and analog-to-synchronization method and system, adopt a new digital twin solution, realize dynamic and real-time updating of network configuration data through real-time / quasi-real-time acquisition and control, and compare data with the network element configuration twin database to ensure error-free data collection and data accuracy, thereby effectively solving the above-mentioned problems existing in the prior art.

[0006] In the first aspect, a method for synchronizing network element device information collection is proposed, and the steps are as follows:

[0007] S1. Accessing a network element configuration buffer database, collecting configuration information data of the current stage from the network element configuration buffer database, and preprocessing the configuration information data and storing it in a network element configuration twin database;

[0008] S2. Read the first incremental log data from the network element configuration twin database, perform modular conversion on the incremental log data to obtain second incremental log data, and store the data in the network element configuration modular conversion database;

[0009] S3, performing data processing on the second incremental log data and transferring it into a unified resource model database;

[0010] S4. Connect the network element configuration twin database and the unified resource model database to the application layer, configure the collection task template in the application layer, configure the collection indicators of the corresponding network element equipment, and complete the collection.

[0011] In a further embodiment of the first aspect, the configuration information data is preprocessed in step S1, including: modeling the configuration information data according to a unified model: each type of data is modeled according to the assigned major category specifications and detailed category specifications and specifications, and the attribute range is modeled according to the fields defined by this specification.

[0012] In a further embodiment of the first aspect, in step S2, the first incremental log data is read from the network element configuration twin database, including: identifying changes in resource data collected between different batches as incremental data; marking and pushing the incremental data.

[0013] In a further embodiment of the first aspect, in step S2, converting the incremental log data to obtain second incremental log data includes:

[0014] The configuration information data is compared with the current model transfer library: if the model transfer library does not have it, it is a new tag; if the model transfer library has it and the data has changed, it is an update tag; if the model transfer library has it and the data has not changed, it is a keep tag; if the model transfer library has it but the latest collection does not have this data, it is a delete tag.

[0015] In a further embodiment of the first aspect, in step S3, data processing is performed on the second incremental log data and then transferred to a unified resource model database, including performing resource data splicing:

[0016] Collect the relationship data between entity resource data and the link data between the physical device ports on the az side:

[0017] When the physical device port is associated with the physical resource data, a relationship between the device and the entity is established;

[0018] When there is a link connection between the ports of the physical device, a link between the ports is established.

[0019] In a further embodiment of the first aspect, when there is a link connection between the ports of the physical device, a link between the ports is established, including:

[0020] The A-end port in the link is collected, and the relevant information of the Z-end port is collected. The acquisition and control is responsible for supplementing the Z-end port information into the link to form a complete link and then push the link information.

[0021] A second aspect of the present invention provides a network element device information acquisition analog-to-synchronous system, the network element device information acquisition analog-to-synchronous system comprising an acquisition control platform and an information sharing platform;

[0022] The acquisition and control platform includes the network element configuration buffer database; the resource sharing platform includes the network element configuration twin database, the network element configuration model conversion database, and the unified resource model database;

[0023] In addition, the network element equipment information collection analog conversion synchronization system also includes:

[0024] A first processing unit, configured to collect configuration information data of the current stage from the network element configuration buffer database, and store the configuration information data in the network element configuration twin database after preprocessing the configuration information data;

[0025] A second processing unit is used to read the first incremental log data from the network element configuration twin database, perform modular conversion on the incremental log data to obtain second incremental log data, and store the second incremental log data in the network element configuration modular conversion database;

[0026] A third processing unit, configured to perform data processing on the second incremental log data and transfer the data into a unified resource model database;

[0027] The fourth processing unit is used to connect the network element configuration twin database and the unified resource model database to the application layer, configure the collection task template in the application layer, configure the collection indicators of the corresponding network element equipment, and complete the collection.

[0028] According to a third aspect of the present invention, an electronic device is provided, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the network element device information collection analog-to-analog synchronization method as described in the first aspect is implemented.

[0029] The fourth aspect of the present invention proposes a computer-readable storage medium, characterized in that the storage medium stores at least one executable instruction, and when the executable instruction is run on an electronic device, the electronic device executes the network element device information collection analog-to-synchronization method as described in the first aspect.

[0030] Beneficial effect: The present invention provides a network element equipment information collection and analog-to-synchronization method, which realizes dynamic and real-time updating of network configuration data through real-time / quasi-real-time collection and control, and ensures error-free collection and data accuracy by comparing data with the network element configuration twin database; makes data traceable and controllable in the whole process, and greatly ensures the transparency and visualization of data processing in each link, thereby improving data quality and enhancing data availability. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart of the network element equipment information acquisition model conversion synchronization method proposed in an embodiment of the present invention.

[0032] Figure 2 It is a data flow diagram of the network element equipment information collection model conversion synchronization method proposed in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features known in the art are not described. Example 1

[0034] This embodiment discloses a method for synchronizing network element equipment information acquisition and analog conversion. The method is divided into two stages: acquisition and control and information sharing.

[0035] Acquisition and control stage: collect data on network element equipment, panels, ports, etc., store them in the "network element configuration twin database", and store them in the "network element configuration simulation database" after simulation conversion according to the unified model and agreed simulation conversion method. Finally, the acquisition and control interact with resources to store the data in the "unified resource model database". This method is used to build a digital twin library of network elements and networks. Based on the incremental log change mechanism of "network element->network element configuration twin database->network element configuration simulation database->unified resource model database", the acquisition and control and resource sharing platforms perform their respective duties to achieve real-time and accurate data twin processing.

[0036] The acquisition and control system periodically collects and normalizes network element resource data (resource data conversion: the data collected by the acquisition and control system is the data of each manufacturer. Different manufacturers have different data naming specifications. This step is to convert the resource data of different manufacturers through a unified model to achieve normalized storage of data from different manufacturers according to the unified model specifications). The network element configuration data is normalized by pulling the resource data from the unified resource model database, comparing it and marking it with the AUDK mark. The result is notified to the resource data sharing platform (hereinafter referred to as the resource sharing platform) through Kafka messages. The resource sharing platform enters the data into the network element configuration data conversion library (intermediate library) and the unified resource model database (original library).

[0037] Data processing division of labor:

[0038] Procurement and control is responsible for the collection of network element configuration data: real-time collection of manufacturers' existing network resource data;

[0039] Enter the twin database: After collection, the unconverted data is stored in the twin database according to the business model;

[0040] Normalization: Resource data is converted to a model. The data collected is from various manufacturers. Different manufacturers have different data naming specifications. This step is to convert the resource data of different manufacturers through a unified model to achieve normalized storage of data from different manufacturers according to the unified model specifications. The collected data is converted according to the unified model. Each type of data is converted according to the assigned major and minor specifications, and the attribute range is converted according to the fields defined by this specification.

[0041] For example

[0042] The collected router equipment analog conversion: the major category specification is equipment, the detailed category rule is IP equipment, the specification is router, and the analog conversion attributes include (code, name, specification ID, business status ID, manufacturer ID, working status ID, network management name, network access date, activation date, deactivation date, network exit date, etc.).

[0043] The collected switch equipment analog conversion: the major category specification is equipment, the detailed category rule is IP equipment, the specification is switch, and the analog conversion attributes include (code, name, specification ID, business status ID, manufacturer ID, working status ID, network management name, network access date, activation date, deactivation date, network exit date, etc.).

[0044] The collected port device analog conversion: the major category specification is port, the detailed category rule is the transport layer basic port, the specification is Ethernet port, and the analog conversion attributes include (code, name, physical device ID, logical device ID, hardware ID, network management code, network management name, physical status ID, blocking status ID, port type ID, working status ID, etc.).

[0045] Incremental identification: Identify changes in resource data collected between different batches, and label and push these incremental data. Compare the collected data with the current model transfer library. If the model transfer library does not have it, add a label; if the model transfer library has it and the data has changed, update the label; if the model transfer library has it and the data has not changed, keep the label; if the model transfer library has it but the latest collection does not have this data, delete the label.

[0046] Model transfer into model transfer library: after model transfer, the unified model data will be stored in the model transfer library;

[0047] Stitching within the twin library of network element configuration: Stitching of resource data in the twin library within the same network, including relationship data between entity resource data and link data between physical device ports at the az end; when entities are connected with entity data, for example, there is a physical B board on physical device A, the relationship between the device and the entity is established; or when there is a link connection between ports, the link between the ports is established. Usually, the collection will collect the A-end port in the link and collect relevant information of the Z-end port. The collection and control is responsible for supplementing the Z-end port information into the link to form a complete link and then push the link information.

[0048] Resource sharing stage: Compare the differences between the network element configuration model database and the unified resource model database to ensure that the data in the original database and the twin database are consistent;

[0049] Enter the unified resource model library according to the rules: configure the resource entry business primary key, field, and some verification rules according to the resource type, and read these configuration rules for processing before entering the library;

[0050] Manual processing mechanism for differential data: When the data in the original database and the twin database are inconsistent, they need to be displayed and provided with a manual processing channel;

[0051] Data splicing of twin libraries across network element configurations: Splicing of resource data in twin libraries across networks, including relationship data between entity resource data and link data between physical device ports on the az side.

[0052] Manual maintenance data: Provides manual maintenance data channel;

[0053] Automatically stitching together collected data: Automatically stitching together relational data and link data in cross-network data.

[0054] In the new generation of cloud network operation system, the acquisition and control platform is required to provide information collection and control services for various types of equipment. For IDC computer rooms, the equipment that needs to provide acquisition and control services mainly includes routers, switches, firewalls, IDS / IPS and other security equipment used in IDC computer rooms, star-level computer room internal quality test target servers (hereinafter referred to as "target servers"), etc. The specific collection content includes the resources, performance, configuration, alarm and other information of all the above equipment; the control aspect includes the basic resource configuration, main parameter configuration and business configuration of the above equipment. At the same time, in order to facilitate the use of upper-level applications, it should be possible to configure the collection indicators of the corresponding device model through the collection task template, and then complete the collection.

[0055] For data usage scenarios where the collected data is complex, the amount of collected data is large, and the real-time requirements are high, this invention is used to perform collection model conversion.

[0056] Database planning: Configure twin libraries and model-to-model libraries using different PG library instances. Apply for your own schema on demand on each PG library instance based on the professional network. Refer to the hardware configuration of the recently delivered PG library physical machine (2-way 16-core, 384G memory, 6*1.8T SAS data disks, 1.4T available storage per disk). Based on comprehensive considerations of resources and performance, initially build up to 3 schemas for each PG instance. Each schema plans to mount 1 disk as a table space. Of the remaining 3 disks, 1 is for PAAS self-use, and 2 are reserved for table space expansion.

[0057] Resource planning: To meet the implementation requirements of the IDC data communication twin solution, apply for the following resources:

[0058] 1. Two PG library instances (one master and two slaves) are used to deploy the twin library and the model conversion library respectively.

[0059] 2. Create an IDC data communication schema in each of the two PG library instances, and mount the tablespace as an exclusive drive letter. Example 2

[0060] This embodiment discloses a network element device information acquisition and analog conversion synchronization system, which includes a collection and control platform and a resource sharing platform; the collection and control platform includes a network element configuration buffer database; the resource sharing platform includes a network element configuration twin database, a network element configuration analog conversion database, and a unified resource model database;

[0061] In addition, the network element device information collection and analog conversion synchronization system also includes: a first processing unit, which is used to collect the configuration information data of the current stage from the network element configuration buffer database, and store the configuration information data in the network element configuration twin database after pre-processing; a second processing unit, which is used to read the first incremental log data from the network element configuration twin database, analog-convert the incremental log data to obtain the second incremental log data, and store it in the network element configuration analog conversion database; a third processing unit, which is used to perform data processing on the second incremental log data and then transfer it to a unified resource model database; a fourth processing unit, which is used to connect the network element configuration twin database and the unified resource model database to the application layer, configure the collection task template at the application layer, configure the collection indicators of the corresponding network element equipment, and complete the collection. Example 3

[0062] This embodiment provides an electronic device, which includes a processor, a memory, a communication interface and a communication bus. The processor, the memory and the communication interface communicate with each other through the communication bus. The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the network element device information collection analog conversion synchronization method disclosed in the above embodiment.

[0063] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication may be performed through an input / output (I / O) interface. In addition, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter. The network adapter communicates with other modules of the electronic device through a bus. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc. Example 4

[0064] This embodiment proposes a computer-readable storage medium, in which at least one executable instruction is stored. When the executable instruction is executed on an electronic device, the electronic device executes the network element device information acquisition analog-to-synchronization method as described in the above embodiment. More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0065] In summary of the above embodiments, the present invention is mainly to solve the problems of the current acquisition and control connection scheme that adopts timed collection, comparison of production library and intermediate library for AUDK labeling, and deletion processing according to the number of collections, such as long collection and comparison cycle, the collection data cannot be guaranteed to be error-free, the division of labor between resource sharing and acquisition control is unclear, and a large amount of invalid garbage data cannot be cleaned up in time. The pilot project now adopts a new digital twin solution, which realizes dynamic and real-time update of network configuration data through real-time / quasi-real-time collection of acquisition and control, and compares data with the network element configuration twin database to ensure error-free collection and data accuracy.

[0066] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes may be made to it in form and detail without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A network element equipment information acquisition analog conversion synchronization method, It is characterized in that include: S1. Accessing a network element configuration buffer database, collecting configuration information data of the current stage from the network element configuration buffer database, and preprocessing the configuration information data and storing it in a network element configuration twin database; S2. Read the first incremental log data from the network element configuration twin database, perform modular conversion on the incremental log data to obtain second incremental log data, and store the data in the network element configuration modular conversion database; Reading first incremental log data from the network element configuration twin database includes: identifying changes in resource data collected between different batches as incremental data; marking and pushing the incremental data; S3, performing data processing on the second incremental log data and transferring it into a unified resource model database, including performing resource data splicing: Collect the relationship data between entity resource data and the link data between the physical device ports on the az side: When the physical device port is associated with the physical resource data, a relationship between the device and the entity is established; When there is a link connection between the ports of the physical device, a link between the ports is established; S4. Connect the network element configuration twin database and the unified resource model database to the application layer, configure the collection task template in the application layer, configure the collection indicators of the corresponding network element equipment, and complete the collection.

2. The network element device information acquisition analog conversion synchronization method according to claim 1, It is characterized in that In step S1, the configuration information data is preprocessed, including: the configuration information data is modeled according to a unified model: each type of data is modeled according to the assigned major category specifications and detailed category specifications, and the attribute range is modeled according to the fields defined by the specifications.

3. The network element device information acquisition analog conversion synchronization method according to claim 1, It is characterized in that In step S2, the incremental log data is converted to obtain second incremental log data, including: The configuration information data is compared with the current model transfer library: if the model transfer library does not have it, it is a new tag; if the model transfer library has it and the data has changed, it is an update tag; if the model transfer library has it and the data has not changed, it is a keep tag; if the model transfer library has it but the latest collection does not have this data, it is a delete tag.

4. The network element device information acquisition analog conversion synchronization method according to claim 1, It is characterized in that When there is a link connection between the ports of the physical device, a link between the ports is established, including: The A-end port in the link is collected, and the relevant information of the Z-end port is collected. The acquisition and control is responsible for supplementing the Z-end port information into the link to form a complete link and then push the link information.

5. A network element equipment information collection analog conversion synchronization system, It is characterized in that Including acquisition control platform and information sharing platform; The acquisition and control platform includes a network element configuration buffer database; The resource sharing platform includes a network element configuration twin database, a network element configuration model conversion database, and a unified resource model database; The network element equipment information collection model conversion synchronization system also includes: A first processing unit, configured to collect configuration information data of the current stage from the network element configuration buffer database, and store the configuration information data in the network element configuration twin database after pre-processing; The second processing unit is used to read the first incremental log data from the network element configuration twin database, perform modular conversion on the incremental log data to obtain second incremental log data, and store the data in the network element configuration modular conversion database; reading the first incremental log data from the network element configuration twin database includes: identifying changes in resource data collected between different batches as incremental data; marking and pushing the incremental data; The third processing unit is used to perform data processing on the second incremental log data and then transfer it into the unified resource model database, including performing resource data splicing: Collect the relationship data between entity resource data and the link data between the physical device ports on the az side: When the physical device port is associated with the physical resource data, a relationship between the device and the entity is established; When there is a link connection between the ports of the physical device, a link between the ports is established; The fourth processing unit is used to connect the network element configuration twin database and the unified resource model database to the application layer, configure the collection task template in the application layer, configure the collection indicators of the corresponding network element equipment, and complete the collection.

6. An electronic device, It is characterized in that The device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the network element device information collection analog-to-synchronization method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium, It is characterized in that The storage medium stores at least one executable instruction. When the executable instruction is executed on the electronic device, the electronic device executes the network element device information acquisition analog-to-synchronization method according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • Database data unified acquisition and distribution method and system

    CN110413673A

  • Comprehensive processing method and system for collected data, storage medium and electronic equipment

    CN112231380A

  • Distributed data acquisition system and method

    CN114253914A

  • Twin service assembly based on real-time data model and method thereof

    CN115080275A

  • Automated standardized location digital twin and location digital twin method factoring in dynamic data at different construction levels, and system thereof

    WO2023079139A1