Information collection model conversion synchronization method and system for network element device

Abstract

The present invention relates to the technical field of data synchronization. Provided are an information collection model conversion synchronization method and system for a network element device. The method comprises: accessing a network element configuration buffer database, collecting configuration information data of the current stage from the network element configuration buffer database, pre-processing the configuration information data, and then storing same in a network element configuration twin database; reading first incremental log data from the network element configuration twin database, performing model conversion on the incremental log data to obtain second incremental log data, and storing same in a network element configuration model conversion database; executing data processing on the second incremental log data, and then transferring same to a unified resource model database; and making the network element configuration twin database and the unified resource model database access an application layer, configuring a collection task template on the application layer, and configuring a collection index of a corresponding network element device, so as to complete collection.

Description

Network element equipment information acquisition analog conversion synchronization method and system Technical Field

[0001] The present invention relates to the technical field of data synchronization, and in particular to a method and system for synchronizing network element equipment information acquisition and analog conversion. Background Art

[0002] The cloud-network digital twin is the core cornerstone of the next-generation cloud-network operations system. Based on a unified data model for telecom cloud-network resources, it enables cross-disciplinary, cross-regional, and cross-tier data integration and correlation of network resources (network operation data + resource data). This enables the visualization and management of all cloud-network resources, enabling centralized scheduling of two-tier resources. This intensive management improves the operational efficiency of cloud-network services. It provides fundamental data support and services for business scenarios such as precision marketing, targeted investment, speed increases and fee reductions, network optimization and data transfer, base station energy conservation, mobile asset downsizing, and tower assessment.

[0003] The NE configuration twin database primarily stores NE configuration information and converts resource-related data into the NE configuration twin database according to a unified model mapping. Resource data includes devices, hardware, links, ports, code numbers, and other resources and relationship data that can be collected and converted. Network operation data such as alarms, performance, and logs are still collected and shared through existing collection channels for distribution and application.

[0004] The current acquisition and control connection scheme that uses scheduled collection, comparison and AUDK labeling of production and intermediate libraries, and deletion based on the number of collections has problems such as long collection and comparison cycles, the inability to guarantee error-free collection of collected data, unclear division of labor between resource sharing and acquisition and control, and the inability to clean up a large amount of invalid garbage data in a timely manner. Summary of the Invention

[0005] Purpose of the invention: To propose a network element equipment information acquisition and analog-to-synchronization method and system, adopting a new digital twin solution, through real-time / quasi-real-time acquisition and control, to realize dynamic and real-time updating of network configuration data, and by comparing data with the network element configuration twin database, to ensure error-free data collection and data accuracy, thereby effectively solving the above-mentioned problems existing in the prior art.

[0006] In the first aspect, a method for synchronizing network element device information collection is proposed, and the steps are as follows:

[0007] S1. Accessing a network element configuration buffer database, collecting configuration information data of the current stage from the network element configuration buffer database, pre-processing the configuration information data, and storing it in a network element configuration twin database;

[0008] S2. Read the first incremental log data from the network element configuration twin database, perform modular conversion on the incremental log data to obtain second incremental log data, and store the data in the network element configuration modular conversion database;

[0009] S3. Perform data processing on the second incremental log data and transfer it to a unified resource model database;

[0010] S4. Connect the network element configuration twin database and the unified resource model database to the application layer, configure the collection task template in the application layer, configure the collection indicators of the corresponding network element equipment, and complete the collection.

[0011] In a further embodiment of the first aspect, the configuration information data is preprocessed in step S1, including: modeling the configuration information data according to a unified model: each type of data is modeled according to the assigned major category specifications and minor category specifications and specifications, and the attribute range is modeled according to the fields defined by this specification.

[0012] In a further embodiment of the first aspect, step S2 reads the first incremental log data from the network element configuration twin database, including: identifying changes in resource data collected between different batches as incremental data; marking and pushing the incremental data.

[0013] In a further embodiment of the first aspect, performing modular transformation on the incremental log data to obtain second incremental log data in step S2 includes:

[0014] The configuration information data is compared with the current model transfer library: if the model transfer library does not have it, it is labeled as a new one; if the model transfer library has it and the data has changed, it is labeled as an update; if the model transfer library has it and the data has not changed, it is labeled as a keep; if the model transfer library has it but the latest collection does not have this data, it is labeled as a delete.

[0015] In a further embodiment of the first aspect, in step S3, data processing is performed on the second incremental log data and then transferred to a unified resource model database, including performing resource data splicing:

[0016] Collect the relationship data between entity resource data and the link data between the physical device ports on the az side:

[0017] When the physical device port is associated with the physical resource data, a relationship between the device and the entity is established;

[0018] When a link connection exists between the ports of the physical devices, a link between the ports is established.

[0019] In a further embodiment of the first aspect, when a link connection exists between the ports of the physical device, establishing a link between the ports includes:

[0020] The A-end port in the link is collected, and the relevant information of the Z-end port is collected. The acquisition and control is responsible for supplementing the Z-end port information into the link to form a complete link and then push the link information.

[0021] A second aspect of the present invention provides a network element device information acquisition and analog-to-synchronization system, the network element device information acquisition and analog-to-synchronization system comprising an acquisition and control platform and an information sharing platform;

[0022] The acquisition and control platform includes a network element configuration buffer database; the resource sharing platform includes a network element configuration twin database, a network element configuration model conversion database, and a unified resource model database;

[0023] In addition, the network element equipment information collection analog conversion synchronization system also includes:

[0024] A first processing unit is configured to collect configuration information data of the current stage from the network element configuration buffer database, pre-process the configuration information data, and then store it in the network element configuration twin database;

[0025] A second processing unit is configured to read the first incremental log data from the network element configuration twin database, perform modular conversion on the incremental log data to obtain second incremental log data, and store the data in the network element configuration modular conversion database;

[0026] a third processing unit, configured to perform data processing on the second incremental log data and transfer the data into a unified resource model database;

[0027] The fourth processing unit is used to connect the network element configuration twin database and the unified resource model database to the application layer, configure the collection task template in the application layer, configure the collection indicators of the corresponding network element equipment, and complete the collection.

[0028] According to a third aspect of the present invention, an electronic device is provided, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the network element device information collection analog-to-synchronization method as described in the first aspect is implemented.

[0029] The fourth aspect of the present invention proposes a computer-readable storage medium, which stores at least one executable instruction. When the executable instruction is run on an electronic device, the electronic device executes the network element device information collection and analog-to-synchronization method as described in the first aspect.

[0030] Beneficial effects: The present invention provides a network element equipment information collection and analog-to-synchronization method, which realizes dynamic and real-time updating of network configuration data through real-time / quasi-real-time collection and control, and ensures error-free data collection and data accuracy by comparing data with the network element configuration twin database; makes data traceable and controllable in the whole process, and greatly ensures the transparency and visualization of data processing in each link, thereby improving data quality and enhancing data availability. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] FIG1 is a flow chart of a method for synchronizing network element device information collection and conversion according to an embodiment of the present invention.

[0032] FIG2 is a data flow diagram of the network element device information acquisition model conversion synchronization method proposed in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In the following description, numerous specific details are provided to provide a more thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without one or more of these details. In other instances, certain technical features well known in the art have not been described to avoid confusion with the present invention.

[0034] Example 1:

[0035] This embodiment discloses a method for synchronizing network element device information acquisition and conversion, which is divided into two stages: acquisition and control and information sharing.

[0036] Acquisition and control stage: collect data on network element devices, panels, ports, etc., store them in the "network element configuration twin database", and store them in the "network element configuration simulation database" after simulation conversion according to the unified model and agreed simulation conversion method. Finally, the acquisition and control interact with resources to store the data in the "unified resource model database". This method is used to build a digital twin library of network elements and networks. Based on the incremental log change mechanism of "network element -> network element configuration twin database -> network element configuration simulation database -> unified resource model database", the acquisition and control and resource sharing platforms perform their respective duties to achieve real-time and accurate data twin processing.

[0037] The procurement and control system regularly collects and normalizes network element resource data (resource data conversion: the data collected by procurement and control is data from various manufacturers. Different manufacturers have different data naming conventions. This step is to convert resource data from different manufacturers through a unified model to achieve normalized storage of data from different manufacturers according to the unified model specification). The network element configuration data is normalized, and resource data from the unified resource model database is pulled for comparison and marked with AUDK. The results are notified to the resource data sharing platform (hereinafter referred to as the resource sharing platform) through Kafka messages. The resource sharing platform enters the data into the network element configuration data conversion library (intermediate library) and the unified resource model database (original library).

[0038] Data processing division of labor:

[0039] Procurement and control is responsible for collecting network element configuration data: real-time collection of manufacturers' existing network resource data;

[0040] Entering the twin database: After collection, the unmodified data is stored in the twin database according to the business model;

[0041] Normalization: Resource data is converted. The collected data is data from various manufacturers. Different manufacturers have different data naming specifications. This step is to convert the resource data of different manufacturers through a unified model to achieve normalized storage of data from different manufacturers according to the unified model specifications; the collected data is converted according to the unified model. Each type of data is converted according to the assigned major and minor specifications and specifications, and the attribute range is converted according to the fields defined by this specification.

[0042] For example

[0043] The collected router device analog conversion: the major category specification is equipment, the detailed category rule is IP equipment, the specification is router, and the analog conversion attributes include (code, name, specification ID, business status ID, manufacturer ID, working status ID, network management name, network access date, activation date, deactivation date, network exit date, etc.).

[0044] The collected switch equipment analog conversion: the major category specification is equipment, the detailed category rule is IP equipment, the specification is switch, and the analog conversion attributes include (code, name, specification ID, business status ID, manufacturer ID, working status ID, network management name, network access date, activation date, deactivation date, network exit date, etc.).

[0045] The collected port device analog conversion: the major category specification is port, the detailed category rule is the transport layer basic port, the specification is Ethernet port, and the analog conversion attributes include (coding, name, physical device ID, logical device ID, hardware ID, network management code, network management name, physical status ID, blocking status ID, port type ID, working status ID, etc.).

[0046] Incremental identification: Identify changes in resource data collected between different batches, label and push these incremental data, and compare the collected data with the current model transfer library. If the model transfer library does not have this data, add a new label; if the model transfer library has this data and the data has changed, update the label; if the model transfer library has this data and the data has not changed, keep the label; if the model transfer library has this data but the latest collection does not have this data, delete the label.

[0047] Model transfer into model transfer library: after model transfer, the unified model data will be stored in the model transfer library;

[0048] Splicing within the network element configuration twin library: Splicing of resource data in the twin library within the same network, including relationship data between entity resource data and link data between the az-end entity device ports; when the entity and entity data are related, for example, there is an entity B board on the entity A device, then the relationship between the device and the entity is established; or when a link connection is established between ports, then the link between the ports is established. Usually, the collection will collect the A-end port in the link and collect relevant information of the Z-end port. The collection and control is responsible for supplementing the Z-end port information into the link to form a complete link and then push the link information.

[0049] Resource sharing stage: Compare the differences between the network element configuration model database and the unified resource model database to ensure that the data in the original database and the twin database are consistent;

[0050] Importing into the unified resource model library according to rules: Configure the resource import business primary key, fields, and some verification rules according to the resource type. These configuration rules are read and processed before importing into the library;

[0051] Manual processing mechanism for discrepant data: When the data in the original database and the twin database are inconsistent, they need to be displayed and provided with a manual processing channel;

[0052] Data splicing of twin libraries across network element configurations: Splicing of resource data in twin libraries across networks, including relationship data between physical resource data and link data between physical device ports on the az side.

[0053] Manual maintenance data: provides manual maintenance data channel;

[0054] Automatically stitching together collected data: Automatically stitching together relational data and link data in cross-network data.

[0055] In the new generation of cloud network operation systems, a procurement and control platform is required to provide information collection and control services for various types of equipment. For IDC computer rooms, the equipment that needs to provide procurement and control services mainly includes the routers, switches, firewalls, IDS / IPS and other security equipment used in the IDC computer room, and the target server for quality testing in the star-level computer room (hereinafter referred to as the "target server"). The specific collection content includes the resources, performance, configuration, alarms and other information of all the above equipment; the control aspect is the basic resource configuration, main parameter configuration and business configuration of the above equipment. At the same time, in order to facilitate the use of upper-level applications, it should be possible to configure the collection indicators of the corresponding device model through the collection task template, and then complete the collection.

[0056] This invention is used to perform collection model conversion for data usage scenarios where the collected data is complex, the amount of collected data is large, and the real-time requirements are high.

[0057] Database Planning: Configuring the twin database and the model-to-model database each uses a different PG database instance. Each PG database instance applies for its own schema on demand through a dedicated network. Refer to the hardware configuration of the recently delivered PG database physical machine (2-way 16-core, 384GB of memory, 6 x 1.8TB SAS data disks, with 1.4TB of available storage per disk). Based on comprehensive resource and performance considerations, initially create a maximum of three schemas per PG instance. Each schema is planned to have one disk mounted as a tablespace. Of the remaining three disks, one is for PAAS's own use, and two are reserved for tablespace expansion.

[0058] Resource planning: To meet the implementation requirements of the IDC data communication twin solution, apply for the following resources:

[0059] 1. Two PG library instances (one master and two slaves) are used to deploy the twin library and the model conversion library respectively.

[0060] 2. Create an IDC data communication schema in each of the two PG library instances, and mount the tablespace as an exclusive drive letter.

[0061] Example 2:

[0062] This embodiment discloses a network element device information acquisition and analog conversion synchronization system, which includes a collection and control platform and a resource sharing platform; the collection and control platform includes a network element configuration buffer database; the resource sharing platform includes a network element configuration twin database, a network element configuration analog conversion database, and a unified resource model database;

[0063] In addition, the network element device information collection and analog conversion synchronization system also includes: a first processing unit, which is used to collect the configuration information data of the current stage from the network element configuration buffer database, and store the configuration information data in the network element configuration twin database after pre-processing; a second processing unit, which is used to read the first incremental log data from the network element configuration twin database, analog-convert the incremental log data to obtain the second incremental log data, and store it in the network element configuration analog conversion database; a third processing unit, which is used to perform data processing on the second incremental log data and then transfer it to the unified resource model database; a fourth processing unit, which is used to connect the network element configuration twin database and the unified resource model database to the application layer, configure the collection task template at the application layer, configure the collection indicators of the corresponding network element device, and complete the collection.

[0064] Example 3:

[0065] This embodiment provides an electronic device comprising a processor, a memory, a communication interface, and a communication bus. The processor, memory, and communication interface communicate with each other via the communication bus. The memory is configured to store at least one executable instruction that causes the processor to execute the network element device information acquisition and analog-to-synchronization method disclosed in the above embodiment.

[0066] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter. The network adapter communicates with other modules of the electronic device via a bus. It should be understood that, although not shown in the figures, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0067] Example 4:

[0068] This embodiment provides a computer-readable storage medium storing at least one executable instruction. When the executable instruction is executed on an electronic device, the electronic device executes the network element device information acquisition and analog-to-synchronization method described in the above embodiment. More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. The computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such a propagated data signal may take various forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0069] In summary, the present invention is mainly intended to solve the problems of the current acquisition and control connection scheme that adopts timed acquisition, AUDK labeling of production library and intermediate library comparison, and deletion processing according to the number of acquisitions, such as long acquisition and comparison cycle, the collection data cannot be guaranteed to be error-free, the division of labor between resource sharing and acquisition control is unclear, and a large amount of invalid garbage data cannot be cleaned up in time. The pilot program now adopts a new digital twin solution, which realizes dynamic and real-time update of network configuration data through real-time / quasi-real-time acquisition and control, and compares data with the network element configuration twin database to ensure error-free data collection and data accuracy.

[0070] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes may be made to it in form and detail without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A method for analog-to-synchronous acquisition of network element device information, characterized in that, Including: S1. Configure the buffer database for the access network element, collect the configuration information data in the current stage from the buffer database for network element configuration, preprocess the configuration information data, and store it in the twin database for network element configuration; S2. Read the first incremental log data from the twin database for network element configuration, perform modulo conversion on the incremental log data to obtain the second incremental log data, and store it in the modulo conversion database for network element configuration; S3. Perform data processing on the second incremental log data and transfer it to the unified resource model database; S4. Connect the twin database for network element configuration and the unified resource model database to the application layer, configure the collection task template in the application layer, configure the collection metrics for the corresponding network element devices, and complete the collection.

2. The method for analog-to-synchronous conversion of network element device information collection according to claim 1, characterized in that In step S1, preprocessing the configuration information data includes: performing modulo conversion on the configuration information data according to the unified model: each type of data is modulo-converted according to the allocated major category specifications, minor category specifications, and specifications, and the attribute range is modulo-converted according to the fields defined by this specification.

3. The network element device information collection analog-to-synchronous method according to claim 1, characterized in that In step S2, reading the first incremental log data from the twin database for network element configuration includes: identifying the changes in the collection resource data between different batches as incremental data; tagging and pushing the incremental data.

4. The method for collecting and converting network element device information and synchronizing according to claim 1, wherein In step S2, performing modulo conversion on the incremental log data to obtain the second incremental log data includes: Comparing the configuration information data with the current modulo conversion library: if it does not exist in the modulo conversion library, it is a new tag; if it exists in the modulo conversion library and the data has changed, it is an update tag, if it exists in the modulo conversion library and the data has not changed, it is a keep tag; if it exists in the modulo conversion library but there is no such data in the latest collection, it is a delete tag.

5. The method for analog-to-synchronous conversion of network element device information collection according to claim 1, characterized in that In step S3, performing data processing on the second incremental log data and transferring it to the unified resource model database includes performing resource data splicing: Collecting the relationship data between the entity resource data and the link data between the az-side entity device ports; When there is a connection between the entity device port and the entity resource data, establish a relationship between the device and the entity; When there is a link connection between the entity device ports, establish a link between the ports.

6. The method for collecting and synchronizing analog-to-digital conversion of network element device information according to claim 5, characterized in that, When there is a link connection between the entity device ports, establishing a link between the ports includes: Collecting the A-side port in the link and collecting the relevant information of the Z-side port, and the collection and control is responsible for supplementing the information of the Z-side port into the link to form a complete link and then pushing the link information.

7. A module conversion synchronization system for collecting network element device information, characterized in that, Including the collection and control platform and the resource sharing platform; The collection and control platform includes a buffer database for network element configuration; The resource sharing platform includes a twin database for network element configuration, a modulo conversion database for network element configuration, and a unified resource model database; The network element device information collection, modulo conversion, and synchronization system further includes: The first processing unit is used to collect the configuration information data in the current stage from the buffer database for network element configuration, preprocess the configuration information data, and store it in the twin database for network element configuration; The second processing unit is used to read the first incremental log data from the twin database for network element configuration, perform modulo conversion on the incremental log data to obtain the second incremental log data, and store it in the modulo conversion database for network element configuration; A third processing unit, configured to perform data processing on the second incremental log data and then transfer it to the unified resource model database; A fourth processing unit, configured to access the network element configuration twin database and the unified resource model database to the application layer, configure a collection task template in the application layer, configure collection metrics for corresponding network element devices, and complete collection.

8. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the network element device information collection model conversion synchronization method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that, At least one executable instruction is stored in the storage medium, and when the executable instruction runs on an electronic device, the electronic device is caused to execute the network element device information collection model conversion synchronization method according to any one of claims 1 to 6.

Images