Method, system and device for quasi-real-time collection of 24-hour historical performance data of transmission network element and storage medium

Abstract

The invention discloses a method, system and device for quasi-real-time collection of transmission network element 24-hour historical performance data and a storage medium, and belongs to the technical field of data processing and collection, and the method comprises the steps: building a data model for the generation time of the transmission network element 24-hour historical performance data; recording equipment information of the transmission network element; embedding the established data model of the generation time into a data acquisition foundation base; analyzing at regular time points, and waiting for issuing and collecting historical performance data for 24 hours; automatically issuing a 24-hour historical performance data acquisition task, analyzing the acquired 24-hour historical performance data, and storing the analyzed data into a data acquisition foundation base; through an open API interface, a user can use an upper application data platform to query and look up result data on the data acquisition foundation base. According to the invention, the timeliness of checking data by a user can be improved, and the query time of the user is shortened.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing and collection, and in particular relates to a method, system, device and storage medium for quasi-real-time collection of 24-hour historical performance data of a transmission network element. Background Art

[0002] With the continuous development of network technology, the number and types of network devices are increasing, and the network topology is becoming more and more complex. In order to better manage and maintain these devices, it is necessary to collect and transmit the performance data of network elements. The current method is that when the upper-layer application or user needs it, for example, to query the optical power or error performance of the device port within 3 days, first send a collection task, then call the EMS transmission network management northbound interface to collect the 24-hour historical performance file data of a certain device or device port accessed at a certain time; then the EMS transmission northbound interface asynchronously performs data aggregation according to the request parameters, uploads the file to the specified FTP server through FTP, and informs the caller through a message notification that the data file has been prepared or failed. The upper-layer application continuously monitors the message notification of the EMS network management within a period of time, such as 2 hours, or regularly scans the FTP server to see if there are any new FTP performance files; if there are new files, the FTP historical performance file interpretation and storage is triggered; the upper-layer application or user will regularly query the progress of the collection task, or wait for 2 hours until the second and third steps are completed; then query the relevant performance data.

[0003] The above method results in poor user experience. The data viewed by users is not timely. It is impossible to complete data collection, data processing, and application effect presentation within 1 minute. In addition, the data time recognition is low. Some transmission network elements cannot return the 24-hour performance of the day. However, users want to see the latest data of this device, or users think that data collection is not timely or has problems. It takes a lot of time to explain the reasons for the data to users, which is limited by the network management of EMS manufacturers. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a method, system, device and storage medium for collecting 24-hour historical performance data of transmission network elements in quasi-real time, which can improve the timeliness of user viewing data and shorten user query time.

[0005] The present invention provides the following technical solutions:

[0006] In a first aspect, a method for collecting 24-hour historical performance data of a transmission network element in quasi-real time is provided, comprising the following steps:

[0007] Establish a data model for the generation time of 24-hour historical performance data of transmission network elements;

[0008] Record the equipment information of transmission network elements;

[0009] Embed the established data model of generation time into the data acquisition foundation;

[0010] Perform analysis at regular time points and wait for the delivery and collection of 24-hour historical performance data;

[0011] Automatically issue 24-hour historical performance data collection tasks, and store the collected 24-hour historical performance data in the data collection base after analysis;

[0012] Through the open API interface, users can use the upper-level application data platform to query and view the result data on the data collection base.

[0013] Optionally, the equipment information of the transmission network element includes: the equipment model, serial number, configuration parameters of the transmission network element, and the generation time point, generation date and the time point of the most recent successful collection of the 24-hour historical performance data.

[0014] Optionally, the recording of the device information of the transmission network element specifically includes:

[0015] Collect device information, organize device information, retain device information, verify device information and update device information.

[0016] Optionally, the arranging device information specifically includes:

[0017] Sorting the collected device information according to preset categories; the preset categories include device hardware information, device software information and device configuration information;

[0018] The retained device information specifically includes:

[0019] Recording the sorted device information in a preset recording method, wherein the preset recording method includes a text file, a spreadsheet, or a database;

[0020] The verification device information specifically includes:

[0021] Verifying and checking the device information through a preset verification method; the preset verification method includes comparing whether the actual configuration parameters of the device are consistent with the recorded parameters and verifying the accuracy of the device information by executing a preset command on the device;

[0022] The updating device information specifically includes:

[0023] The verified device information is updated regularly at a preset frequency; the preset frequency includes daily, weekly or monthly.

[0024] Optionally, the analysis is performed at regular time points while waiting for the delivery and collection of 24-hour historical performance data, and the regular time points are 05 minutes, 20 minutes, 35 minutes or 50 minutes of each hour; the analysis is performed at regular time points, including analyzing whether the collection time generated by the data model has arrived, analyzing whether there are data files to be collected, analyzing whether all network element devices that need to be collected are in normal working condition, and analyzing whether there is any data missing.

[0025] Optionally, the automatic dispatching of the 24-hour historical performance task and storing the collected 24-hour historical performance data into the data collection base after parsing includes collecting the 24-hour historical performance data and parsing the 24-hour historical performance data;

[0026] The specific process of collecting the 24-hour historical performance data is: querying the historical performance data of each transmission network element in the past 24 hours according to the collection time generated by the data model, and summarizing the data.

[0027] The specific process of parsing the 24-hour historical performance data is: converting the collected 24-hour historical performance data into a unified format, and cleaning and deleting redundant information from the 24-hour historical performance data in the unified format.

[0028] Optionally, it also includes: when the date queried by the upper-layer application data platform is temporarily unable to generate 24-hour historical performance data, a prompt is given in the set language.

[0029] In a second aspect, a system for collecting 24-hour historical performance data of transmission network elements in quasi-real time is provided, comprising:

[0030] A data model building module is used to build a data model for the generation time of 24-hour historical performance data of transmission network elements;

[0031] A recording module, used to record the equipment information of the transmission network element;

[0032] An embedding module, used for embedding the established data model of generation time into a data acquisition basic base;

[0033] The analysis waiting module is used to perform analysis at regular time points and wait for the delivery and collection of 24-hour historical performance data;

[0034] The automated collection module is used to automatically issue 24-hour historical performance data collection tasks, and store the collected 24-hour historical performance data into the data collection base after analysis;

[0035] The service module is used to enable users to query and view the result data on the data collection base through the open API interface using the upper-level application data platform.

[0036] According to a third aspect, a computer device is provided, including a processor and a memory; wherein, when the processor executes the computer program stored in the memory, the steps of the method for collecting 24-hour historical performance data of a transmission network element in quasi-real time as described in any one of the first aspects are implemented.

[0037] In a fourth aspect, a computer-readable storage medium is provided, characterized in that it is used to store a computer program; when the computer program is executed by a processor, the steps of the method for quasi-real-time collection of 24-hour historical performance data of transmission network elements as described in any one of the first aspects are implemented.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The method provided by the present invention can make up for the timeliness of the 24-hour historical performance collection of the transmission network element, better support the timeliness of the upper-layer application query result data, and have predictability for the generation results of the 24-hour historical performance file of each transmission network element, thereby providing strong support for the optimization and upgrading of the equipment, so that customers can more conveniently obtain the 24-hour historical performance data of the equipment, better manage and maintain the network equipment, and at the same time, timely discover the failure or abnormal situation of the equipment, and improve the operating efficiency and stability of the equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of the method for collecting 24-hour historical performance data of transmission network elements in quasi-real time according to the present invention;

[0041] Figure 2 It is a structural block diagram of the system for collecting 24-hour historical performance data of transmission network elements in quasi-real time according to the present invention. DETAILED DESCRIPTION

[0042] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0043] It should be noted that the term "comprises" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or apparatuses.

[0044] Example 1

[0045] like Figure 1 As shown, a method for collecting 24-hour historical performance data of a transmission network element in quasi-real time includes the following steps:

[0046] Step S1: Establish a data model for the generation time of 24-hour historical performance data of transmission network elements.

[0047] The data model of generation time can define the generation time and data collection cycle of each 24-hour historical performance data. Specifically, in this embodiment, it can be a generation time model based on a fixed period, for example, 05 minutes, 20 minutes, 35 minutes or 50 minutes per hour.

[0048] Step S2: Record the equipment information of the transmission network element.

[0049] Specifically, the equipment information of the transmission network element includes: the equipment model, serial number, configuration parameters of the transmission network element, and the generation time point, generation date and the time point of the most recent successful collection of the 24-hour historical performance data.

[0050] Step S2 specifically includes the following sub-steps:

[0051] Step S21: Collect device information.

[0052] Step S22: Organize device information.

[0053] In this embodiment, the collected device information is specifically sorted according to preset categories; the preset categories include device hardware information, device software information and device configuration information.

[0054] Step S23: Save device information.

[0055] In this embodiment, the sorted device information is specifically recorded in a preset recording manner, and the preset recording manner includes a text file, a spreadsheet, or a database.

[0056] Step S24: Verify device information.

[0057] In this embodiment, it specifically includes verifying and checking the device information through a preset verification method; the preset verification method includes comparing whether the actual configuration parameters of the device are consistent with the recorded parameters and verifying the accuracy of the device information by executing a preset command on the device.

[0058] Step S25: Update device information.

[0059] In this embodiment, it specifically includes regularly updating the verified device information at a preset frequency; the preset frequency includes daily, weekly or monthly.

[0060] Step S3: embed the established data model of generation time into the data acquisition basic base.

[0061] In this embodiment, a data collection base architecture is provided, which generally includes modules such as a data storage system, a data collection engine, a task scheduling, an API interface, and data analysis and visualization. The method of embedding the data model of the generation time into the data collection base can adopt the existing technology.

[0062] Step S4: Perform analysis at regular time points and wait for the 24-hour historical performance data to be sent and collected.

[0063] The regular time point is 05 minutes, 20 minutes, 35 minutes or 50 minutes of each hour. Of course, other time points can also be selected as needed.

[0064] Perform analysis at regular time points, including analyzing whether the collection time for data model generation has arrived, analyzing whether there are data files to be collected, analyzing whether all network element devices that need to be collected are in normal working condition, and analyzing whether there is any data missing.

[0065] Step S5: Automatically issue a 24-hour historical performance data collection task, and store the collected 24-hour historical performance data into the data collection base after analysis.

[0066] Step S5 includes the following sub-steps:

[0067] Step S51: issuing and accepting 24-hour historical performance collection tasks.

[0068] According to the collection time generated by the data model of the generation time, the collection task is automatically sent to the target device, and the target device responds to the collection task.

[0069] Step S52: Collect 24-hour historical performance data.

[0070] The specific process is: query the historical performance data of each transmission network element in the past 24 hours according to the collection time generated by the data model, and summarize them.

[0071] The historical performance data of transmission network elements usually includes multiple key performance indicators, such as traffic data, delay data, packet loss data, or CPU usage and memory occupancy.

[0072] Step S53: Analyze the 24-hour historical performance data.

[0073] Specifically, the collected 24-hour historical performance data is converted into a unified format for later storage or query; for example, the performance data of a string is converted into a numeric type, etc.

[0074] It also includes cleaning and deleting redundant information from the 24-hour historical performance data in a unified format.

[0075] Step S54: storing the collected 24-hour historical performance data in the database.

[0076] As an option, the parsed 24-hour historical performance data is stored in a database according to a preset table structure.

[0077] Step S6: Through the open API interface, users can use the upper-level application data platform to query and view the result data on the data collection base.

[0078] Commonly used API interfaces include RESTful API, GraphQL API, etc.

[0079] In some other embodiments, the method for collecting 24-hour historical performance data of transmission network elements in quasi-real time also includes: Step S7: When the date queried by the upper-layer application data platform is temporarily unable to generate 24-hour historical performance data, a prompt is given in the set language.

[0080] As an option, the date data can be queried after 30 minutes, or the manufacturer has not yet generated the data.

[0081] Example 2

[0082] A method for collecting 24-hour historical performance data of a transmission network element in quasi-real time, comprising the following steps:

[0083] Step A1: Establish a data model for the 24-hour historical performance generation time of the transmission network element;

[0084] Step A2: Collecting device information of transmission network elements, including device model, serial number, configuration parameters, as well as the time point of generating 24-hour historical performance, the date of generating 24-hour historical performance, the time point of the most recent successful collection, and other information;

[0085] Step A3: Arrange and classify the collected device information, and arrange the device information according to different categories, including device hardware information, device software information, and device configuration information;

[0086] Step A4: recording the sorted device information in a preset recording method, wherein the preset recording method includes a text file, a spreadsheet or a database;

[0087] Step A5: verifying and checking the device information by a preset verification method. A variety of methods may be used for verification. The preset verification method includes comparing whether the actual configuration parameters of the device are consistent with the recorded parameters and verifying the accuracy of the device information by executing a preset command on the device.

[0088] Step A6: Regularly update the verified device information at a preset frequency, which may include daily, weekly or monthly;

[0089] Step A7: embedding the data collection base based on the data model;

[0090] Step A8: The regular analysis time point is 20 minutes of every hour, waiting for the 24-hour historical performance file to be sent;

[0091] Step A9: Automatically issue the 24-hour historical performance task, and parse and store the 24-hour historical performance into the data collection base;

[0092] Step A10: When the upper-layer application data platform needs to query the 24-hour historical performance of the relevant transmission network element, the result data in the basic base is queried through the open API interface, and the 24-hour historical performance data of the relevant transmission network element that needs to be queried is displayed;

[0093] Step A11: If the queried date cannot generate 24-hour historical performance temporarily, it will prompt that the data for this date should be "queried again in 35 minutes" or "the manufacturer has not generated data yet".

[0094] Example 3

[0095] A method for collecting 24-hour historical performance data of a transmission network element in quasi-real time, comprising the following steps:

[0096] Step E1: Establish a data model for the 24-hour historical performance generation time of the transmission network element;

[0097] Step E2: Collecting device information of transmission network elements, including device model, serial number, configuration parameters, as well as the time point of generating 24-hour historical performance, the date of generating 24-hour historical performance, the time point of the most recent successful collection, and other information;

[0098] Step E3: sorting and classifying the collected device information, and sorting the device information according to different categories, including device hardware information, device software information, and device configuration information;

[0099] Step E4: recording the sorted device information in a preset recording method, wherein the preset recording method includes a text file, a spreadsheet or a database;

[0100] Step E5: verifying and checking the device information by a preset verification method. A variety of methods may be used for verification. The preset verification method includes comparing whether the actual configuration parameters of the device are consistent with the recorded parameters and verifying the accuracy of the device information by executing a preset command on the device.

[0101] Step E6: regularly updating the verified device information at a preset frequency, which may include daily, weekly or monthly;

[0102] Step E7: embedding a data collection base based on the data model;

[0103] Step E8: The regular analysis time point is 50 minutes of each hour, and the 24-hour historical performance file is collected and sent;

[0104] Step E9: Automatically issue the 24-hour historical performance task, and parse and store the 24-hour historical performance into the data collection base;

[0105] Step E10: When the upper-layer application data platform needs to query the 24-hour historical performance of the relevant transmission network element, the result data in the basic base is queried through the open API interface, and the 24-hour historical performance data of the relevant transmission network element that needs to be queried is displayed;

[0106] Step E11: If the queried date cannot generate 24-hour historical performance temporarily, it will prompt that the data for this date should be "queried again after 60 minutes" or "the manufacturer has not generated data yet".

[0107] Example 4

[0108] like Figure 2 As shown, a system for collecting 24-hour historical performance data of transmission network elements in quasi-real time includes:

[0109] A model building module is used to build a data model for the generation time of 24-hour historical performance data of transmission network elements;

[0110] A recording module, used to record the equipment information of the transmission network element;

[0111] An embedding module, used for embedding the established data model of generation time into a data acquisition basic base;

[0112] The analysis waiting module is used to perform analysis at regular time points and wait for the delivery and collection of 24-hour historical performance data;

[0113] The automated collection module is used to automatically issue 24-hour historical performance data collection tasks, and store the collected 24-hour historical performance data into the data collection base after analysis;

[0114] The service module is used to enable users to query and view the result data on the data collection base through the open API interface using the upper-level application data platform.

[0115] For more specific processes of the above method, please refer to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0116] Example 5

[0117] The present invention provides a computer device, comprising a processor and a memory; wherein, when the processor executes a computer program stored in the memory, the steps of the method for collecting 24-hour historical performance data of a transmission network element in quasi-real time are implemented.

[0118] For more specific processes of the above method, please refer to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0119] Example 6

[0120] The present invention provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, the steps of the method for collecting 24-hour historical performance data of a transmission network element in quasi-real time are implemented.

[0121] For more specific processes of the above method, please refer to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0122] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system, device, and storage medium disclosed in the embodiment, since they correspond to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0123] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention or some parts of the embodiments.

[0124] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A method for collecting 24-hour historical performance data of transmission network elements in quasi-real time, characterized in that: The following steps are involved: Establish a data model for the generation time of 24-hour historical performance data of transmission network elements; Record the equipment information of transmission network elements; Embed the established data model of generation time into the data acquisition foundation; Perform analysis at regular time points and wait for the delivery and collection of 24-hour historical performance data; Automatically issue 24-hour historical performance data collection tasks, and store the collected 24-hour historical performance data in the data collection base after analysis; Through the open API interface, users can use the upper-level application data platform to query and view the result data on the data collection base.

2. The method for collecting 24-hour historical performance data of transmission network elements in quasi-real time according to claim 1 is characterized in that: The equipment information of the transmission network element includes: the equipment model, serial number, configuration parameters of the transmission network element, and the generation time point, generation date and the time point of the most recent successful collection of 24-hour historical performance data.

3. The method for collecting 24-hour historical performance data of transmission network elements in quasi-real time according to claim 1 is characterized in that: The recording of the device information of the transmission network element specifically includes: Collect device information, organize device information, retain device information, verify device information and update device information.

4. The method for collecting 24-hour historical performance data of transmission network elements in quasi-real time according to claim 3 is characterized in that: The collated device information specifically includes: Sorting the collected device information according to preset categories; the preset categories include device hardware information, device software information and device configuration information; The retained device information specifically includes: Recording the sorted device information in a preset recording method, wherein the preset recording method includes a text file, a spreadsheet, or a database; The verification device information specifically includes: Verifying and checking the device information through a preset verification method; the preset verification method includes comparing whether the actual configuration parameters of the device are consistent with the recorded parameters and verifying the accuracy of the device information by executing a preset command on the device; The updating device information specifically includes: The verified device information is updated regularly at a preset frequency; the preset frequency includes daily, weekly or monthly.

5. The method for collecting 24-hour historical performance data of transmission network elements in quasi-real time according to claim 1 is characterized in that: The analysis is performed at regular time points, and the 24-hour historical performance data is waiting for the collection and distribution. The regular time points are 05 minutes, 20 minutes, 35 minutes or 50 minutes of each hour. The analysis performed at regular time points includes analyzing whether the collection time for data model generation has arrived, analyzing whether there are data files to be collected, analyzing whether all network element devices that need to be collected are in normal working state, and analyzing whether there is data missing.

6. The method for collecting 24-hour historical performance data of transmission network elements in quasi-real time according to claim 1, characterized in that: The automatic issuance of 24-hour historical performance tasks and storage of the collected 24-hour historical performance data into the data collection base after analysis include collecting the 24-hour historical performance data and analyzing the 24-hour historical performance data; The specific process of collecting the 24-hour historical performance data is: querying the historical performance data of each transmission network element in the past 24 hours according to the collection time generated by the data model, and summarizing the data. The specific process of parsing the 24-hour historical performance data is: converting the collected 24-hour historical performance data into a unified format, and cleaning and deleting redundant information from the 24-hour historical performance data in the unified format.

7. The method for collecting 24-hour historical performance data of transmission network elements in quasi-real time according to claim 1, characterized in that: Also includes: When the upper-layer application data platform is temporarily unable to generate 24-hour historical performance data for the date queried, a prompt will be given in the set language.

8. A system for collecting 24-hour historical performance data of transmission network elements in quasi-real time, characterized in that: include: A model building module is used to build a data model for the generation time of 24-hour historical performance data of transmission network elements; A recording module, used to record the equipment information of the transmission network element; An embedding module, used for embedding the established data model of generation time into a data acquisition basic base; The analysis waiting module is used to perform analysis at regular time points and wait for the delivery and collection of 24-hour historical performance data; The automated collection module is used to automatically issue 24-hour historical performance data collection tasks, and store the collected 24-hour historical performance data into the data collection base after analysis; The service module is used to enable users to query and view the result data on the data collection base through the open API interface using the upper-level application data platform.

9. A computer device, characterized in that: It comprises a processor and a memory; wherein, when the processor executes the computer program stored in the memory, the steps of the method for quasi-real-time collection of 24-hour historical performance data of a transmission network element as described in any one of claims 1-7 are implemented.

10. A computer-readable storage medium, characterized in that: Used to store computer programs; when the computer programs are executed by the processor, the steps of the method for quasi-real-time collection of 24-hour historical performance data of transmission network elements as described in any one of claims 1-7 are implemented.

Images