System performance parameter transmission methods, devices, management equipment, and storage media

Abstract

This application proposes a method, apparatus, management device, and storage medium for transmitting system performance parameters. The method includes: determining system performance parameters, wherein the system performance parameters include key performance indicator (KPI) information and the reliability of the KPI information; and transmitting the system performance parameters.

Description

Technical Field

[0001] This application relates to the field of communications, specifically to methods, apparatus, management devices, and storage media for transmitting system performance parameters. Background Technology

[0002] Currently, mobile communication technology has entered the fifth generation (5G era). Managing 5G networks includes basic management functions such as configuration, performance, and alarms. In terms of performance management, performance measurement data can be sent at very short intervals to reflect system performance in real time. Furthermore, key performance indicators (KPIs) calculated from performance measurement data are also important indicators reflecting system performance.

[0003] However, in actual operating networks, performance measurement data collection often fails due to various reasons, resulting in the inability to collect performance measurement data for certain network elements (or functions) at certain times, meaning the completeness of the performance measurement data cannot be guaranteed. Current methods for calculating KPIs based on performance measurement data do not consider the possibility of missing data, leading to poor accuracy of the resulting KPIs and thus failing to accurately reflect system performance. Summary of the Invention

[0004] This application provides a method, apparatus, management device, and storage medium for transmitting system performance parameters.

[0005] This application provides a method for sending system performance parameters, including:

[0006] Determine system performance parameters, including key performance indicators (KPIs) and the reliability of the KPI information;

[0007] Send the system performance parameters.

[0008] This application provides a device for transmitting system performance parameters, comprising:

[0009] The module is configured to determine system performance parameters, which include key performance indicators (KPIs) and the reliability of the KPI information.

[0010] The sending module is configured to send the system performance parameters.

[0011] This application provides a management device, including:

[0012] One or more processors;

[0013] Storage device for storing one or more programs;

[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the methods in the embodiments of this application.

[0015] This application provides a storage medium storing a computer program, which, when executed by a processor, implements any of the methods described in this application.

[0016] Further details regarding the above embodiments and other aspects of this application, as well as their implementations, are provided in the accompanying drawings, detailed description, and claims. Attached Figure Description

[0017] Figure 1 A flowchart illustrating a method for sending system performance parameters provided in an embodiment of this application;

[0018] Figure 2 A 5G network management hierarchy diagram provided in this application embodiment;

[0019] Figure 3 A flowchart illustrating another method for sending system performance parameters provided in this application embodiment;

[0020] Figure 4 A schematic diagram of the structure of a system performance parameter transmission device provided in an embodiment of this application;

[0021] Figure 5 This is a schematic diagram of the structure of a management device provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be arbitrarily combined with each other.

[0023] The steps illustrated in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases the steps shown or described may be performed in a different order than that presented here.

[0024] In one exemplary implementation Figure 1This is a flowchart illustrating a method for sending system performance parameters according to an embodiment of this application. This method is applicable to situations where system performance parameters are determined and sent. The method can be executed by a device for sending system performance parameters. This device can be implemented in software and / or hardware and integrated into a management device. The management device can be a device that manages the performance of the network elements involved. For example, the management device can calculate KPI information based on the designed network elements.

[0025] The method for transmitting system performance parameters provided in this application can be considered a management method for a wireless communication system, particularly a management method for 5G networks in the field of communications.

[0026] Currently, mobile communication technology has entered the 5G era, and network slicing is an important component of 5G. Network slicing mainly refers to an instantiated, complete logical network with specific network characteristics, composed of a set of network functions (including network resources supporting those functions). This logical network is used to meet the needs of a specific type of network service. Examples of these network characteristics include ultra-low latency and ultra-high reliability.

[0027] When a network slice is complex, it can be decomposed into sub-slices (Network Slice Subnets) for easier management. A sub-slice is a logical network with specific network characteristics, consisting of a set of network functions and the network resources supporting those functions. A network slice can contain zero, one, or more sub-slices. The management and orchestration of sub-slices are handled by the Network Slice Subnet Management Function (NSSMF). NSSMF instantiates sub-slices and controls their lifecycle based on the sub-slice blueprint definition.

[0028] For 5G networks, network slicing is designed to support specific communication services. Therefore, from the perspective of overall 5G network slicing management, it also includes the Communication Service Management Function (CSMF). Thus, 5G network slicing management is divided into three layers: CSMF, NSMF, and NSMF. Figure 2 This application provides a 5G network management hierarchy diagram. Figure 2 The 5G network management hierarchy is shown, which includes communication service management function 1, slice management function 2, and sub-slice management function 3.

[0029] Management of 5G networks will also include basic management functions such as configuration, performance, and alarms, but will be improved or enhanced according to new management requirements. Regarding performance management, the concept of performance datastreaming has been proposed, which refers to real-time performance data. This means that performance measurement data can be reported at very short intervals, reflecting system performance in real time. The management device in this application can receive the reported system performance data, thereby determining system performance parameters that reflect system performance.

[0030] For 5G network performance management, KPIs calculated from performance measurement data are an important indicator reflecting system performance. For specific 5G network KPI definitions, please refer to 3GPP TS 28.554. For KPI definitions related to other mobile networks, please refer to 3GPP TS 32.410, TS 32.450, TS 32.454 and TS 32.455.

[0031] With performance measurement data reported in real time via data streams, if KPIs can also be generated in real time, the performance status of the 5G network system can be reflected more promptly. In actual network operation, the collection of performance measurement data often encounters various issues that prevent data collection from certain network elements at certain times, compromising data integrity.

[0032] The method for sending system performance parameters provided in this application can effectively ensure the accuracy of the calculated KPIs. Specifically, for example... Figure 1 As shown, the present application provides a method for transmitting system performance parameters, including S110 and S120.

[0033] S110. Determine system performance parameters, including key performance indicator (KPI) information and the reliability of the KPI information.

[0034] System performance parameters can be understood as parameters that reflect system performance. The system can be a system composed of management devices and network elements involved in those devices. The management device can manage at least one network element in a subnet or slice, and the managed network element can be considered as one of the network elements involved in the management device. System performance parameters include, but are not limited to: performance measurement data; KPI information; and the reliability of the KPI information. Performance measurement data can be data reflecting performance sent by network elements or network functions to the management device, such as data reflecting network performance. KPI information can include values ​​obtained after calculating KPIs using the KPI calculation formula.

[0035] When determining KPI information, the management device can first detect which performance measurement data is needed to calculate the KPI, such as determining the specific content included in the performance measurement data; it also needs to determine which network elements are involved in calculating the KPI information. Then, it receives the performance measurement data sent by the corresponding network elements. This application can identify the accuracy of the KPI information calculated based on the performance measurement data by using a trust rating to verify the integrity of the received performance measurement data.

[0036] This section does not limit the specific methods for determining KPI information; the calculation formula for KPI information can be set in the definition of KPI. For example, KPI information can be calculated based on the calculation formula and the time window for KPI calculation.

[0037] When the performance measurement data required for calculating KPI information is incomplete, such as when some network elements have not sent performance measurement data, this application can supplement the performance measurement data of those network elements. For example, if missing data is identified, the missing data can be used as the performance measurement data sent by that network element. Therefore, performance measurement data can include data obtained from network elements or it can include missing data. When calculating KPI information, it can be calculated based on performance measurement data including missing data or based on performance measurement data received from network elements.

[0038] To ensure users have a better understanding of the accuracy of KPI information, this application can determine the reliability of the KPI information. This reliability can indicate the accuracy of the KPI information, and the means of determining reliability are not limited here. For example, it can be determined by the completeness of performance measurement data; or by comparing the ratio of missing data to performance measurement data received directly from the network element; furthermore, the method of missing data completion also affects the reliability of the KPI information, for example, using the average of adjacent data to complete the data may be more reliable than using default values. Adjacent data can be performance measurement data collected by the network element corresponding to the missing data before or after the time the missing data was collected. For example, it could be the performance measurement data from the first n reports or the last m reports from the network element that contained the missing data. Here, n and m can be positive numbers.

[0039] S120, Send the system performance parameters.

[0040] Once the system performance parameters are determined, this application can send these parameters to other devices. There are no restrictions on which devices are allowed; any device authorized to access the system performance data of the system corresponding to the management device can participate, such as sending them to a higher-level management device or other network elements. These other devices can then more accurately understand and apply the KPI information based on the KPI information and its corresponding reliability within the system performance parameters.

[0041] For example, assuming that KPI information needs to be calculated at the slice level, the slice management function (which may be located in the slice management device) first checks which performance measurement data is needed for the KPI information to be calculated and which network elements are involved. Then, it ensures that the corresponding performance measurement data collection task has been started (i.e., the performance measurement data collection task use case collects the corresponding performance measurement data), and then calculates the KPI information.

[0042] Specifically: Suppose we want to calculate the uplink throughput of a slice instance at the slice level. This requires the uplink throughput of all User Port Function (UPF) network functions (which can be considered as network elements) under this slice instance. Therefore, the slice management function of this slice instance needs to first check whether the uplink throughput performance measurement task of the relevant UPF already exists. If it does not exist, it creates the measurement task. The uplink throughput of each UPF is collected through the relevant performance measurement task, which is the performance measurement data. Then, the slice management function calculates the KPI information at a preset time (that is, calculates the sum of the uplink throughput of each UPF. Missing data can also be filled in before the calculation).

[0043] It is understandable that the period for calculating KPI information can be greater than or equal to the period for collecting performance measurement data. The period for calculating KPI information can be N times the period for collecting performance measurement data, where N is a positive integer.

[0044] This application provides a method for sending system performance parameters, which determines system performance parameters, including key performance indicators (KPIs) and the reliability of the KPIs; and sends the system performance parameters. This method can more accurately determine KPI information, thereby helping other devices to determine the system performance of the system corresponding to the management device.

[0045] Based on the above embodiments, modified embodiments of the above embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in the modified embodiments.

[0046] In one embodiment, determining the system performance parameters includes:

[0047] Obtain performance measurement data;

[0048] Based on the performance measurement data, determine the KPI information;

[0049] Determine the credibility of the KPI information.

[0050] In this application, when the system performance parameters are determined, performance measurement data can be obtained first. The performance measurement data can be the performance measurement data sent by the network element in the corresponding system of the management device. Here, it is not limited that the performance measurement data is sent by the network element, but can also be sent by other network functions. The specific performance measurement data to be obtained can be determined based on the calculated KPI information, such as the type of performance measurement data required by the management device when detecting and calculating KPI information, and the network element or network function that needs to collect the performance measurement data.

[0051] If performance measurement data is not successfully obtained, the identified missing data can be used as a supplement to the obtained performance measurement data, so that KPI information can be determined based on performance measurement data including the missing data; alternatively, if the missing data is uncertain, KPI information can be determined directly based on the successfully obtained performance measurement data.

[0052] Given a defined system performance parameter, the reliability of KPI information can be determined based on performance measurement data. For example, reliability can be determined by comparing the actual number of data points obtained from the performance measurement data with the total number of data points required to calculate the KPI information.

[0053] If the performance measurement data does not include missing data, the number of performance measurement data points obtained is directly determined as the actual number of real data points obtained. If the performance measurement data includes missing data, the number of data points other than the missing data points can be determined as the actual number of real data points obtained.

[0054] The technical means for determining the number of data points in the performance measurement data, excluding missing data, are not limited here. For example, missing data can be stored separately, or a label can be set for missing data to identify it as missing. The content of the label is not limited, as long as it can be used to identify missing data.

[0055] In one embodiment, the performance measurement data includes missing data, which is performance measurement data that was not successfully acquired.

[0056] In cases where performance measurement data is not successfully acquired, this application can use missing data to supplement the unacquired performance measurement data, that is, to use the missing data as a supplement to the acquired performance measurement data. The means of determining the missing data are not limited; for example, it could be the missing data of the network element that failed to send the performance measurement data. Different network elements may have different missing data.

[0057] In one embodiment, acquiring performance measurement data includes: in the event that performance measurement data is not successfully acquired, determining missing data based on data at a preset time, wherein the preset time includes at least one of the following: a first preset time, the first preset time being the acquisition time corresponding to the performance measurement data acquired before the time when the missing data is acquired; and a second preset time, the second preset time being the acquisition time corresponding to the performance measurement data acquired after the time when the missing data is acquired.

[0058] In the process of determining KPI information in this application, performance measurement data of at least one network element can be obtained, meaning that at least one set of performance measurement data is obtained. If performance measurement data is not successfully obtained, the missing data corresponding to that network element is determined based on the data from a preset time period for the network element for which performance measurement data was not successfully obtained.

[0059] This application can determine missing data based on data acquired at a preset time, i.e., performance measurement data. The time when missing data is acquired can be considered the time when the performance measurement data corresponding to the missing data was not successfully acquired. The first time can be considered the acquisition time corresponding to the acquired performance measurement data before the time when the performance measurement data corresponding to the missing data was not successfully acquired, i.e., the acquisition time when the performance measurement data of the network element corresponding to the performance measurement data that was not successfully acquired is acquired.

[0060] For example, to calculate KPI information, the management device needs to acquire performance measurement data from network element 1, network element 2, and network element 3. The time interval for acquiring performance measurement data can be shorter than the time interval for calculating KPI information. For example, performance measurement data can be acquired every 1 minute, and KPI information can be calculated every 6 minutes. During the acquisition of performance measurement data, if the management device successfully acquires performance measurement data from network element 1 and network element 2, but fails to acquire performance measurement data from network element 3, the management device can determine the missing data based on the performance measurement data of network element 3 at a preset time, and use the missing data as the performance measurement data of network element 3.

[0061] If the performance measurement data for network element 3 is not successfully acquired in the 5th minute, the missing data will be determined based on the performance measurement data acquired before the 5th minute; or based on the performance measurement data acquired after the 5th minute; or based on the performance measurement data acquired before / after the 5th minute. For example, the missing data can be determined based on the average of the performance measurement data from the 3rd and 4th minutes, and / or based on the average of the performance measurement data from the 6th to the 9th minutes, or based on the average of the performance measurement data from the 4th and 6th minutes.

[0062] It should be noted that there are no restrictions on which acquisition time before the missing data acquisition time is designated as the first setting time, or which selection time after the missing data acquisition time is designated as the second setting time.

[0063] How to determine missing data based on data at preset times is not limited here; for example, the average of the data at preset times can be used as the missing data.

[0064] In one embodiment, determining missing data based on data at a preset time includes: determining the mean of the data at the preset time as the missing data.

[0065] In one embodiment, acquiring performance measurement data includes: if performance measurement data is not successfully acquired, determining a preset default value as missing data.

[0066] This application can also directly define a preset default value as missing data. Different network elements can be set with different default values. The default value can be calculated and determined based on the historical performance measurement data of the network element. The calculation method is not limited here, as long as the default value can identify the performance of the network element.

[0067] In one embodiment, before determining KPI information based on the performance measurement data, the method further includes: if the performance measurement data includes missing data, removing the identification information of the missing data to obtain new performance measurement data that can be used for KPI information calculation.

[0068] This application can determine KPI information based on performance measurement data containing missing data. Missing data may contain identification information during storage, i.e., information used to identify data as missing. Therefore, during the calculation of KPI information, if missing data affects the calculation of LPI information, the identification information in the missing data can be removed first, and the data after removing the identification information can be used to update the corresponding performance measurement data, thereby obtaining new performance measurement data. The new performance measurement data may include directly obtained performance measurement data and the missing data after removing the identification information. If the identification information is a symbol, in order not to affect the calculation of KPI information, the identification data in the missing data is removed, and the KPI information is calculated based on the data after removal. In this application, all missing data have their identification information removed, and then the new performance measurement data is formed by combining it with directly obtained performance measurement data.

[0069] For example, if performance measurement data for network element 3 is not successfully obtained, missing data is identified and marked with a negative sign to indicate that the data is missing. Therefore, before calculating KPI information, the missing data can be removed to form new performance measurement data for network element 3. Then, the KPI information is calculated by combining the directly obtained performance measurement data for network elements 1 and 2.

[0070] In one embodiment, determining the credibility of the KPI information includes: determining the number of target data in the performance measurement data, wherein the target data is data excluding missing data; and determining the credibility of the KPI information as the percentage of the number of target data to the total number of all performance measurement data required to calculate the KPI information.

[0071] The target data can be considered as actual acquired data, not missing data. The total number of all required performance measurement data can be determined based on the calculation formula needed to calculate KPIs, and is not limited here.

[0072] The following is an exemplary description of the method for sending system performance parameters provided in this application. Figure 3 This is a flowchart illustrating another method for sending system performance parameters provided in an embodiment of this application.

[0073] This application, when calculating KPIs, first performs an integrity check on the performance measurement data used, and then identifies the calculated KPI results based on the integrity rate of the performance measurement data. In other words, it appropriately identifies the KPI information, such as through a credibility rating. This allows users of the KPI information to have a more accurate understanding and application of the KPIs. This application overcomes the problem in existing mobile network management technologies that do not identify the credibility of KPIs based on data integrity when calculating KPIs, thus affecting the accurate application of KPIs. It proposes a method that first performs an integrity check on performance measurement data when calculating KPIs and then identifies the credibility of KPIs based on the integrity rate of the performance measurement data.

[0074] Figure 3 A flowchart illustrating another method for sending system performance parameters provided in this application embodiment is shown below. Figure 3 As shown, the method includes S310-S360.

[0075] S310. Determine the calculation formula for KPI information and the time window for calculating KPI.

[0076] S320: Collect the performance measurement data needed to calculate KPIs.

[0077] S330. Calculate the reliability of KPIs.

[0078] S340, Identify missing data.

[0079] S350. Calculate KPI information according to the calculation formula for KPI information.

[0080] S360, report system performance parameters.

[0081] Specifically, KPI information is calculated and the KPI results, i.e., system performance parameters, are reported. The KPI calculation results include KPI values, i.e., KPI information and their reliability (reliability is generally the completeness rate of the data used to calculate the KPI). KPIs can be calculated according to a predefined KPI calculation formula (specific KPIs and their calculation formulas can be found in 3GPP TS 28.554). Reliability can be calculated based on the completeness of the performance measurement data used in the KPI calculation formula.

[0082] The steps for calculating KPIs include:

[0083] Determine the KPI calculation formula and the time window for KPI calculation. The KPI calculation formula is set in the KPI definition (see 3GPP TS 28.554 for details). Specifically, the KPI calculation formula is obtained by looking up the corresponding KPI definition from the KPI name included in the received KPI calculation command. The KPI calculation time window refers to the time range from which the KPI is calculated, i.e., the time interval from which the KPI is calculated. It is usually specified in the KPI calculation task or KPI calculation command, and will not be detailed here.

[0084] Determine the data types needed to calculate KPIs based on the KPI calculation formula, and then collect the performance measurement data required for KPI calculation based on the time window for KPI calculation.

[0085] The collected data undergoes an integrity check to calculate the data integrity rate. Here, the data integrity rate is:

[0086] Missing data needs to be processed. Missing data can be considered as performance measurement data that was not successfully acquired. The processing of missing data can be carried out in different ways depending on the situation: for example, missing data can be imputed to obtain the missing data. Imputation processing includes taking the average of adjacent data, taking the default value of the data type for the network element corresponding to the missing data, etc.

[0087] If missing data was already completed during data collection but was marked (e.g., with a negative sign), then the marking process (e.g., removing the negative sign) is performed before calculating the KPI. The KPI information and its reliability (i.e., data completeness rate) are then reported.

[0088] In the steps described above, the calculation of KPIs is provided by a KPI calculation service or a KPI control service. The KPI calculation service or KPI control service is generally located in the KPI calculation or KPI control function module, but it can also be located in the network element management module, subnet management module, sub-slice management module, or slice management module.

[0089] The purpose of using the method described in this application is to overcome the problem of low accuracy in calculating KPIs in existing mobile network management technologies. This is achieved by proposing a method that first checks the integrity of performance measurement data before calculating KPIs and then uses the integrity rate of the performance measurement data to indicate the reliability of the KPIs.

[0090] This application also provides a device for transmitting system performance parameters. Figure 4 This is a schematic diagram of a system performance parameter transmission device provided in an embodiment of this application. This device can be integrated into a management device. Figure 4 As shown, the present application provides an apparatus including: a determining module 41, configured to determine system performance parameters, the system performance parameters including key performance indicator (KPI) information and the reliability of the KPI information; and a sending module 42, configured to send the system performance parameters.

[0091] The system performance parameter sending device provided in this embodiment is used to implement the system performance parameter sending method of this application embodiment. The implementation principle and technical effect of the system performance parameter sending device provided in this embodiment are similar to those of the system performance parameter sending method of this application embodiment, and will not be repeated here.

[0092] Based on the above embodiments, modified embodiments of the above embodiments are proposed. It should be noted that, in order to keep the description brief, only the differences from the above embodiments are described in the modified embodiments.

[0093] In one embodiment, determining module 41 includes:

[0094] The acquisition unit is configured to acquire performance measurement data.

[0095] The first determining unit is configured to determine KPI information based on the performance measurement data;

[0096] The second determining unit is configured to determine the credibility of the KPI information.

[0097] In one embodiment, the performance measurement data includes missing data, which is performance measurement data that was not successfully acquired.

[0098] In one embodiment, the acquisition unit is specifically configured to: determine missing data based on data at a preset time when performance measurement data is not successfully acquired, wherein the preset time includes at least one of the following: a first preset time, the first preset time being the acquisition time corresponding to the performance measurement data acquired before the time of acquiring the missing data; and a second preset time, the second preset time being the acquisition time corresponding to the performance measurement data acquired after the time of acquiring the missing data.

[0099] In one embodiment, when the acquisition unit determines missing data based on data at a preset time, it is specifically configured to: determine the mean of the data at the preset time as the missing data.

[0100] In one embodiment, the acquisition unit is specifically configured to: determine a preset default value as missing data if performance measurement data is not successfully acquired.

[0101] In one embodiment, the determining module 41 in the device further includes a removal unit, which, before determining KPI information based on the performance measurement data, removes the identification information of the missing data to obtain new performance measurement data if the performance measurement data includes missing data.

[0102] In one embodiment, the second determining unit is specifically configured to: determine the number of target data in the performance measurement data, wherein the target data is data excluding missing data; and determine the reliability of the KPI information as the percentage of the number of target data to the total number of all performance measurement data required to calculate the KPI information.

[0103] This application provides a management device. Figure 5 This is a schematic diagram of the structure of a management device provided in an embodiment of this application. Figure 5 As shown, the management device provided in this application includes one or more processors 51 and a storage device 52; the processors 51 in the management device may be one or more. Figure 5 Taking a processor 51 as an example; storage device 52 is used to store one or more programs; the one or more programs are executed by the one or more processors 51, causing the one or more processors 51 to implement the method as described in the embodiments of this application.

[0104] The management equipment also includes: a communication device 53, an input device 54, and an output device 55.

[0105] The processor 51, storage device 52, communication device 53, input device 54, and output device 55 in the management device can be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0106] Input device 54 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the management device. Output device 55 may include display devices such as a display screen.

[0107] The communication device 53 may include a receiver and a transmitter. The communication device 53 is configured to perform information transmission and reception communication under the control of the processor 51. The information includes, but is not limited to, system performance parameters.

[0108] Storage device 52, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the methods described in the embodiments of this application (e.g., the determining module 41 and sending module 42 in the system performance parameter sending device). Storage device 52 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the management device, etc. Furthermore, storage device 52 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, storage device 52 may further include memory remotely located relative to processor 51, and these remote memories can be connected to the management device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0109] This application embodiment also provides a storage medium storing a computer program. When executed by a processor, the computer program implements the method for sending system performance parameters as described in any of the embodiments of this application. The method includes: determining system performance parameters, the system performance parameters including key performance indicator (KPI) information and the reliability of the KPI information; and sending the system performance parameters.

[0110] The computer storage medium in this application embodiment can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be—but is not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable CD-ROM, optical storage device, magnetic storage device, or any suitable combination thereof. The computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0111] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device.

[0112] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, radio frequency (RF), etc., or any suitable combination thereof.

[0113] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0114] The above description is merely an exemplary embodiment of this application and is not intended to limit the scope of protection of this application.

[0115] Those skilled in the art will understand that the term "terminal" encompasses any suitable type of wireless user equipment, such as mobile phones, portable data processing devices, portable web browsers, or vehicle-mounted mobile stations.

[0116] Generally, the various embodiments of this application can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. For example, some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device, although this application is not limited thereto.

[0117] Embodiments of this application can be implemented by executing computer program instructions through the data processor of a mobile device, for example, in a processor entity, or through hardware, or through a combination of software and hardware. The computer program instructions can be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages.

[0118] Any block diagram of logical flow in the accompanying drawings of this application may represent program steps, or may represent interconnected logic circuits, modules, and functions, or may represent a combination of program steps and logic circuits, modules, and functions. The computer program may be stored on memory. Memory may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as, but not limited to, read-only memory (ROM), random access memory (RAM), optical storage devices and systems (Digital Video Disc (DVD) or Compact Disk (CD)), etc. Computer-readable media may include non-transitory storage media. The data processor may be of any type suitable to the local technical environment, such as, but not limited to, general-purpose computers, special-purpose computers, microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and processors based on multi-core processor architectures.

[0119] A detailed description of exemplary embodiments of this application has been provided above through exemplary and non-limiting examples. However, various modifications and adjustments to the above embodiments will be apparent to those skilled in the art when considered in conjunction with the accompanying drawings and claims, without departing from the scope of this application. Therefore, the proper scope of this application will be determined by the claims.

Claims

1. A method for transmitting system performance parameters, characterized in that, include: Determine system performance parameters, which include key performance indicators (KPIs) and the reliability of the KPIs; wherein, the system performance parameters are measurement data used for KPI calculation. Send the system performance parameters; The determination of system performance parameters includes: Obtain performance measurement data; Based on the performance measurement data, determine the KPI information; Determine the credibility of the KPI information; wherein, the credibility of the KPI information refers to the KPI data integrity rate, which is to perform an integrity check on the collected data; The performance measurement data includes missing data, which refers to performance measurement data that was not successfully acquired. The acquisition of performance measurement data includes: In the event that performance measurement data is not successfully acquired, missing data is determined based on data acquired at a preset time. The preset time includes at least one of the following: a first preset time, which is the acquisition time corresponding to the performance measurement data acquired before the time when the missing data is acquired; and a second preset time, which is the acquisition time corresponding to the performance measurement data acquired after the time when the missing data is acquired.

2. The method according to claim 1, characterized in that, The step of determining missing data based on data at a preset time includes: The mean of the data at a preset time point is determined as the missing data.

3. The method according to claim 1, characterized in that, The acquisition of performance measurement data includes: If performance measurement data is not successfully obtained, the preset default value will be identified as missing data.

4. The method according to claim 1, characterized in that, Before determining the KPI information based on the performance measurement data, the process also includes: If the performance measurement data includes missing data, the identification information of the missing data is removed to obtain new performance measurement data.

5. The method according to claim 1, characterized in that, Determining the credibility of the KPI information includes: Determine the number of target data in the performance measurement data, wherein the target data is the data excluding missing data; The reliability of the KPI information is determined by the percentage of the number of target data points to the total number of all performance measurement data points required to calculate the KPI information.

6. A device for transmitting system performance parameters, characterized in that, include: The determination module is configured to determine system performance parameters, which include key performance indicator (KPI) information and the reliability of the KPI information; wherein, the system performance parameters are measurement data used for KPI calculation. The sending module is configured to send the system performance parameters; The module to be determined includes: The acquisition unit is configured to acquire performance measurement data. The first determining unit is configured to determine KPI information based on the performance measurement data; The second determining unit is configured to determine the credibility of the KPI information; wherein, the credibility of the KPI information refers to the KPI data integrity rate, which is to perform an integrity check on the collected data; The performance measurement data includes missing data, which refers to performance measurement data that was not successfully acquired. The acquisition unit is specifically configured to: determine missing data based on data at a preset time when performance measurement data is not successfully acquired, wherein the preset time includes at least one of the following: a first preset time, which is the acquisition time corresponding to the performance measurement data acquired before the time when the missing data is acquired; and a second preset time, which is the acquisition time corresponding to the performance measurement data acquired after the time when the missing data is acquired.

7. A management device, characterized in that, include: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

8. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1-5.

Images