5G industrial application quality comprehensive availability evaluation method and device, and medium

Abstract

The invention relates to a 5G industrial application quality comprehensive availability evaluation method and apparatus, and a medium. The method comprises the steps of obtaining index values of each 5G industrial application quality comprehensive evaluation index in a current period and a historical period; determining the influence degree of each network quality index in each period according to the service standard maximum value and the service standard minimum value of each network quality index, and determining the weight and the evaluation reference value of each network quality index according to the influence degree; evaluating the network quality value according to the index value, the weight and the evaluation reference value of each network quality index in the current period; evaluating the service quality value according to the index value of each service quality index in the current period; and carrying out comprehensive availability evaluation on the 5G industrial application quality according to the network quality value, the service quality value and the index value of each network wireless index in the current period. According to the method, 5G industrial application quality evaluation and industrial application are combined, and 5G industrial application quality evaluation is effectively carried out.

Description

Technical Field

[0001] The present invention relates to the field of industrial network communication technology, and in particular to a method, device, and medium for evaluating the comprehensive availability of 5G industrial application quality. Background Art

[0002] The communication network technology used in industrial sites should meet the more stringent requirements of industrial applications such as real-time, determinism, availability, reliability, and anti-interference. Different industries have different requirements for 5G. 5G is gradually improving the determinism and real-time performance of the network through slicing and key data QoS guarantee. However, compared with wired networks, it still has high latency and large jitter. Although the performance of 5G is constantly improving, it still cannot fully meet the needs of some industrial core applications.

[0003] In the process of 5G industrial applications, the current 5G network construction method is to build first and then test. There is a risk that the network cannot meet business needs. After the network is built, there is a lack of unified evaluation methods. The network is tested using methods such as Ping, which cannot truly reflect the performance of the network in carrying multiple services. Whether 5G meets the requirements of industrial application scenarios requires continuous debugging after the business is launched, resulting in long construction periods and increased costs. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a 5G industrial application quality comprehensive availability evaluation method, device, and medium, which solves the technical problems that the prior art evaluation method is single, not combined with industrial application services, and cannot effectively evaluate the quality of 5G industrial applications in industrial field environments.

[0006] (II) Technical solution

[0007] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0008] In a first aspect, an embodiment of the present invention provides a method for evaluating the comprehensive availability of 5G industrial application quality, including:

[0009] Obtain comprehensive evaluation indicators of the quality of 5G industrial applications, and the indicator values ​​in the current cycle and historical cycle; among them, the comprehensive evaluation indicators of the quality of 5G industrial applications include network wireless indicators, network quality indicators and service quality indicators. The network quality indicators include: delay reliability, jitter, packet loss rate, service rate, synchronization accuracy, and the service quality indicators include: availability and stability;

[0010] Determine the influence of each network quality indicator in each period according to the maximum value and minimum value of the service standard of each network quality indicator, and determine the weight and evaluation benchmark value of each network quality indicator according to the influence;

[0011] Evaluate the network quality value of 5G industrial applications based on the indicator value, weight and evaluation benchmark value of each network quality indicator in the current cycle;

[0012] Evaluate the service quality value of 5G industrial applications based on the indicator values ​​of each service quality indicator in the current cycle;

[0013] Based on the network quality value, service quality value and the index values ​​of each network wireless indicator in the current period, a comprehensive availability evaluation of 5G industrial application quality is conducted.

[0014] Optionally, the network quality value of the 5G industrial application is evaluated according to the indicator value, weight and evaluation reference value of each network quality indicator in the current cycle, including:

[0015] Evaluate the network quality value of 5G industrial applications through the network quality evaluation formula;

[0016] The network quality evaluation formula is:

[0017] Q n is the network quality value of 5G industrial applications, z is the current cycle identifier, is the index value of delay reliability in the current cycle, es D is the evaluation benchmark value of delay reliability, w D is the weight of delay reliability, is the index value of jitter in the current cycle, es J is the evaluation reference value of jitter, w J is the jitter weight, is the indicator value of packet loss rate in the current period, es R is the evaluation benchmark value of packet loss rate, w R is the weight of packet loss rate, is the indicator value of the service rate in the current period, es S is the evaluation benchmark value of the service rate, w S is the weight of the service rate, is the index value of synchronization accuracy in the current cycle, es T is the evaluation benchmark value of synchronization accuracy, w T is the weight of synchronization accuracy;

[0018] According to the indicator values ​​of each service quality indicator in the current cycle, the service quality value of 5G industrial applications is evaluated, including:

[0019] Evaluate the service quality value of 5G industrial applications through the service quality evaluation formula;

[0020] The service quality evaluation formula is:

[0021] Q b is the service quality value of 5G industrial applications, is the indicator value of availability in the current cycle, es A is the evaluation benchmark value of availability, w A is the weight of availability, is the indicator value of stability in the current period, es C is the evaluation benchmark value of stability, w C is the weight of stability.

[0022] Optionally, the indicator value of the network quality indicator in the current period of each 5G industrial application quality comprehensive evaluation indicator is obtained, including:

[0023] Obtain the latency and bandwidth of industrial applications at each monitoring moment in the current cycle, and determine the latency reliability index value in the current cycle based on the latency and bandwidth Among them, z is the current cycle identifier;

[0024] Obtain the degree of change in the delay of the industrial application at each monitoring moment in the current cycle, and determine the degree of change as the index value of the jitter in the current cycle

[0025] Obtain the number of lost data packets and the total number of transmitted data packets of the industrial application in the current cycle, and determine the quotient of the number of lost data packets and the total number of transmitted data packets as the indicator value of the packet loss rate in the current cycle

[0026] Obtain the average data size and average data sending time of the industrial application in the current cycle, and determine the quotient of the average data size and the average data sending time as the indicator value of the business rate in the current cycle

[0027] Get the index value of synchronization accuracy in the current period through the synchronization signal

[0028] Obtain the indicator values ​​of the service quality indicators in the current cycle of each 5G industrial application quality comprehensive evaluation indicator, including:

[0029] Get the average time it takes for the industrial application device to resume network communication after restarting in the current cycle, and determine the average time as the indicator value of availability in the current cycle

[0030] Obtain the operating time of the equipment in the industrial application in the current cycle, and determine the operating time as the indicator value of stability in the current cycle

[0031] Optionally, the method further comprises:

[0032] Determine the nature of the 5G industrial application, where the nature is a fixed application or a non-fixed application; if the 5G industrial application is fixed in the device, the nature of the 5G industrial application is a fixed application, otherwise the nature of the 5G industrial application is a non-fixed application;

[0033] If the nature of the 5G industrial application is a fixed application, the network delay of the equipment where the 5G industrial application is located is tested at each monitoring time, and the network delay is determined as the delay at the monitoring time;

[0034] If the nature of the 5G industrial application is a non-fixed application, the network delay of the equipment where the 5G industrial application is located is tested at each monitoring moment, and the difference between the network delay and the preset application delay compensation value is determined as the delay at the monitoring moment.

[0035] Optionally, the network latency of the equipment where the 5G industrial application is located is tested at each monitoring time, including:

[0036] At any monitoring moment, test the network latency of the device where the 5G industrial application is located by following the steps below:

[0037] Perform clock synchronization;

[0038] Sending a test case to the device where the 5G industrial application is located through the test device, wherein the test case includes business data, and a timestamp is written in the business communication protocol data field or the reserved field of the message header of the sending test case;

[0039] Calculate the difference between the service data in the test case from the test device to the device where the 5G industrial application is located based on the timestamp, and determine the difference as the network delay of the device where the 5G industrial application is located; or calculate the difference between the service data in the test case from the test device to the device where the 5G industrial application is located, and the difference from the device where the 5G industrial application is located to the test device based on the timestamp, and determine both differences as the network delay of the device where the 5G industrial application is located;

[0040] If the monitoring period corresponding to the current monitoring time is not greater than the network delay, the bandwidth is determined as the product of the amount of data sent between any monitoring time and the adjacent previous monitoring time and the sending frequency;

[0041] If the monitoring period corresponding to the current monitoring moment is greater than the network delay, any monitoring moment will no longer be used as a monitoring moment. A new monitoring period is determined based on the network delay, and a new monitoring moment is determined based on the adjacent previous monitoring moment and the new monitoring period.

[0042] Optionally, determining the influence of each network quality indicator in each period includes:

[0043] like The influence of any network quality index i in any historical period x is in, is the index value of any network quality index i in any historical period x, i min is the minimum value of the service standard for any network quality indicator i, i max The maximum value of the service standard for any network quality indicator i;

[0044] like but

[0045] like but x is the historical cycle identifier;

[0046] The weight and evaluation benchmark value of each network quality indicator are determined according to the impact, including:

[0047] According to the preset network quality indicator priority, the weight of each network quality indicator is determined in turn according to the impact; among them, the weight of any network quality indicator i is the weight adjustment value of any network quality index i, when the average value of the influence of any network quality index i in each period is μ i =0 o'clock otherwise σ i is the standard deviation of the influence of any network quality indicator i in each cycle, y is the network quality indicator with a determined weight, n i =5-the number of network quality indicators with determined weights;

[0048] Determine the standard value of each network quality indicator; among them, the standard value of any network quality indicator i

[0049] The evaluation benchmark value of each network quality indicator is determined according to the standard value of each network quality indicator.

[0050] Optionally, determining an evaluation reference value of each network quality indicator according to a standard value of each network quality indicator includes:

[0051] For any network quality indicator i, its evaluation benchmark value is determined through the following steps:

[0052] Determine the adjustment ratio of any network quality indicator i

[0053] Determine the evaluation benchmark value es of any network quality indicator i i =min{bs i ×(1+ch i ),i max}.

[0054] In a second aspect, an embodiment of the present invention provides a 5G industrial application quality comprehensive availability evaluation device, including:

[0055] The acquisition module is used to obtain the comprehensive evaluation indicators of the quality of 5G industrial applications, and the indicator values ​​in the current cycle and the historical cycle; among them, the comprehensive evaluation indicators of the quality of 5G industrial applications include network wireless indicators, network quality indicators and service quality indicators. The network quality indicators include: delay reliability, jitter, packet loss rate, service rate, synchronization accuracy, and the service quality indicators include: availability and stability;

[0056] A determination module, used to determine the influence of each network quality indicator in each period according to the maximum value and minimum value of the business standard of each network quality indicator, and determine the weight and evaluation reference value of each network quality indicator according to the influence;

[0057] The first evaluation module is used to evaluate the network quality value of 5G industrial applications according to the indicator value, weight and evaluation benchmark value of each network quality indicator in the current cycle;

[0058] The second evaluation module is used to evaluate the service quality value of 5G industrial applications according to the indicator value of each service quality indicator in the current cycle;

[0059] The third evaluation module is used to conduct a comprehensive availability evaluation on the quality of 5G industrial applications based on the network quality value, service quality value and the index values ​​of each network wireless index in the current period.

[0060] In a third aspect, an embodiment of the present invention provides an electronic device, including:

[0061] Memory;

[0062] Multi-core processors; and

[0063] Computer programs;

[0064] Wherein, the multi-core processor includes a plurality of idle processing cores;

[0065] The computer program is stored in the memory and is configured to execute a 5G industrial application quality comprehensive availability evaluation method as described in the first aspect above.

[0066] In a fourth aspect, an embodiment of the present invention provides a computer-readable medium having computer executable instructions stored thereon, which, when executed by a processor, implements a method for comprehensive availability evaluation of 5G industrial application quality as described in the first aspect above.

[0067] (III) Beneficial effects

[0068] The beneficial effects of the present invention are as follows: the present invention relates to a method, device, equipment, and medium for evaluating the comprehensive availability of 5G industrial application quality. The method obtains the comprehensive evaluation index of each 5G industrial application quality, and the index value in the current cycle and the historical cycle; according to the maximum value of the business standard and the minimum value of the business standard of each network quality index, the influence of each network quality index in each cycle is determined, and the weight and evaluation benchmark value of each network quality index are determined according to the influence; according to the index value, weight, and evaluation benchmark value of each network quality index in the current cycle, the network quality value of 5G industrial application is evaluated; according to the index value of each business quality index in the current cycle, the business quality value of 5G industrial application is evaluated; according to the network quality value, the business quality value, and the index value of each network wireless index in the current cycle, the quality of 5G industrial application is comprehensively evaluated for availability. The method of the present invention performs a comprehensive availability evaluation on the quality of 5G industrial application according to the network quality value, the business quality value, and the index value of each network wireless index in the current cycle, realizes the combination of 5G industrial application quality evaluation and industrial application, and effectively evaluates the quality of 5G industrial application. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 A flow chart of a 5G industrial application quality comprehensive availability evaluation method provided by the present invention;

[0070] Figure 2 A schematic diagram of communication between a test device and a terminal to be tested provided by the present invention;

[0071] Figure 3 A schematic diagram of the structure of a 5G industrial application quality comprehensive availability evaluation device provided by the present invention;

[0072] Figure 4 The present invention provides a schematic structural diagram of an electronic device. DETAILED DESCRIPTION

[0073] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0074] In the process of 5G industrial applications, the current 5G network construction method is to build first and then test. There is a risk that the network cannot meet business needs. After the network is built, there is a lack of unified evaluation methods. The network is tested using methods such as Ping, which cannot truly reflect the performance of the network in carrying multiple services. Whether 5G meets the requirements of industrial application scenarios requires continuous debugging after the business is launched, resulting in long construction periods and increased costs.

[0075] To this end, the present invention relates to a 5G industrial application quality comprehensive availability evaluation method, device, equipment, and medium. The method obtains the comprehensive evaluation index of each 5G industrial application quality, and the index value in the current cycle and the historical cycle; according to the maximum value of the business standard and the minimum value of the business standard of each network quality index, the influence of each network quality index in each cycle is determined, and the weight and evaluation benchmark value of each network quality index are determined according to the influence; according to the index value, weight and evaluation benchmark value of each network quality index in the current cycle, the network quality value of 5G industrial application is evaluated; according to the index value of each business quality index in the current cycle, the business quality value of 5G industrial application is evaluated; according to the network quality value, the business quality value and the index value of each network wireless index in the current cycle, the quality of 5G industrial application is comprehensively evaluated for availability. The method of the present invention performs a comprehensive availability evaluation on the quality of 5G industrial applications according to the network quality value, the business quality value and the index value of each network wireless index in the current cycle, realizes the combination of 5G industrial application quality evaluation and industrial application, and effectively evaluates the quality of 5G industrial applications.

[0076] See also Figure 1 This embodiment provides a 5G industrial application quality comprehensive availability evaluation method, and the implementation process of the method is as follows:

[0077] 101. Obtain comprehensive evaluation indicators of the quality of various 5G industrial applications, and the indicator values ​​in the current cycle and historical cycles.

[0078] In specific implementation, the method provided in this embodiment will be executed every preset period (i.e., evaluation period) to perform comprehensive availability evaluation on the quality of 5G industrial applications. Therefore, in step 101, the index value of each 5G industrial application quality comprehensive evaluation index in the current period (such as the current period is the zth period) will be obtained, as well as the index value of each 5G industrial application quality comprehensive evaluation index in each period obtained historically (i.e., the 1st to z-1 periods).

[0079] Each time a period obtains the indicator value within the period, it will be stored. Therefore, the indicator value of the historical period can be directly read from the reading position.

[0080] Among them, the comprehensive evaluation indicators of 5G industrial application quality include network wireless indicators, network quality indicators and service quality indicators.

[0081] 1. Network wireless indicators

[0082] Network wireless indicators are indicators used to describe the quality of the network environment, including but not limited to: field strength, multipath delay, multipath attenuation and other indicators.

[0083] The acquisition of network wireless indicators can be achieved by using existing solutions.

[0084] 2. Network quality indicators

[0085] Network quality indicators include: delay reliability (D), jitter (J), packet loss rate (R), service rate (S), and synchronization accuracy (T).

[0086] If the current cycle is the zth cycle, the process of obtaining the network quality indicator is:

[0087] Obtain the latency and bandwidth of industrial applications at each monitoring moment in the current cycle, and determine the latency reliability index value in the current cycle based on the latency and bandwidth For example, the number of monitoring moments at which the delay is not greater than the delay threshold and the bandwidth is not greater than the bandwidth threshold is determined from the delay and bandwidth at each monitoring moment, and the quotient of this number and the total number of monitoring moments is determined as the indicator value of the delay reliability in the current cycle.

[0088] Obtain the degree of change in the delay of the industrial application at each monitoring moment in the current cycle, and determine the degree of change as the index value of the jitter in the current cycle

[0089] Obtain the number of lost data packets and the total number of transmitted data packets of the industrial application in the current cycle, and determine the quotient of the number of lost data packets and the total number of transmitted data packets as the indicator value of the packet loss rate in the current cycle

[0090] Obtain the average data size and average data sending time of the industrial application in the current cycle, and determine the quotient of the average data size and the average data sending time as the indicator value of the business rate in the current cycle That is, the service rate is determined based on the size of the transmitted data and the transmission time.

[0091] Get the index value of synchronization accuracy in the current period through the synchronization signal

[0092] In each cycle (i.e., evaluation cycle), multiple delay and bandwidth monitoring will be performed at a certain period (i.e., monitoring period). For the convenience of description, the monitoring period in the evaluation cycle is named monitoring period. In other words, the delay and bandwidth will be monitored when the monitoring time is reached according to the monitoring period.

[0093] The monitoring process is as follows:

[0094] 1) Determine the nature of 5G industrial applications.

[0095] The nature is a fixed application or a non-fixed application. If the 5G industrial application is fixed in the device, the nature of the 5G industrial application is a fixed application, otherwise the nature of the 5G industrial application is a non-fixed application.

[0096] For example, an application-fixed device means that the application is fixed on the terminal to be tested and cannot be changed or erased; while an application-non-fixed device means that the application on the terminal to be tested is not fixed and can be changed or erased.

[0097] 2) If the nature of the 5G industrial application is a fixed application, the network delay of the equipment where the 5G industrial application is located is tested at each monitoring moment, and the network delay is determined as the delay at the monitoring moment (this delay is the network delay).

[0098] 3) If the nature of 5G industrial applications is non-fixed applications, such as some devices with built-in modules that generally cannot monitor application delays, the application delay can be estimated through network delay. That is, when the application delay cannot be tested, the application delay can be evaluated by testing the network delay, that is, testing the network delay of the device where the 5G industrial application is located at each monitoring moment, and determining the difference between the network delay and the preset application delay compensation value as the delay at the monitoring moment (this delay is the application delay). That is, application delay = network delay - preset application delay compensation value.

[0099] For example, due to the limitations of industrial equipment, in order not to affect operational safety, new applications are generally not allowed to be tested. For example, for devices such as PLC controllers, the application delay can only be estimated by testing the network delay, and different empirical values ​​are compensated for processors with different computing capabilities, such as microcontrollers and ARM.

[0100] like Figure 2 As shown in the figure, network latency refers to the time required for a network layer information transmission between a terminal under test and a test device that have a communication relationship; application latency refers to the time required for a network layer information transmission between a terminal under test and a test device that have a communication relationship. The terminal under test and the test device can communicate through a cellular network or a wired network.

[0101] When performing latency monitoring, different delays will be monitored based on the nature of the 5G industrial application, and regardless of the type of delay, network delay will be monitored.

[0102] For any monitoring moment, the monitoring process of network delay is:

[0103] Among them, latency refers to the time required for information transmission at the network layer between a test device with a communication relationship and the device where the 5G industrial application is located. The latency monitoring process is shown in steps 201 to 205.

[0104] 201, perform clock synchronization.

[0105] Clock synchronization between the test equipment and the equipment where the 5G industrial application is located can ensure test accuracy.

[0106] 202. Send a test case to the device where the 5G industrial application is located through the test device, wherein the test case includes business data, and a timestamp is written in the business communication protocol data field or the reserved field of the message header of the sending test case.

[0107] 203, calculate the difference of the service data in the test case from the test device to the device where the 5G industrial application is located according to the timestamp, and determine the difference as the network delay of the device where the 5G industrial application is located. Alternatively, calculate the difference of the service data in the test case from the test device to the device where the 5G industrial application is located, and the difference from the device where the 5G industrial application is located to the test device according to the timestamp, and determine both differences as the network delay of the device where the 5G industrial application is located.

[0108] Among them, the process of calculating the difference between the service data in the test case from the test device to the device where the 5G industrial application is located according to the timestamp, and determining the difference as the network delay of the device where the 5G industrial application is located is a one-way monitoring process. For example, for test equipment using 5G CPE or 5G gateway, the test delay is obtained by calculating the difference between the service data in the test case from the sender to the receiver.

[0109] In specific implementation, network delay can also be compensated. For example, after calculating the difference between the business data in the test case from the test device to the device where the 5G industrial application is located based on the timestamp, the difference between the difference and the preset application delay compensation value is used as the network delay of the device where the 5G industrial application is located.

[0110] In specific implementation, the application delay compensation value ranges from 500ns to 1ms.

[0111] 204. If the monitoring period corresponding to the current monitoring time is not greater than the network delay, the bandwidth is determined as the product of the amount of data sent between any monitoring time and the adjacent previous monitoring time and the sending frequency.

[0112] 205. If the monitoring period corresponding to the current monitoring moment is greater than the network delay, any monitoring moment will no longer be used as a monitoring moment. A new monitoring period is determined based on the network delay (i.e., the new monitoring period is determined to be greater than the network delay). A new monitoring moment is determined based on the adjacent previous monitoring moment and the new monitoring period.

[0113] That is, if the monitoring period corresponding to the current monitoring time is greater than the network delay, the currently obtained network delay is discarded, and when it is determined that a new monitoring time has arrived, steps 201 to 205 are re-executed to perform new monitoring.

[0114] For example, the current monitoring cycle is Cy0. If Cy0 is not greater than the network delay obtained in step 203, the bandwidth is obtained based on the amount of data sent and the frequency of sending (e.g., bandwidth = amount of data sent × number of times sent per second). If Cy0 is greater than the network delay obtained in step 203, Cy0 is reset to a value greater than the network delay obtained in step 203, the network delay obtained in step 203 is discarded, and the time after Cy0 is reset from the last monitoring time is determined as the new monitoring time. When the new monitoring time is reached, steps 201 to 205 are executed again to obtain a new monitoring value.

[0115] 3. Business quality indicators include: availability (T) and stability (C).

[0116] If the current cycle is the zth cycle, the process of obtaining the service quality indicator is as follows:

[0117] Get the average time it takes for the industrial application device to resume network communication after restarting in the current cycle, and determine the average time as the indicator value of availability in the current cycle

[0118] Obtain the running time of the equipment where the industrial application is located in the current cycle (i.e. the total duration of the current cycle minus the duration of interruption, shutdown, etc. of the equipment where the industrial application is located in the current cycle), and determine the running time as the indicator value of stability in the current cycle

[0119] It should be noted that the basic indicator values ​​of the above-mentioned 5G industrial application quality comprehensive evaluation indicators, including: network wireless indicators, network wireless indicators, network quality indicators and service quality indicator values ​​(such as the duration of interruptions and downtime in the current period of stability), are all collected using existing indicator collection equipment and are not limited here.

[0120] The centralized collection equipment is provided below for illustrative purposes only. The equipment provided below does not limit the method provided in this embodiment. That is to say, the method provided in this embodiment can collect basic indicator values ​​through the following equipment or through other methods.

[0121] Data acquisition industrial applications: The business data of data acquisition industrial applications include equipment operation data and sensor data.

[0122] Visual industrial applications: The business data of visual industrial applications include video data and image data.

[0123] Control-type industrial applications: The business data of control-type industrial applications includes control instruction data.

[0124] Hybrid industrial applications, the business data running in hybrid industrial applications includes at least two of video data, image data and control instruction data.

[0125] Data acquisition applications mainly focus on large-bandwidth, high-concurrency services, which are insensitive to latency and packet loss; vision applications mainly focus on visual inspection and cloud control services; control applications mainly focus on control and collaborative services; hybrid applications refer to business types that contain at least two of ABC.

[0126] The data is collected through test equipment, where the test equipment includes actual business terminal equipment or test equipment with similar performance. The test equipment can be actual equipment using 5G communication, such as 5G+AGV, 5G+controller, 5G+video equipment, 5G+robot, 5G gateway and other equipment. If the actual business equipment cannot be tested, it is necessary to use simulation equipment with similar performance to replace the test to evaluate the business performance. Therefore, the test equipment uses 5G CPE or 5G gateway or built-in 5G module to access the 5G network. Furthermore, the test case is formatted data containing video, pictures, text, business data or unformatted data containing video, pictures, text, business data.

[0127] 102. Determine the influence of each network quality indicator in each period according to the maximum value and the minimum value of the service standard of each network quality indicator, and determine the weight and evaluation reference value of each network quality indicator according to the influence.

[0128] In step 102, the influence of each network quality indicator in each period is determined based on the maximum and minimum service standards of each network quality indicator, and then the weight and evaluation reference value of each network quality indicator are determined based on the influence.

[0129] The implementation details of each process are as follows:

[0130] 1. According to the maximum and minimum service standards of each network quality indicator, determine the implementation details of the impact of each network quality indicator in each cycle:

[0131] like The influence of any network quality index i in any historical period x is in, is the index value of any network quality index i in any historical period x, i min is the minimum value of the service standard for any network quality indicator i, i max The maximum value of the service standard for any network quality indicator i.

[0132] like but

[0133] like but

[0134] Among them, x is the historical cycle identifier.

[0135] In a specific implementation, i is a network quality indicator identifier, such as i=delay reliability, or i=jitter, or i=packet loss rate, or i=service rate, or i=synchronization accuracy.

[0136] The service standard maximum value and service standard minimum value of each network quality indicator are pre-set based on service requirements. The specific value determination scheme can adopt existing schemes, such as those determined based on experience, and will not be described in detail here.

[0137] If for any network quality indicator i, the minimum service standard is preset to i min , the maximum value of the business standard is i max , then if in any history x This means that the value of any network quality indicator i in this period meets the business requirements and is normal.

[0138] if This means that the value of any network quality indicator i in this period does not meet the business requirements, and the value of this indicator is affected, so It represents the influence of the indicator value in this period. This means that the value of any network quality indicator i in this period does not meet the business requirements, and the value of this indicator is affected. Therefore, Characterizes the degree of influence of the indicator value within this period.

[0139] 2. Implementation details of determining the weight and evaluation benchmark value of each network quality indicator based on the impact:

[0140] 1) According to the preset network quality indicator priority, the weight of each network quality indicator is determined in turn according to the impact.

[0141] Among them, the weight of any network quality index i is the weight adjustment value of any network quality index i, when the average value of the influence of any network quality index i in each period is μ i =0 (i.e. Where X is the total number of historical cycles) otherwise σ i is the standard deviation of the influence of any network quality indicator i in each cycle (i.e. y is the network quality indicator with a determined weight, n i 5 - the number of network quality indicators for which weights have been determined.

[0142] For different 5G industrial applications and / or different business requirements, the tolerance for the impact of each network quality indicator is different. For example, for real-time related (such as real-time positioning) 5G industrial applications and / or services, very precise synchronization is required to obtain accurate location information, so the tolerance for the network quality indicator of synchronization accuracy is low (that is, although the impact of synchronization accuracy is small, it is also considered that the quality is low and cannot meet the needs). For example, for data transmission related (such as sensor return data), extremely low packet loss rate is required to accurately transmit data, so the tolerance for the network quality indicator of packet loss rate is low (that is, although the impact of packet loss rate is small, it is also considered that the quality is low and cannot meet the needs). Therefore, before executing the method provided in this embodiment, the priority of each network quality indicator will be determined based on the tolerance of each network quality indicator for the 5G industrial application and / or business demand to be evaluated. The lower the tolerance, the more important the indicator is for the evaluation and the higher the priority.

[0143] in addition, is the weight adjustment value of any network quality indicator i, which is obtained based on the influence of any network quality indicator i in each historical period. Therefore, Characterizes the influence of any network quality indicator i.

[0144] In this step, the weights of network quality indicators with lower tolerance (i.e., more important for evaluation) are processed first. For example, the priority of each network quality indicator, from priority to non-priority, is delay reliability, jitter, packet loss rate, service rate, and synchronization accuracy.

[0145] For example, any network quality indicator i = packet loss rate, the weight of packet loss rate

[0146] in, is the weight adjustment value of any network quality indicator i, which is determined according to the influence of any network quality indicator i.

[0147] Still taking i = packet loss rate as an example, if the average impact of packet loss rate in each period is μ 丢包率 =0, it means that the packet loss rate index value meets the business requirements from a historical perspective and does not need to be adjusted. Otherwise, it means that the packet loss rate indicator value does not meet the business requirements from a historical perspective.

[0148]

[0149] in, y is the network quality indicator with a determined weight (Before determining the weight of the packet loss rate, the delay reliability and jitter, which are more prioritized than the packet loss rate, have already determined their weights. Therefore, y is the delay reliability, jitter, The number of network quality indicators with determined weights is 2, i.e. n 丢包率 is 5-2=3).

[0150] In this way, the weight of each network quality indicator will be determined in order of priority, and the weight of any network quality indicator is not fixed, but is determined based on its influence in the historical period and the weight of the already determined network quality indicators. The weight of each network quality indicator not only takes into account the situation of the network quality indicator itself, but also considers the value of the network quality indicator for the quality evaluation of 5G industrial applications, which can improve the accuracy of Suizhou's evaluation results.

[0151] 2) Determine the evaluation benchmark value of each network quality indicator.

[0152] In specific implementation, the benchmark value of each indicator provided by the user can be obtained, and the benchmark value can be determined as the evaluation benchmark value of each network quality indicator. If there is an indicator for which the user does not provide a benchmark value, the typical value of the indicator in the industry can be obtained, and the typical value can be used as the evaluation benchmark value of the indicator.

[0153] In addition to the above methods for determining the evaluation benchmark value, the following methods can also be used to determine the evaluation benchmark value of each network quality indicator:

[0154] (1) Determine the standard value of each network quality indicator.

[0155] Among them, the standard value of any network quality index i

[0156] (2) Determine the evaluation benchmark value of each network quality indicator based on the standard value of each network quality indicator.

[0157] For example, for any network quality indicator i, its evaluation benchmark value is determined by the following steps: Determine the adjustment ratio of any network quality indicator i Determine the evaluation benchmark value es of any network quality indicator i i =min{bs i ×(1+ch i ),i max}.

[0158] because is the weight adjustment value of any network quality indicator i, which is obtained based on the influence of any network quality indicator i in each historical period. Therefore, It represents the influence degree of any network quality index i. When the evaluation benchmark value of any network quality index i is adjusted, The larger the value is, the more impact any network quality indicator i has, and the evaluation benchmark value needs to be improved. i It reflects whether any network quality indicator i meets the business requirements as a whole. The larger the value, the less it meets the business requirements as a whole, and the evaluation benchmark value needs to be improved. Therefore, the adjustment ratio Adjust the evaluation benchmark value of any network quality index i to es i =min{bs i ×(1+ch i ),i max}, so that when conducting comprehensive availability evaluation of 5G industrial application quality, the comparison benchmark is based on the dynamic changes of historical network quality indicators, which is more in line with the actual industrial application of network quality indicators, and realizes the combination of 5G industrial application quality evaluation and industrial application.

[0159] The above-mentioned method of determining the evaluation benchmark value of each network quality indicator according to the standard value of each network quality indicator can effectively provide a scheme for determining the evaluation benchmark value for indicators for which the user has not provided a benchmark value, or for which the user has not provided a benchmark value and the typical value in the industry cannot be determined (such as indicators that cannot be measured and therefore the typical value in the industry cannot be determined).

[0160] 103. Evaluate the network quality value of 5G industrial applications based on the indicator value, weight and evaluation benchmark value of each network quality indicator in the current period.

[0161] In step 103, the network quality value of 5G industrial applications will be evaluated through the network quality evaluation formula.

[0162] The network quality evaluation formula is:

[0163] Q n is the network quality value of 5G industrial applications, z is the current cycle identifier, is the index value of delay reliability in the current cycle, es D is the evaluation benchmark value of delay reliability, w D is the weight of delay reliability, is the index value of jitter in the current cycle, es J is the evaluation reference value of jitter, w J is the jitter weight, is the indicator value of packet loss rate in the current period, es R is the evaluation benchmark value of packet loss rate, w R is the weight of packet loss rate, is the indicator value of the service rate in the current period, es S is the evaluation benchmark value of the service rate, w S is the weight of the service rate, is the index value of synchronization accuracy in the current cycle, es T is the evaluation benchmark value of synchronization accuracy, w T is the weight of synchronization accuracy.

[0164] The weight and evaluation reference value of each network quality indicator are obtained in step 102 .

[0165] 104. Evaluate the service quality value of 5G industrial applications according to the indicator value of each service quality indicator in the current cycle.

[0166] In step 104, the service quality value of the 5G industrial application will be evaluated through the service quality evaluation formula.

[0167] The service quality evaluation formula is:

[0168] Q b is the service quality value of 5G industrial applications, is the indicator value of availability in the current cycle, es A is the evaluation benchmark value of availability (such as the preset maximum time to restore network communication after restart), w A is the weight of availability, is the indicator value of stability in the current period, es C is the evaluation benchmark value of stability (such as the maximum continuous operation time within a preset cycle, for example, the time is a value less than 36 hours), w C is the stability weight (the weight here is pre-set, as long as w A +wC =1, such as w A =w C =0.5).

[0169] 105. Based on the network quality value, service quality value and the index value of each network wireless index in the current period, a comprehensive availability evaluation is conducted on the quality of 5G industrial applications.

[0170] For example, the network quality value calculated according to step 103 is matched with a pre-set network quality value grading standard (such as network quality is divided into three levels, namely, good network quality availability, average network quality availability, and poor network availability) to determine the grading result corresponding to the network quality value calculated in step 103.

[0171] The service quality value calculated according to step 104 is matched with a preset service quality value grading standard (e.g., service quality is divided into three levels, namely, good service quality availability, general service quality availability, and poor service quality availability) to determine the grading result corresponding to the service quality value calculated in step 104.

[0172] Then, according to the existing network wireless quality evaluation method, the network wireless quality classification results corresponding to the indicator values ​​of each network wireless indicator in the current cycle are determined (for example, the network wireless quality is divided into three levels, namely, good network wireless quality, average network wireless quality, and poor network wireless quality).

[0173] Finally, the three evaluation results are combined as the comprehensive availability evaluation result of 5G industrial application quality. For example, the comprehensive availability evaluation result of 5G industrial application quality is: the network quality availability is good, but the network wireless quality is poor, and the service quality availability is average.

[0174] Alternatively, the worst result between the network quality grading result and the network wireless quality grading result is selected as the network evaluation result. For example, if the network quality availability is good and the network wireless quality is poor, the network evaluation result is determined to be poor, and the comprehensive availability evaluation result of 5G industrial application quality is obtained as: the network is poor, but the service quality availability is average.

[0175] It should be noted that the above is a comprehensive availability evaluation of the quality of 5G industrial applications based on the network quality value, service quality value and the indicator value of each network wireless indicator in the current cycle, that is, a comprehensive availability evaluation is performed from three dimensions: network quality, service quality and wireless indicators. In the specific implementation, the evaluation dimension can also be increased, or the evaluation dimension can be flexibly adjusted to flexibly adjust the method provided in this embodiment, so that the 5G industrial application quality comprehensive availability evaluation method provided in the embodiment can more accurately evaluate the comprehensive availability of the 5G industrial application quality provided in the embodiment.

[0176] Among them, the added dimension can also obtain the indicator value of the evaluation indicator under the dimension in the current cycle and the historical cycle through step 101, determine the weight and evaluation benchmark value of the indicator under the dimension through step 102, and evaluate the network quality value of 5G industrial applications according to the indicator value, weight and evaluation benchmark value of each indicator under the dimension in the current cycle, as well as the indicator value, weight and evaluation benchmark value of each network quality indicator under the above-mentioned network quality, service quality and wireless indicator dimensions through step 103.

[0177] The 5G industrial application quality comprehensive availability evaluation method provided in this embodiment can be used to evaluate the quality comprehensive availability of 5G industrial applications that meet customer business needs and are consistent with actual operation scenarios based on customer needs, achieving the purpose of targeted evaluation and reducing the difficulty of evaluation. At the same time, when evaluating the quality comprehensive availability of 5G industrial applications, the business performance of the equipment where the 5G industrial application is located, such as timeliness, data volume, and system stability, is fully considered, thereby improving the efficiency and accuracy of the quality comprehensive availability evaluation of 5G industrial applications.

[0178] This embodiment provides a method for evaluating the comprehensive availability of 5G industrial application quality, obtaining the index values ​​of each 5G industrial application quality comprehensive evaluation index in the current cycle and the historical cycle; determining the influence of each network quality index in each cycle according to the maximum value and minimum value of the business standard of each network quality index, and determining the weight and evaluation benchmark value of each network quality index according to the influence; evaluating the network quality value of 5G industrial applications according to the index value, weight and evaluation benchmark value of each network quality index in the current cycle; evaluating the service quality value of 5G industrial applications according to the index value of each service quality index in the current cycle; and performing a comprehensive availability evaluation on the quality of 5G industrial applications according to the network quality value, service quality value and the index value of each network wireless index in the current cycle. The method provided in this embodiment performs a comprehensive availability evaluation on the quality of 5G industrial applications according to the network quality value, service quality value and the index value of each network wireless index in the current cycle, realizing the combination of 5G industrial application quality evaluation and industrial application, and effectively evaluating the quality of 5G industrial applications.

[0179] Based on the same inventive concept of the 5G industrial application quality comprehensive availability evaluation method, this embodiment provides a 5G industrial application quality comprehensive availability evaluation device, such as Figure 3 As shown, the device comprises:

[0180] The acquisition module 301 is used to obtain the comprehensive evaluation indicators of the quality of 5G industrial applications, and the indicator values ​​in the current cycle and the historical cycle. Among them, the comprehensive evaluation indicators of the quality of 5G industrial applications include network wireless indicators, network quality indicators and service quality indicators. The network quality indicators include: delay reliability, jitter, packet loss rate, service rate, synchronization accuracy, and the service quality indicators include: availability and stability.

[0181] The determination module 302 is used to determine the influence of each network quality indicator in each period according to the maximum value and minimum value of the service standard of each network quality indicator, and determine the weight and evaluation reference value of each network quality indicator according to the influence.

[0182] The first evaluation module 303 is used to evaluate the network quality value of 5G industrial applications according to the indicator value, weight and evaluation reference value of each network quality indicator in the current cycle.

[0183] The second evaluation module 304 is used to evaluate the service quality value of 5G industrial applications according to the indicator value of each service quality indicator in the current cycle.

[0184] The third evaluation module 305 is used to perform a comprehensive availability evaluation on the quality of 5G industrial applications based on the network quality value, service quality value and the index values ​​of each network wireless index in the current cycle.

[0185] Among them, the first evaluation module 303 is used to evaluate the network quality value of 5G industrial applications through a network quality evaluation formula.

[0186] The network quality evaluation formula is:

[0187] Q n is the network quality value of 5G industrial applications, z is the current cycle identifier, is the index value of delay reliability in the current cycle, es D is the evaluation benchmark value of delay reliability, w D is the weight of delay reliability, is the index value of jitter in the current cycle, es J is the evaluation reference value of jitter, w J is the jitter weight, is the indicator value of packet loss rate in the current period, es R is the evaluation benchmark value of packet loss rate, w R is the weight of packet loss rate, is the indicator value of the service rate in the current period, es S is the evaluation benchmark value of the service rate, w S is the weight of the service rate, is the index value of synchronization accuracy in the current cycle, es T is the evaluation benchmark value of synchronization accuracy, w T is the weight of synchronization accuracy.

[0188] The second evaluation module 304 is used to evaluate the service quality value of 5G industrial applications through a service quality evaluation formula.

[0189] The service quality evaluation formula is:

[0190] Q b is the service quality value of 5G industrial applications, is the indicator value of availability in the current cycle, es A is the evaluation benchmark value of availability, w A is the weight of availability, is the indicator value of stability in the current period, es C is the evaluation benchmark value of stability, w C is the weight of stability.

[0191] The acquisition module 301 is used to obtain the delay and bandwidth of the industrial application at each monitoring time in the current cycle, and determine the index value of the delay reliability in the current cycle according to the delay and bandwidth. Among them, z is the current cycle identifier.

[0192] Obtain the degree of change in the delay of the industrial application at each monitoring moment in the current cycle, and determine the degree of change as the index value of the jitter in the current cycle

[0193] Obtain the number of lost data packets and the total number of transmitted data packets of the industrial application in the current cycle, and determine the quotient of the number of lost data packets and the total number of transmitted data packets as the indicator value of the packet loss rate in the current cycle

[0194] Obtain the average data size and average data sending time of the industrial application in the current cycle, and determine the quotient of the average data size and the average data sending time as the indicator value of the business rate in the current cycle

[0195] Get the index value of synchronization accuracy in the current period through the synchronization signal

[0196] Get the average time it takes for the industrial application device to resume network communication after restarting in the current cycle, and determine the average time as the indicator value of availability in the current cycle

[0197] Obtain the operating time of the equipment in the industrial application in the current cycle, and determine the operating time as the indicator value of stability in the current cycle

[0198] The device further includes: a delay monitoring module, which is used to determine the nature of the 5G industrial application, wherein the nature is a fixed application or a non-fixed application. If the 5G industrial application is fixed in the device, the nature of the 5G industrial application is a fixed application, otherwise the nature of the 5G industrial application is a non-fixed application.

[0199] If the nature of the 5G industrial application is a fixed application, the network delay of the equipment where the 5G industrial application is located is tested at each monitoring moment, and the network delay is determined as the delay at that monitoring moment.

[0200] If the nature of the 5G industrial application is a non-fixed application, the network delay of the equipment where the 5G industrial application is located is tested at each monitoring moment, and the difference between the network delay and the preset application delay compensation value is determined as the delay at the monitoring moment.

[0201] Among them, the network latency of the equipment where 5G industrial applications are located is tested at each monitoring time, including:

[0202] At any monitoring moment, test the network latency of the device where the 5G industrial application is located by following the steps below:

[0203] Perform clock synchronization.

[0204] A test case is sent to the device where the 5G industrial application is located through a test device, wherein the test case includes business data, and a timestamp is written in the business communication protocol data field or the reserved field of the message header of the sending test case.

[0205] The difference between the service data in the test case from the test device to the device where the 5G industrial application is located is calculated based on the timestamp, and the difference is determined as the network delay of the device where the 5G industrial application is located. Alternatively, the difference between the service data in the test case from the test device to the device where the 5G industrial application is located, and the difference from the device where the 5G industrial application is located to the test device are calculated based on the timestamp, and both differences are determined as the network delay of the device where the 5G industrial application is located.

[0206] If the monitoring period corresponding to the current monitoring time is not greater than the network delay, the bandwidth is determined as the product of the amount of data sent between any monitoring time and the adjacent previous monitoring time and the sending frequency.

[0207] If the monitoring period corresponding to the current monitoring moment is greater than the network delay, any monitoring moment will no longer be used as a monitoring moment. A new monitoring period is determined based on the network delay, and a new monitoring moment is determined based on the adjacent previous monitoring moment and the new monitoring period.

[0208] Determine module 302, for The influence of any network quality index i in any historical period x is in, is the index value of any network quality index i in any historical period x, i min is the minimum value of the service standard for any network quality indicator i, i max The maximum value of the service standard for any network quality indicator i. but like but x is the historical period identifier.

[0209] According to the preset network quality indicator priority, the weight of each network quality indicator is determined in turn according to the impact. Among them, the weight of any network quality indicator i is the weight adjustment value of any network quality index i, when the average value of the influence of any network quality index i in each period is μ i =0 o'clock otherwise σ i is the standard deviation of the influence of any network quality indicator i in each cycle, y is the network quality indicator with a determined weight, n i 5 - the number of network quality indicators for which weights have been determined.

[0210] Determine the standard value of each network quality indicator. Among them, the standard value of any network quality indicator i

[0211] The evaluation benchmark value of each network quality indicator is determined according to the standard value of each network quality indicator.

[0212] For any network quality indicator i, its evaluation benchmark value is determined through the following steps:

[0213] Determine the adjustment ratio of any network quality indicator i

[0214] Determine the evaluation benchmark value es of any network quality indicator i i =min{bs i ×(1+ch i ),i max}.

[0215] The device provided in this embodiment performs a comprehensive availability evaluation on the quality of 5G industrial applications based on the network quality value, service quality value and the index values ​​of each network wireless index in the current period, thereby combining the 5G industrial application quality evaluation with industrial applications and effectively evaluating the quality of 5G industrial applications.

[0216] Based on the same inventive concept of the 5G industrial application quality comprehensive availability evaluation method, this embodiment provides an electronic device, such as Figure 4 As shown, it includes: a memory 401, a processor 402, and a computer program.

[0217] Among them, the computer program is stored in the memory 401 and is configured to be executed by the processor 402 to implement the above-mentioned 5G industrial application quality comprehensive availability evaluation method.

[0218] Specifically,

[0219] Obtain the comprehensive evaluation indicators of 5G industrial application quality in the current cycle and historical cycle. Among them, the comprehensive evaluation indicators of 5G industrial application quality include network wireless indicators, network quality indicators and service quality indicators. Network quality indicators include: delay reliability, jitter, packet loss rate, service rate, synchronization accuracy, and service quality indicators include: availability and stability.

[0220] According to the maximum and minimum business standard values ​​of each network quality indicator, the influence of each network quality indicator in each period is determined, and the weight and evaluation benchmark value of each network quality indicator are determined according to the influence.

[0221] The network quality value of 5G industrial applications is evaluated based on the indicator value, weight and evaluation benchmark value of each network quality indicator in the current period.

[0222] Evaluate the service quality value of 5G industrial applications based on the indicator values ​​of each service quality indicator in the current cycle.

[0223] Based on the network quality value, service quality value and the index values ​​of each network wireless indicator in the current period, a comprehensive availability evaluation of 5G industrial application quality is conducted.

[0224] Optionally, the network quality value of the 5G industrial application is evaluated according to the indicator value, weight and evaluation reference value of each network quality indicator in the current cycle, including:

[0225] The network quality value of 5G industrial applications is evaluated through the network quality evaluation formula.

[0226] The network quality evaluation formula is:

[0227] Q n is the network quality value of 5G industrial applications, z is the current cycle identifier, is the index value of delay reliability in the current cycle, es D is the evaluation benchmark value of delay reliability, w Dis the weight of delay reliability, is the index value of jitter in the current cycle, es J is the evaluation reference value of jitter, w J is the jitter weight, is the indicator value of packet loss rate in the current period, es R is the evaluation benchmark value of packet loss rate, w R is the weight of packet loss rate, is the indicator value of the service rate in the current period, es S is the evaluation benchmark value of the service rate, w S is the weight of the service rate, is the index value of synchronization accuracy in the current cycle, es T is the evaluation benchmark value of synchronization accuracy, w T is the weight of synchronization accuracy.

[0228] According to the indicator values ​​of each service quality indicator in the current cycle, the service quality value of 5G industrial applications is evaluated, including:

[0229] The service quality value of 5G industrial applications is evaluated through the service quality evaluation formula.

[0230] The service quality evaluation formula is:

[0231] Q b is the service quality value of 5G industrial applications, is the indicator value of availability in the current cycle, es A is the evaluation benchmark value of availability, w A is the weight of availability, is the indicator value of stability in the current period, es C is the evaluation benchmark value of stability, w C is the weight of stability.

[0232] Optionally, the indicator value of the network quality indicator in the current period of each 5G industrial application quality comprehensive evaluation indicator is obtained, including:

[0233] Obtain the latency and bandwidth of industrial applications at each monitoring time in the current cycle, and determine the latency reliability index value in the current cycle based on the latency and bandwidth Among them, z is the current cycle identifier.

[0234] Obtain the degree of change in the delay of the industrial application at each monitoring moment in the current cycle, and determine the degree of change as the index value of the jitter in the current cycle

[0235] Obtain the number of lost data packets and the total number of transmitted data packets of the industrial application in the current cycle, and determine the quotient of the number of lost data packets and the total number of transmitted data packets as the indicator value of the packet loss rate in the current cycle

[0236] Obtain the average data size and average data sending time of the industrial application in the current cycle, and determine the quotient of the average data size and the average data sending time as the indicator value of the business rate in the current cycle

[0237] Get the index value of synchronization accuracy in the current period through the synchronization signal

[0238] Obtain the indicator values ​​of the service quality indicators in the current cycle of each 5G industrial application quality comprehensive evaluation indicator, including:

[0239] Get the average time it takes for the industrial application device to resume network communication after restarting in the current cycle, and determine the average time as the indicator value of availability in the current cycle

[0240] Obtain the operating time of the equipment in the industrial application in the current cycle, and determine the operating time as the indicator value of stability in the current cycle

[0241] Optionally, the method further comprises:

[0242] Determine the nature of the 5G industrial application, where the nature is a fixed application or a non-fixed application. If the 5G industrial application is fixed in the device, the nature of the 5G industrial application is a fixed application, otherwise the nature of the 5G industrial application is a non-fixed application.

[0243] If the nature of the 5G industrial application is a fixed application, the network delay of the equipment where the 5G industrial application is located is tested at each monitoring moment, and the network delay is determined as the delay at that monitoring moment.

[0244] If the nature of the 5G industrial application is a non-fixed application, the network delay of the equipment where the 5G industrial application is located is tested at each monitoring moment, and the difference between the network delay and the preset application delay compensation value is determined as the delay at the monitoring moment.

[0245] Optionally, the network latency of the equipment where the 5G industrial application is located is tested at each monitoring time, including:

[0246] At any monitoring moment, test the network latency of the device where the 5G industrial application is located by following the steps below:

[0247] Perform clock synchronization.

[0248] A test case is sent to the device where the 5G industrial application is located through a test device, wherein the test case includes business data, and a timestamp is written in the business communication protocol data field or the reserved field of the message header of the sending test case.

[0249] The difference between the service data in the test case from the test device to the device where the 5G industrial application is located is calculated based on the timestamp, and the difference is determined as the network delay of the device where the 5G industrial application is located. Alternatively, the difference between the service data in the test case from the test device to the device where the 5G industrial application is located, and the difference from the device where the 5G industrial application is located to the test device are calculated based on the timestamp, and both differences are determined as the network delay of the device where the 5G industrial application is located.

[0250] If the monitoring period corresponding to the current monitoring time is not greater than the network delay, the bandwidth is determined as the product of the amount of data sent between any monitoring time and the adjacent previous monitoring time and the sending frequency.

[0251] If the monitoring period corresponding to the current monitoring moment is greater than the network delay, any monitoring moment will no longer be used as a monitoring moment. A new monitoring period is determined based on the network delay, and a new monitoring moment is determined based on the adjacent previous monitoring moment and the new monitoring period.

[0252] Optionally, determining the influence of each network quality indicator in each period includes:

[0253] like The influence of any network quality index i in any historical period x is in, is the index value of any network quality index i in any historical period x, i min is the minimum value of the service standard for any network quality indicator i, i max The maximum value of the service standard for any network quality indicator i.

[0254] like but

[0255] like but x is the historical period identifier.

[0256] The weight and evaluation benchmark value of each network quality indicator are determined according to the impact, including:

[0257] According to the preset network quality indicator priority, the weight of each network quality indicator is determined in turn according to the impact. Among them, the weight of any network quality indicator i is the weight adjustment value of any network quality index i, when the average value of the influence of any network quality index i in each period is μi =0 o'clock otherwise σ i is the standard deviation of the influence of any network quality indicator i in each cycle, y is the network quality indicator with a determined weight, n i 5 - the number of network quality indicators for which weights have been determined.

[0258] Determine the standard value of each network quality indicator. Among them, the standard value of any network quality indicator i

[0259] The evaluation benchmark value of each network quality indicator is determined according to the standard value of each network quality indicator.

[0260] Optionally, determining an evaluation reference value of each network quality indicator according to a standard value of each network quality indicator includes:

[0261] For any network quality indicator i, its evaluation benchmark value is determined through the following steps:

[0262] Determine the adjustment ratio of any network quality indicator i

[0263] Determine the evaluation benchmark value es of any network quality indicator i i =min{bs i ×(1+ch i ),i max}.

[0264] The electronic device provided in this embodiment has a computer program executed by a processor to perform a comprehensive availability evaluation on the quality of 5G industrial applications based on the network quality value, service quality value and the index values ​​of each network wireless index in the current cycle, thereby realizing the combination of 5G industrial application quality evaluation and industrial application, and effectively evaluating the quality of 5G industrial applications.

[0265] Based on the same inventive concept of the 5G industrial application quality comprehensive availability evaluation method, this embodiment provides a computer-readable storage medium, and a computer program is stored thereon. The computer program is executed by a processor to implement the above-mentioned 5G industrial application quality comprehensive availability evaluation method.

[0266] Specifically,

[0267] Obtain the comprehensive evaluation indicators of 5G industrial application quality in the current cycle and historical cycle. Among them, the comprehensive evaluation indicators of 5G industrial application quality include network wireless indicators, network quality indicators and service quality indicators. Network quality indicators include: delay reliability, jitter, packet loss rate, service rate, synchronization accuracy, and service quality indicators include: availability and stability.

[0268] According to the maximum and minimum business standard values ​​of each network quality indicator, the influence of each network quality indicator in each period is determined, and the weight and evaluation benchmark value of each network quality indicator are determined according to the influence.

[0269] The network quality value of 5G industrial applications is evaluated based on the indicator value, weight and evaluation benchmark value of each network quality indicator in the current period.

[0270] Evaluate the service quality value of 5G industrial applications based on the indicator values ​​of each service quality indicator in the current cycle.

[0271] Based on the network quality value, service quality value and the index values ​​of each network wireless indicator in the current period, a comprehensive availability evaluation of 5G industrial application quality is conducted.

[0272] Optionally, the network quality value of the 5G industrial application is evaluated according to the indicator value, weight and evaluation reference value of each network quality indicator in the current cycle, including:

[0273] The network quality value of 5G industrial applications is evaluated through the network quality evaluation formula.

[0274] The network quality evaluation formula is:

[0275] Q n is the network quality value of 5G industrial applications, z is the current cycle identifier, is the index value of delay reliability in the current cycle, es D is the evaluation benchmark value of delay reliability, w D is the weight of delay reliability, is the index value of jitter in the current cycle, es J is the evaluation reference value of jitter, w J is the jitter weight, is the indicator value of packet loss rate in the current period, es R is the evaluation benchmark value of packet loss rate, w R is the weight of packet loss rate, is the indicator value of the service rate in the current period, es S is the evaluation benchmark value of the service rate, w S is the weight of the service rate, is the index value of synchronization accuracy in the current cycle, es T is the evaluation benchmark value of synchronization accuracy, w T is the weight of synchronization accuracy.

[0276] According to the indicator values ​​of each service quality indicator in the current cycle, the service quality value of 5G industrial applications is evaluated, including:

[0277] The service quality value of 5G industrial applications is evaluated through the service quality evaluation formula.

[0278] The service quality evaluation formula is:

[0279] Q b is the service quality value of 5G industrial applications, is the indicator value of availability in the current cycle, es A is the evaluation benchmark value of availability, w A is the weight of availability, is the indicator value of stability in the current period, es C is the evaluation benchmark value of stability, w C is the weight of stability.

[0280] Optionally, the indicator value of the network quality indicator in the current period of each 5G industrial application quality comprehensive evaluation indicator is obtained, including:

[0281] Obtain the latency and bandwidth of industrial applications at each monitoring moment in the current cycle, and determine the latency reliability index value in the current cycle based on the latency and bandwidth Among them, z is the current cycle identifier.

[0282] Obtain the degree of change in the delay of the industrial application at each monitoring moment in the current cycle, and determine the degree of change as the index value of the jitter in the current cycle

[0283] Obtain the number of lost data packets and the total number of transmitted data packets of the industrial application in the current cycle, and determine the quotient of the number of lost data packets and the total number of transmitted data packets as the indicator value of the packet loss rate in the current cycle

[0284] Obtain the average data size and average data sending time of the industrial application in the current cycle, and determine the quotient of the average data size and the average data sending time as the indicator value of the business rate in the current cycle

[0285] Get the index value of synchronization accuracy in the current period through the synchronization signal

[0286] Obtain the indicator values ​​of the service quality indicators in the current cycle of each 5G industrial application quality comprehensive evaluation indicator, including:

[0287] Get the average time it takes for the industrial application device to resume network communication after restarting in the current cycle, and determine the average time as the indicator value of availability in the current cycle

[0288] Obtain the operating time of the equipment in the industrial application in the current cycle, and determine the operating time as the indicator value of stability in the current cycle

[0289] Optionally, the method further comprises:

[0290] Determine the nature of the 5G industrial application, where the nature is a fixed application or a non-fixed application. If the 5G industrial application is fixed in the device, the nature of the 5G industrial application is a fixed application, otherwise the nature of the 5G industrial application is a non-fixed application.

[0291] If the nature of the 5G industrial application is a fixed application, the network delay of the equipment where the 5G industrial application is located is tested at each monitoring moment, and the network delay is determined as the delay at that monitoring moment.

[0292] If the nature of the 5G industrial application is a non-fixed application, the network delay of the equipment where the 5G industrial application is located is tested at each monitoring moment, and the difference between the network delay and the preset application delay compensation value is determined as the delay at the monitoring moment.

[0293] Optionally, the network latency of the equipment where the 5G industrial application is located is tested at each monitoring time, including:

[0294] At any monitoring moment, test the network latency of the device where the 5G industrial application is located by following the steps below:

[0295] Perform clock synchronization.

[0296] A test case is sent to the device where the 5G industrial application is located through a test device, wherein the test case includes business data, and a timestamp is written in the business communication protocol data field or the reserved field of the message header of the sending test case.

[0297] The difference between the service data in the test case from the test device to the device where the 5G industrial application is located is calculated based on the timestamp, and the difference is determined as the network delay of the device where the 5G industrial application is located. Alternatively, the difference between the service data in the test case from the test device to the device where the 5G industrial application is located, and the difference from the device where the 5G industrial application is located to the test device are calculated based on the timestamp, and both differences are determined as the network delay of the device where the 5G industrial application is located.

[0298] If the monitoring period corresponding to the current monitoring time is not greater than the network delay, the bandwidth is determined as the product of the amount of data sent between any monitoring time and the adjacent previous monitoring time and the sending frequency.

[0299] If the monitoring period corresponding to the current monitoring moment is greater than the network delay, any monitoring moment will no longer be used as a monitoring moment. A new monitoring period is determined based on the network delay, and a new monitoring moment is determined based on the adjacent previous monitoring moment and the new monitoring period.

[0300] Optionally, determining the influence of each network quality indicator in each period includes:

[0301] like The influence of any network quality index i in any historical period x is in, is the index value of any network quality index i in any historical period x, i min is the minimum value of the service standard for any network quality indicator i, i max The maximum value of the service standard for any network quality indicator i.

[0302] like but

[0303] like but x is the historical period identifier.

[0304] The weight and evaluation benchmark value of each network quality indicator are determined according to the impact, including:

[0305] According to the preset network quality indicator priority, the weight of each network quality indicator is determined in turn according to the impact. Among them, the weight of any network quality indicator i is the weight adjustment value of any network quality index i, when the average value of the influence of any network quality index i in each period is μ i =0 o'clock otherwise σ i is the standard deviation of the influence of any network quality indicator i in each cycle, y is the network quality indicator with a determined weight, n i 5 - the number of network quality indicators for which weights have been determined.

[0306] Determine the standard value of each network quality indicator. Among them, the standard value of any network quality indicator i

[0307] The evaluation benchmark value of each network quality indicator is determined according to the standard value of each network quality indicator.

[0308] Optionally, determining an evaluation reference value of each network quality indicator according to a standard value of each network quality indicator includes:

[0309] For any network quality indicator i, its evaluation benchmark value is determined through the following steps:

[0310] Determine the adjustment ratio of any network quality indicator i

[0311] Determine the evaluation benchmark value es of any network quality indicator i i =min{bs i ×(1+ch i ),i max}.

[0312] The computer-readable storage medium provided in this embodiment has a computer program executed by a processor to perform a comprehensive availability evaluation on the quality of 5G industrial applications based on the network quality value, service quality value, and the index values ​​of each network wireless index in the current cycle, thereby combining the 5G industrial application quality evaluation with industrial applications and effectively evaluating the quality of 5G industrial applications.

[0313] Since the system / device described in the above embodiments of the present invention is a system / device used to implement the method of the above embodiments of the present invention, a person skilled in the art can understand the specific structure and deformation of the system / device based on the method described in the above embodiments of the present invention, and thus will not be described in detail here. All systems / devices used in the method of the above embodiments of the present invention belong to the scope of protection of the present invention.

[0314] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0315] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions.

[0316] It should be noted that in the claims, any reference numerals placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In the claims enumerating several means, several of these means may be embodied by the same hardware. The use of the words first, second, third, etc., is for convenience of expression only and does not indicate any order. These words may be understood as part of the component name.

[0317] In addition, it should be noted that, in the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0318] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments after knowing the basic creative concept. Therefore, the claims should be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0319] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention should also include these modifications and variations.

Claims

1. A 5G industrial application quality comprehensive availability evaluation method, characterized in that: include: Obtain comprehensive evaluation indicators of the quality of 5G industrial applications, and the indicator values ​​in the current cycle and the historical cycle; wherein the comprehensive evaluation indicators of the quality of 5G industrial applications include network wireless indicators, network quality indicators and service quality indicators, the network quality indicators include: delay reliability, jitter, packet loss rate, service rate, synchronization accuracy, and the service quality indicators include: availability and stability; Determine the influence of each network quality indicator in each period according to the maximum value and minimum value of the service standard of each network quality indicator, and determine the weight and evaluation benchmark value of each network quality indicator according to the influence; Evaluate the network quality value of 5G industrial applications based on the indicator value, weight and evaluation benchmark value of each network quality indicator in the current cycle; Evaluate the service quality value of 5G industrial applications based on the indicator values ​​of each service quality indicator in the current cycle; Based on the network quality value, service quality value and the index values ​​of each network wireless index in the current period, a comprehensive availability evaluation is performed on the quality of 5G industrial applications.

2. The method according to claim 1, characterized in that The evaluating the network quality value of 5G industrial applications according to the indicator value, weight and evaluation reference value of each network quality indicator in the current cycle includes: Evaluate the network quality value of 5G industrial applications through the network quality evaluation formula; The network quality evaluation formula is: Q n is the network quality value of 5G industrial applications, z is the current cycle identifier, is the index value of delay reliability in the current cycle, es D is the evaluation benchmark value of delay reliability, w D is the weight of delay reliability, is the index value of jitter in the current cycle, es J is the evaluation reference value of jitter, w J is the jitter weight, is the indicator value of packet loss rate in the current period, es R is the evaluation benchmark value of packet loss rate, w R is the weight of packet loss rate, is the indicator value of the service rate in the current period, es S is the evaluation benchmark value of the service rate, w S is the weight of the service rate, is the index value of synchronization accuracy in the current cycle, es T is the evaluation benchmark value of synchronization accuracy, w T is the weight of synchronization accuracy; The evaluating the service quality value of 5G industrial applications according to the indicator value of each service quality indicator in the current cycle includes: Evaluate the service quality value of 5G industrial applications through the service quality evaluation formula; The service quality evaluation formula is: Q b is the service quality value of 5G industrial applications, is the indicator value of availability in the current cycle, es A is the evaluation benchmark value of availability, w A is the weight of availability, is the indicator value of stability in the current period, es C is the evaluation benchmark value of stability, w C is the weight of stability.

3. The method according to claim 1, characterized in that Obtain the index values ​​of network quality indicators in the current cycle of each 5G industrial application quality comprehensive evaluation index, including: Obtain the delay and bandwidth of the industrial application at each monitoring time in the current cycle, and determine the indicator value of the delay reliability in the current cycle based on the delay and bandwidth Among them, z is the current cycle identifier; Obtain the degree of change in the delay of industrial applications at each monitoring moment in the current cycle, and determine the degree of change as the index value of jitter in the current cycle Obtain the number of lost data packets and the total number of transmitted data packets of the industrial application in the current cycle, and determine the quotient of the number of lost data packets and the total number of transmitted data packets as the indicator value of the packet loss rate in the current cycle Obtain the average data size and average data sending time of the industrial application in the current cycle, and determine the quotient of the average data size and the average data sending time as the indicator value of the business rate in the current cycle Get the index value of synchronization accuracy in the current period through the synchronization signal Obtain the indicator values ​​of the service quality indicators in the current cycle of each 5G industrial application quality comprehensive evaluation indicator, including: Get the average time it takes for the industrial application device to resume network communication after restarting in the current cycle, and determine the average time as the indicator value of availability in the current cycle Obtain the operating time of the equipment in the industrial application in the current cycle, and determine the operating time as the indicator value of stability in the current cycle 4. The method according to claim 3, characterized in that The method further comprises: Determine the nature of the 5G industrial application, wherein the nature is a fixed application or a non-fixed application; if the 5G industrial application is fixed in the device, the nature of the 5G industrial application is a fixed application, otherwise the nature of the 5G industrial application is a non-fixed application; If the nature of the 5G industrial application is a fixed application, the network delay of the device where the 5G industrial application is located is tested at each monitoring time, and the network delay is determined as the delay at the monitoring time; If the nature of the 5G industrial application is a non-fixed application, the network delay of the equipment where the 5G industrial application is located is tested at each monitoring moment, and the difference between the network delay and the preset application delay compensation value is determined as the delay at the monitoring moment.

5. The method according to claim 4, characterized in that Testing the network latency of the equipment where the 5G industrial application is located at each monitoring moment includes: At any monitoring moment, test the network latency of the device where the 5G industrial application is located by following the steps below: Perform clock synchronization; Sending a test case to the device where the 5G industrial application is located through the test device, wherein the test case includes business data, and a timestamp is written in the business communication protocol data field or the reserved field of the message header of the sending test case; Calculate the difference of the service data in the test case from the test device to the device where the 5G industrial application is located according to the timestamp, and determine the difference as the network delay of the device where the 5G industrial application is located; or calculate the difference of the service data in the test case from the test device to the device where the 5G industrial application is located, and the difference from the device where the 5G industrial application is located to the test device according to the timestamp, and determine both differences as the network delay of the device where the 5G industrial application is located; If the monitoring period corresponding to the current monitoring moment is not greater than the network delay, the bandwidth is determined as the product of the amount of data sent between any monitoring moment and the adjacent previous monitoring moment and the sending frequency; If the monitoring period corresponding to the current monitoring moment is greater than the network delay, any monitoring moment will no longer be used as a monitoring moment, and a new monitoring period will be determined based on the network delay, and the new monitoring moment will be determined based on the adjacent previous monitoring moment and the new monitoring period.

6. The method according to claim 1, characterized in that Determining the influence of each network quality indicator in each period includes: like The influence of any network quality index i in any historical period x is in, is the index value of any network quality index i in any historical period x, i min is the minimum value of the service standard for any network quality indicator i, i max The maximum value of the service standard for any network quality indicator i; like but like but x is the historical cycle identifier; Determining the weight and evaluation reference value of each network quality indicator according to the impact includes: According to the preset network quality indicator priority order, the weight of each network quality indicator is determined in turn according to the impact degree; wherein the weight of any network quality indicator i is the weight adjustment value of any network quality indicator i, when the average value μ of the influence degree of any network quality indicator i in each period is i =0 o'clock otherwise σ i is the standard deviation of the influence of any network quality indicator i in each cycle, y is the network quality indicator with a determined weight, n i =5-the number of network quality indicators with determined weights; Determine the standard value of each network quality indicator; among them, the standard value of any network quality indicator i The evaluation benchmark value of each network quality indicator is determined according to the standard value of each network quality indicator.

7. The method according to claim 6, characterized in that Determining the evaluation reference value of each network quality indicator according to the standard value of each network quality indicator includes: For any network quality indicator i, its evaluation benchmark value is determined through the following steps: Determine the adjustment ratio of any network quality indicator i Determine the evaluation benchmark value es of any network quality indicator i i =min{bs i ×(1+ch i ),i max }.

8. A 5G industrial application quality comprehensive availability evaluation device, characterized in that: include: An acquisition module is used to obtain the comprehensive evaluation indicators of the quality of 5G industrial applications, and the indicator values ​​in the current cycle and the historical cycle; wherein the comprehensive evaluation indicators of the quality of 5G industrial applications include network wireless indicators, network quality indicators and service quality indicators, the network quality indicators include: delay reliability, jitter, packet loss rate, service rate, synchronization accuracy, and the service quality indicators include: availability and stability; A determination module, used to determine the influence of each network quality indicator in each period according to the maximum value and minimum value of the business standard of each network quality indicator, and determine the weight and evaluation reference value of each network quality indicator according to the influence; The first evaluation module is used to evaluate the network quality value of 5G industrial applications according to the indicator value, weight and evaluation benchmark value of each network quality indicator in the current cycle; The second evaluation module is used to evaluate the service quality value of 5G industrial applications according to the indicator value of each service quality indicator in the current cycle; The third evaluation module is used to perform a comprehensive availability evaluation on the quality of 5G industrial applications based on the network quality value, service quality value and the index values ​​of each network wireless index in the current period.

9. An electronic device, characterized in that: include: Memory; Multi-core processors; as well as Computer programs; Wherein, the multi-core processor includes a plurality of idle processing cores; The computer program is stored in the memory and is configured to execute a 5G industrial application quality comprehensive availability evaluation method as described in any one of claims 1-7.

10. A computer-readable medium having computer-executable instructions stored thereon, characterized in that: When the executable instructions are executed by the processor, a 5G industrial application quality comprehensive availability evaluation method as described in any one of claims 1-7 is implemented.

Images