Multi-system multi-type tradeoff evaluation method and system based on scores

Abstract

The invention discloses a score-based multi-system multi-type tradeoff evaluation method and system, and belongs to the technical field of radio and television system evaluation. Comprising the steps that S1, a to-be-tested list of a to-be-tested system is acquired, and the to-be-tested list comprises a plurality of to-be-tested devices; s2, monitoring the to-be-tested equipment through a monitoring assembly, and evaluating the to-be-tested equipment according to a monitoring result to obtain evaluation values of a plurality of evaluation dimensions; and S3, calculating a total evaluation value of the to-be-tested system according to the evaluation value. The technical scheme has the beneficial effects that multiple evaluation dimensions are covered, multiple types of equipment can be evaluated, the flexibility is relatively high, and the evaluation accuracy is high.

Description

Technical Field

[0001] The present invention relates to the technical field of radio and television system evaluation, and in particular to a trade-off evaluation method and system. Background Art

[0002] The concept of intelligent operation and maintenance is becoming increasingly popular in system management and maintenance. In the radio and television industry, the complexity and diversity of equipment are particularly prominent. Various systems are nested with each other, and they face the challenges of a large number of equipment, a wide range of types, a wide range of physical distribution, and a large span of new and old equipment purchases. Taking traditional provincial TV stations as an example, the broadcast control system is usually located inside the TV station. Due to historical reasons, most TV stations are located in the city center. Units with satellite services often choose to be located in suburbs far away from the city due to electromagnetic interference problems. The transmitting and receiving towers of the radio station are also built on the edge of the city or in scenic spots to avoid interference. Therefore, an ordinary system may contain hundreds of devices, which are scattered in a city, even dozens of kilometers apart. Some of these devices are newly added, and some have been in operation for several years.

[0003] Although these devices have the same brand, their software and hardware models, access interfaces and protocols often have significant differences. For such a complex system, there are many ways to evaluate its comprehensive performance. The most common approach is to count the total number of alarms over a period of time and regard the system with a small number of alarms as stable. However, in a system with a large number of devices, even in the same environment, the probability of alarms is the same, and the total number of alarms is bound to be higher than that of a small system with fewer devices. In addition, some units may make judgments based on age. Although this method is scientific in the same type of system, it still has many limitations between different systems. Summary of the invention

[0004] The purpose of the present invention is to provide a score-based multi-system multi-type trade-off evaluation method to solve the above technical problems;

[0005] The present invention also aims to provide a score-based multi-system multi-type trade-off evaluation system to solve the above technical problems;

[0006] A score-based multi-system multi-type trade-off evaluation method, including:

[0007] Step S1, obtaining a list of devices to be tested of the system to be tested, wherein the list includes multiple devices to be tested;

[0008] Step S2, monitoring the device under test through a monitoring component, and evaluating the device under test according to the monitoring result to obtain evaluation values ​​of multiple evaluation dimensions;

[0009] Step S3, calculating the total evaluation value of the system to be tested according to the evaluation value.

[0010] Preferably, the evaluation values ​​in step S2 include a stability evaluation value, an availability evaluation value, a response speed evaluation value, an impact evaluation value, a durability evaluation value and an operation and maintenance quality evaluation value.

[0011] Preferably, the stability evaluation value is obtained by using the following formula:

[0012] S=(a×0.7+b×0.3)×0.2+c×0.8;

[0013] Wherein, S represents the stability evaluation value, S∈[0,5], a represents the number of alarms monitored by the monitoring component, b represents the number of alarm prompts monitored by the monitoring component, and c represents the number of key alarms monitored by the monitoring component.

[0014] Preferably, the usability evaluation value is obtained by using the following formula:

[0015] U = (t1-t0) × 5 / H;

[0016] Wherein, U represents the availability evaluation value, U∈[0,5], t1 represents the current time of the device under test, t0 represents the last failure time or startup time of the device under test, and H represents the number of hours of the operation and maintenance cycle.

[0017] Preferably, the response speed evaluation value is obtained by using the following formula:

[0018]

[0019] Wherein, R represents the response speed evaluation value, T represents the detection time of the device under test, N represents the number of detections, and T avg represents the average value of the detection time.

[0020] Preferably, the durability evaluation value is obtained by using the following formula:

[0021] D=(t1-T MFG ) / Y;

[0022] Where D represents the durability evaluation value, D∈[0,5], t1 represents the current time of the device under test, T MFG represents the production date of the device under test, and Y represents the total number of days the device under test is scrapped.

[0023] Preferably, the operation and maintenance quality evaluation value is obtained by using the following formula:

[0024]

[0025] Where Q represents the operation and maintenance quality evaluation value, Q∈[0,5], I represents the number of inspections, Itotal represents the total amount to be inspected, E represents the number of emergency drills, and E total represents the total amount of emergency drills, P represents the number of orders to be closed, and P total Indicates the total number of events, T E The average time to close an event ticket, T single Indicates the time required to complete a shutdown, A total Indicates the total number of problem tickets, A avg Indicates the average time it takes for a ticket to be closed.

[0026] Preferably, the system operation and maintenance quality of the system to be tested is obtained by using the following formula:

[0027]

[0028] Among them, Q S Indicates the system operation and maintenance quality, Q S ∈[0,5], Q1 represents the operation and maintenance quality evaluation value of the device under test with alarm records, M1 represents the total number of the devices under test with alarm records, Q2 represents the operation and maintenance quality evaluation value of the device under test without alarm records, and M2 represents the total number of the devices under test without alarm records;

[0029] The total evaluation value is obtained by using the following formula:

[0030] V=(S×0.3+U×0.1+R×0.2+F×0.1+D×0.1+Q S ×0.2)×5;

[0031] Wherein, V represents the total evaluation value, V∈[0,5], S represents the stability evaluation value, U represents the availability evaluation value, R represents the response speed evaluation value, F represents the impact evaluation value, D represents the durability evaluation value, Q S Indicates the system operation and maintenance quality.

[0032] Preferably, the method further comprises step S4 of performing integer discretization processing on the total evaluation value to obtain a rounded evaluation value.

[0033] A score-based multi-system multi-type trade-off evaluation system, including:

[0034] A data collection module, used to collect a list of systems to be tested, wherein the list includes multiple devices to be tested;

[0035] A monitoring module, connected to the data collection module, for monitoring the device under test;

[0036] A multi-dimensional evaluation module, connected to the monitoring module, for evaluating the device under test according to the monitoring result of the monitoring module to obtain evaluation values ​​of multiple evaluation dimensions;

[0037] The total evaluation calculation module is connected to the multi-dimensional evaluation module and calculates the total evaluation value of the system to be tested based on the evaluation value.

[0038] The beneficial effects of the present invention are: covering multiple evaluation dimensions, being able to evaluate various types of equipment, having strong flexibility, and high evaluation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a step diagram of the multi-system multi-type trade-off evaluation method of the present invention;

[0040] Figure 2 It is a schematic diagram of the multi-system multi-type trade-off evaluation system of the present invention.

[0041] In the attached figure: 1. Data collection module; 2. Monitoring module; 3. Multi-dimensional evaluation module; 4. Overall evaluation calculation module. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0044] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0045] A score-based multi-system multi-type trade-off evaluation method, such as Figure 1 As shown, including,

[0046] Step S1, obtaining a list of the system to be tested, the list including multiple devices to be tested;

[0047] Step S2, monitoring the device under test through the monitoring component, and evaluating the device under test according to the monitoring result to obtain evaluation values ​​of multiple evaluation dimensions;

[0048] Step S3, calculating the total evaluation value of the system to be tested according to the evaluation value.

[0049] Specifically, the present invention provides a multi-system multi-type trade-off evaluation method based on scores. First, the system is initialized and a list of all devices in the current system is read. The initial value of each dimension is cleared to 0. Then, the monitoring state is entered to receive parameter values, summary reports, etc. sent by each actual monitoring system at any time. If a parameter value of a certain dimension cannot be received for 10 consecutive seconds, an alarm will be sounded to remind the operator of this system to perform manual verification. Through the evaluation values ​​of multiple evaluation dimensions, the system can comprehensively analyze the performance of the device to be tested, with strong flexibility and high evaluation accuracy.

[0050] In a preferred embodiment, the evaluation values ​​in step S2 include a stability evaluation value, an availability evaluation value, a response speed evaluation value, an impact evaluation value, a durability evaluation value, and an operation and maintenance quality evaluation value.

[0051] Specifically, taking the device to be tested as the basic unit, first compile and organize a list of all devices in the system to be tested that need to participate in the evaluation. For each device, evaluate it from six dimensions: stability, availability, response speed, impact, durability, and operation and maintenance quality.

[0052] In a preferred embodiment, the stability evaluation value is obtained by using the following formula:

[0053] S=(a×0.7+b×0.3)×0.2+c×0.8;

[0054] Where S represents the stability evaluation value, S∈[0,5], a represents the number of alarms monitored by the monitoring component, b represents the number of alarm prompts monitored by the monitoring component, and c represents the number of key alarms monitored by the monitoring component.

[0055] Specifically, stability refers to the "number of alarms" that occur during the operation of a device under test, especially the "number of key alarms".

[0056] If S>5, it will be forced to S=5. The purpose of this is to force the value range to [0,5] to prevent a system from failing in other dimensions due to too many alarms. For example, some radio and television systems will perform various test maintenances at irregular intervals to verify system stability. These maintenances will generate a large number of alarms in a short period of time. These alarms cannot be cleared because they are used for troubleshooting, which will affect the fairness of this evaluation system.

[0057] In a preferred embodiment, the usability evaluation value is obtained by using the following formula:

[0058] U = (t1-t0) × 5 / H;

[0059] Where U represents the availability evaluation value, U∈[0,5], t1 represents the current time of the device under test, t0 represents the last failure time or startup time of the device under test, and H represents the number of hours of the operation and maintenance cycle.

[0060] Specifically, the availability of a device is the normal operation time of the device. Simply put, it is the total time since the last failure of the device. If the device has not experienced any abnormality, it is defined as the total time since the system was turned on, in hours.

[0061] The operation and maintenance cycle hours is the total system operation and maintenance cycle time in hours. For example, some systems are operated and maintained once a week, so the value is 7×24 (hours).

[0062] In a preferred embodiment, the response speed evaluation value is obtained by using the following formula:

[0063]

[0064] Among them, R represents the response speed evaluation value, T represents the detection time of the device under test, n represents the number of detections, and T avg Indicates the average value of the detection time.

[0065] Specifically, the detection time refers to the time required for the device to complete a detection task, reflecting the device's response speed. The average detection time is the average value after multiple detections. And if the value is less than 0.5 or greater than 1.5, it is forced to be R=5, otherwise, the device R=0. Its meaning is: if the time of this round of detection differs from the average detection time by more than 50%, the device's average response speed is forced to be 5, otherwise the device's average response speed is 0. Therefore, the value is either 5 or 0.

[0066] In a preferred embodiment, the durability evaluation value is obtained by using the following formula:

[0067] D=(t1-T MFG ) / Y;

[0068] Where D represents the durability evaluation value, D∈[0,5], t1 represents the current time of the device under test, T MFG represents the production date of the device under test, and Y represents the total number of days the device under test is scrapped.

[0069] Specifically, durability is the current available residual value calculated based on the service life, historical service life and scrapping situation. The time is calculated with the production date as the starting time. The unit is also days.

[0070] The "total number of days for equipment to be scrapped" in the formula is the total time it takes for equipment to be scrapped in days. Generally, the scrapping time for equipment is 5 years, so its "total number of days for equipment to be scrapped" is 5×365 (ignoring leap years).

[0071] In a preferred embodiment, the following formula is used to obtain the operation and maintenance quality evaluation value:

[0072]

[0073] Among them, Q represents the operation and maintenance quality evaluation value, Q∈[0,5], I represents the number of inspections, I total represents the total amount to be inspected, E represents the number of emergency drills, and E total represents the total amount of emergency drills, P represents the number of orders to be closed, and P total Indicates the total number of events, T E The average time to close an event ticket, T single Indicates the time required to complete a shutdown, A total Indicates the total number of problem tickets, A avg Indicates the average time it takes for a ticket to be closed.

[0074] Specifically, a comprehensive judgment is made based on the daily operation and maintenance activities of the front-line operation and maintenance personnel. The content is further divided into 6 sub-items. They are the inspection completion rate of the operation and maintenance personnel (inspection completion status), emergency drill completion rate (emergency drill completion status), incident ticket closure rate (incident ticket closure ratio), incident ticket closure time (the time taken from the opening of the incident ticket to the closing of the incident ticket), problem ticket closure rate (problem ticket closure ratio), and problem ticket closure time (the time taken from the opening of the problem ticket to the closing of the problem ticket).

[0075] The value range of each of the 6 sub-items is [0,1]. The number of inspections specifically refers to the number of equipment inspected by the operation and maintenance personnel, the total number of inspections specifically refers to the total number of equipment that the operation and maintenance personnel should inspect, the number of emergency drills specifically refers to the number of emergency drills completed by the operation and maintenance personnel, and the total number of emergency drills specifically refers to the number of emergency drills that should be conducted within the plan.

[0076] In a preferred embodiment, the system operation and maintenance quality of the system to be tested is obtained by using the following formula:

[0077]

[0078] Among them, Q S Indicates the system operation and maintenance quality, Q S ∈[0,5], Q1 represents the operation and maintenance quality evaluation value of the equipment under test with alarm records, M1 represents the total number of equipment under test with alarm records, Q2 represents the operation and maintenance quality evaluation value of the equipment under test without alarm records, and M2 represents the total number of equipment under test without alarm records.

[0079] Specifically, with the operation and maintenance quality evaluation value of the equipment, the operation and maintenance quality of the system can be calculated. When there is no alarm state, it is only necessary to average the operation and maintenance quality of the above-mentioned equipment. When there is an alarm, the alarm state is very important for the overall evaluation of a system. However, if the averaging algorithm is still used, the alarm will be weakened due to the average. When the total number of devices in the system is large, no matter how many alarms there are, they may be "drowned" under the "false appearance" of normal operation of most devices. Therefore, when an alarm is generated, it is necessary to average the "alarm" and "no alarm" devices separately.

[0080] The total evaluation value is obtained by using the following formula:

[0081]

[0082] Among them, V represents the total evaluation value, V∈[0,5], S represents the stability evaluation value, U represents the availability evaluation value, R represents the response speed evaluation value, F represents the impact evaluation value, D represents the durability evaluation value, Q S Indicates the system operation and maintenance quality.

[0083] Specifically, since the values ​​of the six dimensions are all between 0 and 5, the value range of the total evaluation value is [0,5].

[0084] The impact degree refers to the number of other systems affected by the system. The reason for setting this item is that some equipment belongs to multiple systems (such as GPS timing equipment), and some systems have upstream and downstream relationships. The failure of a certain upstream equipment may affect some downstream equipment or even the entire system. The evaluation standard is to determine whether the business alarm will affect the normal operation of other businesses.

[0085] The impact evaluation value is that if the number of affected systems is less than 5, the impact evaluation value = the number of affected systems. If the number of affected systems is greater than or equal to 5, the impact evaluation value is equal to 5.

[0086] The performance integration value of the system equipment is measured according to the above comprehensive trade-off formula. The final score of the entire system is calculated according to the number of devices. The final score is set between 0 and 5 points to facilitate the decision-making level.

[0087] In a preferred embodiment, the method further includes step S4, performing integer discretization processing on the total evaluation value to obtain a rounded evaluation value.

[0088] Specifically, the following formula is used to obtain the rounded evaluation value:

[0089] C = (V + 0.5) / 5;

[0090] Among them, C represents the rounded evaluation value, and V represents the total evaluation value.

[0091] At this time, the range of the total evaluation value changes from 0 to 5 to 6 discrete values: 0, 1, 2, 3, 4, and 5. When C is 0 or 1, the system under test is judged to be in a safe state; when C is 2 or 3, the system under test is judged to be in a warning state; when C is 4 or 5, the system under test is judged to be in a dangerous state and needs to be rectified.

[0092] The evaluation values ​​and weights of the above evaluation dimensions are shown in the following table. Considering that different devices in the same system and the same device in different systems have different response times for the same type of transactions, the evaluation items for response speed are formulated. The main focus is on the "alarm closure rate" (i.e. the proportion of single alarm closures under each business) and the "alarm closure time" (i.e. the time from the start to the end of the alarm).

[0093]

[0094]

[0095] A score-based multi-system multi-type trade-off evaluation system, referring to Figure 2 ,include,

[0096] Data collection module 1, used to collect a list of systems to be tested, the list including multiple devices to be tested;

[0097] A monitoring module 2, connected to the data collection module 1, is used to monitor the device under test;

[0098] The multi-dimensional evaluation module 3 is connected to the monitoring module 2 and is used to evaluate the device to be tested according to the monitoring result of the monitoring module 2 to obtain the evaluation values ​​of multiple evaluation dimensions;

[0099] The total evaluation calculation module 4 is connected to the multi-dimensional evaluation module 3 and calculates the total evaluation value of the system to be tested based on the evaluation value.

[0100] Specifically, the multi-dimensional evaluation module 3 formulates a comprehensive evaluation table based on six dimensions, namely stability, availability, response speed, impact, durability, and operation and maintenance quality, and each dimension has a specific algorithm. The overall evaluation calculation module 4 separates the devices with alarms and the devices without alarms, and finally calculates the comprehensive evaluation value of the entire system according to the formula.

[0101] Considering the possibility of comprehensive display in real life, a new formula for integer discretization of comprehensive evaluation values ​​has been added. Through the above formula, the trade-off evaluation score of any system can be easily and accurately calculated. With the help of this score, decision makers can be guided to understand the overall stability and security of the system at a macro level, providing a reference for subsequent normal operation, upgrade, repair, and scrapping.

[0102] The above description is only a preferred embodiment of the present invention, and does not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A score-based multi-system multi-type trade-off evaluation method, characterized in that: include, Step S1, obtaining a list of devices to be tested of the system to be tested, wherein the list includes multiple devices to be tested; Step S2, monitoring the device under test through a monitoring component, and evaluating the device under test according to the monitoring result to obtain evaluation values ​​of multiple evaluation dimensions; Step S3, calculating the total evaluation value of the system to be tested according to the evaluation value.

2. The score-based multi-system multi-type trade-off evaluation method according to claim 1, characterized in that: The evaluation values ​​in step S2 include a stability evaluation value, an availability evaluation value, a response speed evaluation value, an impact evaluation value, a durability evaluation value, and an operation and maintenance quality evaluation value.

3. The score-based multi-system multi-type trade-off evaluation method according to claim 2, characterized in that: The stability evaluation value is obtained by using the following formula: S=(a×0.7+b×0.3)×0.2+c×0.8; Wherein, S represents the stability evaluation value, S∈[0,5], a represents the number of alarms monitored by the monitoring component, b represents the number of alarm prompts monitored by the monitoring component, and c represents the number of key alarms monitored by the monitoring component.

4. The score-based multi-system multi-type trade-off evaluation method according to claim 2, characterized in that: The usability evaluation value is obtained by using the following formula: U = (t1-t0) × 5 / H; Wherein, U represents the availability evaluation value, U∈[0,5], t1 represents the current time of the device under test, t0 represents the last failure time or startup time of the device under test, and H represents the number of hours of the operation and maintenance cycle.

5. The score-based multi-system multi-type trade-off evaluation method according to claim 2, characterized in that: The response speed evaluation value is obtained by using the following formula: Wherein, R represents the response speed evaluation value, T represents the detection time of the device under test, N represents the number of detections, and T avg represents the average value of the detection time.

6. The score-based multi-system multi-type trade-off evaluation method according to claim 2, characterized in that: The durability evaluation value is obtained by using the following formula: D=(t1-T MFG ) / Y; Where D represents the durability evaluation value, D∈[0,5], t1 represents the current time of the device under test, T MFG represents the production date of the device under test, and Y represents the total number of days the device under test is scrapped.

7. The score-based multi-system multi-type trade-off evaluation method according to claim 2, characterized in that: The operation and maintenance quality evaluation value is obtained by using the following formula: Where Q represents the operation and maintenance quality evaluation value, Q∈[0,5], I represents the number of inspections, I total represents the total amount to be inspected, E represents the number of emergency drills, and E total represents the total amount of emergency drills, P represents the number of orders to be closed, and P total Represents the total number of events, T E The average time to close an event ticket, T single Indicates the time required to complete a shutdown, A total Indicates the total number of problem tickets, A avg Indicates the average time it takes to close a ticket.

8. The score-based multi-system multi-type trade-off evaluation method according to claim 2, characterized in that: The system operation and maintenance quality of the system under test is obtained by using the following formula: Among them, Q S Indicates the system operation and maintenance quality, Q S ∈[0,5], Q1 represents the operation and maintenance quality evaluation value of the device under test with alarm records, M1 represents the total number of the devices under test with alarm records, Q2 represents the operation and maintenance quality evaluation value of the device under test without alarm records, and M2 represents the total number of the devices under test without alarm records; The total evaluation value is obtained by using the following formula: V=(S×0.3+U×0.1+R×0.2+F×0.1+D×0.1+Q S ×0.2)×5; Wherein, V represents the total evaluation value, V∈[0,5], S represents the stability evaluation value, U represents the availability evaluation value, R represents the response speed evaluation value, F represents the impact evaluation value, D represents the durability evaluation value, Q S Indicates the system operation and maintenance quality.

9. The score-based multi-system multi-type trade-off evaluation method according to claim 1, characterized in that: The method further includes step S4, performing integer discretization processing on the total evaluation value to obtain a rounded evaluation value.

10. A score-based multi-system multi-type trade-off evaluation system, characterized in that: include, A data collection module, used to collect a list of systems to be tested, wherein the list includes multiple devices to be tested; A monitoring module, connected to the data collection module, for monitoring the device under test; A multi-dimensional evaluation module, connected to the monitoring module, for evaluating the device under test according to the monitoring result of the monitoring module to obtain evaluation values ​​of multiple evaluation dimensions; The total evaluation calculation module is connected to the multi-dimensional evaluation module and calculates the total evaluation value of the system to be tested based on the evaluation value.

Images