Device evaluation method, device, device evaluation apparatus, readable storage medium, and program product

Abstract

The application relates to a device evaluation method and device, an apparatus, a readable storage medium and a program product. The method comprises the following steps: obtaining running state data corresponding to each of a plurality of configuration states of a target device; wherein the running state data comprises at least a compatibility data set and a security data set; for each configuration state, performing median aggregation and multi-dimensional weighting based on the compatibility data set in the running state data corresponding to the configuration state to obtain a compatibility score corresponding to the configuration state, and performing maximum aggregation and security failure evaluation based on the security data set in the running state data corresponding to the configuration state to obtain a security score corresponding to the configuration state; and weighting the compatibility scores and the security scores corresponding to the plurality of configuration states to obtain a device evaluation result. The method can improve the comprehensiveness of device evaluation.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a device evaluation method, apparatus, device, readable storage medium, and program product. Background Technology

[0002] With the rapid development of technologies such as the Internet of Things, artificial intelligence, and power automation, intelligent measurement equipment is widely used in key fields such as power, energy, transportation, and industrial control. These devices undertake multiple core tasks, including data acquisition, status monitoring, edge computing, and communication interaction. The complexity, real-time performance, and stability requirements of their software systems are also increasing, posing complex compatibility and security challenges to the software systems of intelligent measurement equipment.

[0003] In current technologies, single tests are usually conducted for the actual application scenarios of intelligent measurement equipment. However, when faced with complex application scenarios, there is a problem of low detection coverage. Therefore, there is an urgent need for a device evaluation method to improve the comprehensiveness of device evaluation. Summary of the Invention

[0004] Therefore, it is necessary to provide a device evaluation method, apparatus, equipment, readable storage medium, and program product that can improve the comprehensiveness of device evaluation in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a device evaluation method, including:

[0006] Acquire the operating status data of the target device under various configuration states; wherein, the operating status data includes at least a compatibility data set and a security data set;

[0007] For each configuration state, based on the compatibility data set under the corresponding running status data of the configuration state, median aggregation and multi-dimensional weighting are performed to obtain the compatibility score corresponding to the configuration state;

[0008] Based on the security data set under the running status data corresponding to the configuration state, maximum value aggregation and security failure assessment are performed to obtain the security score corresponding to the configuration state.

[0009] The device evaluation result is obtained by weighting the compatibility score and security score corresponding to each of the various configuration states.

[0010] In one embodiment, there are multiple sets of running status data corresponding to each configuration state; the compatibility data set contains compatibility data in multiple dimensions;

[0011] The compatibility data set based on the running status data corresponding to the configuration state is aggregated by median and weighted by multiple dimensions to obtain the compatibility score corresponding to the configuration state, including:

[0012] In the compatibility data set under multiple sets of the aforementioned operating status data, the compatibility data under the same dimension are aggregated by median to obtain the median aggregated data corresponding to the configuration status; based on the weight, normal data range, tolerance band width, and deviation sensitivity coefficient of each dimension in the compatibility data set, the data under multiple dimensions included in the median aggregated data are weighted and summed to obtain the compatibility score corresponding to the configuration status.

[0013] In one embodiment, the compatibility score corresponding to the configuration state is obtained by weighted summation of the data under each dimension included in the median aggregated data based on the weights of each dimension in the compatibility data set, the normal data range, the tolerance band width, and the deviation sensitivity coefficient, including:

[0014] Based on the normal data range and tolerance band width of each dimension in the compatibility dataset, the relative tolerance offset distance of each dimension included in the median aggregated data is determined; based on the deviation sensitivity coefficient of each dimension in the compatibility dataset, the relative tolerance offset distance of the same dimension in the median aggregated data is non-linearly processed to determine the compatibility score of each dimension in the median aggregated data; based on the weight of each dimension in the compatibility dataset, the compatibility scores of each dimension in the median aggregated data are weighted and summed to obtain the compatibility score corresponding to the configuration state.

[0015] In one embodiment, there are multiple sets of runtime status data corresponding to each configuration state; the security data set contains security data in multiple dimensions;

[0016] The security data set based on the running status data corresponding to the configuration state is used to perform maximum value aggregation and security failure assessment to obtain the security score corresponding to the configuration state, including:

[0017] In the security data set under multiple sets of the aforementioned operating status data, the security data under the same dimension are aggregated by maximum value to obtain the maximum aggregated data corresponding to the configuration status; based on the risk threshold range and weight of each dimension in the security data set, the maximum aggregated data is weighted to obtain the security score corresponding to the configuration status.

[0018] In one embodiment, the step of weighting the aggregated maximum value data based on the risk threshold ranges and weights of each dimension in the security dataset to obtain the security score corresponding to the configuration state includes:

[0019] Based on the risk threshold range of each dimension in the security dataset, the severity of each dimension in the maximum aggregate data is determined; the severity of each dimension in the maximum aggregate data is weighted based on the weight of each dimension in the security dataset to determine the weighted severity of each dimension in the maximum aggregate data; the maximum value among the weighted severity values ​​of each dimension in the maximum aggregate data is used to determine the risk suppression factor corresponding to the maximum aggregate data; the remaining security margin of the maximum aggregate data is determined based on the weighted severity of each dimension in the maximum aggregate data; and the security score corresponding to the configuration state is determined based on the risk suppression factor and the remaining security margin.

[0020] In one embodiment, the weighted average of compatibility scores and security scores corresponding to each of the various configuration states to obtain the device evaluation result includes:

[0021] A global compatibility score is obtained by weighting the compatibility scores corresponding to each of the multiple configuration states; a global security score is obtained by weighting the security scores corresponding to each of the multiple configuration states; a global comprehensive score is obtained by weighting the global compatibility score and the global security score; a comprehensive device score for each configuration state is obtained by weighting the compatibility score and the security score corresponding to each configuration state; and a device evaluation result is obtained based on the global compatibility score, the global security score, the global comprehensive score, and the comprehensive device score corresponding to each of the multiple configuration states.

[0022] Secondly, this application also provides a device evaluation apparatus, including: an acquisition module, used to acquire the operating status data of the target device under various configuration states; wherein the operating status data includes at least a compatibility data set and a security data set;

[0023] The compatibility data processing module is used to perform median aggregation and multi-dimensional weighting on the compatibility data set under the running status data corresponding to each configuration state to obtain the compatibility score corresponding to the configuration state.

[0024] The security data processing module is used to perform maximum value aggregation and security failure assessment based on the security data set under the running status data corresponding to the configuration status, and obtain the security score corresponding to the configuration status.

[0025] The output module is used to weight the compatibility score and security score corresponding to each of the various configuration states to obtain the device evaluation result.

[0026] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0027] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0028] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0029] The aforementioned equipment evaluation methods, devices, equipment, readable storage media, and program products cover various practical application configuration scenarios of the equipment through multiple configuration states, avoiding the limitations of single configuration evaluation. Simultaneously, for each configuration state, operational status data including compatibility and security datasets is acquired, covering multiple data dimensions. Subsequently, differentiated analysis methods are employed based on the characteristics of the compatibility and security data. Median aggregation avoids interference from extreme values ​​in compatibility data, multi-dimensional weighting balances the importance of different compatibility indicators, and maximum value aggregation accurately captures weak indicators in security data. Security failure assessment quantifies the impact of security risks, ensuring the accuracy and objectivity of both types of indicator evaluations. Finally, through comprehensive weighted integration of multiple configurations and indicators, a full-chain comprehensive coverage is achieved from data collection and indicator analysis to result output, effectively improving the comprehensiveness of equipment testing and ensuring that the evaluation results objectively and comprehensively reflect the overall performance of the equipment in various scenarios. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a diagram illustrating the application environment of a device evaluation method in one embodiment;

[0032] Figure 2 This is a flowchart illustrating a device evaluation method in one embodiment;

[0033] Figure 3This is a schematic diagram illustrating the calculation process of the compatibility score in one embodiment;

[0034] Figure 4 This is a structural block diagram of the device evaluation apparatus in one embodiment;

[0035] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0037] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0038] The device evaluation method provided in this application embodiment can be applied to, for example, Figure 1 The application environment shown is illustrated. Terminal 101 communicates with server 102 via a network. A data storage system can store the data that server 102 needs to process. The data storage system can be integrated onto server 102, or it can be located in the cloud or on another network server.

[0039] Users can initiate a device evaluation task for a target device through terminal 101. After receiving the device evaluation task, server 102 can execute test cases to test the target device, thereby obtaining the operating status data of the target device under various configuration states. The operating status data includes at least a compatibility data set and a security data set. For each configuration state, based on the compatibility data set under the corresponding operating status data, median aggregation and multi-dimensional weighting are performed to obtain the compatibility score for the configuration state. Based on the security data set under the corresponding operating status data, maximum value aggregation and security failure assessment are performed to obtain the security score for the configuration state. The device evaluation result is obtained by weighting the compatibility score and security score corresponding to each of the various configuration states. Finally, server 102 can send the obtained device evaluation result to terminal 101, and terminal 101 can generate an evaluation report based on the device evaluation result for display.

[0040] Terminal 101 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, and projection equipment. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0041] In one exemplary embodiment, such as Figure 2 As shown, a device evaluation method is provided, which is applied to... Figure 1 Taking the server in the example, the explanation includes the following steps 201 to 204. Wherein:

[0042] Step 201: Obtain the operating status data of the target device under various configuration states; wherein, the operating status data includes at least a compatibility data set and a security data set.

[0043] In some embodiments, the target device may be a smart measuring device, such as an industrial IoT smart sensor, smart electricity meter, smart water meter, smart gas meter, smart heat meter, etc., or other devices with data acquisition, processing, communication transmission and status monitoring functions.

[0044] In some embodiments, configuration state refers to the environment state configured for the deployment environment of the target device; different configuration states represent different deployment environment states; for example, taking the target device as an intelligent measurement device, the configuration state of the intelligent measurement device may include different operating system versions, hardware platforms, communication protocols, network topologies, etc., encountered in actual deployment.

[0045] The running status data of the target device in the configuration state refers to the data generated by the target device in the configuration state. In some embodiments, various test cases can be executed to enable the target device to perform various test operations in the configuration state, thereby enriching the collected running status data. For example, by executing functional test cases, the target device can be made to perform corresponding functions in the configuration state; by executing load test cases, the target device can be made to drive load in the configuration state, etc.

[0046] In some embodiments, during the operation of the target device, operational status data can be collected in real time through monitoring agents, log collection tools, embedded performance probes, and dedicated security scanning tools.

[0047] The compatibility data set includes at least one type of compatibility data. Compatibility data refers to data that reflects the compatibility stability and performance of the target device and the configuration environment defined by the configuration state. Taking the intelligent measurement device as an example, compatibility data may include resource usage index data (such as memory usage rate), performance index data (such as response latency, data sampling frequency, and interface call success rate), and abnormal event record data (such as number of crashes and number of timeouts).

[0048] Similarly, security data refers to data that reflects security protection capabilities and potential risk exposure; taking intelligent measurement devices as an example again, security data can include vulnerability scan results, digital signature verification status, privilege escalation event counts, number of unauthorized access attempts, and encryption module operating status, etc.

[0049] In some embodiments, the compatibility field corresponding to the compatibility data and the security field corresponding to the security data can be preset. In this way, after the running status data is collected, the compatibility data can be extracted from the running status data by field matching to obtain a compatibility data set, and the security data can be extracted from the running status data to obtain a security data set.

[0050] In other embodiments, the collected operational status data can be input into a pre-trained classification model, which then performs data classification prediction to output a compatibility dataset and a security dataset.

[0051] Step 202: For each configuration state, based on the compatibility data set under the running status data corresponding to the configuration state, perform median aggregation and multi-dimensional weighting to obtain the compatibility score corresponding to the configuration state.

[0052] In some embodiments, the compatibility dataset may include compatibility data across multiple dimensions; based on this, median aggregation may be performed first to reduce the amount of data in each dimension, and then multiple data dimensions may be merged through multi-dimensional weighting.

[0053] Median aggregation refers to the process where, for data within the same dimension, after sorting according to preset rules, if the sequence length is odd, the value at the middle position is directly taken; if the sequence length is even, the average of the two middle values ​​is taken as the representative value for the current dimension.

[0054] For example, for the first configuration state, its compatibility data set includes multiple dimensions, such as the first dimension a, the second dimension b, and the third dimension c. At the same time, each dimension can also contain multiple sets of compatibility data. For example, the first dimension can include compatibility data a1, a2, and a3. In this way, the median aggregation can be performed on the compatibility data under the first dimension to obtain the uniquely determined compatibility data under the first dimension.

[0055] Multi-dimensional weighting refers to weighting the compatibility data across multiple dimensions after median aggregation, based on the respective weights of each dimension in the compatibility dataset, such as through weighted summation.

[0056] For compatibility data, since compatibility data has the characteristics of random load and sudden fluctuation, the median can effectively filter out outliers caused by occasional extreme high loads or instantaneous fluctuations, thereby highlighting the typical stable operating level of the target device under this configuration state and avoiding the problem of distortion caused by outliers in the mean method.

[0057] Step 203: Based on the security data set under the running status data corresponding to the configuration status, perform maximum value aggregation and security failure assessment to obtain the security score corresponding to the configuration status.

[0058] In some embodiments, the security dataset may include security data across multiple dimensions. Based on this, the maximum value aggregation may be performed on the security dataset first to reduce the amount of data in each dimension, and then a security failure assessment may be performed to integrate the data from multiple dimensions.

[0059] Among them, maximum value aggregation refers to selecting the maximum value among multiple data points under the same dimension as the representative value of the current dimension.

[0060] For example, for the first configuration state, its security data set includes multiple dimensions, such as the first dimension a, the second dimension b, and the third dimension c. At the same time, each dimension can also contain multiple sets of compatibility data. For example, the first dimension can include compatibility data a1, a2, and a3. In this way, the maximum value can be aggregated for the compatibility data under the first dimension, that is, the maximum value among a1, a2, and a3 can be selected as the representative value of the first dimension.

[0061] In some embodiments, security failure assessment can employ multiple assessment methods. For example, security failure assessment can be a failure assessment mode dominated by the weakest link, which determines whether there is a security failure risk based on the data under the target dimension (i.e., the dimension corresponding to the weakest link) in the security dataset. If the data under the target dimension meets the set conditions, it is considered that there is a security failure risk. Alternatively, it can be a failure assessment mode based on the accumulation of multiple risks, which overlays the data under multiple dimensions in the security dataset and determines whether there is a security failure risk based on the overlay result. Of course, it can also be a comprehensive assessment combining two failure assessment modes, which comprehensively considers the data under the target dimension and the overlay result of the data under all dimensions in the security dataset to conduct a security failure assessment.

[0062] For security data, since security assessments must follow a conservative principle, even if the vast majority of test cases perform normally, as long as there is a single serious risk exposure, such as a privilege violation or a vulnerability trigger, it should be considered a potential threat. Therefore, by directly selecting the largest data in each dimension as the representative value of the current dimension through maximum value aggregation, it can be ensured that even if only a single serious privilege violation or vulnerability trigger event occurs, the risk can be included in the security assessment system. This avoids masking key risk points by using aggregation methods such as mean or median, thus conforming to the conservative principle of security assessment.

[0063] Step 204: Weight the compatibility score and security score corresponding to each of the various configuration states to obtain the device evaluation result.

[0064] In some embodiments, the compatibility score and security score corresponding to each configuration state can be weighted to obtain a comprehensive score for that configuration state; then, the comprehensive scores of various configuration states can be combined to obtain the device evaluation result.

[0065] In other embodiments, the compatibility scores corresponding to each of the multiple configuration states can be weighted to obtain a comprehensive compatibility score; at the same time, the security scores corresponding to each of the multiple configuration states can be weighted to obtain a comprehensive security score. Finally, the comprehensive compatibility score and the comprehensive security score are combined to obtain the device evaluation result.

[0066] The results of equipment evaluation can comprehensively reflect the security and compatibility of target equipment under different configuration states. Based on the evaluation results, the adaptability, risk resistance level and stable operation potential of target equipment in complex application scenarios can be judged. This provides a scientific basis for equipment selection, configuration optimization and scenario-based deployment, ensuring that the equipment can meet the requirements of security and compatibility in actual applications.

[0067] The aforementioned equipment evaluation method covers various practical application configuration scenarios of the equipment through multiple configuration states, avoiding the limitations of single configuration evaluation. Simultaneously, for each configuration state, it acquires operational status data including compatibility and security datasets, covering multiple data dimensions. Then, it employs differentiated analysis methods based on the characteristics of the compatibility and security data. Median clustering avoids interference from extreme values ​​in compatibility data, multi-dimensional weighting balances the importance of different compatibility indicators, and maximum value aggregation accurately captures weak indicators in security data. Security failure assessment quantifies the impact of security risks, ensuring the accuracy and objectivity of both types of indicator evaluations. Finally, through comprehensive weighted integration of multiple configurations and indicators, it achieves full-chain comprehensive coverage from data collection and indicator analysis to result output, effectively improving the comprehensiveness of equipment testing and ensuring that the evaluation results objectively and comprehensively reflect the overall performance of the equipment in various scenarios.

[0068] In one exemplary embodiment, there are multiple sets of runtime status data corresponding to each configuration state; the compatibility data set contains compatibility data across multiple dimensions; such as... Figure 3 As shown, Figure 3 The calculation process for the compatibility score is shown, with step 202 including steps 301 to 302. Wherein:

[0069] Step 301: In the compatibility data set under multiple sets of running status data, perform median aggregation on the compatibility data under the same dimension to obtain the median aggregated data corresponding to the configuration status.

[0070] In some embodiments, multiple sets of runtime status data can be obtained through multiple test runs; it is understood that each set of runtime status data will contain a compatibility data set, therefore, multiple sets of runtime status data correspond to multiple sets of compatibility data sets; at the same time, each set of compatibility data sets includes compatibility data in multiple dimensions.

[0071] When performing median aggregation, the median is selected from multiple sets of compatible data corresponding to the same dimension to obtain the median aggregation result under that dimension. The median aggregation results of multiple dimensions are combined to obtain the median aggregated data.

[0072] Step 302: Based on the weights of each dimension, the normal data range, the tolerance band width, and the deviation sensitivity coefficient in the compatibility dataset, the data under multiple dimensions included in the median aggregated data are weighted and summed to obtain the compatibility score corresponding to the configuration status.

[0073] The normal data range of a dimension refers to the normal data range of the data under that dimension; for example, the normal data under the first dimension should be between 1 and 10, then the normal data range of the first dimension is 1 to 10.

[0074] The tolerance band width of a dimension refers to the maximum fluctuation distance between the actual data and the ideal data in that dimension. For example, if the ideal data in the first dimension is 5 and the tolerance band width is 2, then if the actual data is between 3 and 7, the actual data is considered to be within the tolerance band width range.

[0075] The deviation sensitivity coefficient of a dimension refers to the sensitivity of the data in that dimension to data deviation. It is used to measure the correlation between the data skewness of that dimension and the final compatibility score.

[0076] In some embodiments, the weights, normal data ranges, tolerance band widths, and deviation sensitivity coefficients of each dimension in the compatibility dataset are preset; for example, they can be determined through multiple iterations using the Delphi method (an expert group decision-making and prediction method based on multiple rounds of anonymous consultation) based on expert experience.

[0077] In the above embodiments, by first aggregating the median and then weighting it across multiple dimensions, on the one hand, the interference of individual abnormal evaluation data on the scoring results can be effectively avoided through median aggregation, ensuring the objectivity and stability of the initial scores of each dimension and avoiding scoring deviations caused by extreme values; on the other hand, by assigning differentiated weights based on the differences in the degree of influence of different dimensions on compatibility and performing weighted calculations, the core role of key dimensions in compatibility can be accurately highlighted, making the final compatibility score more in line with the adaptation needs of actual application scenarios, and greatly improving the accuracy and credibility of the scoring results.

[0078] In an exemplary embodiment, based on the weights of each dimension in the compatibility dataset, the normal data range, the tolerance band width, and the deviation sensitivity coefficient, the data under each dimension included in the median aggregated data are weighted and summed to obtain the compatibility score corresponding to the configuration state, which may include:

[0079] Based on the normal data range and tolerance band width of each dimension in the compatibility dataset, the relative offset distance of the tolerance for each dimension included in the median aggregated data is determined.

[0080] Among them, the tolerance relative offset distance describes the degree of deviation of the data in a dimension from the ideal data in that dimension.

[0081] Specifically, the first difference can be obtained by subtracting the actual data of each dimension in the median aggregate data from the ideal target data of each dimension; then the absolute value of the first difference can be subtracted from the tolerance band width to obtain the second difference; and the ratio of the second difference to the data width of the normal data range can be used as the relative offset distance of the tolerance.

[0082] For example, the tolerance relative offset distance can be calculated using the following formula:

[0083] .

[0084] Where k represents the kth configuration state. Represents the j-th dimension, The compatibility dataset is in the [number]th [year]. The ideal target value for a dimension is usually taken as the midpoint between the upper and lower limits of the normal data range; This indicates the median aggregate data under the k-th configuration state at the th... Actual data for the dimensions; and These represent the compatibility datasets at the [number]th [year]. The upper and lower limits of the normal data range for a dimension; The compatibility dataset is in the [number]th [year]. The tolerance band width of the dimension.

[0085] Based on the deviation sensitivity coefficients of each dimension in the compatibility dataset, the relative offset distance of tolerance under the same dimension in the median aggregated data is non-linearly processed to determine the compatibility score of each dimension in the median aggregated data.

[0086] Nonlinear processing can include operations such as exponential decay, logarithmic compression, and power transformation.

[0087] In some embodiments, the relative offset distance of the tolerance in the same dimension of the median aggregated data may be weighted first according to the deviation sensitivity coefficient of each dimension, and then the weighted result may be subjected to nonlinear processing.

[0088] For example, the compatibility score for the j-th dimension can be calculated using the following formula:

[0089] .

[0090] in, Represents the first in the compatibility dataset The deviation sensitivity coefficient corresponding to the dimension. This represents the maximum value function, taking the larger of 0 and N to implement a tolerance band mechanism. When the value inside the parentheses is negative, i.e., not exceeding the tolerance band, it is taken as 0, thus making the index input 0 and keeping the score at full value. This represents an exponentially decaying function.

[0091] Based on the weights of each dimension in the compatibility dataset, the compatibility scores of each dimension in the median aggregate data are weighted and summed to obtain the compatibility score corresponding to the configuration status.

[0092] For example, the compatibility score corresponding to the kth configuration state can be calculated using the following formula:

[0093] .

[0094] in, Indicates the first Compatibility score under various configuration states; This represents the summation symbol, which accumulates all dimensions of compatibility-related operational metrics; Indicates the first in the compatibility data set The importance weight of the dimension, with a value range of . And the sum of the weights of each dimension in the compatibility dataset satisfies .

[0095] In the above embodiments, an exponential decay method is used to apply non-linear progressive penalties to the degree of deviation. At the same time, a tolerance band is combined to achieve the practical engineering characteristics of "no penalty for slight fluctuations and rapid decay for severe deviations". The independent sensitivity coefficients of each dimension are used to flexibly adapt to the differences in engineering sensitivity of different indicators, avoiding the evaluation abrupt changes caused by traditional hard thresholds. This achieves differentiated processing of different types of compatibility problems and significantly improves the sensitivity and discrimination of the evaluation.

[0096] In some embodiments, there are multiple sets of runtime status data corresponding to each configuration state; the security data set contains security data across multiple dimensions; based on the security data set under the runtime status data corresponding to the configuration state, maximum value aggregation and security failure assessment are performed to obtain the security score corresponding to the configuration state, including:

[0097] In a security data set with multiple sets of operational status data, the maximum value of security data under the same dimension is aggregated to obtain the maximum value aggregated data corresponding to the configuration status.

[0098] In some embodiments, multiple sets of runtime status data can be obtained through multiple test runs; it is understood that each set of runtime status data will contain a security data set, therefore, multiple sets of runtime status data correspond to multiple sets of security data sets; at the same time, each set of security data sets includes security data in multiple dimensions.

[0099] When performing maximum value aggregation, the maximum value is determined from multiple sets of security data corresponding to the same dimension. The maximum value is used as the maximum value aggregation result for that dimension. The maximum value aggregation results of multiple dimensions are combined to obtain the maximum value aggregation data corresponding to the configuration state.

[0100] Based on the risk threshold ranges and weights of each dimension in the security dataset, the maximum aggregated data is weighted to obtain the security score corresponding to the configuration state.

[0101] The risk threshold range for each dimension is used to describe the normal data range of the data under that dimension. In some embodiments, the weights and risk threshold ranges corresponding to different dimensions of security data can be determined through multiple iterations using the Delphi method based on expert experience.

[0102] In some embodiments, the risk severity of each dimension in the maximum aggregated data may be assessed first based on the risk threshold range of each dimension in the security dataset, and then the severity assessment results of multiple dimensions may be weighted to obtain the security score corresponding to the configuration state.

[0103] In other embodiments, the maximum value data corresponding to the configuration state can be aggregated, and the aggregated maximum value results of each dimension can be mapped to the severity function value. Then, the values ​​can be weighted by various different failure assessment modes to obtain the security score corresponding to the configuration state.

[0104] In the above embodiments, by aggregating the maximum value, the extreme value information that best reflects the configuration risk in each dimension of data can be accurately captured, avoiding the masking of key failure risks by the mean or other aggregation methods, and ensuring the sensitivity and accuracy of identification of potential risks in the configuration status; by weighting, the application scenarios, technical focuses and historical failure data characteristics of different failure assessment modes can be combined to assign each mode an adaptability weight coefficient, so as to achieve differentiated and refined assessment of configuration security.

[0105] In some embodiments, a security score corresponding to a configuration state is obtained by weighting the aggregated data of the maximum value based on the risk threshold range and weight of each dimension in the security dataset. This may include:

[0106] Based on the risk threshold range of each dimension in the security dataset, the severity of each dimension in the maximum aggregated data is determined.

[0107] In some embodiments, the severity of each dimension in the maximum aggregated data can be determined using a piecewise linear approach.

[0108] For example, severity can be calculated using the following formula:

[0109] .

[0110] in, , The first in the security dataset The lower and upper limits of the risk threshold range corresponding to the dimension; The maximum value in the aggregated data representing the k-th configuration state. The severity of the disease; The maximum value in the aggregated data representing the k-th configuration state. Dimensional data.

[0111] The severity of each dimension in the maximum aggregate data is weighted based on the weights of each dimension in the security dataset to determine the weighted severity of each dimension in the maximum aggregate data.

[0112] For example, the maximum value aggregated data corresponding to the k-th configuration state is the first... The weighted severity of a dimension can be expressed as ;in, The first in the security dataset The importance weights corresponding to the dimensions.

[0113] Based on the maximum value among the weighted severity values ​​of each dimension in the maximum aggregated data, the risk suppression factor corresponding to the maximum aggregated data is determined.

[0114] Among them, the maximum value of the weighted severity of each dimension in the aggregated data corresponding to the configuration status is the most critical single risk dimension in the current configuration status.

[0115] In some embodiments, the risk suppression factor corresponding to the maximum aggregated data can be calculated using the following formula:

[0116] .

[0117] in, That is, the maximum value aggregated data corresponding to the kth configuration state. The weighted severity of the dimension, This represents the maximum value function, taking the larger of 0 and N; The first in the security dataset The importance weight of the dimension, with a value range of . Furthermore, the sum of the weights of each dimension in the security dataset is 1.

[0118] It can be seen that, in the presence of a single fatal risk, The larger the risk factor, the smaller the risk suppression factor; in the absence of a single fatal risk. The smaller the value, the larger the risk suppression factor becomes.

[0119] The remaining safety margin of the maximum aggregate data is determined based on the weighted severity of each dimension in the maximum aggregate data.

[0120] Among them, the product of the weighted severity of each dimension in the aggregated data corresponding to the configuration state represents the cumulative threat of various risks in that configuration state.

[0121] In some embodiments, the remaining safety margin can be calculated using the following formula:

[0122] .

[0123] in, The security dataset representing the k-th configuration state contains m dimensions. Representing the One dimension, Indicates the first Residual safety margin in each dimension This is the product symbol.

[0124] Based on the risk suppression factor and the remaining safety margin, the security score corresponding to the configuration state is determined.

[0125] In some embodiments, the security score corresponding to the configuration state can be calculated using the following formula:

[0126] .

[0127] in, Indicates the first Security score for each configuration state. The risk suppression factor calculated in the above embodiments, The remaining safety margin is calculated based on the above embodiments.

[0128] In the above embodiments, the risk suppression factor can sensitively capture the overall security collapse caused by a single fatal vulnerability, and the remaining security margin can reflect the cumulative threat of multiple minor defects. Finally, by combining the risk suppression factor and the remaining security margin, the security score corresponding to the configuration state is determined, which can effectively avoid the situation in traditional methods where a single high-risk risk is underestimated or multiple weaknesses are ignored.

[0129] In some embodiments, the device evaluation results are obtained by weighting the compatibility scores and security scores corresponding to various configuration states, including:

[0130] The compatibility scores corresponding to multiple configuration states are weighted to obtain the global compatibility score.

[0131] The compatibility weights corresponding to each configuration state can be preset; furthermore, the compatibility weights corresponding to each configuration state can be determined based on the actual deployment ratio of each configuration state; the higher the actual deployment ratio of a configuration state, the higher its weight; for example, if the market deployment ratio of the first configuration state is higher than that of the second configuration state, then the weight of the first configuration state is higher than that of the second configuration state.

[0132] The security scores corresponding to multiple configuration states are weighted to obtain the global security score.

[0133] Similarly, the security weights corresponding to each configuration state can be preset; furthermore, the security weights corresponding to each configuration state can be determined based on the actual deployment ratio of each configuration state; the higher the actual deployment ratio of a configuration state, the higher its weight; for example, if the market deployment ratio of the first configuration state is higher than that of the second configuration state, then the weight of the first configuration state is higher than that of the second configuration state.

[0134] The security weight and compatibility weight corresponding to each configuration state can be the same or different.

[0135] The global compatibility score and the global security score are weighted to obtain the global comprehensive score.

[0136] For example, the overall global score can be expressed as:

[0137] .

[0138] in, Represents the overall compatibility score. Represents the overall security score. This represents the relative importance weight, which ranges from 0 to 1. It is used to adjust the contribution ratio of compatibility and security in the overall score. When the relative importance weight is biased towards 1, the overall score reflects compatibility performance more. When the relative importance weight is biased towards 0, it reflects security performance more. When the weight is 0.5, the contributions of the two are balanced.

[0139] The compatibility score and security score corresponding to each configuration state are weighted to obtain the comprehensive device score for the configuration state.

[0140] For example, the overall device score for each configuration state can be expressed as:

[0141] .

[0142] in, Indicates the first A comprehensive device score for each configuration status; Indicates relative importance weight; Indicates the first The compatibility score for each configuration state. Indicates the first Security score for each configuration state.

[0143] The device evaluation results are obtained based on the global compatibility score, global security score, global comprehensive score, and the comprehensive device score corresponding to each of the multiple configuration states.

[0144] In some embodiments, the global compatibility score, global security score, global comprehensive score, and comprehensive device scores corresponding to multiple configuration states can be combined to obtain the device evaluation results.

[0145] In the above embodiments, since the device evaluation results include global compatibility score, global security score, global comprehensive score, and comprehensive device score corresponding to each of the multiple configuration states, the device evaluation results can take into account both the device's global core performance and the local characteristic performance under different configuration states. This avoids the one-sidedness of single-dimensional evaluation and, through the complementary verification of global performance baseline and device performance differences in multiple scenarios, comprehensively and accurately reflects the device's real operating capabilities and adaptation level, providing comprehensive and reliable data support for the device's application scenario adaptation.

[0146] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0147] Based on the same inventive concept, this application also provides a device evaluation apparatus for implementing the device evaluation method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more device evaluation apparatus embodiments provided below can be found in the limitations of the device evaluation method described above, and will not be repeated here.

[0148] In one exemplary embodiment, such as Figure 4 As shown, a schematic diagram of a device evaluation apparatus is provided. The device evaluation apparatus 400 includes:

[0149] The acquisition module 401 is used to acquire the operating status data of the target device under various configuration states; wherein, the operating status data includes at least a compatibility data set and a security data set.

[0150] The compatibility data processing module 402 is used to perform median aggregation and multi-dimensional weighting on the compatibility data set under the running status data corresponding to each configuration state to obtain the compatibility score corresponding to the configuration state.

[0151] The security data processing module 403 is used to perform maximum value aggregation and security failure assessment based on the security data set under the running status data corresponding to the configuration status, and obtain the security score corresponding to the configuration status.

[0152] Output module 404 is used to weight the compatibility score and security score corresponding to various configuration states to obtain the device evaluation result.

[0153] In one embodiment, there are multiple sets of running status data corresponding to each configuration state; the compatibility data set contains compatibility data in multiple dimensions; the compatibility data processing module 402 is specifically used to perform median aggregation on the compatibility data in the same dimension in the compatibility data set under multiple sets of running status data to obtain the median aggregated data corresponding to the configuration state; based on the weight, normal data range, tolerance band width and deviation sensitivity coefficient of each dimension in the compatibility data set, the data in the multiple dimensions included in the median aggregated data are weighted and summed to obtain the compatibility score corresponding to the configuration state.

[0154] In one embodiment, the compatibility data processing module 402 is specifically used to determine the relative offset distance of tolerance for each dimension included in the median aggregated data based on the normal data range and tolerance band width of each dimension in the compatibility data set; to perform nonlinear processing on the relative offset distance of tolerance for the same dimension in the median aggregated data based on the deviation sensitivity coefficient of each dimension in the compatibility data set, and to determine the compatibility score of each dimension in the median aggregated data; and to perform weighted summation of the compatibility scores of each dimension in the median aggregated data based on the weight of each dimension in the compatibility data set, to obtain the compatibility score corresponding to the configuration state.

[0155] In one embodiment, there are multiple sets of running status data corresponding to each configuration state; the security data set contains security data in multiple dimensions; the security data processing module 403 is specifically used to aggregate the security data in the same dimension in the security data set under multiple sets of running status data to obtain the maximum aggregated data corresponding to the configuration state; based on the risk threshold range and weight of each dimension in the security data set, the maximum aggregated data is weighted to obtain the security score corresponding to the configuration state.

[0156] In one embodiment, the security data processing module 403 is specifically used to: determine the severity of each dimension in the maximum aggregate data based on the risk threshold range of each dimension in the security data set; weight the severity of each dimension in the maximum aggregate data based on the weight of each dimension in the security data set to determine the weighted severity of each dimension in the maximum aggregate data; determine the risk suppression factor corresponding to the maximum aggregate data based on the maximum value among the weighted severity of each dimension in the maximum aggregate data; determine the remaining security margin of the maximum aggregate data based on the weighted severity of each dimension in the maximum aggregate data; and determine the security score corresponding to the configuration state based on the risk suppression factor and the remaining security margin.

[0157] In one embodiment, the output module 404 is specifically used to weight the compatibility scores corresponding to each of the multiple configuration states to obtain a global compatibility score; to weight the security scores corresponding to each of the multiple configuration states to obtain a global security score; to weight the global compatibility score and the global security score to obtain a global comprehensive score; to weight the compatibility score and the security score corresponding to each configuration state to obtain a comprehensive device score for the configuration state; and to obtain a device evaluation result based on the global compatibility score, the global security score, the global comprehensive score, and the comprehensive device scores corresponding to each of the multiple configuration states.

[0158] Each module in the aforementioned equipment evaluation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0159] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data related to device evaluation. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a device evaluation method.

[0160] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0161] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0162] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0163] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0164] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0165] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0166] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0167] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for evaluating equipment, characterized in that, The method includes: Acquire the operating status data of the target device under various configuration states; wherein, the operating status data includes at least a compatibility data set and a security data set; For each configuration state, based on the compatibility data set under the corresponding runtime data of the configuration state, median aggregation and multi-dimensional weighting are performed to obtain the compatibility score corresponding to the configuration state, and... Based on the security data set under the running status data corresponding to the configuration state, maximum value aggregation and security failure assessment are performed to obtain the security score corresponding to the configuration state. The device evaluation result is obtained by weighting the compatibility score and security score corresponding to each of the various configuration states.

2. The method according to claim 1, characterized in that, There are multiple sets of running status data corresponding to each configuration state; the compatibility data set contains compatibility data in multiple dimensions; The compatibility data set based on the running status data corresponding to the configuration state is aggregated by median and weighted by multiple dimensions to obtain the compatibility score corresponding to the configuration state, including: In the compatibility data set under multiple sets of the aforementioned operating status data, the compatibility data under the same dimension are aggregated by median to obtain the median aggregated data corresponding to the configuration status. Based on the weights of each dimension, the normal data range, the tolerance band width, and the deviation sensitivity coefficient in the compatibility data set, the data under multiple dimensions included in the median aggregate data are weighted and summed to obtain the compatibility score corresponding to the configuration state.

3. The method according to claim 2, characterized in that, The compatibility score corresponding to the configuration state is obtained by weighting and summing the data under each dimension included in the median aggregated data based on the weights of each dimension, the normal data range, the tolerance band width, and the deviation sensitivity coefficient in the compatibility data set, including: Based on the normal data range and tolerance band width of each dimension in the compatibility data set, the relative offset distance of the tolerance of each dimension included in the median aggregated data is determined. Based on the deviation sensitivity coefficients of each dimension in the compatibility dataset, the relative offset distance of tolerance under the same dimension in the median aggregated data is non-linearly processed to determine the compatibility score of each dimension in the median aggregated data. Based on the weights of each dimension in the compatibility dataset, the compatibility scores of each dimension in the median aggregated data are weighted and summed to obtain the compatibility score corresponding to the configuration state.

4. The method according to claim 1, characterized in that, There are multiple sets of runtime status data corresponding to each configuration state; the security data set contains security data in multiple dimensions; The security data set based on the running status data corresponding to the configuration state is used to perform maximum value aggregation and security failure assessment to obtain the security score corresponding to the configuration state, including: In the security data set under multiple sets of the aforementioned operating status data, the security data under the same dimension are aggregated by maximum value to obtain the maximum value aggregated data corresponding to the configuration status; Based on the risk threshold range and weight of each dimension in the security data set, the maximum value aggregated data is weighted to obtain the security score corresponding to the configuration state.

5. The method according to claim 4, characterized in that, The security score corresponding to the configuration state is obtained by weighting the aggregated data of the maximum value based on the risk threshold range and weight of each dimension in the security data set, including: Based on the risk threshold range of each dimension in the security dataset, the severity of each dimension in the maximum aggregated data is determined; The severity of each dimension in the maximum aggregate data is weighted based on the weight of each dimension in the security dataset to determine the weighted severity of each dimension in the maximum aggregate data. Based on the maximum value among the weighted severity values ​​of each dimension in the maximum aggregated data, the risk suppression factor corresponding to the maximum aggregated data is determined; Based on the weighted severity of each dimension in the maximum value aggregated data, the remaining safety margin of the maximum value aggregated data is determined; Based on the risk suppression factor and the remaining security margin, the security score corresponding to the configuration state is determined.

6. The method according to claim 1, characterized in that, The device evaluation result is obtained by weighting the compatibility score and security score corresponding to each of the various configuration states, including: The compatibility scores corresponding to each of the multiple configuration states are weighted to obtain the global compatibility score; The security scores corresponding to each of the multiple configuration states are weighted to obtain a global security score; The global compatibility score and the global security score are weighted to obtain a global comprehensive score. The compatibility score and security score corresponding to each configuration state are weighted to obtain the comprehensive device score of the configuration state; The device evaluation result is obtained based on the global compatibility score, the global security score, the global comprehensive score, and the comprehensive device score corresponding to each of the multiple configuration states.

7. A device for evaluating equipment, characterized in that, The device includes: The acquisition module is used to acquire the operating status data of the target device under various configuration states; wherein, the operating status data includes at least a compatibility data set and a security data set; The compatibility data processing module is used to perform median aggregation and multi-dimensional weighting on the compatibility data set under the running status data corresponding to each configuration state to obtain the compatibility score corresponding to the configuration state. The security data processing module is used to perform maximum value aggregation and security failure assessment based on the security data set under the running status data corresponding to the configuration status, and obtain the security score corresponding to the configuration status. The output module is used to weight the compatibility score and security score corresponding to each of the various configuration states to obtain the device evaluation result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images