A secondary equipment performance evaluation method and device, electronic equipment and storage medium

Abstract

The application discloses a kind of secondary equipment performance evaluation method, device, electronic equipment and storage medium.The method comprises: according to target debugging item, overall debugging is carried out to secondary equipment, and the signal set of feedback signal generated in the process of overall debugging is obtained to secondary equipment;The feedback signal in signal set is parsed, and the characteristic global significant aggregate representation vector of signal set is determined according to the analysis result;According to the characteristic global significant aggregate representation vector and the performance evaluation model determined in advance, the performance of secondary equipment is evaluated.The above scheme, by analyzing the signal set of feedback signal obtained in the process of overall debugging of secondary equipment, and then obtaining more representative characteristic global significant aggregate representation vector, and according to the performance of secondary equipment is evaluated according to characteristic global significant aggregate representation vector and performance evaluation model, can reduce the influence caused by human factor on performance evaluation, improve the accuracy and reliability of evaluation result.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent substations, and in particular to a secondary equipment performance evaluation method and device, electronic equipment and a storage medium. BACKGROUND

[0002] With the development of the power system, the automation level of substations is also continuously improved. The automation substation realizes real-time monitoring, control and protection of the power grid operation state by using advanced computer technology and communication means. In this process, the stability and reliability of the secondary system directly affect the safe operation of the entire power system. How to efficiently and accurately evaluate the performance of the secondary equipment in these secondary systems has become a problem to be solved. The traditional evaluation method often relies on manual operation, which requires a large number of human resources, and technical personnel need to participate in each link. This method obviously cannot adapt to the large-scale power grid construction and maintenance needs, and manual operation is easily affected by personal skills, experience and attention, especially after a long time of work, the error rate may increase. SUMMARY

[0003] The present application provides a secondary equipment performance evaluation method, device, electronic equipment and storage medium, which can reduce the influence of human factors on performance evaluation, improve the accuracy and reliability of the evaluation results.

[0004] According to one aspect of the present application, a secondary equipment performance evaluation method is provided, the method comprising:

[0005] comprehensively debugging the secondary equipment according to the target debugging item, and obtaining a signal set of feedback signals generated by the secondary equipment in the comprehensive debugging process;

[0006] analyzing the feedback signals in the signal set, and determining a characteristic global significant aggregate representation vector of the signal set according to the analysis result;

[0007] evaluating the performance of the secondary equipment according to the characteristic global significant aggregate representation vector and a pre-determined performance evaluation model.

[0008] According to another aspect of the present application, a secondary equipment performance evaluation device is provided, the device comprising:

[0009] a signal set determination module configured to comprehensively debug the secondary equipment according to the target debugging item, and obtain a signal set of feedback signals generated by the secondary equipment in the comprehensive debugging process;

[0010] a representation vector determination module configured to analyze the feedback signals in the signal set and determine a characteristic global significant aggregated representation vector of the signal set according to an analysis result;

[0011] a device performance evaluation module configured to evaluate the performance of the secondary device according to the characteristic global significant aggregated representation vector and a pre-determined performance evaluation model.

[0012] According to another aspect of the present application, there is provided an electronic device, comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein

[0015] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the secondary device performance evaluation method according to any one of the embodiments of the present application.

[0016] According to another aspect of the present application, there is provided a computer readable storage medium storing computer instructions for enabling a processor to perform the secondary device performance evaluation method according to any one of the embodiments of the present application when executed by the processor.

[0017] The technical solution of the embodiments of the present application comprehensively debugs the secondary device according to a target debug item, obtains a signal set of feedback signals generated by the secondary device during the comprehensive debugging, analyzes the feedback signals in the signal set, and determines a characteristic global significant aggregated representation vector of the signal set according to an analysis result. The performance of the secondary device is evaluated according to the characteristic global significant aggregated representation vector and a pre-determined performance evaluation model. The technical solution of the embodiments of the present application analyzes the signal set of the feedback signals obtained during the comprehensive debugging of the secondary device, and further obtains a more representative characteristic global significant aggregated representation vector. The performance of the secondary device is evaluated according to the characteristic global significant aggregated representation vector and the performance evaluation model, which can reduce the influence of human factors on performance evaluation, and improve the accuracy and reliability of the evaluation result.

[0018] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments description. Obviously, the drawings in the following description only show some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative effort based on these drawings.

[0020] Figure 1 is a flow chart of a secondary equipment performance evaluation method according to the first embodiment of the present application;

[0021] Figure 2 is a flow chart of a secondary equipment performance evaluation method according to the second embodiment of the present application;

[0022] Figure 3 is a structural schematic diagram of a secondary equipment performance evaluation device according to the third embodiment of the present application;

[0023] Figure 4 is a structural schematic diagram of an electronic device implementing the secondary equipment performance evaluation method according to the embodiments of the present application. DETAILED DESCRIPTION

[0024] In order to make the person skilled in the art better understand the present application, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should be within the scope of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0026] Embodiment one

[0027] Figure 1A flowchart of a secondary device performance evaluation method is provided for the first embodiment of the present application. The first embodiment can be applied to the case of evaluating the performance of secondary devices in a substation. The method can be executed by a secondary device performance evaluation device, which can be realized in the form of hardware and / or software, and can be configured in an electronic device. As shown in FIG. 1, the method comprises the following steps. Figure 1

[0028] S110, comprehensively debugging the secondary device according to the target debugging item, and obtaining a signal set of feedback signals generated by the secondary device in the comprehensive debugging process.

[0029] The secondary device refers to a device that performs data processing, control, and state monitoring in the telemetry, remote control, or remote signaling system of a substation, and its function and performance directly affect the effectiveness of the system. The target debugging item refers to a test item set for various functions of the secondary device. The comprehensive debugging refers to testing all testable functions of the secondary device according to the target debugging item.

[0030] The feedback signal is a form of response made by the secondary device to the input signal in the comprehensive debugging process, which can be various state parameters of the secondary device during operation, such as current, voltage, temperature, frequency, and phase.

[0031] In the embodiment of the present application, the secondary device can be comprehensively debugged according to the target debugging item, and a signal set of feedback signals generated by the secondary device in the comprehensive debugging process can be obtained. Specifically, all testable functions of the secondary device can be tested automatically according to the target debugging item through a hardware test device, and the feedback signals generated by the secondary device in the comprehensive debugging process can be automatically collected by installing corresponding sensors, such as current sensors and voltage sensors, on the secondary device. Then, the signal set of feedback signals can be determined based on the collected feedback signals. It can be understood that the collected feedback signals can intuitively reflect the actual performance of the secondary device. By comprehensively debugging the secondary device according to the target debugging item, a large amount of feedback signal data can be obtained, which provides a rich information base for subsequent performance analysis and problem diagnosis. At the same time, it can be ensured that all possible situations and boundary conditions are tested, avoiding potential failures caused by omission of certain situations.

[0032] S120, analyzing the feedback signals in the signal set, and determining a characteristic global significant aggregation representation vector of the signal set according to the analysis result.

[0033] ​The analysis result of the feedback signal contains feature information related to performance evaluation of the secondary device. The feature global significant aggregation representation vector of the signal set refers to a higher quality feature representation obtained by significantly aggregating the feature information corresponding to the feedback signals in the signal set according to the clustering feature correlation contribution. The significant aggregation of the clustering feature correlation contribution involves the evaluation of the feature contribution, the aggregation of the significant features, and the evaluation of the aggregation effect, and the effect and accuracy of the clustering can be improved by effectively aggregating the significant features.

[0034] In the embodiment of the present application, considering that each feedback signal in the signal set may be a complex waveform or contain a large number of data points, the calculation process will be very complex if the original feedback signal data is directly processed, therefore, the feedback signals in the signal set can be analyzed first to extract the feature information related to the performance evaluation of the secondary device in the feedback signals as the analysis result. At the same time, since the feature information of different feedback signals has different importance in the subsequent performance evaluation task, that is, the feature information of different feedback signals will have different contribution degrees to the judgment of whether the secondary device works as expected and the device performance evaluation task, in order to extract the most representative and influential feature information from the numerous feature information and enhance the expression of these key feature information, thereby obtaining a higher quality feature representation, in the technical solution of the present application, the analysis result needs to be further significantly aggregated according to the clustering feature correlation contribution to obtain the feature global significant aggregation representation vector of the signal set.

[0035] Optionally, the specific process of analyzing the feedback signals in the signal set is as follows: first, low-pass filtering is used to filter out the high-frequency noise of each feedback signal in the signal set to ensure the clarity of the feedback signal. Second, feedback signals of different ranges are standardized to the same scale, and long-time sequence feedback signals are segmented according to the time window to improve the quality and availability of the data. Third, the mean, standard deviation, maximum, minimum, skewness and kurtosis of the feedback signal are calculated to reflect the basic trend and fluctuation of the feedback signal. Fourth, the features of each feedback signal are summarized into a numerical array, and each array element corresponds to a feature, thereby organizing a fixed-length feature vector. Fifth, the feature vectors corresponding to all feedback signals are summarized to form a set of feedback signal feature vectors as the analysis result.

[0036] S130, according to the feature global significant aggregation representation vector and the predetermined performance evaluation model, the performance of the secondary device is evaluated.

[0037] The performance evaluation model is a classifier-based machine learning model, and common classifier-based machine learning models include logistic regression, decision tree, support vector machine, random forest, and the like.

[0038] In the embodiment of the present application, the classifier-based machine learning model can be trained in advance according to a training set composed of historical data, a mapping relationship from a feature global significant aggregation representation vector to a performance evaluation result of the secondary device is established, a performance evaluation model is obtained, and the performance of the secondary device is evaluated according to the feature global significant aggregation representation vector and the pre-determined performance evaluation model. Optionally, the performance of the secondary device is evaluated according to the feature global significant aggregation representation vector and the pre-determined performance evaluation model, including: inputting the feature global significant aggregation representation vector into the classifier-based performance evaluation model to obtain a device performance evaluation result. The device performance evaluation result is used to indicate whether the device performance meets a preset standard, and can include excellent, good, general, poor, and the like. By evaluating the performance of the secondary device according to the feature global significant aggregation representation vector and the pre-determined performance evaluation model, the performance evaluation of the secondary device can be more comprehensive, the influence of human factors on the performance evaluation can be reduced, the accuracy and reliability of the evaluation result can be ensured, and support is provided for the stable operation of the substation.

[0039] The technical solution of the embodiment of the present application comprehensively debugs the secondary device according to the target debugging item, obtains a signal set of feedback signals generated by the secondary device in the comprehensive debugging process, analyzes the feedback signals in the signal set, and determines a feature global significant aggregation representation vector of the signal set according to the analysis result. The performance of the secondary device is evaluated according to the feature global significant aggregation representation vector and a pre-determined performance evaluation model. The technical solution of the embodiment of the present application analyzes the signal set of the feedback signals obtained in the comprehensive debugging process of the secondary device, further obtains a more representative feature global significant aggregation representation vector, and evaluates the performance of the secondary device according to the feature global significant aggregation representation vector and the performance evaluation model, which can reduce the influence of human factors on the performance evaluation, improve the accuracy and reliability of the evaluation result.

[0040] Embodiment Two

[0041] Figure 2 A flowchart of a secondary device performance evaluation method provided by the embodiment two of the present application is shown in FIG. 6. The embodiment of the present application is optimized based on the above-described embodiments, and the solutions not described in detail in the embodiment of the present application are described in the above-described embodiments. As shown in FIG. 6, the method includes: Figure 2

[0042] ​S210, for each debugging item in the target debugging item, generate a simulated three-remote test signal corresponding to the debugging item, and send the simulated three-remote test signal to the secondary equipment.

[0043] The simulated three-remote test signal refers to a test signal obtained by simulating the signal type used in the telemetry, remote control and remote signaling system of the substation, and is used to test and verify the transmission, control and monitoring functions of the secondary equipment.

[0044] In the embodiment of the application, for each debugging item in the target debugging item, a signal generator or a hardware test device can be used to generate a simulated three-remote test signal corresponding to the debugging item, and the generated simulated three-remote test signal is sent to the secondary equipment to be tested through an interface or a network to observe and analyze the response characteristics of the secondary equipment. Alternatively, when generating the simulated three-remote test signal corresponding to the debugging item, signals of different frequencies, amplitudes and waveforms can be generated to simulate various situations in actual application scenarios. By generating the simulated three-remote test signal corresponding to the debugging item and sending the simulated three-remote test signal to the secondary equipment, the automatic debugging of the secondary equipment can be realized, the risk in the testing process can be reduced, and the debugging efficiency can be improved.

[0045] S220, collect the feedback signal generated by the secondary equipment after receiving the simulated three-remote test signal, and obtain a signal set according to the collected feedback signal.

[0046] In the embodiment of the application, in order to test the reaction of the secondary equipment of the substation under different conditions, different scenes and conditions can be pre-set in the debugging item to simulate the generation of various three-remote signals (such as circuit breaker state change, current / voltage value, etc.), and the feedback signal generated by the secondary equipment after receiving various simulated three-remote test signals is collected, and then a signal set of feedback signals is obtained. Since the different feedback signals in the signal set can reflect the feedback response of the secondary equipment of the substation under different conditions, by comprehensively analyzing the feedback signals in the signal set, it can be more comprehensive and accurate to understand and judge whether the secondary equipment works as expected, so as to obtain the performance evaluation result of the secondary equipment.

[0047] S230, analyze the feedback signals in the signal set by the signal feature extractor to obtain a feature code representation vector of the feedback signals.

[0048] The feature code representation vector is a vector representation form of the feature information in the feedback signal related to the performance evaluation of the secondary equipment. By converting the feedback signal into a vector form, it is helpful for subsequent signal analysis and performance evaluation tasks.

[0049] In the embodiment of the present application, the signal feature extractor based on the convolutional neural network model can be used to extract features of each feedback signal in the signal set, complete signal analysis of each feedback signal, and obtain the feature encoding representation vector corresponding to each feedback signal. By analyzing the feedback signals in the signal set through the signal feature extractor, it can help to identify which features are closely related to the performance of the secondary equipment. For example, these features may be frequency components, signal strength, duration, etc., which can more directly reflect the state of the secondary equipment.

[0050] S240, generating a feature encoding representation vector set corresponding to the signal set according to the feature encoding representation vector of the feedback signal.

[0051] In the embodiment of the present application, after obtaining the feature encoding representation vector of the feedback signal, the feature encoding representation vector set of the feedback signal can be determined based on the feature encoding representation vector of the feedback signal. It can be understood that the feedback signal in the signal set and the feature encoding representation vector in the feature encoding representation vector set are in a one-to-one correspondence.

[0052] S250, performing significant aggregation of feature correlation contribution of the feature encoding representation vector set to obtain a feature global significant aggregation representation vector of the signal set.

[0053] In the embodiment of the present application, the feature encoding representation vector set needs to be significantly aggregated in the clustering feature correlation contribution, the most representative and influential feature information is extracted from the feature encoding representation vector in the feature encoding representation vector set, and the expression of these key feature information is enhanced, so as to obtain a higher quality feature representation, that is, the feature global significant aggregation representation vector corresponding to the signal set.

[0054] Optionally, the feature encoding representation vector set is significantly aggregated in the clustering feature correlation contribution to obtain the feature global significant aggregation representation vector of the signal set, including: performing clustering analysis on the feature encoding representation vector set to obtain an encoding feature self-supervised clustering representation vector of the feature encoding representation vector set; based on the encoding feature self-supervised clustering representation vector, performing feature clustering field modulation aggregation on the feature encoding representation vector set to obtain the feature global significant aggregation representation vector of the signal set.

[0055] The encoding feature self-supervised clustering representation vector refers to a vector representing the common features of each cluster after clustering analysis of the feature encoding representation vector set in a self-supervised learning framework. The feature clustering field modulation aggregation refers to a process of using the encoding feature self-supervised clustering representation vector to further process the feature encoding representation vector set to enhance or modulate the feature representation.

[0056] In the embodiment of the present application, the feature code representation vector set can be subjected to cluster analysis first, the number of clusters in the feature code representation vector set after cluster analysis and the feature code representation vectors in each cluster can be determined, and then for each cluster, the mean vector of the feature code representation vectors in the cluster can be calculated as the encoding feature self-supervised cluster representation vector of the cluster to obtain the encoding feature self-supervised cluster representation vector of the feature code representation vector set. Subsequently, the feature code representation vector set can be subjected to feature cluster field modulation aggregation based on the encoding feature self-supervised cluster representation vector to obtain the feature global significant aggregation representation vector of the signal set.

[0057] Optionally, the feature code representation vector set is subjected to feature cluster field modulation aggregation based on the encoding feature self-supervised cluster representation vector to obtain the feature global significant aggregation representation vector of the signal set, which includes: determining the weight of each feature code representation vector in the feature code representation vector set according to the encoding feature self-supervised cluster representation vector; and performing position-weighted summation on the feature code representation vectors in the feature code representation vector set according to the weight to obtain the feature global significant aggregation representation vector of the signal set.

[0058] In the embodiment of the present application, the weight of each feature code representation vector can be determined according to the encoding feature self-supervised cluster representation vector, and the feature code representation vectors in the feature code representation vector set are subjected to position-weighted summation according to the determined weight to obtain the feature global significant aggregation representation vector of the signal set. The feature global significant aggregation representation vector obtained through the above steps can provide a comprehensive feature representation, which not only contains the feature information of the original feedback signal, but also fuses the knowledge learned in the clustering process. This comprehensive feature representation helps the performance evaluation model to capture the key features of the data more accurately when processing complex tasks, thereby improving the performance and generalization ability of the performance evaluation model in the secondary device performance evaluation task.

[0059] Optionally, the weight of each feature code representation vector in the feature code representation vector set is determined according to the encoding feature self-supervised cluster representation vector, which includes: calculating the implicit cluster contribution factor of each feature code representation vector in the feature code representation vector set relative to the encoding feature self-supervised cluster representation vector to obtain an implicit cluster contribution factor sequence; determining an encoding feature cluster contribution field distribution vector according to the implicit cluster contribution factor sequence, and determining the weight of the feature code representation vector according to the encoding feature cluster contribution field distribution vector and a pre-determined weight.

[0060] The implicit clustering contribution factor is used to represent the contribution degree of the feature code representation vector to the cluster to which it belongs. Through the implicit clustering contribution factor, it can be identified which feature code representation vector of the feedback signal has greater importance to the cluster center, that is, has greater contribution to the subsequent secondary equipment performance evaluation task. The encoding feature clustering contribution field distribution vector is a vector used to quantify the contribution degree of a feature vector to a cluster center or a cluster representation vector in the field of machine learning and data processing. The weight determination model is a machine learning model based on a self-attention mechanism.

[0061] In the embodiment of the present application, the implicit clustering contribution factor of each feature code representation vector in the feature code representation vector set relative to the encoding feature self-supervised clustering representation vector can be calculated first to obtain a sequence of implicit clustering contribution factors. Then, the sequence of calculated implicit clustering contribution factors is converted into an encoding feature clustering contribution field distribution vector, that is, the sequence of implicit clustering contribution factors is integrated into a vector field to construct a clustering contribution field using data integration. Finally, the weight of the feature code representation vector is determined by the weight determination model based on the self-attention mechanism and the encoding feature clustering contribution field distribution vector. Specifically, the weight vector of the feature code representation vector can be generated by explicitly modeling the encoding feature clustering contribution field distribution vector through the self-attention mechanism. This can strengthen the attention to feature vectors that have a greater contribution to the subsequent secondary equipment performance evaluation task, while suppressing unimportant feature expressions, thereby improving the quality of the feature global significant aggregation representation vector. It can be understood that the greater the implicit feature code representation vector of the feature code representation vector, the higher its weight in the subsequent aggregation process, so that the performance evaluation model can pay more attention to features that have a significant contribution to the cluster.

[0062] Alternatively, the calculation process of the implicit clustering contribution factor is as follows: first, the absolute value of the position division between the feature code representation vector and the encoding feature self-supervised clustering representation vector is calculated to obtain a local time sequence self-supervised clustering interaction correlation feature vector; second, the logarithmic function value of each position feature value in the local time sequence self-supervised clustering interaction correlation feature vector is calculated to obtain a local time sequence self-supervised clustering interaction logarithmic representation vector; third, the weighted sum of each position feature value in the feature code representation vector is calculated by taking each position feature value in the local time sequence self-supervised clustering interaction logarithmic representation vector as a weighting coefficient to obtain a clustering contribution degree representation factor; fourth, the exponential function value of the clustering contribution degree representation factor is calculated with the natural constant e as the base to obtain the implicit clustering contribution factor.

[0063] Optionally, the calculation process of the weight vector of the feature encoding representation vector is as follows: First, calculate the matrix multiplication between the encoded feature cluster contribution field distribution vector and the query weight matrix, key weight matrix, and value weight matrix respectively to obtain the cluster contribution field distribution query vector, cluster contribution field distribution key vector, and cluster contribution field distribution value vector; Second, calculate the multiplication between the cluster contribution field distribution query vector and the transpose of the cluster contribution field distribution key vector, and then divide the resulting matrix by the square root of the length of the cluster contribution field distribution key vector to obtain the query-key cluster contribution field semantic interaction representation matrix; Third, normalize the query-key cluster contribution field semantic interaction representation matrix using the Softmax function to obtain the normalized query-key cluster contribution field semantic interaction representation matrix; Fourth, calculate the multiplication between the cluster contribution field distribution value vector and the query-key cluster contribution field semantic interaction representation matrix to obtain the weight vector of the feature encoding representation vector.

[0064] Optionally, the relevant formulas used in step S250 are as follows:

[0065] X = {x1, x2, ..., x} k , ..., x n};

[0066]

[0067] v d ={d1; d2; ...; d j ;...;d n};

[0068]

[0069] v q =W q v d ;

[0070] v k =W k v d ;

[0071] v v =W b v d ;

[0072]

[0073] Where X is the set of feature encoding representation vectors, x1, x2, x... k x n Let x be the 1st, 2nd, kth, and nth feature encoding representation vectors in the feature encoding representation vector set, respectively. iis the i-th feature encoding representation vector in the feature encoding representation vector set, N is the number of feature encoding representation vectors in the feature encoding representation vector set, x c is the encoding feature self-supervised clustering representation vector, x j is the j-th feature encoding representation vector in the feature encoding representation vector set, is the k-th position feature value in the j-th feature encoding representation vector, is the k-th position feature value in the encoding feature self-supervised clustering representation vector, log represents the logarithmic function value with base 2, exp represents the natural exponential function value, d j is the j-th feature encoding representation vector corresponding to the implicit clustering contribution factor, v d is the encoding feature clustering contribution field distribution vector, W q , W k and W b are the query weight matrix, the key weight matrix and the value weight matrix respectively, v q , v k and v v are the clustering contribution field distribution query vector, the clustering contribution field distribution key vector and the clustering contribution field distribution value vector respectively, L is the length of the clustering contribution field distribution key vector, Softmax(·) is the Softmax function, is the vector multiplication, v w is the weight vector of the feature encoding representation vector, x f is the feature global significant aggregation representation vector.

[0074] S260, according to the feature global significant aggregation representation vector and the pre-determined performance evaluation model, the performance of the secondary equipment is evaluated.

[0075] The technical scheme of the embodiment of the present application generates, for each debugging item in the target debugging item, a simulated three-remote test signal corresponding to the debugging item, and sends the simulated three-remote test signal to the secondary equipment; collects a feedback signal generated by the secondary equipment after receiving the simulated three-remote test signal, and obtains a signal set according to the collected feedback signal; analyzes the feedback signal in the signal set through a signal feature extractor to obtain a feature code representation vector of the feedback signal; generates a feature code representation vector set corresponding to the signal set according to the feature code representation vector of the feedback signal; performs significant aggregation of clustering features on the feature code representation vector set to obtain a feature global significant aggregation representation vector of the signal set; and evaluates the performance of the secondary equipment according to the feature global significant aggregation representation vector and a pre-determined performance evaluation model. The technical scheme of the embodiment of the present application can comprehensively debug the secondary equipment by generating a simulated three-remote test signal, analyze a signal set of the feedback signal obtained in the comprehensive debugging process, obtain a more representative feature global significant aggregation representation vector, and then evaluate the performance of the secondary equipment according to the feature global significant aggregation representation vector and the performance evaluation model, so that an automatic closed-loop process of secondary equipment performance evaluation can be realized, the risk in the test process is reduced, the influence of human factors on performance evaluation is reduced, and the accuracy and reliability of the evaluation result are improved.

[0076] Embodiment three

[0077] Figure 3 A structure schematic diagram of a secondary equipment performance evaluation device provided by the embodiment three of the present application is shown in FIG. 3. Figure 3 As shown in the figure, the device comprises:

[0078] A signal set determination module 310 is configured to comprehensively debug the secondary equipment according to the target debugging item, and obtain a signal set of the feedback signal generated by the secondary equipment in the comprehensive debugging process;

[0079] An expression vector determination module 320 is configured to analyze the feedback signal in the signal set, and determine a feature global significant aggregation expression vector of the signal set according to the analysis result;

[0080] A device performance evaluation module 330 is configured to evaluate the performance of the secondary equipment according to the feature global significant aggregation expression vector and a pre-determined performance evaluation model.

[0081] Optionally, the signal set determination module 310 comprises:

[0082] A comprehensive debugging unit is configured to, for each debugging item in the target debugging item, generate a simulated three-remote test signal corresponding to the debugging item, and send the simulated three-remote test signal to the secondary equipment;

[0083] a signal collection unit, configured to collect a feedback signal generated by the secondary device after receiving the analog three-remote test signal, and obtain the signal set according to the collected feedback signal.

[0084] Optionally, the representation vector determination module 320 comprises:

[0085] a feature coding representation vector determination unit, configured to parse the feedback signal in the signal set by a signal feature extractor to obtain a feature coding representation vector of the feedback signal;

[0086] a feature coding representation vector set determination unit, configured to generate a feature coding representation vector set corresponding to the signal set according to the feature coding representation vector of the feedback signal;

[0087] a feature global significant aggregation representation vector determination unit, configured to perform clustering feature correlation contribution significant aggregation on the feature coding representation vector set to obtain a feature global significant aggregation representation vector of the signal set.

[0088] Optionally, the feature global significant aggregation representation vector determination unit comprises:

[0089] a coding feature self-supervised clustering representation vector determination subunit, configured to perform clustering analysis on the feature coding representation vector set to obtain a coding feature self-supervised clustering representation vector of the feature coding representation vector set;

[0090] the coding feature self-supervised clustering representation vector determination subunit, configured to perform feature clustering field modulation aggregation on the feature coding representation vector set based on the coding feature self-supervised clustering representation vector to obtain the feature global significant aggregation representation vector of the signal set.

[0091] Optionally, the coding feature self-supervised clustering representation vector determination subunit is specifically configured to:

[0092] for each feature coding representation vector in the feature coding representation vector set, determine a weight of the feature coding representation vector according to the coding feature self-supervised clustering representation vector;

[0093] perform position weighted summation on the feature coding representation vectors of the feature coding representation vector set according to the weight to obtain the feature global significant aggregation representation vector of the signal set.

[0094] Optionally, for each feature code representation vector in the set of feature code representation vectors, determining the weight of the feature code representation vector according to the code feature self-supervised cluster representation vector comprises: calculating the implicit cluster contribution factor of each feature code representation vector in the set of feature code representation vectors with respect to the code feature self-supervised cluster representation vector to obtain a sequence of implicit cluster contribution factors; determining a code feature cluster contribution field distribution vector according to the sequence of implicit cluster contribution factors, and determining the weight of the feature code representation vector obtained by the model according to the code feature cluster contribution field distribution vector and a pre-determined weight.

[0095] Optionally, the performance evaluation model is a classifier-based machine learning model; and the device performance evaluation module 330 is specifically configured to: input the feature global salient aggregation representation vector into the classifier-based performance evaluation model to obtain a device performance evaluation result.

[0096] The secondary device performance evaluation apparatus provided by the embodiments of the present application can execute the secondary device performance evaluation method provided by any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0097] Embodiment Four

[0098] Figure 4 A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0099] As Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., communicatively connected to the at least one processor 11, where the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or loaded into the random access memory (RAM) 13 from the storage unit 18. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0100] Various components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, a speaker, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0101] The processor 11 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the secondary device performance evaluation method.

[0102] In some embodiments, the secondary device performance evaluation method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the secondary device performance evaluation method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the secondary device performance evaluation method by any other appropriate means, such as by means of firmware.

[0103] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a load programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0104] Computer programs used to implement the processes of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer program, when executed, can cause instructions defined in the flow charts and / or block diagrams to be implemented on the computer or other programmable apparatus. The computer programs can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0105] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store computer programs for use by or in connection with an instruction execution system, apparatus, or device. Computer-readable storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0106] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0107] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0108] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0109] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be executed in parallel, executed in sequence, or executed in a different order, as long as the desired results of the present disclosure are achieved, and the present disclosure is not limited herein.

[0110] The specific embodiments described above are not intended to be limiting, and persons skilled in the art will appreciate that various modifications, combinations, sub-combinations and alternatives can be made to the specific embodiments without departing from the spirit and principles of the disclosure. Accordingly, the disclosure is not limited to the specific embodiments described above, but only by the scope of the appended claims.

Claims

1. A method of evaluating performance of a secondary device, characterized by, The method comprises: comprehensive debugging of secondary equipment according to a target debugging item, and obtaining a signal set of feedback signals generated by the secondary equipment in the comprehensive debugging process; analyzing the feedback signals in the signal set, and determining a characteristic global significant aggregation representation vector of the signal set according to an analysis result; evaluating the performance of the secondary equipment according to the characteristic global significant aggregation representation vector and a pre-determined performance evaluation model; analyzing the feedback signals in the signal set, and determining a characteristic global significant aggregation representation vector of the signal set according to an analysis result, comprising: analyzing the feedback signals in the signal set by a signal feature extractor to obtain a characteristic code representation vector of the feedback signals; generating a characteristic code representation vector set corresponding to the signal set according to the characteristic code representation vector of the feedback signals; performing significant aggregation of clustering feature correlation contribution on the characteristic code representation vector set to obtain the characteristic global significant aggregation representation vector of the signal set; performing significant aggregation of clustering feature correlation contribution on the characteristic code representation vector set to obtain the characteristic global significant aggregation representation vector of the signal set, comprising: performing clustering analysis on the characteristic code representation vector set to obtain an encoding feature self-supervised clustering representation vector of the characteristic code representation vector set; modulating and aggregating the characteristic code representation vector set based on the encoding feature self-supervised clustering representation vector to obtain the characteristic global significant aggregation representation vector of the signal set.

2. The method of claim 1, wherein, comprehensive debugging of secondary equipment according to a target debugging item, and obtaining a signal set of feedback signals generated by the secondary equipment in the comprehensive debugging process, comprising: generating a simulated three-remote test signal corresponding to each debugging item in the target debugging item, and sending the simulated three-remote test signal to the secondary equipment; collecting feedback signals generated by the secondary equipment after receiving the simulated three-remote test signal, and obtaining the signal set according to the collected feedback signals.

3. The method of claim 1, wherein, modulating and aggregating the characteristic code representation vector set based on the encoding feature self-supervised clustering representation vector to obtain the characteristic global significant aggregation representation vector of the signal set, comprising: determining a weight of each characteristic code representation vector in the characteristic code representation vector set according to the encoding feature self-supervised clustering representation vector; performing position-weighted summation of the characteristic code representation vectors in the characteristic code representation vector set according to the weight to obtain the characteristic global significant aggregation representation vector of the signal set.

4. The method of claim 3, wherein, determining a weight of each characteristic code representation vector in the characteristic code representation vector set according to the encoding feature self-supervised clustering representation vector, comprising: calculating an implicit clustering contribution factor of each characteristic code representation vector in the characteristic code representation vector set relative to the encoding feature self-supervised clustering representation vector to obtain an implicit clustering contribution factor sequence; Determine a coding feature cluster contribution field distribution vector according to the implicit cluster contribution factor sequence, and determine a weight of the feature coding representation vector according to the coding feature cluster contribution field distribution vector and a predetermined weight.

5. The method of claim 1, wherein, The performance evaluation model is a classifier-based machine learning model. According to the feature global significant aggregation representation vector and a predetermined performance evaluation model, the performance of the secondary device is evaluated, including: Input the feature global significant aggregation representation vector into a classifier-based performance evaluation model to obtain a device performance evaluation result.

6. A secondary device performance evaluation apparatus characterized by comprising: The device includes: A signal set determination module is configured to comprehensively debug the secondary device according to a target debugging item, and obtain a signal set of feedback signals generated by the secondary device during the comprehensive debugging; A representation vector determination module is configured to analyze the feedback signals in the signal set, and determine a feature global significant aggregation representation vector of the signal set according to an analysis result; A device performance evaluation module is configured to evaluate the performance of the secondary device according to the feature global significant aggregation representation vector and a predetermined performance evaluation model. The representation vector determination module includes: A feature coding representation vector determination unit is configured to analyze the feedback signals in the signal set by a signal feature extractor to obtain a feature coding representation vector of the feedback signals; A feature coding representation vector set determination unit is configured to generate a feature coding representation vector set corresponding to the signal set according to the feature coding representation vector of the feedback signals; A feature global significant aggregation representation vector determination unit is configured to perform significant aggregation of cluster feature correlation contribution on the feature coding representation vector set to obtain a feature global significant aggregation representation vector of the signal set. The feature global significant aggregation representation vector determination unit includes: A coding feature self-supervised cluster representation vector determination subunit is configured to perform cluster analysis on the feature coding representation vector set to obtain a coding feature self-supervised cluster representation vector of the feature coding representation vector set; A coding feature self-supervised cluster representation vector determination subunit is configured to perform feature cluster field domain modulation aggregation on the feature coding representation vector set based on the coding feature self-supervised cluster representation vector to obtain a feature global significant aggregation representation vector of the signal set.

7. An electronic device, comprising: The electronic device includes: At least one processor; and A memory connected in communication with the at least one processor; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the secondary device performance evaluation method in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to perform the secondary device performance evaluation method in any one of claims 1-5 when executed.

Images