Secondary equipment performance evaluation method and device, electronic equipment and storage medium

Abstract

The invention discloses a secondary equipment performance evaluation method and device, electronic equipment and a storage medium. The method comprises the steps of performing comprehensive debugging on the secondary equipment according to a target debugging item, and obtaining a signal set of feedback signals generated by the secondary equipment in a comprehensive debugging process; analyzing feedback signals in the signal set, and determining a feature global significant aggregation representation vector of the signal set according to an analysis result; and evaluating the performance of the secondary equipment according to the feature global significant aggregation representation vector and a predetermined performance evaluation model. According to the scheme, the signal set of the feedback signals obtained in the comprehensive debugging process of the secondary equipment is analyzed, so that the more representative feature global significant aggregation representation vector is obtained, and the performance of the secondary equipment is evaluated according to the feature global significant aggregation representation vector and the performance evaluation model; the influence of human factors on performance evaluation can be reduced, and the accuracy and reliability of an evaluation result are improved.

Description

Technical Field

[0001] The present invention relates to the field of smart substations, and in particular to a secondary equipment performance evaluation method, device, electronic equipment and storage medium. Background Art

[0002] With the development of power systems, the automation level of substations is also constantly improving. Automated substations use advanced computer technology and communication methods to achieve real-time monitoring, control and protection of the power grid operation status. In this process, the stability and reliability of the secondary system directly affect the safe operation of the entire power system. How to efficiently and accurately evaluate the performance of secondary equipment in these secondary systems has become an urgent problem to be solved. Traditional evaluation methods often rely on manual operation and require a large amount of human resources. Each link requires the personal participation of technicians. This method is obviously unable to adapt to the needs of large-scale power grid construction and maintenance. In addition, manual operation is easily affected by personal skills, experience and attention. Especially after working for a long time, the error rate may increase. Summary of the invention

[0003] The present invention provides a secondary equipment performance evaluation method, device, electronic equipment and storage medium, which can reduce the impact of human factors on performance evaluation and improve the accuracy and reliability of evaluation results.

[0004] According to one aspect of the present invention, a secondary equipment performance evaluation method is provided, the method comprising:

[0005] Performing comprehensive debugging on the secondary device according to the target debugging item, and obtaining a signal set of feedback signals generated by the secondary device during the comprehensive debugging process;

[0006] Analyzing the feedback signal in the signal set, and determining a feature global significant aggregation representation vector of the signal set according to the analysis result;

[0007] The performance of the secondary device is evaluated according to the feature global significant aggregation representation vector and a predetermined performance evaluation model.

[0008] According to another aspect of the present invention, a secondary equipment performance evaluation device is provided, the device comprising:

[0009] A signal set determination module, used to perform comprehensive debugging on the secondary device according to the target debugging item, and obtain a signal set of feedback signals generated by the secondary device during the comprehensive debugging process;

[0010] A representation vector determination module, used to analyze the feedback signal in the signal set, and determine the feature global significant aggregation representation vector of the signal set according to the analysis result;

[0011] The device performance evaluation module is used to evaluate the performance of the secondary device according to the feature global significant aggregation representation vector and a predetermined performance evaluation model.

[0012] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the secondary device performance evaluation method described in any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the secondary device performance evaluation method described in any embodiment of the present invention when executed.

[0017] The technical solution of the embodiment of the present invention comprehensively debugs the secondary device according to the target debugging items, and obtains the signal set of the feedback signal generated by the secondary device during the comprehensive debugging process; analyzes the feedback signal in the signal set, and determines the feature global significant aggregation representation vector of the signal set according to the analysis result; and evaluates the performance of the secondary device according to the feature global significant aggregation representation vector and a predetermined performance evaluation model. The technical solution of the embodiment of the present invention analyzes the signal set of the feedback signal obtained during the comprehensive debugging of the secondary device, thereby obtaining a more representative feature global significant aggregation representation vector, and evaluates the performance of the secondary device according to the feature global significant aggregation representation vector and the performance evaluation model, which can reduce the impact of human factors on performance evaluation and improve the accuracy and reliability of the evaluation results.

[0018] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 is a flow chart of a secondary equipment performance evaluation method provided according to Embodiment 1 of the present invention;

[0021] Figure 2 is a flow chart of a secondary equipment performance evaluation method provided according to Embodiment 2 of the present invention;

[0022] Figure 3 is a structural schematic diagram of a secondary equipment performance evaluation device provided according to Embodiment 3 of the present invention;

[0023] Figure 4 It is a schematic diagram of the structure of an electronic device for implementing the secondary device performance evaluation method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] Embodiment 1

[0027] Figure 1A flowchart of a secondary equipment performance evaluation method is provided for the first embodiment of the present invention. This embodiment is applicable to the case of evaluating the performance of secondary equipment in a substation. The method can be executed by a secondary equipment performance evaluation device. The secondary equipment performance evaluation device can be implemented in the form of hardware and / or software. The secondary equipment performance evaluation device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0028] S110. Comprehensively debug the secondary device according to the target debugging item, and obtain a signal set of feedback signals generated by the secondary device during the comprehensive debugging process.

[0029] Among them, secondary equipment refers to equipment that performs tasks such as data processing, control and status monitoring in the telemetry, remote control or telesignaling system of the substation. Its functions and performance directly affect the effectiveness of the system. Target debugging items refer to test items set for various functions of secondary equipment, and comprehensive debugging refers to testing all testable functions of secondary equipment according to target debugging items.

[0030] Among them, the feedback signal is the expression of the response of the secondary equipment to the input signal during the comprehensive debugging process, which can be various state parameters of the secondary equipment during operation, such as current, voltage, temperature, frequency and phase.

[0031] In an embodiment of the present invention, the secondary device can be fully debugged according to the target debugging item, and the signal set of the feedback signal generated by the secondary device during the comprehensive debugging process can be obtained. Specifically, all testable functions of the secondary device can be automatically tested according to the target debugging item through the hardware testing device, and the feedback signal generated by the secondary device during the comprehensive debugging process can be automatically collected by installing corresponding sensors on the secondary device, such as current sensors, voltage sensors, etc., and then the signal set of the feedback signal is determined based on the collected feedback signal. It can be understood that the collected feedback signal can intuitively reflect the actual performance of the secondary device. By fully debugging the secondary device according to the target debugging item, a large amount of feedback signal data can be obtained. These data provide a rich information basis for subsequent performance analysis and problem diagnosis. At the same time, it can ensure that all possible situations and boundary conditions are tested, avoiding potential failures caused by missing certain situations.

[0032] S120, analyzing the feedback signal in the signal set, and determining a feature global significant aggregation representation vector of the signal set according to the analysis result.

[0033] Among them, the analysis result of the feedback signal contains feature information related to the performance evaluation of the secondary device. The feature global significant aggregation representation vector of the signal set refers to the higher quality feature representation obtained after the feature information corresponding to the feedback signal in the signal set is significantly aggregated by clustering the relevant contribution of the clustered features. Among them, the significant aggregation of the relevant contribution of clustered features involves multiple aspects such as the evaluation of feature contribution, the aggregation of significant features, and the evaluation of aggregation effect. By effectively aggregating significant features, the clustering effect and accuracy can be improved.

[0034] In an embodiment of the present invention, considering that each feedback signal in the signal set may be a complex waveform or contain a large number of data points, directly processing these original feedback signal data will result in a very complicated calculation process. Therefore, the feedback signals in the signal set may be parsed first, and characteristic information related to the performance evaluation of the secondary device in the feedback signal may be extracted as the parsing result. At the same time, since the characteristic information of different feedback signals has different importance in the subsequent performance evaluation task, that is, the characteristic information of different feedback signals will have different contributions to judging whether the secondary device works as expected and the device performance evaluation task, therefore, in order to extract the most representative and influential characteristic information from a large number of characteristic information, and enhance the expression of these key characteristic information, so as to obtain a higher quality feature representation, in the technical solution of the present invention, it is necessary to further cluster the analysis results to significantly aggregate the relevant contribution of the features, so as to obtain the global significant aggregation representation vector of the features of the signal set.

[0035] Optionally, the specific process of parsing the feedback signals in the signal set is as follows: the first step is to use low-pass filtering to filter out the high-frequency noise of each feedback signal in the signal set to ensure the clarity of the feedback signal. The second step is to standardize the feedback signals of different ranges to the same scale, and segment the feedback signals of long time series according to the time window to improve the quality and availability of the data. The third step is to calculate the statistics such as the mean, standard deviation, maximum, minimum, skewness and kurtosis of the feedback signal to reflect the basic trend and fluctuation of the feedback signal. The fourth step is to summarize the features of each feedback signal into a numerical array, where each array element corresponds to a feature, thereby organizing it into a feature vector of fixed length. The fifth step is to summarize the feature vectors corresponding to all feedback signals to form a set of feature vectors of a feedback signal as the parsing result.

[0036] S130, evaluating the performance of the secondary device according to the feature global significant aggregation representation vector and a predetermined performance evaluation model.

[0037] Among them, the performance evaluation model is a classifier-based machine learning model. Common classifier-based machine learning models include logistic regression, decision tree, support vector machine, random forest, etc.

[0038] In an embodiment of the present invention, a machine learning model based on a classifier can be trained in advance according to a training set composed of historical data, and a mapping relationship from a feature global significant aggregation representation vector to a performance evaluation result of a secondary device can be established to obtain a performance evaluation model, and then the performance of the secondary device is evaluated according to the feature global significant aggregation representation vector and a predetermined performance evaluation model. Optionally, the performance of the secondary device is evaluated according to the feature global significant aggregation representation vector and a predetermined performance evaluation model, including: inputting the feature global significant aggregation representation vector into a performance evaluation model based on a classifier to obtain a device performance evaluation result. Among them, the device performance evaluation result is used to indicate whether the device performance meets the preset standard, and may include excellent, good, general, poor, etc. By evaluating the performance of the secondary device according to the feature global significant aggregation representation vector and a predetermined performance evaluation model, the performance evaluation of the secondary device can be more comprehensively realized, the impact of human factors on the performance evaluation can be reduced, the accuracy and reliability of the evaluation results can be ensured, and support can be provided for the stable operation of the substation.

[0039] The technical solution of the embodiment of the present invention comprehensively debugs the secondary device according to the target debugging items, and obtains the signal set of the feedback signal generated by the secondary device during the comprehensive debugging process; analyzes the feedback signal in the signal set, and determines the feature global significant aggregation representation vector of the signal set according to the analysis result; and evaluates the performance of the secondary device according to the feature global significant aggregation representation vector and a predetermined performance evaluation model. The technical solution of the embodiment of the present invention analyzes the signal set of the feedback signal obtained during the comprehensive debugging of the secondary device, thereby obtaining a more representative feature global significant aggregation representation vector, and evaluates the performance of the secondary device according to the feature global significant aggregation representation vector and the performance evaluation model, which can reduce the impact of human factors on performance evaluation and improve the accuracy and reliability of the evaluation results.

[0040] Embodiment 2

[0041] Figure 2 This is a flow chart of a secondary equipment performance evaluation method provided in the second embodiment of the present invention. The embodiment of the present invention is optimized based on the above embodiment. For solutions not described in detail in the embodiment of the present invention, please refer to the above embodiment. Figure 2 As shown, the method includes:

[0042] S210: For each debugging item in the target debugging items, generate a simulated three-remote control test signal corresponding to the debugging item, and send the simulated three-remote control test signal to the secondary device.

[0043] Among them, the simulated three-remote test signal refers to the test signal obtained by simulating the signal type used in the telemetry, remote control and telesignaling system of the substation, which is used to test and verify the transmission, control and monitoring functions of the secondary equipment.

[0044] In an embodiment of the present invention, for each debugging item in the target debugging item, a signal generator or a hardware testing device can be used to generate a simulated three-remote test signal corresponding to the debugging item, and the generated simulated three-remote test signal is sent to the secondary device to be tested through an interface or a network to observe and analyze the response characteristics of the secondary device. Optionally, when generating the simulated three-remote test signal corresponding to the debugging item, various situations in the actual application scenario can be simulated by generating signals of different frequencies, amplitudes and waveforms. By generating the simulated three-remote test signal corresponding to the debugging item and sending the simulated three-remote test signal to the secondary device, the automated debugging of the secondary device can be achieved, the risk in the testing process can be reduced, and the debugging efficiency can be improved.

[0045] S220, collecting feedback signals generated by the secondary equipment after receiving the simulated three-remote control test signals, and obtaining a signal set according to the collected feedback signals.

[0046] In an embodiment of the present invention, in order to test the response of the secondary equipment of the substation under different circumstances, different scenarios and conditions can be pre-set in the debugging item to simulate and generate various three-remote signals (such as circuit breaker state changes, current / voltage values, etc.), and the feedback signals generated by the secondary equipment after receiving various simulated three-remote test signals are collected to obtain a signal set of feedback signals. Since the different feedback signals in the signal set can reflect the feedback response of the secondary equipment of the substation under different circumstances, by comprehensively analyzing the feedback signals in the signal set, it is possible to more comprehensively and accurately understand and judge whether the secondary equipment is working as expected, thereby obtaining the performance evaluation results of the secondary equipment.

[0047] S230. Analyze the feedback signal in the signal set by a signal feature extractor to obtain a feature coding representation vector of the feedback signal.

[0048] The feature coding representation vector is a vector representation of feature information related to the performance evaluation of the secondary device in the feedback signal. By converting the feedback signal into a vector form, it helps in the execution of subsequent signal analysis and performance evaluation tasks.

[0049] In the embodiment of the present invention, a signal feature extractor based on a convolutional neural network model can be used to extract features from each feedback signal in a signal set, complete signal analysis of each feedback signal, and obtain a feature encoding representation vector corresponding to each feedback signal. Parsing the feedback signals in a signal set by a signal feature extractor can help identify which features are closely related to the performance of the secondary device. For example, these features may be frequency components, signal strength, duration, etc., which can more directly reflect the status of the secondary device.

[0050] S240. Generate a set of feature coding representation vectors corresponding to the signal set according to the feature coding representation vector of the feedback signal.

[0051] In an embodiment of the present invention, after obtaining the feature coding representation vector of the feedback signal, a feature coding representation vector set of the feedback signal can be determined based on the feature coding representation vector of the feedback signal. It can be understood that the feedback signal in the signal set and the feature coding representation vector in the feature coding representation vector set are in a one-to-one correspondence.

[0052] S250, clustering the feature encoding representation vector set to significantly aggregate the relevant contribution of the features, and obtain a globally significant aggregation representation vector of the features of the signal set.

[0053] In an embodiment of the present invention, it is necessary to perform clustering on a feature coding representation vector set to significantly aggregate the relevant contribution of features, extract the most representative and influential feature information from the feature coding representation vectors in the feature coding representation vector set, and enhance the expression of these key feature information, so as to obtain a higher quality feature representation, that is, a feature global domain significantly aggregated representation vector corresponding to the signal set.

[0054] Optionally, the feature coding representation vector set is clustered and the correlation contribution is significantly aggregated to obtain the feature global significant aggregate representation vector of the signal set, including: performing cluster analysis on the feature coding representation vector set to obtain the coding feature self-supervised cluster representation vector of the feature coding representation vector set; based on the coding feature self-supervised cluster representation vector, performing feature clustering field modulation aggregation on the feature coding representation vector set to obtain the feature global significant aggregate representation vector of the signal set.

[0055] The coded feature self-supervised clustering representation vector refers to the vector representing the common features of each cluster after cluster analysis, obtained by clustering the feature coded representation vector set under the self-supervised learning framework. Feature clustering field modulation aggregation refers to the process of further processing the feature coded representation vector set using the coded feature self-supervised clustering representation vector to enhance or modulate the feature representation.

[0056] In an embodiment of the present invention, a cluster analysis can be first performed on a feature coding representation vector set to determine the number of clusters in the feature coding representation vector set after cluster analysis and the feature coding representation vector in each cluster, and then for each cluster, the mean vector of the feature coding representation vector in the cluster is calculated as the coding feature self-supervised cluster representation vector of the cluster to obtain the coding feature self-supervised cluster representation vector of the feature coding representation vector set. Afterwards, based on the coding feature self-supervised cluster representation vector, the feature coding representation vector set can be subjected to feature clustering field modulation aggregation to obtain the feature global significant aggregation representation vector of the signal set.

[0057] Optionally, based on the coding feature self-supervised clustering representation vector, the feature coding representation vector set is subjected to feature clustering field modulation aggregation to obtain a feature global significant aggregation representation vector of the signal set, including: for each feature coding representation vector in the feature coding representation vector set, determining the weight of the feature coding representation vector according to the coding feature self-supervised clustering representation vector; and performing position-weighted summation of the feature coding representation vectors of the feature coding representation vector set according to the weight to obtain a feature global significant aggregation representation vector of the signal set.

[0058] In an embodiment of the present invention, the weight of each feature coding representation vector can be determined based on the coding feature self-supervised clustering representation vector, and the feature coding representation vectors of the feature coding representation vector set are weighted and summed according to the determined weights according to the position to obtain the feature global significant aggregation representation vector of the signal set. The feature global significant aggregation representation vector obtained through the above steps can provide a comprehensive feature representation, which not only contains the feature information of the original feedback signal, but also integrates the knowledge learned in the clustering process. This comprehensive feature representation helps the performance evaluation model to more accurately capture the key features of the data when processing complex tasks, thereby improving the performance and generalization ability of the performance evaluation model in performing secondary equipment performance evaluation tasks.

[0059] Optionally, for each feature coding representation vector in the feature coding representation vector set, the weight of the feature coding representation vector is determined according to the coding feature self-supervised clustering representation vector, including: calculating the implicit clustering contribution factor of each feature coding representation vector in the feature coding representation vector set relative to the coding feature self-supervised clustering representation vector to obtain a sequence of implicit clustering contribution factors; determining the coding feature clustering contribution field distribution vector according to the implicit clustering contribution factor sequence, and obtaining the weight of the feature coding representation vector according to the coding feature clustering contribution field distribution vector and a predetermined weight determination model.

[0060] Among them, the implicit clustering contribution factor is used to characterize the contribution of the feature encoding representation vector to the cluster to which it belongs. Through the implicit clustering contribution factor, it is possible to identify which feedback signal feature encoding representation vectors are more important to the cluster center, that is, they contribute more to the subsequent secondary equipment performance evaluation task. The encoding feature cluster contribution field distribution vector is a vector used in the field of machine learning and data processing to quantify the contribution of the feature vector to the cluster center or cluster representation vector. The weight determination model is a machine learning model based on the self-attention mechanism.

[0061] In an embodiment of the present invention, the implicit clustering contribution factor of each feature encoding representation vector in the feature encoding representation vector set relative to the encoding feature self-supervised clustering representation vector can be calculated first to obtain a sequence of implicit clustering contribution factors, and then the sequence of implicit clustering contribution factors calculated is converted into a distribution vector of the encoding feature clustering contribution field, that is, the sequence of implicit clustering contribution factors is integrated into the vector field by data integration to construct the clustering contribution field, and finally, the weight of the feature encoding representation vector is determined by a weight determination model based on a self-attention mechanism and the encoding feature clustering contribution field distribution vector. Specifically, the encoding feature clustering contribution field distribution vector can be explicitly modeled by a self-attention mechanism to generate a weight vector of the feature encoding representation vector, so that the focus on the feature vectors that contribute greatly to the subsequent secondary equipment performance evaluation task can be strengthened, while suppressing unimportant feature expressions, and improving the quality of the feature global significant aggregation representation vector. It can be understood that the larger the implicit feature encoding representation vector of the feature encoding representation vector, the higher its weight in the subsequent aggregation process, so that the performance evaluation model can pay more attention to the features that contribute significantly to the clustering.

[0062] Optionally, the calculation process of the implicit clustering contribution factor is as follows: the first step is to calculate the absolute value of the positional division between the feature coding representation vector and the coded feature self-supervised clustering representation vector to obtain the local temporal self-supervised clustering interaction correlation feature vector; the second step is to calculate the logarithmic function value of the feature value at each position in the local temporal self-supervised clustering interaction correlation feature vector with a base of 2 to obtain the local temporal self-supervised clustering interaction logarithmic representation vector; the third step is to use the feature value at each position in the local temporal self-supervised clustering interaction logarithmic representation vector as the weighting coefficient, and calculate the weighted sum of the feature value at each position in the feature coding representation vector to obtain the clustering contribution representation factor; the fourth step is to calculate the exponential function value of the clustering contribution representation factor with a natural constant e as the base to obtain the implicit clustering contribution factor.

[0063] Optionally, the calculation process of the weight vector of the feature encoding representation vector is as follows: the first step is to calculate the matrix multiplication between the encoded feature cluster contribution field distribution vector and the query weight matrix, the key weight matrix and the value weight matrix respectively to obtain the cluster contribution field distribution query vector, the cluster contribution field distribution key vector and the cluster contribution field distribution value vector; the second step is to calculate the multiplication between the cluster contribution field distribution query vector and the transposed vector of the cluster contribution field distribution key vector, and then divide the obtained matrix by the square root of the length of the cluster contribution field distribution key vector by position to obtain the query-key cluster contribution field semantic interaction representation matrix; the third step is to use the Softmax function to normalize the query-key cluster contribution field semantic interaction representation matrix to obtain the normalized query-key cluster contribution field semantic interaction representation matrix; the fourth step is to calculate the multiplication between the cluster contribution field distribution value vector and the query-key cluster contribution field semantic interaction representation matrix to obtain the weight vector of the feature encoding representation vector.

[0064] Optionally, the relevant formula used in step S250 is as follows:

[0065] X={x 1 , x 2 , ..., x k , ..., x n};

[0066]

[0067] v d ={d 1 ;d 2 ;...;d j ;...;d n};

[0068]

[0069] v q =W q v d ;

[0070] v k =W k v d ;

[0071] v v =W b v d ;

[0072]

[0073] Among them, X is the feature encoding representation vector set, x 1 , x 2 , x k , x nare the first, second, kth and nth feature encoding representation vectors in the feature encoding representation vector set, respectively, x i is the i-th feature coding representation vector in the feature coding representation vector set, N is the number of feature coding representation vectors in the feature coding representation vector set, x c is the encoding feature self-supervised clustering representation vector, x j is the jth feature encoding representation vector in the feature encoding representation vector set, The jth feature code represents the feature value of the kth position in the vector, The feature value of the kth position in the encoding feature self-supervised clustering vector is represented, log represents the logarithmic function value with base 2, exp represents the natural exponential function value, and d j is the implicit clustering contribution factor corresponding to the j-th feature encoding representation vector, v d Contribution field distribution vector for encoding feature clustering, W q , W k and W b are the query weight matrix, key weight matrix and value weight matrix respectively, v q 、v k and v v are cluster contribution field distribution query vector, cluster contribution field distribution key vector and cluster contribution field distribution value vector respectively, L is the length of cluster contribution field distribution key vector, Softmax(·) is the Softmax function, is vector multiplication, v w is the weight vector of the feature encoding representation vector, x f is the feature global significant aggregation representation vector.

[0074] S260, evaluating the performance of the secondary device according to the feature global significant aggregation representation vector and a predetermined performance evaluation model.

[0075] The technical solution of the embodiment of the present invention generates a simulated three-remote test signal corresponding to each debugging item in the target debugging item, and sends the simulated three-remote test signal to the secondary device; collects the feedback signal generated by the secondary device after receiving the simulated three-remote test signal, and obtains a signal set according to the collected feedback signal; analyzes the feedback signal in the signal set by a signal feature extractor to obtain a feature coding representation vector of the feedback signal; generates a feature coding representation vector set corresponding to the signal set according to the feature coding representation vector of the feedback signal; clusters the feature coding representation vector set to significantly aggregate the feature correlation contribution, and obtains the feature global significant aggregation representation vector of the signal set; evaluates the performance of the secondary device according to the feature global significant aggregation representation vector and a predetermined performance evaluation model. The technical solution of the embodiment of the present invention comprehensively debugs the secondary equipment by generating simulated three-remote control test signals, and analyzes the signal set of feedback signals obtained during the comprehensive debugging process to obtain a more representative feature global significant aggregation representation vector, and then evaluates the performance of the secondary equipment based on the feature global significant aggregation representation vector and the performance evaluation model. This can realize an automated closed-loop process for secondary equipment performance evaluation, reduce risks in the testing process, reduce the impact of human factors on performance evaluation, and improve the accuracy and reliability of evaluation results.

[0076] Embodiment 3

[0077] Figure 3 This is a schematic diagram of the structure of a secondary equipment performance evaluation device provided in the third embodiment of the present invention. Figure 3 As shown, the device comprises:

[0078] A signal set determination module 310 is used to perform comprehensive debugging on the secondary device according to the target debugging item, and obtain a signal set of feedback signals generated by the secondary device during the comprehensive debugging process;

[0079] A representation vector determination module 320, configured to analyze the feedback signal in the signal set and determine a feature global significant aggregation representation vector of the signal set according to the analysis result;

[0080] The device performance evaluation module 330 is used to evaluate the performance of the secondary device according to the feature global significant aggregation representation vector and a predetermined performance evaluation model.

[0081] Optionally, the signal set determination module 310 includes:

[0082] A comprehensive debugging unit, for generating, for each debugging item in the target debugging items, a simulated three-remote control test signal corresponding to the debugging item, and sending the simulated three-remote control test signal to the secondary device;

[0083] The signal acquisition unit is used to acquire the feedback signal generated by the secondary device after receiving the simulated three-remote test signal, and obtain the signal set according to the acquired feedback signal.

[0084] Optionally, the representation vector determination module 320 includes:

[0085] a feature coding representation vector determining unit, configured to parse the feedback signal in the signal set through a signal feature extractor to obtain a feature coding representation vector of the feedback signal;

[0086] a feature coding representation vector set determining unit, configured to generate a feature coding representation vector set corresponding to the signal set according to the feature coding representation vector of the feedback signal;

[0087] The feature global significant aggregation representation vector determination unit is used to perform clustering feature correlation contribution significant aggregation on the feature encoding representation vector set to obtain the feature global significant aggregation representation vector of the signal set.

[0088] Optionally, the feature global significant aggregation representation vector determination unit includes:

[0089] A coding feature self-supervised clustering representation vector determination subunit is used to perform cluster analysis on the feature coding representation vector set to obtain a coding feature self-supervised clustering representation vector of the feature coding representation vector set;

[0090] The coding feature self-supervised clustering representation vector determination subunit is used to perform feature clustering field modulation aggregation on the feature coding representation vector set based on the coding feature self-supervised clustering representation vector to obtain the feature global significant aggregation representation vector of the signal set.

[0091] Optionally, the encoding feature self-supervised clustering representation vector determination subunit is specifically used to:

[0092] For each feature coding representation vector in the feature coding representation vector set, determining a weight of the feature coding representation vector according to the coding feature self-supervised clustering representation vector;

[0093] According to the weights, the feature encoding representation vectors of the feature encoding representation vector set are summed up weighted by position to obtain a feature global significant aggregation representation vector of the signal set.

[0094] Optionally, for each feature coding representation vector in the feature coding representation vector set, the weight of the feature coding representation vector is determined according to the coding feature self-supervised clustering representation vector, including: calculating the implicit clustering contribution factor of each feature coding representation vector in the feature coding representation vector set relative to the coding feature self-supervised clustering representation vector to obtain a sequence of implicit clustering contribution factors; determining the coding feature clustering contribution field distribution vector according to the implicit clustering contribution factor sequence, and obtaining the weight of the feature coding representation vector according to the coding feature clustering contribution field distribution vector and a predetermined weight determination model.

[0095] Optionally, the performance evaluation model is a classifier-based machine learning model; the device performance evaluation module 330 is specifically used to: input the feature global significant aggregation representation vector into the classifier-based performance evaluation model to obtain a device performance evaluation result.

[0096] The secondary equipment performance evaluation device provided in the embodiment of the present invention can execute the secondary equipment performance evaluation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0097] Embodiment 4

[0098] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0099] like Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0100] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0101] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a secondary device performance evaluation method.

[0102] In some embodiments, the secondary device performance evaluation method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the secondary device performance evaluation method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the secondary device performance evaluation method in any other appropriate manner (e.g., by means of firmware).

[0103] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0104] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0105] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0106] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0107] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0108] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0109] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0110] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A secondary equipment performance evaluation method, characterized in that: The method comprises: Performing comprehensive debugging on the secondary device according to the target debugging item, and obtaining a signal set of feedback signals generated by the secondary device during the comprehensive debugging process; Analyzing the feedback signal in the signal set, and determining a feature global significant aggregation representation vector of the signal set according to the analysis result; The performance of the secondary device is evaluated according to the feature global significant aggregation representation vector and a predetermined performance evaluation model.

2. The method according to claim 1, characterized in that: Comprehensively debugging the secondary device according to the target debugging item, and obtaining a signal set of feedback signals generated by the secondary device during the comprehensive debugging process, including: For each debugging item in the target debugging items, generating a simulated three-remote control test signal corresponding to the debugging item, and sending the simulated three-remote control test signal to the secondary device; The feedback signal generated by the secondary device after receiving the simulated three-remote control test signal is collected, and the signal set is obtained according to the collected feedback signal.

3. The method according to claim 1, characterized in that Parsing the feedback signal in the signal set, and determining the feature global significant aggregation representation vector of the signal set according to the parsing result, including: Parsing the feedback signal in the signal set by a signal feature extractor to obtain a feature encoding representation vector of the feedback signal; Generating a set of feature coding representation vectors corresponding to the signal set according to the feature coding representation vector of the feedback signal; The feature encoding representation vector set is clustered and feature-related contribution is significantly aggregated to obtain a feature global significant aggregation representation vector of the signal set.

4. The method according to claim 3, characterized in that: The feature encoding representation vector set is clustered and the relevant contribution of the feature is significantly aggregated to obtain the feature global significant aggregation representation vector of the signal set, including: Performing cluster analysis on the feature coding representation vector set to obtain a coding feature self-supervised cluster representation vector of the feature coding representation vector set; Based on the self-supervised clustering representation vector of the coding feature, the feature coding representation vector set is subjected to feature clustering field modulation aggregation to obtain a feature global significant aggregation representation vector of the signal set.

5. The method according to claim 4, characterized in that Based on the coding feature self-supervised clustering representation vector, the feature coding representation vector set is subjected to feature clustering field modulation aggregation to obtain a feature global significant aggregation representation vector of the signal set, including: For each feature coding representation vector in the feature coding representation vector set, determining a weight of the feature coding representation vector according to the coding feature self-supervised clustering representation vector; According to the weights, the feature encoding representation vectors of the feature encoding representation vector set are summed up weighted by position to obtain a feature global significant aggregation representation vector of the signal set.

6. The method according to claim 5, characterized in that For each feature encoding representation vector in the feature encoding representation vector set, determining a weight of the feature encoding representation vector according to the encoding feature self-supervised clustering representation vector, including: Calculating the implicit clustering contribution factor of each feature encoding representation vector in the feature encoding representation vector set relative to the encoding feature self-supervised clustering representation vector to obtain an implicit clustering contribution factor sequence; The coding feature cluster contribution field distribution vector is determined according to the implicit cluster contribution factor sequence, and the weight of the feature coding representation vector is obtained according to the coding feature cluster contribution field distribution vector and a predetermined weight determination model.

7. The method according to claim 1, characterized in that The performance evaluation model is a machine learning model based on a classifier; The performance of the secondary device is evaluated according to the feature global significant aggregation representation vector and a predetermined performance evaluation model, including: The feature global significant aggregation representation vector is input into a classifier-based performance evaluation model to obtain a device performance evaluation result.

8. A secondary equipment performance evaluation device, characterized in that: The device comprises: A signal set determination module, used to perform comprehensive debugging on the secondary device according to the target debugging item, and obtain a signal set of feedback signals generated by the secondary device during the comprehensive debugging process; A representation vector determination module, used to analyze the feedback signal in the signal set, and determine the feature global significant aggregation representation vector of the signal set according to the analysis result; The device performance evaluation module is used to evaluate the performance of the secondary device according to the feature global significant aggregation representation vector and a predetermined performance evaluation model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the secondary device performance evaluation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the secondary device performance evaluation method according to any one of claims 1 to 7 when executed.

Images