Model performance influencing factor identification method, device, equipment and storage medium

Abstract

The application discloses a model performance influencing factor identification method, device and equipment and a storage medium. The method comprises the following steps: constructing an initial model, training the initial model based on an obtained input error training set and an influencing factor training set, and obtaining a target error model; obtaining an input error data group and a target feature data group to be analyzed; wherein the target feature data group to be analyzed comprises at least one feature to be analyzed; inputting the input error data group and the target feature data group to be analyzed into the target error model to obtain an output error data group; and processing the output error data group to obtain an influence level of each feature to be analyzed on the target error model. The technical scheme of the application can indirectly determine the influence level of each feature to be analyzed on the error of the original model.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and in particular relates to a method, apparatus, device and storage medium for identifying factors affecting model performance. Background Technology

[0002] Currently, the general method for error analysis of the original model is to directly perform error analysis on the original model. The analysis process requires knowledge of the original model's running program and input data, and the original model needs to be re-executed. Furthermore, the analysis can only be performed on the data items used by the original model.

[0003] However, the currently constructed model cannot call the original model's running program, so the original model cannot be re-executed during error analysis. In addition, the data items used in the original model may be missing some input items or have biases in the frequency of some data, which means it is difficult to obtain the input data of the original model again. Therefore, it is currently impossible to analyze the error of the original model. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in related technologies. Therefore, one object of this invention is to provide a method, apparatus, device, and storage medium for identifying factors influencing model performance.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide the following technical solutions:

[0006] A method for identifying factors affecting model performance, comprising:

[0007] An initial model is constructed, and the initial model is trained based on the obtained input error training set and influencing factor training set to obtain the target error model;

[0008] Obtain an input error data set and a target feature data set to be analyzed; wherein the target feature data set to be analyzed includes at least one feature to be analyzed; input the input error data set and the target feature data set to be analyzed into the target error model to obtain an output error data set;

[0009] The output error data set is processed to obtain the influence level of each of the analyzed feature data on the target error model.

[0010] Optionally, the target feature data set to be analyzed includes any one of the following:

[0011] M types of features to be analyzed;

[0012] M-1 types of features and one of the features to be analyzed;

[0013] One of the features to be analyzed;

[0014] Where M≥2, and M is an integer.

[0015] Optionally, before constructing the initial model, the following steps are included:

[0016] Obtain the initial set of feature data to be analyzed and the initial set of input error data;

[0017] Based on the correlation algorithm, the correlation between the initial feature data set to be analyzed and the initial input error data set is calculated, and the initial feature data set to be analyzed is filtered based on the correlation to obtain the first target feature data set to be analyzed.

[0018] Optionally, the step of calculating the correlation between the initial feature data set to be analyzed and the initial input error data set based on the correlation algorithm, and filtering the initial feature data set to be analyzed based on the correlation to obtain a first target feature data set to be analyzed, includes:

[0019] Calculate the correlation between each of the features to be analyzed and the initial input error data set;

[0020] Based on the relevance, the initial set of feature data to be analyzed is filtered to obtain a first target set of feature data to be analyzed; or

[0021] The initial set of feature data to be analyzed is subjected to dimensionality reduction processing to obtain an initial dimensionality-reduced set of feature data to be analyzed;

[0022] Calculate the dimensionality reduction correlation between the initial dimensionality reduction feature data set to be analyzed and the initial input error data set;

[0023] Based on the dimensionality reduction relevance, the dimensionality reduction feature data group to be analyzed is filtered to obtain the first target feature data group to be analyzed.

[0024] Optionally, the step of constructing an initial model and training the initial model based on the acquired input error training set and influencing factor training set to obtain a target error model includes:

[0025] Based on the first target feature data set to be analyzed, obtain the first input error training set and the first influencing factor training set;

[0026] An initial model is constructed, and the initial model is trained based on the first input error training set and the first influencing factor training set to obtain a first target error model;

[0027] The first target error model is iteratively trained to obtain the (N-1)th target error model; where N≥2 and N is an integer;

[0028] Based on the Nth target feature data set to be analyzed, obtain the Nth influencing factor training set and the Nth input error training set;

[0029] The Nth target error model is obtained by training the Nth influencing factor training set and the Nth input error training set.

[0030] Optionally, processing the output error data set to obtain the influence level of each of the analyzed feature data on the target error model includes:

[0031] The Nth output error data set is analyzed based on the evaluation algorithm to obtain the influence value of each of the features to be analyzed on the Nth target error model;

[0032] Based on the influence value, the influence level of each of the features to be analyzed on the Nth objective error model is determined.

[0033] Optionally, after determining the influence level of each of the features to be analyzed on the Nth target error model, the method further includes:

[0034] Based on the influence level of each of the features to be analyzed on the error model of the Nth target, the data of the Nth target feature to be analyzed are filtered to obtain the data group of the N+1th target feature to be analyzed;

[0035] Based on the N+1th target feature data set to be analyzed and the N+1th input error data set, obtain the N+1th influencing factor training set and the N+1th input error training set;

[0036] The Nth target error model is trained for the N+1th time based on the N+1th influencing factor training set and the N+1th input error training set to obtain the N+1th target error model.

[0037] Embodiments of the present invention also provide a device for identifying factors affecting model performance, comprising:

[0038] The training module is used to build an initial model and train the initial model based on the acquired input error training set and influencing factor training set to obtain the target error model.

[0039] An acquisition module is used to acquire an input error data set and a target feature data set to be analyzed; wherein the target feature data set to be analyzed includes at least one feature to be analyzed;

[0040] The output module is used to input the input error data set and the target feature data set to be analyzed into the target error model to obtain the output error data set;

[0041] The calculation module is used to process the output error data set to obtain the influence level of each of the analyzed feature data on the target error model.

[0042] Embodiments of the present invention also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the method described above.

[0043] Embodiments of the present invention also provide a computer-readable storage medium comprising a stored computer program, wherein the computer program, when executed, controls the device containing the computer-readable storage medium to perform the method described above.

[0044] The embodiments of the present invention have the following technical effects:

[0045] The above-mentioned technical solution of the present invention 1) obtains the input error data set based on RTM, and the input error data set may be inconsistent with the input of the original model, and there is no need to re-execute the original model. Therefore, it is not necessary to understand the running procedure of the original model, and the operation is simple. In addition, since the original model and the target error model share the same estimation value set, the influence level of each feature to be analyzed on the error of the original model can be indirectly determined by determining the influence level of each feature to be analyzed on the error of the target error model.

[0046] 2) A target error model was constructed. This target error model is independent of the original model and does not affect the operation of each other. Therefore, the target model of the embodiment of the present invention has strong independence, small interference, and high confidence.

[0047] 3) The target error model is trained multiple times based on the types of features to be analyzed, which improves the confidence of the target error model and helps to improve the accuracy of the influence level of each feature to be analyzed on the error of the original model.

[0048] 4) The target feature data group to be analyzed includes multiple categories, which can be selected according to different user needs to improve the efficiency of data processing and the accuracy of processing results.

[0049] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating a method for identifying factors affecting model performance according to an embodiment of the present invention.

[0051] Figure 2This is the first example of the process of a method for identifying factors affecting model performance provided in this embodiment of the invention;

[0052] Figure 3 This is a second example of the process of a method for identifying factors affecting model performance provided in an embodiment of the present invention;

[0053] Figure 4 This is the third example of the process of a method for identifying factors affecting model performance provided in this embodiment of the invention;

[0054] Figure 5 This is a schematic diagram of the structure of a device for identifying factors affecting model performance provided in an embodiment of the present invention. Detailed Implementation

[0055] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0056] First, to facilitate understanding of the embodiments of the present invention by those skilled in the art, some terms are explained:

[0057] (1) RTM: Real Time Monitor, real-time monitoring system.

[0058] (2) Pearson algorithm: Pearson correlation coefficient; Pearson correlation coefficient is widely used to measure the degree of correlation between two variables, and its value is between -1 and 1.

[0059] (3) PCA algorithm: Principal Component Analysis.

[0060] (4) LASSO: Least absolute shrinkage and selection operator.

[0061] (5) RF: Random Forest.

[0062] (6) MLP: Multi-Layer Perception.

[0063] (7) MSE: Mean Squared Error.

[0064] (8) F1 score: F1 score.

[0065] (9) PDP: parallel distributed processing.

[0066] (10) ICE: Impact Confidence Ease.

[0067] (11) SHAP: SHapleyAdditive exPlanation.

[0068] (12) SOC: State of Charge.

[0069] (13) SOH: State of Health, lifespan status.

[0070] Embodiments of the present invention provide a system for identifying factors affecting model performance, comprising:

[0071] RTM, model builder, processor, and memory;

[0072] Among them, RTM, model builder, processor and memory interact with each other via network.

[0073] In practical applications, the original model is built into the RTM. Therefore, the true value set and the estimated value set are obtained based on the RTM. Specifically, the original model outputs the estimated value set, and then the original model sends the estimated value set to the RTM for storage. The true value set and the estimated value set are then processed to obtain the calculation error set, which is also stored in the RTM. In other words, the estimated value set, the true value set, and the calculation error set can be obtained based on the RTM. The input error data set can then be obtained based on the calculation error set.

[0074] The input data consists of the input error data set and the feature data set to be analyzed that may affect the error of the original model. The input data is then fed into the target error model that is built and trained based on the model builder.

[0075] The target error model is obtained by training the model builder using the input error training set and the influencing factor training set. It is independent of the original model, meaning that when performing error analysis based on the target error model, it is not necessary to re-execute the original model.

[0076] Furthermore, after obtaining the output error data set based on the target error model, the output error data set is input to the processor. The processor analyzes and processes the output error data set based on the built-in evaluation algorithm to obtain the influence level or degree of each feature to be analyzed on the target error model.

[0077] Furthermore, during system operation, the generated data can be stored in the memory through the data interface as needed for subsequent retrieval.

[0078] In the embodiments of the present invention, the input error data set is obtained based on RTM, and the input error data set may be inconsistent with the input of the original model. There is no need to re-execute the original model. Therefore, it is not necessary to understand the running procedure of the original model, and the operation is simple. In addition, since the original model and the target error model share the same set of estimated values, the influence level of each feature to be analyzed on the error of the original model can be indirectly determined by determining the influence level of each feature to be analyzed on the error of the target error model.

[0079] like Figure 1 As shown, embodiments of the present invention also provide a method for identifying factors affecting model performance, applied to the above-mentioned system, including:

[0080] Step S1: Construct an initial model and train the initial model based on the acquired input error training set and influencing factor training set to obtain the target error model;

[0081] In embodiments of the present invention, the true value set, the estimated value set, and the input error data set are obtained from the RTM;

[0082] The true value group includes multiple true values, and each estimated value group includes multiple estimated values. Correspondingly, each true value corresponds to an estimated value, forming a set of data. By analogy, multiple sets of data can be obtained, which are also known as input error data groups.

[0083] The types of features to be analyzed may be obtained based on business knowledge, such as temperature, time, remaining charging capacity, maximum power of charging pile, stability of charging pile, ambient temperature and temperature rise rate, etc. Based on business knowledge, a general range of the types of features to be analyzed can be determined.

[0084] Specifically, taking the actual and estimated values ​​of remaining charging time as examples, based on business knowledge, we can identify various factors that may affect the target error model of remaining charging time, and how these factors influence the error:

[0085] For example: 1) Charging method at high SOC: When the battery is close to full charge, the charging method changes from constant current to constant voltage or multi-stage constant current. At this time, the current decreases and the charging time increases.

[0086] 2) Remaining charging capacity: The amount of electricity still needed to be charged, which is related to SOC and SOH;

[0087] 3) Power limitations: This includes the maximum charging power of the current charging device and whether the current supplied by the device is stable;

[0088] 4) Battery balancing: Battery balancing consumes additional energy, thus affecting charging time;

[0089] 5) Thermal management: Thermal management affects both the additional energy consumption and the charging power through the power meter, thus affecting the charging time;

[0090] 6) Charging power limitation: Charging power limitation is an important factor affecting the remaining charging time, which depends on SOC, temperature and SOH.

[0091] Furthermore, the original model acquires multiple sets of data, analyzes these sets of data, and outputs the input error data set.

[0092] Each input error data group includes multiple input error data points, the number of which corresponds to the number of data groups.

[0093] In an optional embodiment of the present invention, if the true value set and the estimated value set cannot be obtained based on RTM, only the evaluation of the estimated value set can be obtained; wherein, the evaluation of the estimated value set may include large error, small error, qualified or unqualified, etc.

[0094] In practical application scenarios, after obtaining the evaluation of the estimated value group, for example, the estimated value evaluation group includes a subgroup of qualified estimated values ​​and another subgroup of unqualified estimated values.

[0095] Specifically, the estimated value group evaluation can be analyzed and processed based on the following methods:

[0096] Obtain the target feature group to be analyzed, and analyze the correlation between the target feature group to be analyzed and the estimated value evaluation group based on the mean test algorithm;

[0097] Furthermore: If one of the target features to be analyzed in the target feature group is used as a variable, then the data of other features to be analyzed in the target feature group are kept unchanged, and only the data of that specific feature is changed. For example, if the specific feature to be analyzed is temperature, then a gradient transformation is performed on the temperature. For example, by using gradient change to take values, increasing by 5 degrees each time, and taking data between 20 and 70 degrees, multiple target feature groups to be analyzed can be obtained. The mean test algorithm is used to evaluate and analyze each target feature group and the estimated value group, observing the distribution changes of qualified and unqualified estimated value subgroups in the estimated value group evaluation (e.g., t-test algorithm). The correlation between each feature to be analyzed and the estimated value group evaluation can be determined, and thus the correlation between each feature to be analyzed and the original model can be obtained. Based on the value of the t-test, the magnitude of the influence of each feature to be analyzed on the error of the original model can be specifically determined, and the influence level can be determined based on the magnitude of the influence.

[0098] In addition, depending on actual needs, a scatter plot algorithm can be selected to determine the correlation between each feature to be analyzed and the estimated value group.

[0099] Furthermore, before constructing the initial model, the following steps are included:

[0100] Obtain the initial set of feature data to be analyzed and the initial set of input error data;

[0101] Based on the correlation algorithm, the correlation between the initial feature data set to be analyzed and the initial input error data set is calculated, and the initial feature data set to be analyzed is filtered based on the correlation to obtain the first target feature data set to be analyzed.

[0102] Specifically, the step of calculating the correlation between the initial set of feature data to be analyzed and the initial set of input error data based on the correlation algorithm, and filtering the initial set of feature data to be analyzed based on the correlation to obtain the first target set of feature data to be analyzed, includes:

[0103] Calculate the correlation between each of the features to be analyzed and the initial input error data set;

[0104] Based on the relevance, the initial set of feature data to be analyzed is filtered to obtain the first target set of feature data to be analyzed;

[0105] The embodiments of the present invention can be implemented based on the Pearson algorithm. Specifically, the correlation between each feature to be analyzed and the initial input error data set can be determined based on the value of the Pearson algorithm. For example, if the value of the Pearson algorithm is 0, it indicates that there is no linear correlation between the feature to be analyzed and the initial input error data set.

[0106] If the value of the Pearson algorithm is 1, it indicates that there is a good linear correlation between the feature to be analyzed and the initial input error data set;

[0107] ...

[0108] By analogy, the correlation between each feature to be analyzed and the initial input error data set can be determined. Then, a correlation threshold can be set to filter and delete the features to be analyzed whose values ​​do not meet the correlation threshold. The features to be analyzed whose values ​​meet the correlation threshold are retained to form the first target feature data set to be analyzed.

[0109] It should be noted that in the above embodiments, each feature to be analyzed corresponds to multiple feature data to be analyzed.

[0110] In embodiments of the present invention, some features that are poorly correlated with the error influence of a target error model to be input can be deleted, thereby avoiding wasting the system's computing power in subsequent calculations and improving computational efficiency.

[0111] Alternatively, in an embodiment of the present invention, the initial set of feature data to be analyzed is subjected to dimensionality reduction processing to obtain an initial dimensionality-reduced set of feature data to be analyzed;

[0112] Calculate the dimensionality reduction correlation between the initial dimensionality reduction feature data set to be analyzed and the initial input error data set;

[0113] Based on the dimensionality reduction relevance, the dimensionality reduction feature data group to be analyzed is filtered to obtain the first target feature data group to be analyzed.

[0114] The embodiments of the present invention can be implemented based on the PCA algorithm. The PCA algorithm is used to perform dimensionality reduction on the initial target feature data set to be analyzed. Based on the dimensionality reduction result, a set of feature data to be analyzed that is highly correlated with the initial input error data set is obtained from the initial target feature data set to be analyzed. The set of feature data to be analyzed includes several features to be analyzed. The first target feature data set to be analyzed is obtained based on these features to be analyzed.

[0115] In practical applications, either the PCA algorithm or the Pearson algorithm can be selected to filter and process the Nth target feature data group to be analyzed, based on actual needs, to obtain the Nth target feature data group to be analyzed.

[0116] Further, the construction of the initial model, and the training of the initial model based on the acquired input error training set and the influencing factor training set to obtain the target error model, includes:

[0117] Based on the first target feature data set to be analyzed, obtain the first input error training set and the first influencing factor training set;

[0118] An initial model is constructed, and the initial model is trained based on the first input error training set and the first influencing factor training set to obtain a first target error model;

[0119] The first target error model is iteratively trained to obtain the (N-1)th target error model; where N≥2 and N is an integer;

[0120] Based on the Nth target feature data set to be analyzed, obtain the Nth influencing factor training set and the Nth input error training set;

[0121] The Nth target error model is obtained by training the Nth influencing factor training set and the Nth input error training set.

[0122] In an embodiment of the present invention, in order to improve the matching degree between the Nth target error model and the Nth input error data group and the Nth target feature data group to be analyzed, whenever the total number of types of features to be analyzed contained in the Nth target feature data group to be analyzed changes, it is necessary to train the current target error model. For example, the N-1th target error model is trained to obtain the Nth target error model.

[0123] In the embodiments of the present invention, a target error model is constructed. This target error model is independent of the original model and does not affect the operation of the other. Therefore, the target model in the embodiments of the present invention has strong independence, low interference, and high confidence.

[0124] In an optional embodiment of the present invention, the Nth objective error model can specifically be a regression model or a classification model;

[0125] If it is necessary to directly obtain the Nth output error data set based on the Nth objective error model, then the regression model is selected as the Nth objective error model; if it is only necessary to determine the satisfaction level of the estimated value set of the original model, then the classification model can be selected as the Nth objective error model.

[0126] For example, you can choose the Nth objective error model with structures such as LASSO, RF, and MLP.

[0127] During the training of the N-1th objective error model, it is necessary to determine the reliability of the training results in order to ensure the reliability of the obtained Nth objective error model.

[0128] Specifically, a first threshold is set. When the (N-1)th target error model is a regression model, the confidence of the training result is verified based on the MSE algorithm. This step is repeated until the confidence of the training result is greater than the first threshold. Then, the training result corresponding to the confidence is taken as the Nth target error model.

[0129] A second threshold is set. When the (N-1)th target error model is a classification model, the confidence of the training result is verified based on the F1 algorithm. This step is repeated until the confidence of the training result is greater than the second threshold. Then, the training result corresponding to the confidence is taken as the Nth target error model.

[0130] In the embodiments of the present invention, the training results are verified to ensure the confidence level of the Nth output error data group obtained based on the Nth target error model.

[0131] Step S2: Obtain the input error data set and the target feature data set to be analyzed; wherein the target feature data set to be analyzed includes at least one feature to be analyzed;

[0132] Furthermore, the target feature data set to be analyzed includes any one of the following:

[0133] M types of features to be analyzed;

[0134] M-1 types of features and one of the features to be analyzed;

[0135] One of the features to be analyzed;

[0136] Where M≥2, and M is an integer.

[0137] In practical applications, different categories of Nth target analysis feature arrays can be selected according to the actual situation;

[0138] For example: 1) When it is necessary to determine the magnitude of the influence of the gradient change of a feature to be analyzed on the error of the Nth target error model, the Nth target feature data group including the feature to be analyzed can be selected.

[0139] 2) When it is necessary to determine the influence of a certain feature among multiple features to be analyzed on the error of the Nth target error model, a target feature data set of the Nth target can be selected, including M-1 features and one feature to be analyzed. Specifically, the target feature data set of the Nth target includes multiple sets of data. In each set of data, one feature data is modified to obtain a set of feature data to be analyzed. This process is repeated to obtain multiple sets of feature data to be analyzed. Then, the influence of a certain feature to be analyzed on the error of the Nth target error model can be determined.

[0140] By repeating the above steps, the influence level of each feature to be analyzed on the error of the Nth objective error model can be obtained.

[0141] 3) When it is necessary to simultaneously determine the magnitude of the influence of each of the multiple features to be analyzed on the error of the Nth target error model, the Nth target feature data group including M features to be analyzed can be selected.

[0142] In embodiments of the present invention, the target feature data group to be analyzed includes multiple categories, which can enable the selection of different categories of target feature data groups to be analyzed according to different user needs, so as to improve the efficiency of data processing and the accuracy of processing results.

[0143] Step S3: Input the input error data set and the target feature data set to be analyzed into the target error model to obtain the output error data set;

[0144] Specifically, the step of inputting the input error data set and the target feature data set to be analyzed into the target error model to obtain the output error data set includes:

[0145] The Nth target feature data set to be analyzed and the Nth input error data set are input into the Nth target error model to obtain the Nth output error data set; or

[0146] The Nth dimension reduction target feature data set to be analyzed and the Nth input error data set are input into the Nth target error model to obtain the Nth output error data set.

[0147] Step S4: Process the output error data set to obtain the influence level of each of the features to be analyzed on the target error model.

[0148] Specifically, the process of processing the Nth output error data group to obtain the influence level of each of the analyzed feature data on the Nth target error model includes:

[0149] The Nth output error data set is analyzed based on the evaluation algorithm to obtain the influence value of each of the features to be analyzed on the Nth target error model;

[0150] Based on the influence value, the influence level of each of the features to be analyzed on the Nth objective error model is determined.

[0151] In an embodiment of the present invention, after obtaining the Nth input error data set, the Nth input error data set is analyzed based on an evaluation algorithm. The Nth output error data set includes multiple output error data sets. Specifically, the number of input error data sets corresponds to the number of input error data sets.

[0152] For example, the Nth output error data set can be analyzed and processed based on the PDP algorithm or the ICE algorithm. Specifically, it can be based on the following formula:

[0153]

[0154] In the formula, V represents the category of the feature to be analyzed, n represents the number of V-class feature data to be analyzed, and i represents the sequence number of the n V-class feature data to be analyzed; XS represents the feature to be evaluated; XC represents the remaining features to be analyzed; that is, the sum of XS and XC is the total number of types of features to be analyzed contained in the Nth target feature data group.

[0155] In practical applications, the PDP and ICE algorithms output the priority of the correlation between each feature to be analyzed and the Nth input error data. Based on this priority, the influence level of each feature to be analyzed on the Nth target error model can be determined, and correspondingly, the influence level of each feature to be analyzed on the original model can be obtained.

[0156] In an optional embodiment of the present invention, the Nth output error data group can also be analyzed and processed based on the SHAP algorithm to obtain the contribution of each feature to be analyzed to the Nth target error model; based on the contribution, the influence level of each feature to be analyzed on the Nth target error model can be obtained, and correspondingly, the influence level of each feature to be analyzed on the original model can be obtained.

[0157] Further, obtaining the Nth target error model includes:

[0158] Based on the influence level of each of the features to be analyzed on the error model of the Nth target, the data of the Nth target feature to be analyzed are filtered to obtain the data group of the N+1th target feature to be analyzed;

[0159] Based on the N+1th target feature data set to be analyzed and the N+1th input error data set, obtain the N+1th influencing factor training set and the N+1th input error training set;

[0160] The Nth target error model is trained for the N+1th time based on the N+1th influencing factor training set and the N+1th input error training set to obtain the N+1th target error model.

[0161] In embodiments of the present invention, after obtaining the influence level of each feature to be analyzed on the original model, multiple features to be analyzed in the Nth target feature data group can be screened based on the influence level according to actual needs. For example, based on the influence level, it is determined that time and temperature have a small impact on the error of the Nth target error model, with a low influence level and poor correlation. Therefore, in the subsequent analysis of the model error influencing factors, these two features are deleted and no longer analyzed.

[0162] For example, if the Nth target feature data set contains 8 features to be analyzed, after filtering out the temperature and time features, 6 features remain to be analyzed; that is, the N+1th target feature data set contains 6 features to be analyzed. Therefore, if the N+1th target feature data set is input into the Nth target error model for processing, the output result will have low confidence. Thus, it is necessary to obtain the N+1th influencing factor training set and the N+1th input error training set again, and perform the Nth training on the Nth target error model based on the N+1th influencing factor training set and the N+1th input error training set. The Nth training includes multiple training sessions, and the confidence of the training results is tested based on the MSE algorithm or the F1 algorithm until the confidence meets the requirements, thus obtaining the N+1th target error model.

[0163] In the embodiments of the present invention, the target error model is trained multiple times based on the types of features to be analyzed, which improves the confidence of the target error model and helps to improve the accuracy of the influence level of each feature to be analyzed on the error of the original model.

[0164] Embodiments of the present invention can be implemented based on the following methods:

[0165] For example, taking the remaining charging time data in a real-world application scenario as the input error data set, which consists of the true value set and the estimated value set:

[0166] The accuracy of the remaining charging time is affected by a variety of factors. In this embodiment of the invention, the relationship between the error between the estimated remaining time and the actual charging time and the factors that may affect the charging time is analyzed by using a model-free influencing factor analysis method and an error analysis method with the help of a target error model. The main factors affecting the error of the remaining charging time and the influence level of each factor are obtained.

[0167] Among them, based on business knowledge, we can identify factors that may affect the remaining charging time. These factors can be specifically divided into two categories.

[0168] The first category is physical factors, such as the battery's electrical, electrochemical, and thermal parameters; relevant data for these factors can be obtained from RTM.

[0169] The second category is strategic factors, such as the control of charging current and voltage by the control unit of the charging pile or the vehicle control device. The data of these factors can also be obtained from the RTM or calculated based on historical data. However, the method of calculating strategic factor data based on historical data is not within the scope of protection of this invention, so it will not be described in detail here.

[0170] Therefore, in the embodiments of the present invention, the first target feature data group to be analyzed can be selected from the above two types of factors based on business knowledge for analysis of at least one feature to be analyzed.

[0171] Specifically, the influencing factors of the original model of remaining charging time are identified and analyzed based on the Nth target feature data group in three categories:

[0172] 1) such as Figure 2 As shown, the first type of target N's feature data group to be analyzed is as follows:

[0173] Obtain an initial set of feature data to be analyzed. This initial set contains only one feature; for example, if the feature is temperature, the initial set includes multiple temperature values. Alternatively, based on a gradient change algorithm, obtain temperature gradient change data, such as 5 degrees, 10 degrees, 15 degrees, 20 degrees, 25 degrees, 30 degrees, 35 degrees, 40 degrees, ...

[0174] Then, the initial feature data set to be analyzed is filtered based on the Pearson algorithm. It is found that the temperature data at multiple gradients, such as 10 degrees, 15 degrees, 20 degrees, etc., are highly correlated with the initial input error data set. Based on the temperature data at multiple gradients, such as 10 degrees, 15 degrees, 20 degrees, etc., the first target feature data set to be analyzed is obtained.

[0175] An initial model is constructed. Based on the first target feature data set to be analyzed, a first input error training set and a first influencing factor training set are obtained. The initial model is then trained multiple times based on the first input error training set and the first influencing factor training set to obtain the first target error model.

[0176] The first target feature data set to be analyzed and the first input error data set are input into the first target error model, and the first output error data set is output.

[0177] The first output error data set is analyzed based on the PDP algorithm to determine the priority ranking of the influence of multiple temperature values ​​on the error of the first target error model.

[0178] Then, based on priority sorting, the first target feature data group to be analyzed is filtered, and the first 20 temperature values ​​are selected to obtain the second target feature data group to be analyzed.

[0179] Based on the second target feature data set to be analyzed, a second influencing factor training set and a second input error training set are obtained. Based on the second influencing factor training set and the second input error training set, the first target error model is trained multiple times to obtain the second target error model.

[0180] Input the second target feature data set to be analyzed into the second target error model, and output the second output error data set;

[0181] The second output error data set is analyzed based on the PDP algorithm to determine the priority ranking of the influence of multiple temperature values ​​on the error of the second target error model.

[0182] Then, based on priority sorting, the second target feature data group to be analyzed is filtered, and the first 19 temperature values ​​are selected to obtain the third target feature data group to be analyzed.

[0183] ...

[0184] Similarly, based on the Nth target error model, the Nth output error data set is output, and then the contribution of each temperature value in the Nth target feature data set to the Nth target error model is obtained based on the SHAP algorithm.

[0185] In addition, the threshold of N can be set according to actual needs;

[0186] If N equals the threshold value of N, then the process terminates;

[0187] Otherwise, let N = N + 1, continue iterating, and repeat the above process.

[0188] 2) such as Figure 3 As shown, the second type of target N's feature data group to be analyzed is as follows:

[0189] Obtain an initial set of feature data to be analyzed. This initial set of feature data includes only M-1 features and one feature to be analyzed. For example, if the feature to be analyzed is temperature, time, remaining charging capacity, maximum power of the charging pile, stability of the charging pile, ambient temperature, and temperature rise rate, then the initial set of feature data to be analyzed includes multiple sets of data, specifically: one set of M-1 features and data after temperature change, one set of M-1 features and data after time change, one set of M-1 features and data after remaining charging capacity change, one set of M-1 features and data after maximum power of the charging pile change, one set of M-1 features and data after charging pile stability change, one set of M-1 features and data after ambient temperature change, and so on.

[0190] Then, based on the Pearson algorithm or PCA algorithm, these data are filtered to obtain multiple features such as remaining charging capacity, maximum power of charging pile, and stability of charging pile that are highly correlated with the initial input error data set. Based on the data such as remaining charging capacity, maximum power of charging pile, and stability of charging pile, the first target feature data set to be analyzed is obtained.

[0191] An initial model is constructed. Based on the first target feature data set to be analyzed, a first input error training set and a first influencing factor training set are obtained. The initial model is then trained multiple times based on the first input error training set and the first influencing factor training set to obtain the first target error model.

[0192] The first target feature data set to be analyzed and the first input error data set are input into the first target error model, and the first output error data set is output.

[0193] The first output error data set is analyzed based on the PDP algorithm to determine the priority ranking of the influence of multiple temperature values ​​on the error of the first target error model.

[0194] Then, based on priority sorting, the first target feature data group to be analyzed is filtered, and the top 20 features are selected to obtain the second target feature data group to be analyzed; based on the second target feature data group to be analyzed, the second influencing factor training set and the second input error training set are obtained, and the first target error model is trained multiple times based on the second influencing factor training set and the second input error training set to obtain the second target error model;

[0195] Input the second target feature data set to be analyzed into the second target error model, and output the second output error data set;

[0196] The second output error data set is analyzed based on the PDP algorithm to determine the priority ranking of the impact of features such as remaining charging capacity, maximum power of charging pile, and stability of charging pile on the error of the second target error model.

[0197] Then, based on priority sorting, the second target feature data group to be analyzed is filtered, and the first 19 features are selected to obtain the third target feature data group to be analyzed.

[0198] ...

[0199] Similarly, based on the Nth target error model, the Nth output error data set is output, and then the contribution of each feature to be analyzed in the Nth target feature data set to the Nth target error model is obtained based on the SHAP algorithm.

[0200] In addition, the threshold of N can be set according to actual needs;

[0201] If N equals the threshold value of N, then the process terminates;

[0202] Otherwise, let N = N + 1, continue iterating, and repeat the above process.

[0203] 3) such as Figure 4 As shown, the third type of target N's feature data group to be analyzed is as follows:

[0204] Obtain the initial set of feature data to be analyzed, which includes only M types of features to be analyzed. For example, if the M types of features to be analyzed are temperature, time, remaining charging capacity, maximum power of charging pile, stability of charging pile, ambient temperature, and temperature rise rate, then the initial set of feature data to be analyzed includes data such as temperature, time, remaining charging capacity, maximum power of charging pile, stability of charging pile, ambient temperature, and temperature rise rate.

[0205] Then, based on the Pearson algorithm or PCA algorithm, these data are filtered to obtain multiple features such as remaining charging capacity, maximum power of charging pile, and stability of charging pile that are highly correlated with the initial input error data set. Based on the data such as remaining charging capacity, maximum power of charging pile, and stability of charging pile, the first target feature data set to be analyzed is obtained.

[0206] An initial model is constructed. Based on the first target feature data set to be analyzed, a first input error training set and a first influencing factor training set are obtained. The initial model is then trained multiple times based on the first input error training set and the first influencing factor training set to obtain the first target error model.

[0207] The first target feature data set to be analyzed and the first input error data set are input into the first target error model, and the first output error data set is output.

[0208] The first output error data set is analyzed based on the PDP algorithm to determine the priority ranking of the influence of multiple temperature values ​​on the error of the first target error model.

[0209] Then, based on priority sorting, the first target feature data group to be analyzed is filtered, and the top 20 features are selected to obtain the second target feature data group to be analyzed; based on the second target feature data group to be analyzed, the second influencing factor training set and the second input error training set are obtained, and the first target error model is trained multiple times based on the second influencing factor training set and the second input error training set to obtain the second target error model;

[0210] Input the second target feature data set to be analyzed into the second target error model, and output the second output error data set;

[0211] The second output error data set is analyzed based on the PDP algorithm to determine the priority ranking of the impact of features such as remaining charging capacity, maximum power of charging pile, and stability of charging pile on the error of the second target error model.

[0212] Then, based on priority sorting, the second target feature data group to be analyzed is filtered, and the first 19 features are selected to obtain the third target feature data group to be analyzed.

[0213] ...

[0214] Similarly, based on the Nth target error model, the Nth output error data set is output, and then the contribution of each of the multiple features to be analyzed in the Nth target feature data set to the Nth target error model is obtained based on the SHAP algorithm.

[0215] In addition, the threshold of N can be set according to actual needs;

[0216] If N equals the threshold value of N, then the process terminates;

[0217] Otherwise, let N = N + 1, continue iterating, and repeat the above process.

[0218] like Figure 5 As shown, embodiments of the present invention also provide a device 500 for identifying factors affecting model performance, comprising:

[0219] Training module 501 is used to construct an initial model and train the initial model based on the acquired input error training set and influencing factor training set to obtain a target error model;

[0220] The acquisition module 502 is used to acquire the input error data set and the target feature data set to be analyzed; wherein, the target feature data set to be analyzed includes at least one feature to be analyzed;

[0221] Output module 503 is used to input the input error data set and the target feature data set to be analyzed into the target error model to obtain the output error data set;

[0222] The calculation module 504 is used to process the output error data set to obtain the influence level of each of the features to be analyzed on the target error model.

[0223] Optionally, the target feature data set to be analyzed includes any one of the following:

[0224] M types of features to be analyzed;

[0225] M-1 types of features and one of the features to be analyzed;

[0226] One of the features to be analyzed;

[0227] Where M≥2, and M is an integer.

[0228] Optionally, before constructing the initial model, the following steps are included:

[0229] Obtain the initial set of feature data to be analyzed and the initial set of input error data;

[0230] Based on the correlation algorithm, the correlation between the initial feature data set to be analyzed and the initial input error data set is calculated, and the initial feature data set to be analyzed is filtered based on the correlation to obtain the first target feature data set to be analyzed.

[0231] Optionally, the step of calculating the correlation between the initial feature data set to be analyzed and the initial input error data set based on the correlation algorithm, and filtering the initial feature data set to be analyzed based on the correlation to obtain a first target feature data set to be analyzed, includes:

[0232] Calculate the correlation between each of the features to be analyzed and the initial input error data set;

[0233] Based on the relevance, the initial set of feature data to be analyzed is filtered to obtain a first target set of feature data to be analyzed; or

[0234] The initial set of feature data to be analyzed is subjected to dimensionality reduction processing to obtain an initial dimensionality-reduced set of feature data to be analyzed;

[0235] Calculate the dimensionality reduction correlation between the initial dimensionality reduction feature data set to be analyzed and the initial input error data set;

[0236] Based on the dimensionality reduction relevance, the dimensionality reduction feature data group to be analyzed is filtered to obtain the first target feature data group to be analyzed.

[0237] Optionally, the step of constructing an initial model and training the initial model based on the acquired input error training set and influencing factor training set to obtain a target error model includes:

[0238] Based on the first target feature data set to be analyzed, obtain the first input error training set and the first influencing factor training set;

[0239] An initial model is constructed, and the initial model is trained based on the first input error training set and the first influencing factor training set to obtain a first target error model;

[0240] The first target error model is iteratively trained to obtain the (N-1)th target error model; where N≥2 and N is an integer;

[0241] Based on the Nth target feature data set to be analyzed, obtain the Nth influencing factor training set and the Nth input error training set;

[0242] The Nth target error model is obtained by training the Nth influencing factor training set and the Nth input error training set.

[0243] Optionally, processing the output error data set to obtain the influence level of each of the analyzed feature data on the target error model includes:

[0244] The Nth output error data set is analyzed based on the evaluation algorithm to obtain the influence value of each of the features to be analyzed on the Nth target error model;

[0245] Based on the influence value, the influence level of each of the features to be analyzed on the Nth objective error model is determined.

[0246] Optionally, after determining the influence level of each of the features to be analyzed on the Nth target error model, the method further includes:

[0247] Based on the influence level of each of the features to be analyzed on the error model of the Nth target, the data of the Nth target feature to be analyzed are filtered to obtain the data group of the N+1th target feature to be analyzed;

[0248] Based on the N+1th target feature data set to be analyzed and the N+1th input error data set, obtain the N+1th influencing factor training set and the N+1th input error training set;

[0249] The Nth target error model is trained for the N+1th time based on the N+1th influencing factor training set and the N+1th input error training set to obtain the N+1th target error model.

[0250] Embodiments of the present invention also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the method described above.

[0251] Embodiments of the present invention also provide a computer-readable storage medium comprising a stored computer program, wherein the computer program, when executed, controls the device containing the computer-readable storage medium to perform the method described above.

[0252] Furthermore, other configurations and functions of the apparatus in the embodiments of the present invention are known to those skilled in the art, and will not be described in detail here to reduce redundancy.

[0253] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0254] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0255] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0256] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0257] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0258] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0259] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0260] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for identifying factors affecting model performance, characterized in that, The method, applied in the field of electric vehicle management, includes: An initial model is constructed, and the initial model is trained based on the obtained input error training set and influencing factor training set to obtain the target error model; The system acquires an input error data set and a target feature data set to be analyzed; wherein the target feature data set to be analyzed includes at least one feature to be analyzed; the input error data set and the target feature data set to be analyzed are input into the target error model to obtain an output error data set; the features to be analyzed in the target feature data set to be analyzed include temperature, time, remaining charging capacity, maximum power of the charging pile, stability of the charging pile, ambient temperature, and temperature rise rate; the input error data set includes multiple sets of data, each set of data consisting of a true value and an estimated value corresponding to the true value; The output error data set is processed to obtain the influence level of each of the analyzed feature data on the target error model; The process of processing the output error data set to obtain the influence level of each of the analyzed feature data on the target error model includes: The Nth output error data set is analyzed based on the evaluation algorithm to obtain the influence value of each of the features to be analyzed on the Nth target error model; Based on the influence value, determine the influence level of each of the features to be analyzed on the Nth target error model; After determining the influence level of each of the features to be analyzed on the Nth target error model, the method further includes: Based on the influence level of each of the features to be analyzed on the error model of the Nth target, the data of the Nth target feature to be analyzed are filtered to obtain the data group of the N+1th target feature to be analyzed; Based on the N+1th target feature data set to be analyzed and the N+1th input error data set, obtain the N+1th influencing factor training set and the N+1th input error training set; The Nth target error model is trained for the N+1th time based on the N+1th influencing factor training set and the N+1th input error training set to obtain the N+1th target error model.

2. The method according to claim 1, characterized in that, The target feature data set to be analyzed includes any one of the following: M types of features to be analyzed; M-1 types of features and one of the features to be analyzed; One of the features to be analyzed; Where M≥2, and M is an integer.

3. The method according to claim 1, characterized in that, Before constructing the initial model, the following is also included: Obtain the initial set of feature data to be analyzed and the initial set of input error data; Based on the correlation algorithm, the correlation between the initial feature data set to be analyzed and the initial input error data set is calculated, and the initial feature data set to be analyzed is filtered based on the correlation to obtain the first target feature data set to be analyzed.

4. The method according to claim 3, characterized in that, The process of calculating the correlation between the initial set of feature data to be analyzed and the initial set of input error data based on a correlation algorithm, and then filtering the initial set of feature data to be analyzed based on the correlation to obtain a first target set of feature data to be analyzed, includes: Calculate the correlation between each of the features to be analyzed and the initial input error data set; Based on the relevance, the initial set of feature data to be analyzed is filtered to obtain a first target set of feature data to be analyzed; or The initial set of feature data to be analyzed is subjected to dimensionality reduction processing to obtain an initial dimensionality-reduced set of feature data to be analyzed; Calculate the dimensionality reduction correlation between the initial dimensionality reduction feature data set to be analyzed and the initial input error data set; Based on the dimensionality reduction relevance, the dimensionality reduction feature data group to be analyzed is filtered to obtain the first target feature data group to be analyzed.

5. The method according to claim 3, characterized in that, The process of constructing an initial model and training the initial model based on the acquired input error training set and influencing factor training set to obtain a target error model includes: Based on the first target feature data set to be analyzed, obtain the first input error training set and the first influencing factor training set; An initial model is constructed, and the initial model is trained based on the first input error training set and the first influencing factor training set to obtain a first target error model; The first target error model is iteratively trained to obtain the (N-1)th target error model; where N≥2 and N is an integer; Based on the Nth target feature data set to be analyzed, obtain the Nth influencing factor training set and the Nth input error training set; The Nth target error model is obtained by training the Nth influencing factor training set and the Nth input error training set.

6. A device for identifying factors affecting model performance, characterized in that, The device, applied in the field of electric vehicle management, includes: The training module is used to build an initial model and train the initial model based on the acquired input error training set and influencing factor training set to obtain the target error model. The acquisition module is used to acquire an input error data set and a target feature data set to be analyzed; wherein, the target feature data set to be analyzed includes at least one feature to be analyzed; the features to be analyzed in the target feature data set to be analyzed include temperature, time, remaining charging capacity, maximum power of the charging pile, stability of the charging pile, ambient temperature, and temperature rise rate; the input error data set includes multiple sets of data, each set of data consisting of a true value and an estimated value corresponding to the true value; The output module is used to input the input error data set and the target feature data set to be analyzed into the target error model to obtain the output error data set; The calculation module is used to process the output error data set to obtain the influence level of each of the analyzed feature data on the target error model; The process of processing the output error data set to obtain the influence level of each of the analyzed feature data on the target error model includes: The Nth output error data set is analyzed based on the evaluation algorithm to obtain the influence value of each of the features to be analyzed on the Nth target error model; Based on the influence value, determine the influence level of each of the features to be analyzed on the Nth target error model; After determining the influence level of each of the analyzed features on the Nth target error model, the method further includes: filtering the Nth target analyzed feature data based on the influence level of each of the analyzed features on the Nth target error model to obtain the (N+1)th target analyzed feature data set; obtaining the (N+1)th influencing factor training set and the (N+1)th input error training set based on the (N+1)th influencing factor training set and the (N+1)th input error training set; and performing the (N+1)th training on the Nth target error model based on the (N+1)th influencing factor training set and the (N+1)th input error training set to obtain the (N+1)th target error model.

7. An electronic device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the method as claimed in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the method as described in any one of claims 1 to 5.

Images