Neural network-based performance evaluation method, system, device, and storage medium

Abstract

This invention proposes a neural network-based performance evaluation method, system, device, and storage medium. The neural network-based performance evaluation method includes: acquiring a preset neural network model; acquiring preset parameters and configuring the parameters of each neuron in the preset neural network model according to the preset parameters; acquiring training samples and modifying the parameters in the preset neural network model according to the training samples to reduce the loss function value, thereby obtaining a performance evaluation model; acquiring detection parameters and inputting the detection parameters into the performance evaluation model to obtain the performance evaluation result. This application helps improve the accuracy of performance evaluation.

Description

Technical Field

[0001] This invention relates to the field of performance evaluation, and in particular to a performance evaluation method, system, device and storage medium based on neural networks. Background Technology

[0002] Performance evaluation has been applied to all stages of product development. Generally, performance refers to the probability that a system or device can meet operational requirements under specified operating conditions and within a specified time.

[0003] Performance evaluation methods are very diverse, including analytical methods, index methods, statistical methods, computer simulation methods, and discussion methods. Performance evaluation involves not only the theories and methods of analysis and evaluation, but also needs analysis and scenario description, as well as system architecture and model framework issues. Therefore, performance evaluation is a very broad and very challenging research topic. Summary of the Invention

[0004] This invention proposes a performance evaluation method, system, device, and storage medium based on neural networks, which is beneficial for obtaining accurate performance evaluation results.

[0005] Firstly, a performance evaluation method based on neural networks is provided, including:

[0006] Obtain the preset neural network model;

[0007] Obtain preset parameters, and configure the parameters of each neuron in the preset neural network model according to the preset parameters;

[0008] Obtain training samples, modify the parameters in the preset neural network model based on the training samples to reduce the loss function value, and obtain the performance evaluation model;

[0009] Obtain the detection parameters, input the detection parameters into the performance evaluation model, and obtain the performance evaluation results.

[0010] In one embodiment, the performance evaluation method based on neural networks includes obtaining preset parameters, which includes:

[0011] Determine the parameters corresponding to the neurons in the preset neural network model to obtain parameter information;

[0012] Based on the parameter information and according to the preset parameter generation rules, multiple sets of parameter values ​​are generated, and each set of parameter values ​​is arranged in a preset order to obtain multiple solution vectors;

[0013] Establish an objective function, wherein the larger the objective function value of the solution vector, the closer the solution vector is to the optimal solution vector;

[0014] The multiple solution vectors are iteratively evolved multiple times based on the objective function to obtain multiple solution vectors after iterative evolution;

[0015] From the multiple solution vectors after iterative evolution, select the solution vector with the largest objective function value to obtain the optimized solution vector;

[0016] The preset parameters are determined based on the parameter values ​​in the optimized solution vector;

[0017] The iterative evolution includes:

[0018] Obtain multiple solution vectors for the current loop, determine the selection probability based on the objective function value of each solution vector, select a preset number of solution vectors based on the selection probability, and obtain multiple target solution vectors, wherein the selection probability is positively correlated with the objective function value;

[0019] Among the multiple target solution vectors, the corresponding parameter values ​​of two selected target solution vectors are swapped according to the crossover probability to obtain multiple target solution vectors after swapping. The crossover probability is negatively correlated with the target function value.

[0020] In the multiple target solution vectors after the exchange, the parameter values ​​in the target solution vectors are replaced with new parameter values ​​according to the replacement probability to obtain multiple solution vectors for the next cycle.

[0021] In one embodiment, the neural network-based performance evaluation method, wherein obtaining training samples includes:

[0022] Obtain initial training samples and analyze them to determine their features;

[0023] New samples are generated based on the features of the initial training samples;

[0024] The training samples are obtained based on the initial training samples and the new samples.

[0025] In one embodiment, the neural network-based performance evaluation method includes obtaining initial training samples and analyzing the initial training samples to determine their features, including:

[0026] Clustering is performed on the initial training samples to obtain initial training samples of multiple categories;

[0027] The features of the initial training samples are determined based on their categories.

[0028] In one embodiment, the neural network-based performance evaluation method, wherein generating new samples based on the features of the initial training samples includes:

[0029] Obtain the number of initial training samples in each category, and calculate the ratio of the number of initial training samples in each category to the total number of initial training samples to obtain the proportion of each category;

[0030] According to the proportion of each category, the initial training samples are randomly copied from the initial training samples of each category to obtain new samples.

[0031] In one embodiment, the neural network-based performance evaluation method, wherein generating new samples based on the features of the initial training samples includes:

[0032] The initial training samples for each category are copied separately to obtain intermediate samples;

[0033] Calculate the variance of the intermediate samples for each category to obtain the category variance;

[0034] For each intermediate sample, a random number is generated, and the average absolute value of the random number is less than or equal to the class variance corresponding to the intermediate sample;

[0035] The intermediate sample is added to the corresponding random number to obtain a new sample.

[0036] In one embodiment, the neural network-based performance evaluation method, wherein modifying the parameters of the preset neural network model based on the training samples to reduce the loss function value and obtain the performance evaluation model includes:

[0037] The training samples are divided into a first class of samples and a second class of samples;

[0038] The first type of sample is input into the preset neural network model to obtain multiple first output values;

[0039] Multiple first loss function values ​​are determined based on multiple first output values ​​and their corresponding true values, and the parameters in the preset neural network model are adjusted in the direction of decreasing first loss function values.

[0040] The second type of sample is input into the neural network model to obtain multiple second output values;

[0041] The second loss function value is determined based on the average of the multiple second output values ​​and the average of the multiple corresponding true values, and the parameters in the preset neural network model are adjusted in the direction of decreasing the second loss function value.

[0042] Secondly, a performance evaluation system based on neural networks is provided, comprising:

[0043] The acquisition module is used to acquire a preset neural network model, preset parameters, training samples, and detection parameters.

[0044] A configuration module is used to configure the parameters of each neuron in a preset neural network model according to the preset parameters;

[0045] The training module is used to modify the parameters in the preset neural network model based on the training samples, so as to reduce the loss function value and obtain the performance evaluation model.

[0046] The evaluation module is used to input the detection parameters into the performance evaluation model to obtain the performance evaluation results.

[0047] Thirdly, an electronic device is provided, including a memory and a processor, the memory storing a computer program executable on the processor, wherein when the processor executes the program, the steps of the neural network-based performance evaluation method described above are implemented.

[0048] Fourthly, a storage medium is provided on which a computer program is stored, wherein when the computer program is executed by a processor, it implements the steps of the neural network-based performance evaluation method as described above.

[0049] This invention trains a pre-defined neural network model to obtain a performance evaluation model, and then uses this model to perform performance evaluation. By employing a performance evaluation model, this invention helps reduce errors caused by subjective evaluation indicators, thereby improving the accuracy of performance evaluation. Attached Figure Description

[0050] Various other advantages and benefits of the present invention will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0051] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0052] Figure 1 This is a flowchart of a neural network-based performance evaluation method according to an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the structure of a neural network-based performance evaluation system according to an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Implementation

[0055] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0057] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0058] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Example

[0059] Figure 1 This is a flowchart illustrating the performance evaluation method based on neural networks in this implementation. Please refer to [link / reference]. Figure 1 The performance evaluation method based on neural networks includes steps 10, 20, 30 and 40.

[0060] Step 10: Obtain the preset neural network model.

[0061] The preset neural network model is a deep learning-based neural network. Optionally, the preset neural network model is a backpropagation (BP) neural network. The structure of a BP neural network consists of an input layer, an output layer, and multiple hidden layers. In a three-layer BP network, the input layer contains x neurons, the hidden layers contain y neurons, and the output layer contains z neurons, where x, y, and z are all positive integers. It has been proven that a three-layer BP neural network can fit any complex function by adjusting the number of neurons in the hidden layers.

[0062] Step 20: Obtain preset parameters and configure the parameters of each neuron in the preset neural network model according to the preset parameters.

[0063] In this embodiment, the initial parameters in the preset neural network model are not random, but preset. Specifically, the parameters of each neuron in the preset neural network model are modified to the corresponding preset parameters to obtain a preset neural network model with optimized initial parameters. Then, the preset neural network model is trained based on the preset neural network model with optimized initial parameters.

[0064] Optionally, different parameter acquisition rules can be determined according to different objectives, and preset parameters can be obtained according to the parameter acquisition rules. For example, based on the objective of finding the optimal parameters globally, preset parameters can be generated through certain parameter generation rules, and a preset neural network model can be configured to reduce the defects of local minima. As another example, based on the objective of improving convergence speed, preset parameters can be generated through certain parameter generation rules, and a preset neural network model can be configured to improve the training convergence speed.

[0065] For example, the operation of a relatively simple neuron can be expressed by the following formula: y = wx + b, where y is the output value of the neuron, x is the input value, w is the weight, and b is the threshold. The preset parameters refer to the weight w and the threshold b; that is, they are set by setting the initial weight w and the initial threshold b in the preset neural network model as preset parameters.

[0066] Step 30: Obtain training samples, modify the parameters in the preset neural network model based on the training samples to reduce the loss function value, and obtain the performance evaluation model.

[0067] The loss function value represents the degree of deviation between the true value and the predicted value. The greater the difference between the true value and the predicted value, the larger the loss function value; the smaller the difference between the true value and the predicted value, the smaller the loss function value.

[0068] Modifying the parameters in the preset neural network model based on training samples involves training the preset neural network model multiple times with training samples, modifying the preset neural network model multiple times, and continuously optimizing the parameters in the preset neural network model so that the difference between the real value and the predicted value is continuously reduced.

[0069] The preset neural network model is trained using training samples. The parameters in the preset neural network model are modified to make the predicted values ​​closer to the true values. The corresponding loss function value is continuously reduced, and finally the trained neural network model, i.e. the performance evaluation model, is obtained.

[0070] Step 40: Obtain the detection parameters, input the detection parameters into the performance evaluation model, and obtain the performance evaluation results.

[0071] Detection parameters are typically test or simulation data of the object being evaluated, used to assess the object's effectiveness. Optionally, the detection parameters are performance parameters being evaluated; for example, in vehicle transportation effectiveness evaluation, the detection parameters could be the vehicle's maneuverability parameters. By inputting the detection parameters into the effectiveness evaluation model, the effectiveness evaluation results are output.

[0072] Furthermore, in some scenarios, performance evaluation models may require input indicator parameters. Performance evaluation models can analyze the performance of the evaluated object based on indicator parameters and detection parameters, and output performance evaluation results.

[0073] This invention trains a pre-defined neural network model to obtain a performance evaluation model, and then uses this model to perform performance evaluation. By employing a performance evaluation model, this invention helps reduce errors caused by subjective evaluation indicators, thereby improving the accuracy of performance evaluation.

[0074] In one embodiment, obtaining preset parameters includes steps 210, 220, 230, 240, 250, and 260.

[0075] Step 210: Determine the parameters corresponding to the neurons in the preset neural network model and obtain parameter information.

[0076] Identify which parameters in the pre-defined neural network model need optimization, and compile the parameter information for these parameters. The parameter information can be a set of parameters arranged in a pre-defined order.

[0077] Step 220: Generate multiple sets of parameter values ​​based on the parameter information and according to the preset parameter generation rules, and arrange each set of parameter values ​​in a preset order to obtain multiple solution vectors.

[0078] Each set of parameter values ​​corresponds to all the parameter values ​​of a preset neural network model. Multiple sets of parameter values ​​can range from 5 to 1000 sets, such as 30 sets.

[0079] The preset parameter generation rules can be to randomly generate multiple sets of parameter values ​​within the parameter value range, or to generate multiple sets of parameter values ​​within the parameter value range according to the arithmetic progression rule.

[0080] Optionally, during the process of arranging multiple parameter values ​​in each set of parameter values ​​in a preset order, the multiple parameter values ​​are sorted according to the position of neurons in the preset neural network model. For example, the parameters of the input layer are arranged first, the parameters of the hidden layer are arranged in the middle, and the parameters of the output layer are arranged last.

[0081] The solution vector can take the following form: [x1, x2, x3, y1, y2, y3, z1, z2, z3]. Here, x1, x2, and x3 represent three parameters in the input layer, y1, y2, and y3 represent three parameters in the hidden layer, and z1, z2, and z3 represent three parameters in the output layer. x1, x2, x3, y1, y2, y3, z1, z2, and z3 are all randomly generated within their respective value ranges. In essence, each solution vector corresponds to a set of parameter values.

[0082] Step 230: Establish the objective function, where the larger the objective function value of the solution vector, the closer the solution vector is to the optimal solution vector.

[0083] The objective function is used to evaluate the solution vector, and it is generally a non-negative value. For example, the closer the solution vector is to the optimal solution vector, the higher the score of the solution vector.

[0084] Optionally, when the optimal solution vector is the solution vector with the largest value for each parameter, the objective function can be the sum of squares of each parameter value.

[0085] Of course, in some other embodiments, an objective function is established, wherein the smaller the objective function value of the solution vector, the closer the solution vector is to the optimal solution vector.

[0086] Step 240: Iterate and evolve multiple solution vectors multiple times according to the objective function to obtain multiple solution vectors after iterative evolution.

[0087] During the iterative evolution process, new vector solutions are continuously generated by performing operations such as crossover and mutation on the solution vectors. The optimal solution vectors are then selected based on the objective function to obtain multiple solution vectors after iterative evolution.

[0088] Step 250: Select the solution vector with the largest objective function value from the multiple solution vectors after iterative evolution to obtain the optimized solution vector.

[0089] Calculate the objective function value for each solution vector, and select the solution vector with the largest value as the optimized solution vector.

[0090] Step 260: Determine the preset parameters based on the parameter values ​​in the optimized solution vector.

[0091] It is understandable that the parameter values ​​in the optimized solution vector are the preset parameters.

[0092] Optionally, the iterative evolution includes steps 241, 242, and 243.

[0093] Step 241: Obtain multiple solution vectors for the current loop, determine the selection probability based on the objective function value of each solution vector, select a preset number of solution vectors based on the selection probability, and obtain multiple objective solution vectors. The selection probability is positively correlated with the objective function value.

[0094] The multiple solution vectors in the current loop can be the solution vectors output by the previous loop, or they can be the initial solution vectors.

[0095] Based on the selection probability, a subset of solution vectors are selected from the multiple solution vectors in the current loop to obtain multiple target solution vectors. In other words, the number of target solution vectors is less than the number of solution vectors in the current loop.

[0096] A larger objective function value results in a higher probability of selection. Optionally, the probability of selection is directly proportional to the objective function value.

[0097] For example, the selection probabilities are as follows:

[0098] P i Let T be the selection probability of the i-th solution vector. i Let T be the objective function value of the i-th solution vector, and n be the total number of solution vectors. j Let be the objective function value of the j-th solution vector.

[0099] Step 242: Among the multiple target solution vectors, the corresponding parameter values ​​of two selected target solution vectors are swapped according to the crossover probability to obtain multiple target solution vectors after swapping. The crossover probability is negatively correlated with the target function value.

[0100] The target solution vectors are paired up, and the parameter values ​​of the paired target solution vectors are swapped.

[0101] Optionally, during the pairing of target solution vectors, the probability of successful pairing is determined based on the objective function value. The closer the objective function values ​​of two target solution vectors are, the higher the probability of successful pairing. The probability of success is less than 1.

[0102] Optionally, the crossover probability can be set as needed. During the process of swapping corresponding parameter values ​​in two selected target solution vectors, the crossover probability is first used to determine whether the parameters of the target solution vectors will be swapped, and then the parameters that will be swapped are moved to different positions.

[0103] Optionally, the crossover probability is negatively correlated with the objective function value. The crossover probability is inversely proportional to the objective function value. For solution vectors with larger objective function values, parameter swapping is less likely.

[0104] Step 243: In the multiple target solution vectors after the exchange, the parameter values ​​in the target solution vectors are replaced with new parameter values ​​according to the replacement probability to obtain multiple solution vectors for the next cycle.

[0105] Optionally, the new parameter value is randomly generated within the corresponding parameter range, or calculated using the following formula.

[0106] A=[α×A a ×(1-P)+β×A b ×P]÷2

[0107] Where A is the new parameter value, α is the first weighting coefficient, and β is the second weighting coefficient. a Let P be the selection probability, and A be the parameter values ​​randomly generated within the corresponding parameter range. b The original parameter values ​​are: α + β = 1, 0 ≤ α ≤ 1, 0 ≤ β ≤ 1, for example, 0.1 ≤ α ≤ 0.9. α and β can be set as needed, or they can be a function calculation formula.

[0108] Optionally, the replacement probability can be randomly generated or negatively correlated with the objective function value. The replacement probability is inversely proportional to the objective function value. For solution vectors with larger objective function values, parameter replacement is less likely to occur.

[0109] Optionally, the replacement probability can be set as needed. During the process of swapping corresponding parameter values ​​in two selected target solution vectors, the replacement probability is first used to determine whether each parameter of the target solution vector will be replaced. Then, the new parameter value replaces the parameter that will be replaced, while the remaining parameters remain unchanged.

[0110] In one embodiment, the performance evaluation method based on neural networks includes obtaining training samples, comprising steps 310, 320, and 330.

[0111] Step 310: Obtain the initial training samples and analyze them to determine their features.

[0112] The initial training samples are the raw data obtained by testing or detecting the evaluation object, such as vehicle motion performance parameters.

[0113] Optionally, the parameters of the same type in the initial training samples are analyzed to determine the distribution characteristics of the parameters or other parameter characteristics, thereby obtaining the characteristics of the initial training samples. For example, the parameters in the initial training samples exhibit a normal distribution.

[0114] Step 320: Generate new samples based on the features of the initial training samples.

[0115] New samples are generated based on the features of the initial training samples, ensuring that the new samples have the same or similar features as the initial training samples. For example, if the initial training samples exhibit a normal distribution, then the resulting new samples will also exhibit a normal distribution.

[0116] Step 330: Based on the initial training samples and the new samples, obtain the training samples.

[0117] Optionally, the initial training samples and the new samples can be merged to form a training sample.

[0118] Optionally, the initial training samples are used as a group of samples, and the new samples are divided into one or more groups to obtain new grouped samples, such as the second group of samples, the third group of samples, etc. The initial training samples and the new grouped samples are then merged according to the groups to form the grouped training samples.

[0119] In one implementation, initial training samples are obtained and analyzed to determine the features of the initial training samples, including steps 311 and 312.

[0120] Step 311: Cluster the initial training samples to obtain initial training samples of multiple categories.

[0121] The initial training samples are classified by performing cluster analysis on parameters of the same type. For example, clustering is performed based on the similarity between initial training samples, grouping initial training samples with high similarity into the same category.

[0122] Step 312: Determine the features of the initial training samples based on their categories.

[0123] The initial training sample features are the classification information of the initial training samples. For example, the initial training sample features might include five categories and the distribution of the number of samples in each of the five categories.

[0124] In one implementation, new samples are generated based on the features of the initial training samples, including steps 321 and 322.

[0125] Step 321: Obtain the number of initial training samples in each category, and calculate the ratio of the number of initial training samples in each category to the total number of initial training samples to obtain the proportion of each category.

[0126] Optionally, the formula for calculating the proportion of each category is as follows:

[0127] Where B is the proportion of the k-th category, and N k Let N be the number of initial training samples for the k-th category, and N be the total number of initial training samples.

[0128] Step 322: Randomly copy the initial training samples from the initial training samples of each category according to the proportion of each category to obtain new samples.

[0129] For example, the number of new samples in each category is calculated based on the proportion of each category. Then, new samples are copied from the initial training samples of the corresponding category according to the number of new samples in each category. The proportion of each category in the new samples is the same as the proportion of each category in the initial training samples.

[0130] In other words, the new sample has the same or similar class features as the initial training sample.

[0131] In one implementation, new samples are generated based on the features of the initial training samples, including steps 323, 324, 325, and 326.

[0132] Step 323: Copy the initial training samples of each category to obtain intermediate samples.

[0133] Optionally, intermediate samples can be obtained in accordance with steps 321 and 322 to obtain intermediate samples with the same features as the initial training samples; alternatively, the initial training samples can be directly copied to obtain intermediate samples that are exactly the same as the initial training samples.

[0134] Step 324: Calculate the variance of the intermediate samples for each category to obtain the category variance.

[0135] Based on the replication and classification of the initial training samples, the classification of the intermediate samples can be obtained. Then, the variance of the intermediate samples for each category is calculated to obtain the category variance.

[0136] Step 325: Obtain a random number for each intermediate sample. The average absolute value of the random number is less than or equal to the class variance corresponding to the intermediate sample.

[0137] The formula for generating random numbers is shown below.

[0138] |R m |≤δ n

[0139] Among them, R m Let δ be a random number for the m-th intermediate sample. n Let be the class variance of the nth class.

[0140] Step 326: Add the intermediate sample to the corresponding random number to obtain a new sample.

[0141] Specifically, the m-th intermediate sample and its corresponding random number R m The samples are added together to obtain a new sample. It is evident that the new sample may differ from the initial training sample.

[0142] In one implementation, the parameters in a preset neural network model are modified based on training samples to reduce the loss function value, thereby obtaining a performance evaluation model, including:

[0143] Step 341: Divide the training samples into Class I samples and Class II samples.

[0144] Optionally, the first type of sample consists of 80% of the training samples, and the second type of sample consists of 20% of the training samples.

[0145] Step 342: Input the first type of sample into the preset neural network model to obtain multiple first output values.

[0146] The first type of samples can be divided into multiple batches, which are then input into a pre-defined neural network model according to the batches. Each first type of sample corresponds to a first output value.

[0147] Step 343: Determine multiple first loss function values ​​based on the multiple first output values ​​and their corresponding true values, and adjust the parameters in the preset neural network model in the direction of decreasing first loss function values.

[0148] The first loss function value is calculated by substituting multiple first output values ​​and their corresponding true values ​​into the loss function.

[0149] Step 344: Input the second type of samples into the neural network model to obtain multiple second output values.

[0150] The second type of samples can be divided into multiple batches and input into a preset neural network model according to the batches. Each second type of sample corresponds to a second output value.

[0151] Step 345: Determine the second loss function value based on the average of multiple second output values ​​and the average of multiple corresponding true values, and adjust the parameters in the preset neural network model in the direction of decreasing second loss function value.

[0152] In other words, the average of the second output value and the average of multiple corresponding true values ​​are substituted into the loss function to calculate the second loss function value. Optionally, by grouping, the average of the output values ​​of each group can be obtained, thereby determining multiple second loss function values.

[0153] By adjusting the parameters in the preset neural network model to make the loss function converge, the training of the preset neural network model is completed, and the performance evaluation model is obtained. Example

[0154] Figure 2 This is a schematic diagram of the structure of a performance evaluation system based on a neural network according to this embodiment, as shown below. Figure 2 As shown, the neural network-based performance evaluation system 50 includes: an acquisition module 501, a configuration module 502, a training module 503, and an evaluation module 504.

[0155] The acquisition module 501 is used to acquire a preset neural network model, preset parameters, training samples, and detection parameters.

[0156] The configuration module 502 is used to configure the parameters of each neuron in the preset neural network model according to preset parameters.

[0157] The training module 503 is used to modify the parameters in the preset neural network model based on the training samples, so as to reduce the loss function value and obtain the performance evaluation model.

[0158] The evaluation module 504 is used to input the detection parameters into the performance evaluation model to obtain the performance evaluation results.

[0159] This invention trains a pre-defined neural network model to obtain a performance evaluation model, and then uses this model to perform performance evaluation. By employing a performance evaluation model, this invention helps reduce errors caused by subjective evaluation indicators, thereby improving the accuracy of performance evaluation.

[0160] In one embodiment, the acquisition module 501 is further configured to determine the parameters corresponding to the neurons in the preset neural network model and obtain parameter information; generate multiple sets of parameter values ​​according to the parameter information and according to the preset parameter generation rules, and arrange each set of parameter values ​​in a preset order to obtain multiple solution vectors; establish an objective function, wherein the larger the objective function value of the solution vector, the closer the solution vector is to the optimal solution vector; iterate and evolve the multiple solution vectors multiple times according to the objective function to obtain multiple solution vectors after iterative evolution; select the solution vector with the largest objective function value from the multiple solution vectors after iterative evolution to obtain the optimized solution vector; and determine the preset parameters according to the parameter values ​​in the optimized solution vector.

[0161] Optionally, the acquisition module 501 is further configured to acquire multiple solution vectors of the current loop, determine the selection probability based on the objective function value of each solution vector, select a preset number of solution vectors based on the selection probability, and obtain multiple target solution vectors, wherein the selection probability is positively correlated with the objective function value; among the multiple target solution vectors, the corresponding parameter values ​​of two selected target solution vectors are swapped according to the crossover probability, and multiple target solution vectors after swapping are obtained, wherein the crossover probability is negatively correlated with the objective function value; among the multiple target solution vectors after swapping, the parameter values ​​in the target solution vectors are replaced with new parameter values ​​according to the replacement probability, and multiple solution vectors of the next loop are obtained.

[0162] In one embodiment, the acquisition module 501 is further configured to acquire initial training samples, analyze the initial training samples, determine the features of the initial training samples, generate new samples based on the features of the initial training samples, and obtain training samples based on the initial training samples and the new samples.

[0163] In one embodiment, the acquisition module 501 is further configured to perform clustering processing on the initial training samples to obtain initial training samples of multiple categories; and determine the features of the initial training samples based on the categories of the initial training samples.

[0164] In one embodiment, the acquisition module 501 is further configured to acquire the number of initial training samples in each category, calculate the ratio of the number of initial training samples in each category to the total number of initial training samples, and obtain the proportion of each category; and randomly copy the initial training samples from the initial training samples of each category according to the proportion of each category to obtain new samples.

[0165] In one embodiment, the acquisition module 501 is further configured to copy the initial training samples of each category to obtain intermediate samples; calculate the variance corresponding to the intermediate samples of each category to obtain the category variance; obtain a random number for each intermediate sample, wherein the average absolute value of the random number is less than or equal to the category variance corresponding to the intermediate sample; and add the intermediate sample to the corresponding random number to obtain a new sample.

[0166] In one embodiment, the training module 503 is further configured to divide the training samples into a first type of samples and a second type of samples; input the first type of samples into a preset neural network model to obtain multiple first output values; determine multiple first loss function values ​​based on the multiple first output values ​​and their corresponding true values, and adjust the parameters in the preset neural network model in the direction of decreasing first loss function values; input the second type of samples into the neural network model to obtain multiple second output values; determine a second loss function value based on the average of the multiple second output values ​​and the average of the multiple corresponding true values, and adjust the parameters in the preset neural network model in the direction of decreasing second loss function values.

[0167] The neural network performance evaluation system 50 in this embodiment is a system corresponding to the neural network performance evaluation method. The operating principle of the neural network performance evaluation system 50 can be referred to the neural network performance evaluation system 50 method described above, and will not be repeated here. Example

[0168] Figure 3 This is a schematic diagram of the structure of an electronic device according to the present invention. The electronic device includes a memory 601 and a processor 602. The memory 601 stores a computer program that can run on the processor 602, wherein when the processor 602 executes the program, it implements the steps of the above-described neural network-based performance evaluation method.

[0169] The electronic device includes a memory 601 and a processor 602 that are interconnected via a system bus 603. It should be noted that only an electronic device with components 601-603 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components may be implemented alternatively. Those skilled in the art will understand that the electronic device described herein is one capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0170] Electronic devices can be computing devices such as desktop computers, laptops, PDAs, and cloud servers. These devices can interact with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0171] The memory 601 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 601 may be an internal storage unit of the device, such as the hard disk or memory of the device. In other embodiments, the memory 601 may also be an external storage device of the device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Of course, the memory 601 may include both internal storage units and external storage devices of the device. In this embodiment, the memory 601 is typically used to store the operating system and various application software installed on the device. Furthermore, the memory 601 can also be used to temporarily store various types of data that have been output or will be output.

[0172] In some embodiments, the processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is typically used to control the overall operation of the device. In this embodiment, the processor is used to execute computer-readable instructions stored in memory or to process data. Example

[0173] The present invention provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described neural network-based performance evaluation method.

[0174] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this application.

[0175] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A performance evaluation method based on neural networks, characterized in that, include: Obtain the preset neural network model; Obtain preset parameters, and configure the parameters of each neuron in the preset neural network model according to the preset parameters; Obtain training samples, modify the parameters in the preset neural network model based on the training samples to reduce the loss function value, and obtain the performance evaluation model; Obtain the detection parameters, input the detection parameters into the performance evaluation model, and obtain the performance evaluation results; The performance evaluation involves all stages of product development. Performance refers to the probability that a system or device will meet the application requirements under specified working conditions and within a specified time. The detection parameters are test data or simulation data of the evaluation object, used to evaluate the performance of the evaluation object; The detection parameters are the performance parameters being evaluated; in the vehicle transportation efficiency assessment, the detection parameters are the vehicle's maneuverability parameters. By inputting the detection parameters into the performance evaluation model, the performance evaluation results are output. The performance evaluation model also requires input indicator parameters; The performance evaluation model analyzes the performance of the evaluated object based on the indicator parameters and detection parameters, and outputs the performance evaluation results. The process of obtaining preset parameters includes: Determine the parameters corresponding to the neurons in the preset neural network model to obtain parameter information; Based on the parameter information and according to the preset parameter generation rules, multiple sets of parameter values ​​are generated. Each set of parameter values ​​is arranged in a preset order to obtain multiple solution vectors. Establish an objective function, wherein the larger the objective function value of the solution vector, the closer the solution vector is to the optimal solution vector; The multiple solution vectors are iteratively evolved multiple times based on the objective function to obtain multiple solution vectors after iterative evolution; From the multiple solution vectors after iterative evolution, select the solution vector with the largest objective function value to obtain the optimized solution vector; The preset parameters are determined based on the parameter values ​​in the optimized solution vector; The iterative evolution includes: Obtain multiple solution vectors for the current loop, determine the selection probability based on the objective function value of each solution vector, select a preset number of solution vectors based on the selection probability, and obtain multiple target solution vectors, wherein the selection probability is positively correlated with the objective function value; Among the multiple target solution vectors, the corresponding parameter values ​​of two selected target solution vectors are swapped according to the crossover probability to obtain multiple target solution vectors after swapping. The crossover probability is negatively correlated with the target function value. In the multiple target solution vectors after the exchange, the parameter values ​​in the target solution vectors are replaced with new parameter values ​​according to the replacement probability to obtain multiple solution vectors for the next cycle.

2. The performance evaluation method based on neural networks according to claim 1, characterized in that, The acquisition of training samples includes: Obtain initial training samples and analyze them to determine their features; New samples are generated based on the features of the initial training samples; The training samples are obtained based on the initial training samples and the new samples.

3. The performance evaluation method based on neural networks according to claim 1, characterized in that, Obtain initial training samples and analyze them to determine their features, including: Clustering is performed on the initial training samples to obtain initial training samples of multiple categories; The features of the initial training samples are determined based on their categories.

4. The performance evaluation method based on neural networks according to claim 3, characterized in that, Based on the features of the initial training samples, new samples are generated, including: Obtain the number of initial training samples in each category, and calculate the ratio of the number of initial training samples in each category to the total number of initial training samples to obtain the proportion of each category; According to the proportion of each category, the initial training samples are randomly copied from the initial training samples of each category to obtain new samples.

5. The performance evaluation method based on neural networks according to claim 3, characterized in that, Based on the features of the initial training samples, new samples are generated, including: The initial training samples for each category are copied separately to obtain intermediate samples; Calculate the variance of the intermediate samples for each category to obtain the category variance; For each intermediate sample, a random number is generated, and the average absolute value of the random number is less than or equal to the class variance corresponding to the intermediate sample; The intermediate sample is added to the corresponding random number to obtain a new sample.

6. The performance evaluation method based on neural networks according to claim 1, characterized in that, The step of modifying the parameters in the preset neural network model based on the training samples to reduce the loss function value and obtain an effectiveness evaluation model includes: The training samples are divided into a first class of samples and a second class of samples; The first type of sample is input into the preset neural network model to obtain multiple first output values; Multiple first loss function values ​​are determined based on multiple first output values ​​and their corresponding true values, and the parameters in the preset neural network model are adjusted in the direction of decreasing first loss function values. The second type of sample is input into the neural network model to obtain multiple second output values; The second loss function value is determined based on the average of the multiple second output values ​​and the average of the multiple corresponding true values, and the parameters in the preset neural network model are adjusted in the direction of decreasing the second loss function value.

7. A neural network-based performance evaluation system for implementing the neural network-based performance evaluation method as described in any one of claims 1 to 6, characterized in that, include: The acquisition module is used to acquire a preset neural network model, preset parameters, training samples, and detection parameters. A configuration module is used to configure the parameters of each neuron in a preset neural network model according to the preset parameters; The training module is used to modify the parameters in the preset neural network model based on the training samples, so as to reduce the loss function value and obtain the performance evaluation model. The evaluation module is used to input the detection parameters into the performance evaluation model to obtain the performance evaluation results.

8. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the neural network-based performance evaluation method as described in any one of claims 1 to 6.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the neural network-based performance evaluation method as described in any one of claims 1 to 6.

Images