Project evaluation method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a project evaluation method and device, computer equipment, a storage medium and a program product. The method comprises the steps of obtaining target project data of a to-be-evaluated project, evaluation demand information of the to-be-evaluated project, and performance attribute information of a plurality of candidate evaluation models; selecting a target evaluation model from the plurality of candidate evaluation models according to the evaluation demand information of the to-be-evaluated project and the performance attribute information of the plurality of candidate evaluation models; and inputting the target project data into the target evaluation model to obtain an evaluation result of the to-be-evaluated project. According to the scheme, the model suitable for the to-be-evaluated project can be selected from the plurality of candidate evaluation models according to the evaluation demand information of the to-be-evaluated project and the performance attribute information of the plurality of candidate evaluation models, so that the advantages of the evaluation models can be brought into full play, and the accuracy of project evaluation is further improved.

Description

Technical Field

[0001] This application relates to the technical field of project evaluation, and particularly to a project evaluation method, device, computer device, storage medium, and program product. Background Art

[0002] An informatization project refers to a project involving the application of information technology carried out in various fields to achieve goals such as digital transformation and improved management efficiency, such as an enterprise's digital transformation project, a government's e-government construction project, an informatization management project in the medical industry, and an online teaching platform construction project in the education field.

[0003] In the early stage of a project, by evaluating aspects such as the technical feasibility, economic rationality, and social impact of the project, a scientific basis can be provided for project decision-making. During the implementation process of the project, through regular evaluation, problems and risks existing in the project, such as schedule delays, cost overruns, and technical difficulties, can be discovered in a timely manner, so as to take measures for adjustment and improvement in a timely manner to ensure that the project can proceed smoothly according to the plan. After the project is completed, evaluating the actual effect of the project can measure whether the project has achieved the expected goals and whether the expected economic and social benefits have been realized. At the same time, through evaluation, the experience and lessons of the project can also be summarized to provide reference for the implementation of similar projects in the future.

[0004] In traditional technologies, an artificial intelligence model is usually used to evaluate a project based on the characteristic data of the project. However, due to the complex and diverse characteristics of project data, there is a problem that the artificial intelligence model adopted is difficult to adapt to the characteristic data of the project, which in turn leads to inaccurate project evaluation. Summary of the Invention

[0005] Based on this, in view of the above technical problems, it is necessary to provide a project evaluation method, device, computer device, storage medium, and program product that can improve the accuracy of project evaluation.

[0006] In a first aspect, this application provides a project evaluation method, including:

[0007] Obtain the target project data of the project to be evaluated, the evaluation requirement information of the project to be evaluated, and the performance attribute information of multiple candidate evaluation models; wherein, the multiple candidate evaluation models are at least two of a multi-layer perceptron model, a convolutional neural network model, and a recurrent neural network model;

[0008] Select a target evaluation model from the multiple candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of the multiple candidate evaluation models;

[0009] Input the target project data into the target evaluation model to obtain the evaluation result of the project to be evaluated.

[0010] In one embodiment, selecting a target evaluation model from multiple candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of the multiple candidate evaluation models includes:

[0011] When the evaluation requirement information includes real-time evaluation requirements, respectively compare the inference duration in the performance attribute information of each candidate evaluation model with a preset duration threshold; wherein, the inference duration of each candidate evaluation model is determined based on the ratio between the total amount of computation required for the forward propagation of the candidate evaluation model and the amount of computation of the hardware device running the candidate evaluation model per unit time;

[0012] If there is a candidate evaluation model whose inference duration is less than the preset duration threshold, then use the candidate evaluation model with an inference duration less than the preset duration threshold as the target evaluation model;

[0013] If there are multiple candidate evaluation models whose inference durations are less than the preset duration threshold, then screen out the target evaluation model from the multiple candidate evaluation models according to the first prediction accuracy, mean square error, inference duration, algorithm complexity, and resource consumption in the performance attribute information of the multiple candidate evaluation models whose inference durations are less than the preset duration threshold; wherein, the first prediction accuracy of each candidate evaluation model is the prediction accuracy determined by the candidate evaluation model based on the sample data.

[0014] In one embodiment, screening out the target evaluation model from multiple candidate evaluation models according to the first prediction accuracy, mean square error, inference duration, algorithm complexity, and resource consumption in the performance attribute information of the multiple candidate evaluation models whose inference durations are less than the preset duration threshold includes:

[0015] For each candidate evaluation model whose inference duration is less than the preset duration threshold, perform weighted processing on the first prediction accuracy, mean square error, inference duration, algorithm complexity, and resource consumption of the candidate evaluation model to obtain the performance weighted value of the candidate evaluation model;

[0016] Use the candidate evaluation model corresponding to the largest performance weighted value among the performance weighted values as the target evaluation model.

[0017] In one embodiment, selecting a target evaluation model from multiple candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of the multiple candidate evaluation models includes:

[0018] In the case where the evaluation requirements information includes the robustness evaluation requirements for abnormal data, the candidate evaluation model corresponding to the second highest prediction accuracy rate in the performance attribute information of each candidate evaluation model is used as the target evaluation model; wherein, the second prediction accuracy rate of each candidate evaluation model is the prediction accuracy rate determined by the candidate evaluation model based on adversarial sample data.

[0019] In one embodiment, the method further includes:

[0020] If the candidate evaluation model is a multi-layer perceptron model, the algorithm complexity of the multi-layer perceptron model is determined according to the number of network layers, the number of input neurons in each layer, and the number of output neurons in each layer of the multi-layer perceptron model;

[0021] If the candidate evaluation model is a convolutional neural network model, the algorithm complexity of the convolutional neural network model is determined according to the size of the input feature map corresponding to the convolutional neural network model, the convolutional kernel parameters, the number of input channels, and the number of output channels;

[0022] If the candidate evaluation model is a recurrent neural network model, the algorithm complexity of the recurrent neural network model is determined according to the number of network layers and the number of input neurons in each layer of the recurrent neural network model.

[0023] In one embodiment, obtaining the target project data of the project to be evaluated includes:

[0024] Taking the difference between the entropy value of the target project data of the project to be evaluated and the entropy value of the global project data being less than a preset difference as a constraint condition, sampling the preset category data in the global project data at each time window according to the sampling ratio corresponding to each time window, to obtain the target project data; wherein, the sampling ratio corresponding to each time window is the ratio of the total data frequency of the preset category data within the time window to the total data frequency of the preset category data in the global project data.

[0025] In a second aspect, the present application further provides a project evaluation device, including:

[0026] An obtaining module, configured to obtain the target project data of the project to be evaluated, the evaluation requirements information of the project to be evaluated, and the performance attribute information of a plurality of candidate evaluation models; wherein, the plurality of candidate evaluation models are at least two of a multi-layer perceptron model, a convolutional neural network model, and a recurrent neural network model;

[0027] A selection module, configured to select a target evaluation model from the plurality of candidate evaluation models according to the evaluation requirements information of the project to be evaluated and the performance attribute information of the plurality of candidate evaluation models;

[0028] An evaluation module, configured to input the target project data into the target evaluation model to obtain the evaluation result of the project to be evaluated.

[0029] In a third aspect, the present application also provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0030] Obtain the target project data of the project to be evaluated, the evaluation requirement information of the project to be evaluated, and the performance attribute information of multiple candidate evaluation models; wherein, the multiple candidate evaluation models are at least two of a multi-layer perceptron model, a convolutional neural network model, and a recurrent neural network model;

[0031] Select a target evaluation model from the multiple candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of the multiple candidate evaluation models;

[0032] Input the target project data into the target evaluation model to obtain the evaluation result of the project to be evaluated.

[0033] In a fourth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0034] Obtain the target project data of the project to be evaluated, the evaluation requirement information of the project to be evaluated, and the performance attribute information of multiple candidate evaluation models; wherein, the multiple candidate evaluation models are at least two of a multi-layer perceptron model, a convolutional neural network model, and a recurrent neural network model;

[0035] Select a target evaluation model from the multiple candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of the multiple candidate evaluation models;

[0036] Input the target project data into the target evaluation model to obtain the evaluation result of the project to be evaluated.

[0037] In a fifth aspect, the present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0038] Obtain the target project data of the project to be evaluated, the evaluation requirement information of the project to be evaluated, and the performance attribute information of multiple candidate evaluation models; wherein, the multiple candidate evaluation models are at least two of a multi-layer perceptron model, a convolutional neural network model, and a recurrent neural network model;

[0039] Select a target evaluation model from the multiple candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of the multiple candidate evaluation models;

[0040] Input the target project data into the target evaluation model to obtain the evaluation result of the project to be evaluated.

[0041] The above project evaluation method, device, computer device, storage medium and program product obtain target project data of the project to be evaluated, evaluation requirement information of the project to be evaluated, and performance attribute information of multiple candidate evaluation models; select a target evaluation model from the multiple candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of the multiple candidate evaluation models; input the target project data into the target evaluation model to obtain an evaluation result of the project to be evaluated. The above solution can select a model suitable for the project to be evaluated from multiple candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of the multiple candidate evaluation models, which can give full play to the advantages of the evaluation model and thus improve the accuracy of project evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for describing the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 It is a schematic flowchart of the project evaluation method in one embodiment;

[0044] Figure 2 It is a schematic flowchart of selecting a target evaluation model in one embodiment;

[0045] Figure 3 It is a schematic flowchart of selecting a target evaluation model in another embodiment;

[0046] Figure 4 It is a signaling diagram of the project evaluation method in one embodiment;

[0047] Figure 5 It is a structural block diagram of the project evaluation device in one embodiment;

[0048] Figure 6 It is an internal structure diagram of a computer device in one embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] In order to make the objectives, technical solutions and advantages of the present application more clear, the following further describes the present application in detail with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] The project evaluation method provided by the embodiments of this application can be applied to the application scenario of evaluating informatization projects to predict possible risks and problems of the projects and propose corresponding solutions.

[0051] This method can be executed by a server or by a terminal with a certain computing power. Among them, the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc.

[0052] In an exemplary embodiment, as Figure 1 shown, a project evaluation method is provided. Taking the application of this method to a server as an example, the method includes the following steps:

[0053] S101, obtain the target project data of the project to be evaluated, the evaluation requirement information of the project to be evaluated, and the performance attribute information of multiple candidate evaluation models.

[0054] Exemplarily, the project to be evaluated can be an informatization project, which can be understood as a project that takes information technology as the core and uses modern information technology means such as computer technology, network technology, and communication technology to optimize, transform, or innovate the business processes, management models, service methods, etc. of enterprises, organizations, or society to improve efficiency, reduce costs, enhance competitiveness, and achieve business goals. For example, the digital transformation project of an enterprise, the e-government construction project of a government, the informatization management project in the medical industry, and the online teaching platform construction project in the education field, etc. Specifically, such as online transaction risk control, intelligent detection of industrial products, financial fraud detection, medical diagnosis systems, etc.

[0055] Exemplarily, the target project data of the project to be evaluated can be the historical data of the project to be evaluated, including code quality, development progress, test results, and operation data, etc. Usually, during the process of the project, the project data of the project will be recorded, and the target project data of the project to be evaluated can be determined according to the historical records of the project data.

[0056] The evaluation requirement information of the project to be evaluated can be real-time analysis of the data of the project to be evaluated or analysis of the security of the project to be evaluated. For example, projects such as online transaction risk control and industrial intelligent detection have high requirements for real-time analysis, while projects such as financial fraud detection and medical diagnosis systems have high requirements for the recognition accuracy of abnormal data. The evaluation requirement information of the project to be evaluated can be determined according to information such as the type and use of the project.

[0057] The performance attribute information of the candidate evaluation models is used to reflect information such as the calculation speed and accuracy of the candidate evaluation models. Sample data can be used in advance to train and validate each candidate evaluation model to obtain the performance attribute information of each candidate evaluation model.

[0058] Exemplarily,

[0059] Among them, multiple candidate evaluation models are at least two of a multi-layer perceptron model, a convolutional neural network model, and a recurrent neural network model.

[0060] Exemplarily, the sample data can be divided into a training set, a validation set, and a test set according to the ratio of 70%, 15%, and 15%, so that the training set is used for model parameter optimization, the validation set is used to monitor the generalization ability during the training process, and the test set is used to finally evaluate the model performance.

[0061] For the multi-layer perceptron model, the model adopts a fully connected neuron structure and uses the backpropagation algorithm for weight optimization. During the weight update process of the neurons in the l-th layer, calculate the gradient of the loss function L with respect to the weight :

[0062]

[0063] Among them, is the learning rate, represents the partial derivative of the loss function with respect to the weight. The model continuously adjusts the parameters through the gradient descent method to make the loss function converge and finally obtains the optimized weight matrix.

[0064] For the convolutional neural network model, during the training process, first perform a convolution operation, extract features from the input feature map X by setting the convolution kernel K, and output a feature map:

[0065]

[0066] Among them, m1 and n1 are used to determine the positions of the elements in the output feature map Y, representing the row and column indices of the elements in the output feature map; i and j are index variables used when the convolutional kernel K traverses the input feature map X, and are used to traverse each element in the convolutional kernel K. During the convolutional operation, starting from the upper left corner of the convolutional kernel, i and j take values in sequence, multiply with the elements at the corresponding positions in the input feature map X and accumulate them to calculate the weighted sum of the local area, thereby completing the feature extraction. This process calculates the weighted sum of the local area through a sliding window to achieve local perception and feature extraction, then performs a pooling operation to reduce the feature dimension, and performs classification or regression calculations through a fully connected layer.

[0067] For the recurrent neural network model, during the training process, the time dependence of the sequence data is utilized. The model memorizes historical information through the hidden state h t , and its update method is as follows:

[0068]

[0069] Among them, W h and W x are the weight matrices of the hidden state and the input data respectively, b is the bias term, f is the activation function, and x t is the input data at time t, ensuring that the model can effectively learn the time series features. During the entire training process, metrics such as accuracy and mean squared error on the validation set are continuously monitored, training parameters are adjusted to avoid overfitting, and the generalization ability of the model is improved to ensure that the selected neural network algorithm has stable performance in the subsequent evaluation stage.

[0070] For the multi-layer perceptron model, the determination of the number of neurons in the hidden layer directly affects the expressive ability and computational complexity of the model. If the number of neurons in the hidden layer is too small, the model may not be able to effectively fit the data. If the number of neurons is too large, it is prone to overfitting. The number of neurons H in the hidden layer can be estimated based on the number of neurons I in the input layer and the number of neurons O in the output layer, using the empirical formula:

[0071]

[0072] This formula can balance the complexity and computational efficiency of the model to a certain extent, ensuring that the network not only has sufficient non-linear expressive ability but also does not affect the generalization performance due to too many neurons. During the parameter initialization stage, the weight matrix is initialized using Xavier initialization or He initialization, so that the input and output variances of the neurons are consistent, avoiding the problems of gradient disappearance or gradient explosion.

[0073] For the convolutional neural network model, the size k×k of the convolutional kernel directly affects the feature extraction ability, and the stride s determines the size of the output feature map. Assuming the size of the input feature map is W×H, then the size of the output feature map after convolution The calculation is as follows:

[0074]

[0075] When initializing the parameters, appropriate k and s need to be selected according to the characteristic scale of the data. Larger convolutional kernels can capture global features, while smaller convolutional kernels are helpful for extracting local details. The choice of the stride affects the degree of feature dimensionality reduction, and it is necessary to ensure a balance between information retention and computational efficiency of the output feature map.

[0076] For recurrent neural network models, the parameter selection of the gating mechanism determines the model's ability to process long-sequence data. For common Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) structures, the weight matrices of the input gate, forget gate, and output gate need to be reasonably initialized to ensure that the model can effectively learn the dependencies of time series in the initial stage. During the parameter initialization process, the model construction and testing unit combines the distribution characteristics of the data and the actual application requirements to ensure that different neural network structures have good learning ability and generalization performance in the subsequent training stage.

[0077] S102, select a target evaluation model from multiple candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of the multiple candidate evaluation models.

[0078] Exemplarily, according to the evaluation requirements of the model to be evaluated and the performance attribute information of multiple candidate evaluation models, it can be determined which candidate evaluation model is more suitable for evaluating the model to be evaluated. For example, if the evaluation requirement information of the model to be evaluated is that fast and accurate evaluation of the project is required, a candidate evaluation model with high accuracy and fast calculation speed can be selected; if the evaluation requirement information of the model to be evaluated is that abnormal data in the project needs to be accurately identified, a candidate evaluation model with high robustness to abnormal data can be selected. Furthermore, the selected candidate evaluation model can be used as the target evaluation model.

[0079] S103, input the target project data into the target evaluation model to obtain the evaluation result of the project to be evaluated.

[0080] Furthermore, the target project data can be input into the target evaluation model so that the target evaluation model predicts the target project data to obtain the evaluation result of the project to be evaluated. Among them, the evaluation result of the project to be evaluated can be used to analyze whether there is abnormal data in the project and the future development trend of the project, so as to timely discover existing problems and then facilitate the timely adoption of remedial measures; or it can be used to pre-know possible faults or problems that may occur in the project to be evaluated in the future, and timely take corresponding measures to avoid them to ensure the smooth progress of the project to be evaluated.

[0081] The above project evaluation method obtains the target project data of the project to be evaluated, the evaluation requirement information of the project to be evaluated, and the performance attribute information of multiple candidate evaluation models; selects a target evaluation model from multiple candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of multiple candidate evaluation models; inputs the target project data into the target evaluation model to obtain the evaluation result of the project to be evaluated. The above solution can select a model suitable for the project to be evaluated from multiple candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of multiple candidate evaluation models, which can give full play to the advantages of the evaluation model and thus improve the accuracy of project evaluation.

[0082] In some optional implementation manners, when the evaluation requirement information includes real-time evaluation requirements, a target evaluation model can be selected according to the performance attribute information of each candidate evaluation model.

[0083] Based on this, referring to Figure 2 , Figure 2 a schematic flowchart of a process for selecting a target evaluation model is provided, which specifically includes the following steps:

[0084] S201, when the evaluation requirement information includes real-time evaluation requirements, compare the inference duration in the performance attribute information of each candidate evaluation model with a preset duration threshold respectively.

[0085] When the evaluation requirement information includes real-time evaluation requirements, it means that the project to be evaluated has relatively high requirements for calculation accuracy and calculation speed. Then, the inference duration in the performance attribute information of each candidate evaluation model can be compared with the preset duration threshold respectively to obtain the magnitude relationship between the inference duration of each candidate evaluation model and the preset duration threshold. Among them, the preset duration threshold can be determined according to different projects to be evaluated and is used to measure the upper limit value of the duration for calculating the evaluation result of the project to be evaluated. The inference duration of each candidate evaluation model is determined based on the ratio between the total amount of calculations required for the forward propagation of the candidate evaluation model and the amount of calculations per unit time of the hardware device running the candidate evaluation model.

[0086] Exemplarily, when the project to be evaluated has high real-time requirements, the inference time T of the model inf needs to be reduced as much as possible. The inference time is mainly affected by the neural network structure, computational complexity, and hardware parallel computing ability. The inference time T of the model inf can measure the real-time performance of different algorithms and is specifically determined by the following calculation formula:

[0087]

[0088] where C opRepresents the total computational amount required for the forward propagation of the model, P hw Represents the computational power of the target hardware, that is, the computational amount of the hardware device running the candidate evaluation model per unit time. When T inf is small, it means that the model can complete inference at a faster speed on the target hardware, which is suitable for scenarios with high real-time requirements, such as online transaction risk control, industrial intelligent detection, and other applications. When comparing different algorithms, it is necessary to comprehensively consider the number of layers of the model, the number of parameters, and the utilization rate of the computing unit to ensure that the selected algorithm can meet the low-latency inference requirements.

[0089] S202, if there is a candidate evaluation model whose inference duration is less than the preset duration threshold, then use the candidate evaluation model with an inference duration less than the preset duration threshold as the target evaluation model.

[0090] If there is only one candidate evaluation model whose inference duration is less than the preset duration threshold, that is, it means that only one candidate evaluation model meets the real-time evaluation requirements of the project to be evaluated, then the candidate evaluation model with an inference duration less than the preset duration threshold can be directly used as the target evaluation model, that is, use this candidate evaluation model to evaluate the project to be evaluated.

[0091] S203, if there are multiple candidate evaluation models whose inference durations are less than the preset duration threshold, then screen out the target evaluation model from the multiple candidate evaluation models according to the first prediction accuracy, mean square error, inference duration, algorithm complexity, and resource consumption in the performance attribute information of the multiple candidate evaluation models with inference durations less than the preset duration threshold.

[0092] If there are multiple candidate evaluation models whose inference durations are less than the preset duration threshold, any one of the candidate evaluation models can be selected as the target evaluation model; or the target evaluation model can be screened out from the multiple candidate evaluation models according to the first prediction accuracy, mean square error, inference duration, algorithm complexity, and resource consumption in the performance attribute information of the multiple candidate evaluation models with inference durations less than the preset duration threshold. For example, the total performance of the candidate evaluation models can be calculated according to the first prediction accuracy, mean square error, inference duration, algorithm complexity, and resource consumption in the performance attribute information of the candidate evaluation models, and then the candidate evaluation model with the best performance can be used as the target evaluation model, which can make the selected candidate evaluation model have higher computational efficiency and computational accuracy. Among them, the first prediction accuracy of each candidate evaluation model is the prediction accuracy determined by the candidate evaluation model based on the sample data.

[0093] In the embodiments of the present application, when the evaluation requirement information of the project to be evaluated includes real-time evaluation requirements, according to the performance attribute information of the candidate evaluation models, a model with higher prediction accuracy and faster computational efficiency is screened out, meeting the evaluation requirements of the project to be evaluated.

[0094] In some alternative implementation manners, when the inference durations of multiple candidate evaluation models are less than a preset duration threshold, the comprehensive performance of the candidate evaluation models can be determined according to the weights corresponding to the performance attribute information of the candidate evaluation models.

[0095] Based on this, referring to Figure 3 , Figure 3 a schematic flowchart of another process for selecting a target evaluation model is provided, which specifically includes the following steps:

[0096] S301. For each candidate evaluation model whose inference duration is less than the preset duration threshold, perform weighted processing on the first prediction accuracy, mean square error, inference duration, algorithm complexity, and resource consumption of the candidate evaluation model to obtain the performance weighted value of the candidate evaluation model.

[0097] Exemplarily, the weights corresponding to the first prediction accuracy, mean square error, inference duration, algorithm complexity, and resource consumption of the candidate evaluation model can be determined by methods such as the expert scoring method, the analytic hierarchy process, and the principal component analysis method, comprehensively considering the importance of factors such as prediction accuracy, operation speed, algorithm complexity, and resource consumption in the project.

[0098] Exemplarily, the performance weighted value of the candidate evaluation model can be determined by the following calculation formula:

[0099]

[0100] where S is the performance weighted value of the candidate evaluation model, Acc is the first prediction accuracy of the candidate evaluation model, MSE is the mean square error of the candidate evaluation model, T inf is the inference duration of the candidate evaluation model, is the algorithm complexity of the candidate evaluation model, M mem is the resource consumption of the candidate evaluation model, , , , and are the weights corresponding to the first prediction accuracy, mean square error, inference duration, algorithm complexity, and resource consumption respectively.

[0101] Among them, the first prediction accuracy of the candidate evaluation model can be determined by the following calculation formula:

[0102]

[0103] Among them, TP and TN respectively represent the number of correctly classified positive-class samples and negative-class samples, and FP and FN respectively represent the number of misclassified positive-class samples and negative-class samples. This metric reflects the overall prediction ability of the model for the target project data; in information-based projects, when tasks involve scenarios with high precision requirements such as user behavior analysis and risk assessment, algorithms with higher accuracy need to be considered first.

[0104] For regression tasks, the mean squared error calculation formula is as follows:

[0105]

[0106] Among them, y i is the true value, is the model prediction value, and n2 is the number of true values. This metric is used to measure the fitting ability of the model for numerical data. When the project to be evaluated involves tasks such as business transaction prediction and system performance evaluation, a lower mean squared error means stronger prediction stability of the model.

[0107] S302. Use the candidate evaluation model corresponding to the largest performance weighting value among the performance weighting values as the target evaluation model.

[0108] Exemplarily, use the candidate evaluation model corresponding to the largest performance weighting value among the performance weighting values as the target evaluation model. In this way, the candidate evaluation model with the optimal model performance for the project to be evaluated can be selected to improve the efficiency and accuracy of the evaluation of the project to be evaluated.

[0109] In some alternative implementation manners, when the evaluation requirement information includes the robustness evaluation requirement for abnormal data, the target evaluation model can be determined according to the prediction accuracy of each candidate evaluation model based on adversarial sample data.

[0110] Exemplarily, when the evaluation requirement information includes the robustness evaluation requirement for abnormal data, the candidate evaluation model corresponding to the largest second prediction accuracy in the performance attribute information of each candidate evaluation model can be used as the target evaluation model.

[0111] Among them, the second prediction accuracy of each candidate evaluation model is the prediction accuracy determined by the candidate evaluation model based on adversarial sample data, that is, the second prediction accuracy can reflect the accuracy of the candidate evaluation model in identifying abnormal data.

[0112] When the evaluation requirement information includes the robustness evaluation requirement for abnormal data, using the candidate evaluation model corresponding to the largest second prediction accuracy in the performance attribute information of each candidate evaluation model as the target evaluation model can screen out the candidate evaluation model with the highest accuracy in identifying abnormal data and can maximally meet the evaluation requirements of the model to be evaluated.

[0113] Exemplarily, when the robustness requirement of the project to be evaluated for abnormal data is relatively high, it is necessary to focus on analyzing the stability of the model in the abnormal data environment. Generally, adversarial sample attack testing and outlier detection methods are used for evaluation. For classification tasks, the adversarial sample generation algorithm (Fast Gradient Sign Method, FGSM) can be used to measure the robustness of the model and calculate the adversarial perturbation:

[0114]

[0115] where is the perturbation coefficient, is the gradient of the loss function J with respect to the input x, y is the true label, represents the model parameters. By calculating the prediction change of the model after perturbation, the stability of different candidate evaluation models under abnormal data interference can be measured. When the candidate evaluation model can still maintain a high prediction accuracy after adding perturbation, it indicates that its robustness to abnormal data is strong and it is suitable for application scenarios with high security requirements, such as financial fraud detection, medical diagnosis systems, etc. In the process of finally selecting the candidate evaluation model, it is necessary to comprehensively consider the inference time, robustness test results and other key indicators to ensure that the selected algorithm meets both the real-time requirement and can maintain stable performance in the abnormal data environment.

[0116] In some alternative implementation manners, the algorithm complexity of different candidate evaluation models can be determined in the following way:

[0117] If the candidate evaluation model is a multi-layer perceptron model, then according to the number of network layers, the number of input neurons and the number of output neurons in each layer of the multi-layer perceptron model, the algorithm complexity of the multi-layer perceptron model is determined.

[0118] Exemplarily, the algorithm complexity of the multi-layer perceptron model can be determined by the following calculation formula:

[0119]

[0120] where is the algorithm complexity of the multi-layer perceptron model, L1 is the number of network layers of the multi-layer perceptron model, N i1 is the number of input neurons in each layer, and N0 is the number of output neurons in each layer. When the number of network layers is relatively deep or the number of neurons in each layer is large, the computational amount increases accordingly, affecting the training speed and inference efficiency of the model.

[0121] If the candidate evaluation model is a convolutional neural network model, then the algorithm complexity of the convolutional neural network model can be determined according to the size of the input feature map, the convolutional kernel parameters, the number of input channels and the number of output channels corresponding to the convolutional neural network model.

[0122] Exemplarily, the algorithm complexity of the convolutional neural network model can be determined by the following calculation formula:

[0123]

[0124] where is the algorithm complexity of the convolutional neural network model, W×H is the size of the input feature map, C is the number of input channels, is the number of output channels, and k×k is the size of the convolutional kernel.

[0125] If the candidate evaluation model is a recurrent neural network model, the algorithm complexity of the recurrent neural network model is determined according to the number of network layers of the recurrent neural network model and the number of input neurons in each layer. A larger convolutional kernel and more output channels will significantly increase the computational amount. Therefore, when selecting a candidate evaluation model, the configuration of the convolutional layer needs to be comprehensively considered to balance the computational overhead and model performance.

[0126] Exemplarily, the space complexity is used to describe the storage space required by the algorithm during the calculation process, usually including the storage requirements of model parameters and intermediate calculation results. The algorithm complexity of the recurrent neural network model can be determined by the following calculation formula:

[0127]

[0128] where is the algorithm complexity (space complexity) of the recurrent neural network model, L2 is the number of network layers of the recurrent neural network model, and N i2 is the number of input neurons in each layer.

[0129] In the embodiments of the present application, the calculation methods of the algorithm complexities of different candidate evaluation models are given, and the algorithm complexities of each candidate evaluation model can be more accurately determined according to the parameters of the model, so as to more accurately screen out the candidate evaluation models that meet the evaluation requirements of the project to be evaluated.

[0130] In some alternative implementation manners, in order to extract representative target project data from the global project data of the project to be evaluated, the global project data of the project to be evaluated can be sampled according to the entropy value of the global project data and the sampling ratio corresponding to each time window, so as to obtain the target project data of the project to be evaluated.

[0131] Exemplarily, during the process of sampling the preset category data in the global project data, a suitable sampling method can be selected according to the distribution characteristics of the data. For example, simple random sampling is applicable to the scenario where the data is evenly distributed, ensuring that all data points have the same probability of being selected. After setting the sample size n4, the data processing unit randomly selects n4 data points from the dataset. Assuming the global data volume is N, the probability of each data point being selected is:

[0132]

[0133] When there is a problem of class imbalance in the data, stratified sampling can ensure that each category is presented proportionally in the target project data. Assuming the global project data D contains q categories of data, and the proportion of category i in the dataset is P(d i ), in order to ensure that the target project data after sampling still conforms to the overall data distribution, the sampling quantity n i of each category of data can be calculated according to the proportion of each category, and the calculation formula is as follows:

[0134]

[0135] Systematic sampling is applicable to the scenario where the data volume is large and the data storage has sequential characteristics. The sampling interval can be based on:

[0136]

[0137] to determine the selection index. After determining the initial index r, data points are sequentially selected from the dataset at intervals of j . Systematic sampling can evenly cover the data sequence, improve the sampling efficiency, and reduce the consumption of computing resources at the same time. The sampling strategy can be dynamically adjusted according to the distribution characteristics of the data to enhance the representativeness of the target project data. When selecting sample data, the above sampling strategy can also be used to select sample data to ensure the stability and generalization ability of the data input during the modeling stage based on the sample data.

[0138] Exemplarily, it can be constrained that the difference between the entropy value of the target project data of the project to be evaluated and the entropy value of the global project data is less than the preset difference, and the preset category data in the global project data is sampled under each time window according to the sampling ratio corresponding to each time window to obtain the target project data.

[0139] Among them, the entropy value of the global project data can be determined by the following calculation formula:

[0140]

[0141] Among them, is the entropy value of the global project data, is the probability distribution of the i-th type of data in the global project data, and n3 is the number of data categories in the global project data. The preset difference can be set according to actual needs. For example, it can be set to 0.1. Taking the difference between the entropy value of the target project data of the project to be evaluated and the entropy value of the global project data being less than the preset difference as a constraint condition, it can be understood that it is used to constrain the entropy value of the target project data of the project to be evaluated to be close to the entropy value of the global project data.

[0142] In addition, the sampling ratio corresponding to each time window is the ratio of the total data frequency of the preset category data within the time window to the total data frequency of the preset category data in the global project data. Among them, the preset category data can be the category data that can be representative selected from the global project data.

[0143] For example, the sampling ratio corresponding to each time window can be determined by the following calculation formula:

[0144]

[0145] Among them, is the sampling ratio corresponding to the time window T w corresponding sampling ratio, is the total data frequency of the preset category data within the time window T w and is the total data frequency of the preset category data in the global project data, and D is the global project data.

[0146] In the embodiments of the present application, by setting constraint conditions and sampling the preset category data in the global project data according to the sampling ratio corresponding to each time window, the obtained target project data can reflect the business cycle characteristics and be representative, so that the evaluation result of the model to be evaluated based on the target project data is more reliable.

[0147] In some alternative implementation manners, the embodiments of the present application further provide a project evaluation system, which includes a data processing unit, a model training unit, and an evaluation unit. The project evaluation method provided by the embodiments of the present application will be introduced below in combination with the project evaluation system.

[0148] Exemplarily, referring to Figure 4 , Figure 4 a signaling diagram of a project evaluation method is provided. Specifically, it includes the following steps:

[0149] S401, trigger an instruction to obtain sample data.

[0150] S402, the data processing unit extracts multiple groups of sample data from the global data of the project by using sampling techniques.

[0151] S403, The data processing unit selects a sampling method in combination with the data structure of the global data of the project.

[0152] Among them, the sampling methods include but are not limited to simple random sampling, stratified sampling, and systematic sampling.

[0153] S404, The data processing unit takes the difference between the entropy value of the sample data and the entropy value of the global project data being less than a preset difference as a constraint condition to ensure that the entropy values of each group of sample data are close to the entropy value of the global project data.

[0154] S405, The data processing unit sets a sliding window sampling mechanism to take into account short-term and long-term trends.

[0155] S406, The data processing unit transmits multiple groups of sample data to the model training unit.

[0156] S407, The model training unit uses each group of sample data to train different candidate evaluation models.

[0157] Among them, the candidate evaluation models include but are not limited to multi-layer perceptron models, convolutional neural network models, and recurrent neural network models.

[0158] S408, The model training unit inputs the performance attribute information of each candidate evaluation model into the evaluation unit.

[0159] Among them, the performance attribute information of the candidate evaluation models includes prediction accuracy, mean square error, inference duration, algorithm complexity, resource consumption, etc.

[0160] S409, The evaluation unit selects a target evaluation model from multiple candidate evaluation models according to the evaluation requirement information of the project and the performance attribute information of multiple candidate evaluation models, and conducts project evaluation on the project to be evaluated based on the target project data of the project to be evaluated.

[0161] S410, Return the target evaluation model and the evaluation result.

[0162] In the embodiment of the present application, in the data extraction stage, the data processing unit uses data sampling technology to extract multiple groups of sample data covering various feature types from the global project data. By balancing data randomness and diversity, it selects a sampling method in combination with data categories, time spans, and statistical distribution characteristics, and uses information entropy to measure data feature diversity and sets a sliding window sampling mechanism to ensure that the selected data can comprehensively reflect the overall feature distribution of the global project data, providing a reliable data basis for subsequent algorithm selection.

[0163] Moreover, in the preliminary modeling and testing phase, for each set of sample data, the model training unit respectively uses different neural network algorithms such as multi-layer perceptron models, convolutional neural network models, and recurrent neural network models for modeling and training, sets training parameters according to the actual project evaluation settings, divides the training set, validation set, and test set, and initializes the parameters according to the characteristics and applicable scenarios of different algorithm structures. For example, determine the number of neurons in the hidden layer of the multi-layer perceptron, set the size and stride of the convolutional kernel of the convolutional neural network, select the gating mechanism parameters of the recurrent neural network, etc. At the same time, monitor the performance metrics on the validation set to avoid overfitting and improve the generalization ability of the model.

[0164] Furthermore, in the model evaluation and selection phase, the evaluation unit comprehensively evaluates the algorithm suitability from multiple dimensions according to the performance metrics of different algorithms on the same sample data, combined with the actual evaluation requirements of the project, such as prediction accuracy, operation speed requirements, etc., taking into account factors such as algorithm complexity and resource consumption. By calculating the time complexity and space complexity, monitoring indicators such as CPU usage and memory occupancy, establish a comprehensive scoring system, calculate the weighted score, and finally select the algorithm with the highest score.

[0165] In addition, during the evaluation process, fully consider the real-time requirements and the robustness requirements for abnormal data of the project. For the real-time requirements, measure the real-time performance of the algorithm by calculating the ratio of the total computational amount of the model forward propagation to the computing power of the target hardware; for the robustness requirements for abnormal data, adopt adversarial sample attack testing and outlier detection methods, such as using the Fast Gradient Sign Method (FGSM) algorithm to calculate the adversarial perturbation to measure the robustness of the model. Integrate the inference time, robustness test results, and other key indicators to ensure that the selected algorithm achieves the optimal performance in a specific application scenario.

[0166] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or in turn with at least some of the steps or stages in other steps or other steps.

[0167] Based on the same inventive concept, an embodiment of the present application further provides a project evaluation device for implementing the above-mentioned project evaluation method. The solution provided by this device for solving problems is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the project evaluation device provided below can refer to the limitations on the project evaluation method in the above text, and will not be elaborated here.

[0168] In an exemplary embodiment, as Figure 5 shown, a project evaluation device is provided, including:

[0169] An acquisition module 10, configured to acquire target project data of the project to be evaluated, evaluation requirement information of the project to be evaluated, and performance attribute information of a plurality of candidate evaluation models; wherein, the plurality of candidate evaluation models are at least two of a multi-layer perceptron model, a convolutional neural network model, and a recurrent neural network model;

[0170] A selection module 20, configured to select a target evaluation model from the plurality of candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of the plurality of candidate evaluation models;

[0171] An evaluation module 30, configured to input the target project data into the target evaluation model to obtain an evaluation result of the project to be evaluated.

[0172] The above project evaluation device acquires target project data of the project to be evaluated, evaluation requirement information of the project to be evaluated, and performance attribute information of a plurality of candidate evaluation models; selects a target evaluation model from the plurality of candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of the plurality of candidate evaluation models; and inputs the target project data into the target evaluation model to obtain an evaluation result of the project to be evaluated. The above solution can select a model suitable for the project to be evaluated from the plurality of candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of the plurality of candidate evaluation models, so as to give full play to the advantages of the evaluation model, thereby improving the accuracy of project evaluation.

[0173] In one of the embodiments, the selection module 20 specifically includes:

[0174] A comparison unit, configured to compare the inference duration in the performance attribute information of each candidate evaluation model with a preset duration threshold when the evaluation requirement information includes a real-time evaluation requirement; wherein, the inference duration of each candidate evaluation model is determined based on the ratio between the total amount of calculations required for the forward propagation of the candidate evaluation model and the amount of calculations per unit time of the hardware device running the candidate evaluation model;

[0175] A determination unit, configured to: if there is a candidate evaluation model whose inference duration is less than a preset duration threshold, use the candidate evaluation model with an inference duration less than the preset duration threshold as the target evaluation model; if there are multiple candidate evaluation models whose inference durations are less than the preset duration threshold, screen out the target evaluation model from the multiple candidate evaluation models according to the first prediction accuracy, mean square error, inference duration, algorithm complexity, and resource consumption in the performance attribute information of the multiple candidate evaluation models whose inference durations are less than the preset duration threshold; wherein, the first prediction accuracy of each candidate evaluation model is the prediction accuracy determined by the candidate evaluation model based on sample data.

[0176] In one embodiment, the determination unit is specifically configured to:

[0177] For each candidate evaluation model whose inference duration is less than the preset duration threshold, perform weighted processing on the first prediction accuracy, mean square error, inference duration, algorithm complexity, and resource consumption of the candidate evaluation model to obtain a performance weighted value of the candidate evaluation model; use the candidate evaluation model corresponding to the largest performance weighted value among the performance weighted values as the target evaluation model.

[0178] In one embodiment, the selection module 20 is specifically configured to:

[0179] When the evaluation requirement information includes the robustness evaluation requirement for abnormal data, use the candidate evaluation model corresponding to the largest second prediction accuracy in the performance attribute information of each candidate evaluation model as the target evaluation model; wherein, the second prediction accuracy of each candidate evaluation model is the prediction accuracy determined by the candidate evaluation model based on adversarial sample data.

[0180] In one embodiment, the determination unit is further configured to:

[0181] If the candidate evaluation model is a multi-layer perceptron model, determine the algorithm complexity of the multi-layer perceptron model according to the number of network layers, the number of input neurons in each layer, and the number of output neurons in each layer of the multi-layer perceptron model; if the candidate evaluation model is a convolutional neural network model, determine the algorithm complexity of the convolutional neural network model according to the size of the input feature map, convolutional kernel parameters, number of input channels, and number of output channels corresponding to the convolutional neural network model; if the candidate evaluation model is a recurrent neural network model, determine the algorithm complexity of the recurrent neural network model according to the number of network layers and the number of input neurons in each layer of the recurrent neural network model.

[0182] In one embodiment, the acquisition module 10 is specifically configured to:

[0183] Taking the difference between the entropy value of the target project data of the project to be evaluated and the entropy value of the global project data being less than a preset difference as a constraint condition, sampling the preset category data in the global project data according to the sampling ratio corresponding to each time window, to obtain the target project data; wherein, the sampling ratio corresponding to each time window is the ratio of the sum of the data frequencies of the preset category data within the time window to the sum of the data frequencies of the preset category data in the global project data.

[0184] Each module in the above project evaluation device can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0185] In an exemplary embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 6 shown. The computer device includes a processor, a memory, an input / output (I / O) interface, and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store project data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a project evaluation method.

[0186] Those skilled in the art can understand that Figure 6 the structure shown in

[0187] merely represents a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0188] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the project evaluation method described in any of the above embodiments are implemented.

[0189] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps of the project evaluation method described in any of the above embodiments are implemented.

[0190] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0191] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., without limitation.

[0192] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in the present application.

[0193] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation to the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A project evaluation method, characterized in that, The method includes: Obtaining target project data of the project to be evaluated, evaluation requirement information of the project to be evaluated, and performance attribute information of multiple candidate evaluation models; wherein, the multiple candidate evaluation models are at least two of a multi-layer perceptron model, a convolutional neural network model, and a recurrent neural network model; Selecting a target evaluation model from the multiple candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of the multiple candidate evaluation models; Inputting the target project data into the target evaluation model to obtain an evaluation result of the project to be evaluated.

2. The method according to claim 1, characterized in that, The selecting a target evaluation model from the multiple candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of the multiple candidate evaluation models includes: When the evaluation requirement information includes a real-time evaluation requirement, respectively comparing the inference duration in the performance attribute information of each candidate evaluation model with a preset duration threshold; wherein, the inference duration of each candidate evaluation model is determined based on the ratio between the total amount of calculations required for the forward propagation of the candidate evaluation model and the amount of calculations per unit time of the hardware device running the candidate evaluation model; If there is a candidate evaluation model whose inference duration is less than the preset duration threshold, then taking the candidate evaluation model with an inference duration less than the preset duration threshold as the target evaluation model; If there are multiple candidate evaluation models whose inference durations are less than the preset duration threshold, then screening out a target evaluation model from the multiple candidate evaluation models according to the first prediction accuracy, mean square error, inference duration, algorithm complexity, and resource consumption in the performance attribute information of the multiple candidate evaluation models whose inference durations are less than the preset duration threshold; wherein, the first prediction accuracy of each candidate evaluation model is the prediction accuracy determined by the candidate evaluation model based on sample data.

3. The method according to claim 2, wherein The screening out a target evaluation model from the multiple candidate evaluation models according to the first prediction accuracy, mean square error, inference duration, algorithm complexity, and resource consumption in the performance attribute information of the multiple candidate evaluation models whose inference durations are less than the preset duration threshold includes: For each candidate evaluation model whose inference duration is less than the preset duration threshold, performing weighted processing on the first prediction accuracy, mean square error, inference duration, algorithm complexity, and resource consumption of the candidate evaluation model to obtain a performance weighted value of the candidate evaluation model; Taking the candidate evaluation model corresponding to the largest performance weighted value among the performance weighted values as the target evaluation model.

4. The method according to claim 1, characterized in that, The selecting a target evaluation model from the multiple candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of the multiple candidate evaluation models includes: When the evaluation requirement information includes a robustness evaluation requirement for abnormal data, then taking the candidate evaluation model corresponding to the largest second prediction accuracy in the performance attribute information of each candidate evaluation model as the target evaluation model; wherein, the second prediction accuracy of each candidate evaluation model is the prediction accuracy determined by the candidate evaluation model based on adversarial sample data.

5. The method according to claim 3, characterized in that, The method further includes: If the candidate evaluation model is a multi-layer perceptron model, determine the algorithm complexity of the multi-layer perceptron model according to the number of network layers of the multi-layer perceptron model, the number of input neurons in each layer, and the number of output neurons in each layer; If the candidate evaluation model is a convolutional neural network model, determine the algorithm complexity of the convolutional neural network model according to the size of the input feature map corresponding to the convolutional neural network model, the convolutional kernel parameters, the number of input channels, and the number of output channels; If the candidate evaluation model is a recurrent neural network model, determine the algorithm complexity of the recurrent neural network model according to the number of network layers of the recurrent neural network model and the number of input neurons in each layer.

6. The method according to claim 1, characterized in that, Obtain the target project data of the project to be evaluated, including: Taking the difference between the entropy value of the target project data of the project to be evaluated and the entropy value of the global project data being less than a preset difference as a constraint condition, sample the preset category data in the global project data at each time window according to the sampling ratio corresponding to each time window to obtain the target project data; where the sampling ratio corresponding to each time window is the ratio of the total data frequency of the preset category data within the time window to the total data frequency of the preset category data in the global project data.

7. A project evaluation device, characterized in that, The device includes: An acquisition module, configured to acquire the target project data of the project to be evaluated, the evaluation requirement information of the project to be evaluated, and the performance attribute information of a plurality of candidate evaluation models; where the plurality of candidate evaluation models are at least two of a multi-layer perceptron model, a convolutional neural network model, and a recurrent neural network model; A selection module, configured to select a target evaluation model from the plurality of candidate evaluation models according to the evaluation requirement information of the project to be evaluated and the performance attribute information of the plurality of candidate evaluation models; An evaluation module, configured to input the target project data into the target evaluation model to obtain the evaluation result of the project to be evaluated.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images