Software access assessment method and device, equipment, program product and storage medium

Abstract

The invention discloses a software admission evaluation method and device, equipment, a program product and a storage medium, the method comprises the steps that a historical acceptance result set is created, the historical acceptance result set is a set of N acceptance results obtained by utilizing an acceptance standard database to accept N historical software to be accepted, and N is a positive integer; determining a training data set based on the historical acceptance result set, and training a first network model by using the training data set; and based on the trained first network model, evaluating whether the software meets an access standard or not.

Description

Technical Field

[0001] The present application relates to the field of infrastructure and IT support technology, and in particular to a software access assessment method and apparatus, equipment, program product, and storage medium. Background Art

[0002] Agile development and continuous delivery have become popular concepts in software project management. They are used to achieve rapid iteration and automated delivery of software versions, which can greatly improve the efficiency of project delivery. However, in the overall project delivery process, the balance between quality and efficiency often becomes a difficult problem for the entire delivery team. At the same time, in the cloud computing industry, market competition is fierce, and the number of software and related services delivered by related manufacturers is increasing. There is an urgent need for a method to evaluate whether software and products meet the entry requirements. Summary of the invention

[0003] In order to solve the above technical problems, the present application provides a software access evaluation method and device, equipment, program product, and storage medium.

[0004] The software access assessment method provided by the present application includes:

[0005] Create a historical acceptance result set, where the historical acceptance result set is a set of N acceptance results obtained by accepting N historical software to be accepted using the acceptance standard database, where N is a positive integer;

[0006] Determine a training data set based on the historical acceptance result set, and train the first network model using the training data set;

[0007] Evaluate whether the software meets the admission criteria based on the trained first network model.

[0008] The software access assessment device provided by the present application comprises:

[0009] A creation unit, used to create a historical acceptance result set, where the historical acceptance result set is a set of N acceptance results obtained by accepting N historical software to be accepted using the acceptance standard database, where N is a positive integer;

[0010] A determination unit, configured to determine a training data set based on a historical acceptance result set;

[0011] A training unit, configured to train a first network model using a training data set;

[0012] An evaluation unit is used to evaluate whether the software meets the admission criteria based on the trained first network model.

[0013] The software access assessment device provided in the present application includes: a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the above method.

[0014] A computer program product provided by the present application includes: a computer program, and the computer program implements the above method when executed by a processor.

[0015] The computer-readable storage medium provided in the present application is used to store a computer program, and the computer program enables a computer to execute the above method.

[0016] In the technical solution of the present application, a historical acceptance result set is created, which is a set of N acceptance results obtained by accepting N historical software to be accepted using an acceptance standard database, where N is a positive integer; a training data set is determined based on the historical acceptance result set, and a first network model is trained using the training data set; and whether the software meets the access standard is evaluated based on the trained first network model. In this way, it is possible to accurately and efficiently evaluate whether the software is connected to the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0018] Figure 1 It is a flowchart of the software access assessment method provided in the embodiment of the present application;

[0019] Figure 2 It is a schematic diagram of an LSTM neural network constructed in an embodiment of the present application;

[0020] Figure 3 It is a flowchart of a software access assessment method in a public cloud scenario based on automated inspection rule matching and a deep learning model provided in an embodiment of the present application;

[0021] Figure 4 It is a schematic diagram of the structural composition of the software access assessment device provided in an embodiment of the present application;

[0022] Figure 5 is a schematic structural diagram of a software access assessment device provided in an embodiment of the present application;

[0023] Figure 6 It is a schematic structural diagram of the chip of an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0026] It should also be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here. The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the front and back associated objects are in an "or" relationship. It should also be understood that the "indication" mentioned in the embodiments of the present application can be a direct indication, an indirect indication, or an indication of an association relationship. For example, A indicates B, which can mean that A directly indicates B, for example, B can be obtained through A; it can also mean that A indirectly indicates B, for example, A indicates C, and B can be obtained through C; it can also mean that there is an association relationship between A and B. It should also be understood that the "correspondence" mentioned in the embodiments of the present application may indicate a direct or indirect correspondence between the two, or an association between the two, or a relationship of indication and being indicated, configuration and being configured, etc.

[0027] To facilitate understanding of the technical solutions of the embodiments of the present application, the relevant technologies of the embodiments of the present application are described below. The following related technologies can be arbitrarily combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0028] At present, in order to ensure the stability and reliability of software products and services after they are put into the network and reduce the maintenance complexity of the existing network, it is necessary to establish a set of standardized inspection items involving multiple fields at the beginning of the entire project, and establish a multi-dimensional machine learning algorithm model to evaluate whether the software can meet the existing network access conditions. There are currently several solutions:

[0029] (1) During the overall project implementation, it is necessary to determine the relevant inspection items in advance, write relevant scripts for inspection of different middleware and products involved, and determine whether they have passed. During the project delivery process, this method can evaluate the software product through relevant acceptance criteria and manual inspection of relevant configuration items to determine whether it can be put online;

[0030] (2) Implement an acceptance platform based on the established acceptance rules. After completing the software integration development and deployment in the project, submit it to the acceptance platform for acceptance and output relevant results. This method can improve the acceptance efficiency to a certain extent, record the acceptance criteria on a platform, realize acceptance automation to a certain extent, and partially reflect whether the software meets the admission conditions.

[0031] (3) Based on the historical software failure impact data, the historical software quality impact data is preprocessed to form a training sample set, and the model is trained with the training sample set to obtain a software quality assessment model. The quality of the software to be assessed is then assessed with the trained model.

[0032] The above three solutions mainly have the following problems:

[0033] (1) The first method is to achieve acceptance by writing scripts and manually comparing them. Different versions of products and components are involved. It takes a lot of manpower and time to evaluate and determine the inspection items for different products and components and to write corresponding acceptance scripts. At the same time, there is a lack of quantitative evaluation of whether the software meets the entry requirements, and it mainly depends on the experience of relevant evaluators.

[0034] (2) The second method is to enter the established acceptance criteria into the acceptance platform. Although it can standardize the acceptance of a software product or component and achieve a certain degree of automation, it does not combine multiple dimensions to evaluate whether the software can be accepted, and lacks evaluation models and methods.

[0035] (3) The third method is to establish a model training sample set based on historical software quality data, and establish a software quality assessment method through the training model. However, it does not propose standards for software access, and the evaluation dimension is relatively single, which cannot comprehensively measure whether the software can meet the access requirements.

[0036] Therefore, how to effectively and accurately evaluate whether the software is allowed to enter the market becomes a problem that needs to be considered. To this end, the following technical solutions of the embodiments of the present application are proposed.

[0037] To facilitate understanding of the technical solutions of the embodiments of the present application, the technical solutions of the present application are described in detail below through specific embodiments. The above related technologies can be combined arbitrarily with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application. The embodiments of the present application include at least part of the following contents.

[0038] Figure 1 is a flowchart of a software access assessment method provided in an embodiment of the present application, such as Figure 1 As shown, the software access assessment method includes the following steps:

[0039] Step 101: Create a historical acceptance result set, where the historical acceptance result set is a set of N acceptance results obtained by accepting N historical software to be accepted using an acceptance standard database, where N is a positive integer.

[0040] Step 102: Determine a training data set based on a historical acceptance result set, and use the training data set to train a first network model.

[0041] Step 103: Evaluate whether the software meets the admission criteria based on the trained first network model.

[0042] In some implementations, the first network model is used to evaluate whether the software meets the admission criteria. Prior to this, the first network model needs to be trained, and a training data set is obtained by creating a historical acceptance result set, and then the first network model is trained based on the training data set. The historical acceptance result set is a set of N acceptance results obtained by accepting N historical software to be accepted using the acceptance criteria database, and N is a positive integer.

[0043] In some embodiments, before creating a set of historical acceptance results, the method also includes: receiving a first request, the first request indicating a request to add a first software acceptance standard, the first request including the first software acceptance standard; determining a judgment result based on the first software acceptance standard and a first judgment rule; if the judgment result is that the first judgment rule is satisfied, determining first information of the first software acceptance standard according to a second judgment rule and a third judgment rule, the first information including an acceptance category and an acceptance item; generating a first acceptance standard set according to the first information and the first software acceptance standard, and storing the first acceptance standard set in an acceptance standard database.

[0044] In order to obtain a set of historical acceptance results, it is necessary to create an acceptance criteria database. Here, the example of adding the first software acceptance criteria to the acceptance criteria database is used for illustration.

[0045] In some embodiments, the first software acceptance standard is used as a new acceptance standard. First, it is necessary to receive a request for adding a new software acceptance standard, that is, a first request, for requesting the addition of the first software acceptance standard, and the request includes the first software acceptance standard. Then, it is determined whether the first software acceptance standard meets the first judgment rule based on the first software acceptance standard and the first judgment rule. If the first software acceptance standard meets the first judgment rule, the second judgment rule and the third judgment rule are used to obtain the first information of the first software acceptance standard, wherein the first information includes the acceptance category and the acceptance item name included in the first software acceptance standard determined according to the second judgment rule, and also includes the specific acceptance item composition included in the first software acceptance standard determined according to the third judgment rule. A first acceptance standard set is generated based on the first software acceptance standard and the first information of the first software acceptance standard, and then it is determined to belong to a specific acceptance category through review by the first reviewer and the second reviewer, and is stored in the acceptance standard database.

[0046] In some implementations, acceptance categories include product baseline configuration, high availability, software architecture, performance, security protection, etc.

[0047] Among them, the first judgment rule, the second judgment rule and the third judgment rule are preset rules, and the first judgment rule is used to preliminarily judge whether the newly added acceptance criteria can be stored in the acceptance criteria database, and then judge the specific acceptance category and specific acceptance items of the newly added acceptance criteria according to the second judgment rule and the third judgment rule. The specific first judgment rule, the second judgment rule and the third judgment rule can be determined according to the actual situation, and this application does not make specific restrictions on this.

[0048] In some implementations, the acceptance criteria in the acceptance criteria database are stored in the form of an acceptance criteria set.

[0049] In some implementations, the acceptance criteria database divides the acceptance criteria sets into acceptance categories. For example, if the first acceptance criteria set belongs to the product base station configuration dimension, the first acceptance criteria set is divided into the product baseline configuration dimension.

[0050] In some implementations, creating a historical acceptance result set includes: receiving N second requests, where the N second requests represent requests for N historical software to be accepted to access the network; selecting M acceptance standard sets from an acceptance standard database based on the N second requests; where M is a positive integer, and M is greater than or equal to N; and determining N acceptance results based on the M acceptance standard sets and the N historical software to be accepted.

[0051] In some embodiments, after establishing the acceptance criteria database, it is necessary to use the acceptance criteria database to accept the historical software to be accepted and obtain the corresponding historical acceptance results. First, it is necessary to receive N historical software to be accepted for network access acceptance, that is, N second requests. According to the N second requests, M acceptance criteria sets corresponding to the N historical software to be accepted are selected from the acceptance criteria database, where M is a positive integer, M is greater than or equal to N. The N historical software to be accepted are accepted using the M acceptance criteria sets to obtain N acceptance results. It can be understood that, according to the acceptance request, at least one acceptance criteria set is selected to accept the software and the acceptance result corresponding to the software is obtained. Exemplarily, the acceptance items in the acceptance criteria set include acceptance items in multiple dimensions such as product baseline configuration, high availability, software architecture, performance, and security protection. According to the request, the acceptance dimension that the software needs to select is determined, and the corresponding acceptance criteria set in the acceptance dimension is obtained, and the software is accepted using the acceptance criteria set. Therefore, the acceptance result of each historical software to be accepted includes the acceptance result of at least one dimension.

[0052] In some implementations, different weights are assigned to the acceptance results of multiple dimensions, and the software is comprehensively evaluated based on the weights of the acceptance results of different dimensions to obtain a true score for the software to be accepted.

[0053] In some implementations, the historical acceptance result set includes the actual score of the software to be accepted.

[0054] In some embodiments, determining a training data set based on the historical acceptance result set includes: determining N feature vectors based on the historical acceptance result set through a second network model, wherein each feature vector includes text information of each acceptance result and frequency information corresponding to the text information; determining a training data set based on the N feature vectors.

[0055] In some embodiments, the second network model is a word vector model, which is used to determine a feature vector for each acceptance result based on a set of historical acceptance results, wherein each feature vector includes text information of each acceptance result and frequency information corresponding to the text information, and then a training data set is determined based on the N feature vectors.

[0056] Specifically, after forming a historical acceptance result set (also called a historical acceptance result database), the text data therein needs to be converted into vectors for training the deep learning model. The embodiment of the present application uses a word vector model to process text data, which can extract the frequency of occurrence of different types of words in a single text and convert them into feature vectors, which is more conducive to improving the prediction accuracy of the model. Based on this, the specific process is described as follows:

[0057] The text in the historical acceptance result set is segmented by dimension, product, system, component, inspection category, inspection item name, acceptance script, expected result, and actual result to obtain the vocabulary table V.

[0058] Assume that a single piece of text data consists of n words belonging to different categories, which can be recorded as [w1,w2,...,w n ], through the word vector model, we can count the frequency of occurrence of words of different categories in the text and construct a feature vector representation. Each dimension of the feature vector corresponds to a category of words, which is used to represent the number of times the word appears in the text. Its expression is shown in formula (1):

[0059]

[0060] Among them, δ(w j ,v i ) is an indicator function, which takes the value 1 when the vocabulary w and v are the same, and 0 otherwise.

[0061] Through the above method, each text in the database can be represented as a fixed-length feature vector, which reflects the frequency of occurrence of different types of words in the text. Through the word vector model, the historical acceptance result set can be converted into an acceptance result training set for training the deep learning model, that is, training the first network model.

[0062] In some implementations, the feature vector of a single text is obtained through the word vector model, that is, f [i] , then construct the input sequence of the first network model, which is expressed as follows (2):

[0063] X=(f [1] ,f [2] ,...,f [M] ) (2);

[0064] Among them, f [M] Represents the Mth acceptance result of software acceptance.

[0065] In some embodiments, training the first network model using a training data set includes: training the first network model according to the training data set and a first parameter set, wherein the training data set is the input of the first network model, and the first parameter set includes a loss function, a number of model iterations, and a learning rate.

[0066] In some embodiments, the first network model is a long short-term memory network (LSTM), and the first network model is trained using the aforementioned training data set according to the settings of the first parameter set, wherein the first parameter set is the setting parameters for the first network model, including the loss function, the number of model iterations, and the learning rate.

[0067] In some implementations, the loss function used in the embodiments of the present application is shown in the following formula (3):

[0068]

[0069] Where M is the number of acceptance results, It is expressed as the model prediction value of the i-th acceptance result, f i Represents the true value of the i-th acceptance result.

[0070] Specifically, assume that the evaluation score of the software to be accepted through the first network model is It can be expressed as shown in formula (4):

[0071]

[0072] Among them, V i The expression of (t) is as shown in the following formula (5):

[0073]

[0074] Among them, tanh() is the hyperbolic tangent function, is the weight connecting the input y(t) and the jth hidden neuron, and M represents the number of hidden neurons. After the calculation of the tanh function, the output v of each hidden neuron i,j The range is [-1,1], and j represents the jth hidden neuron. i (t) is the output set of each hidden neuron, which is actually a state vector. i is to connect hidden neurons to output The weight between them. i (t) and z i The linear combination of (t) can represent the true value f of the score of the software to be accepted. i Estimated value of The actual value of the score of the software to be accepted is obtained from the historical acceptance result set.

[0075] In some embodiments, the constructed first network model includes an input layer, two LSTM layers, two Droput layers, a Dense layer, and an output layer.

[0076] In some implementations, during the iterative training of the model, in order to solve the problems of a large number of training iterations and overfitting of the training results, it is necessary to perform gradient descent in each iteration, and its expression is shown in the following formula (6):

[0077]

[0078] Wherein, α represents the learning rate, which is used to control the step size of parameter update. During the model iteration process, the number of model iterations and overfitting can be controlled by changing the learning rate. Selecting an appropriate learning rate can improve convergence and stability during model training, avoid overfitting, and jump out of local optimum. The specific learning rate and number of model iterations can be determined according to actual conditions, and this application does not make specific restrictions on this. Wherein W and b are parameters of the model, and the specific contents can be determined according to actual conditions, and the parameters of the model are updated by formula (6).

[0079] In some embodiments, the method further includes: determining an evaluation result based on a first indicator set and a first threshold set, the first indicator set including a recall rate and an F1 value; and the evaluation result is used to evaluate the training status of the first network model.

[0080] In some implementations, the training result of the first network model is obtained by comparing indicators such as recall rate and F1 value with corresponding thresholds.

[0081] In some embodiments, the method further includes: if the evaluation result indicates that the first network model is not available, retraining the first network model; if the evaluation result indicates that the first network model is available, saving the first network model.

[0082] In some implementations, the model performance is evaluated based on indicators such as recall rate and F1 value. If the training model does not achieve good results after reaching the maximum number of iterations, the relevant parameters are initialized and retrained; if the training model reaches the relevant indicators, the trained model is saved to the system for software evaluation.

[0083] In some implementations, when the new software to be evaluated passes the acceptance of the acceptance standard database, a corresponding acceptance result is obtained, and the first network model is updated according to the acceptance result.

[0084] The technical solution of the embodiment of the present application creates a historical acceptance result set, which is a set of N acceptance results obtained by accepting N historical software to be accepted using an acceptance standard database, where N is a positive integer; determines a training data set based on the historical acceptance result set, trains a first network model using the training data set; and evaluates whether the software meets the access standard based on the trained first network model. In this way, it is possible to accurately and efficiently evaluate whether the software is connected to the network.

[0085] The technical solution of the embodiment of the present application is illustrated below with reference to specific examples.

[0086] The embodiment of the present application proposes a software access assessment method in a public cloud scenario based on an automated inspection rule matching set and a machine learning algorithm. The embodiment of the present application first establishes a fast and effective acceptance standard set generation system for the inspection item evaluation and corresponding script writing that require a lot of work before the software enters the network in the public cloud scenario; secondly, the acceptance standard set can be divided into multiple dimensions such as A, B, C, D, etc., and the acceptance results of the acceptance standard sets of different dimensions are set to different weights, and a standard database is established based on historical acceptance results, which is used to train the deep learning model, extract the relevant features of the software to be evaluated, and improve the evaluation accuracy of the software to be evaluated; it provides a new solution for the software access assessment method in the public cloud scenario.

[0087] At present, with the continuous development of the cloud computing market, software products such as software operation services (SaaS) and platform as a service (PAAS) in the public cloud field are also increasing. At the same time, in order to meet customer needs, the frequency of software version iterations and function launches has increased rapidly, which has brought new challenges to the stability of the public cloud, customer experience, and related operation and maintenance requirements. Therefore, the embodiment of the present application establishes a standardized automated acceptance system, summarizes the acceptance results of different dimensions, and forms a set of historical acceptance results; and uses word embedding technology to convert relevant acceptance results into relevant vector representations, perform feature extraction, and construct a model training set; use the training data set to train the LSTM model, and select the mean square error as the loss function and the adaptive learning rate as the optimization algorithm. Finally, the model is used to verify the evaluation quality of the software to be evaluated by cross-validation and performance evaluation.

[0088] A software access assessment method in a public cloud scenario based on automated inspection rule matching and deep learning models first needs to establish assessment methods and models of different dimensions for software products in public cloud scenarios. The embodiment of the present application classifies different acceptance items from multiple dimensions such as product baseline configuration, high availability, software architecture, performance, and security protection in a public cloud scenario, and evaluates the weights of acceptance results in different dimensions. The specific description is as follows:

[0089] If a request for additional software acceptance criteria is accepted, the relevant additional acceptance criteria must meet the first judgment rule;

[0090] After the newly added acceptance criteria meet the first judgment rule, the acceptance category and acceptance item name included in the newly added acceptance criteria are determined according to the second judgment rule;

[0091] After the newly added acceptance criteria meet the first judgment rule, the composition of the acceptance items included in the newly added acceptance criteria is determined according to the third judgment rule;

[0092] After the acceptance categories and specific acceptance items included in the newly added acceptance criteria are determined through the third rule, an acceptance criteria set is generated, which is reviewed by the first reviewer and the second reviewer respectively to confirm that it belongs to a specific category and is stored in the existing acceptance criteria set database.

[0093] After establishing acceptance inspection standards of different dimensions through the standardized automated acceptance system, the historical acceptance results of the automated acceptance platform can be summarized into a historical acceptance result set. The specific instructions are as follows:

[0094] Accept the software network acceptance request initiated by the delivery side.

[0095] According to the software network acceptance request, select the corresponding acceptance standard set, which includes multiple dimensions such as product baseline configuration, high availability, software architecture, performance, and security protection.

[0096] According to the acceptance criteria items in the acceptance criteria set, the software to be accepted is accepted through the automated acceptance platform.

[0097] After the acceptance is completed, the acceptance results will be summarized into the historical acceptance result set.

[0098] After forming a set of historical acceptance results, the text data in it needs to be converted into vectors for training deep learning models. The embodiment of the present application uses a word vector model to process text data, which can extract the frequency of occurrence of different types of words in a single text and convert them into feature vectors, which is more conducive to improving the prediction accuracy of the model. According to the above description, the specific process is as follows:

[0099] The text in the historical acceptance result set is segmented according to dimension, product, system, component, acceptance category, acceptance item name, acceptance script, expected result, and actual result to obtain the vocabulary table V.

[0100] Assume that a single piece of text data consists of n words belonging to different categories, which can be recorded as [w1,w2,...,w n ], through the word vector model, we can count the frequency of occurrence of words of different categories in the text and construct a feature vector representation. Each dimension of the feature vector corresponds to a category of words, which is used to represent the number of times the word appears in the text. Its expression is shown in formula (1).

[0101] Through the above method, each text in the database can be represented as a fixed-length feature vector, which reflects the frequency of occurrence of different types of words in the text. Through the word vector model, the historical acceptance result set can be converted into an acceptance result training set for training deep learning models.

[0102] The embodiment of this application selects the LSTM model to evaluate whether the software can meet the existing network access standards. The features extracted by the word vector model and the related weight settings of different dimensions can improve the accuracy of software evaluation and improve efficiency. The specific instructions are as follows:

[0103] The feature vector of a single text has been obtained through the word vector model, that is, f [i] , then construct the input sequence of the LSTM model, and its expression is shown in formula (2). Assume that the evaluation score of the software is The expression of is shown in formula (4).

[0104] Figure 2 Schematic diagram of the LSTM neural network constructed in the embodiment of the present application. Figure 2 As shown, the LSTM neural network includes an input layer, two LSTM layers, two Droput layers, a Dense layer and an output layer.

[0105] In the process of neural network training, due to problems such as overfitting, it is necessary to choose a suitable loss function to measure the estimated value. With the true value f i According to the results of the actual model training process, the loss function selected is the mean square error, and its expression is shown in formula (3).

[0106] In the iterative training process of the model, in order to solve the problems of too many training iterations and overfitting of training results, it is necessary to perform gradient descent in each iteration. The expression is shown in formula (6).

[0107] Based on this, a flowchart of a software access assessment method in a public cloud scenario based on automated inspection rule matching and deep learning models is shown below: Figure 3 The specific implementation process is as follows:

[0108] Step 301: Establish a standard acceptance set;

[0109] According to preset rules, a standard acceptance set is established and uploaded to the automated acceptance platform.

[0110] Step 302: Aggregate the acceptance results into a historical acceptance result set;

[0111] The acceptance results of the automated acceptance platform are summarized into a historical acceptance result set.

[0112] Step 303: Divide the standard acceptance items according to preset dimensions;

[0113] Divide the standard acceptance items in the historical acceptance result set according to preset dimensions.

[0114] Step 304: extracting features using a word vector model;

[0115] Through the word vector model, features of each acceptance result in the historical acceptance result set are extracted and converted into a one-dimensional vector.

[0116] Step 305: converting the extracted features into a feature matrix;

[0117] The extracted feature vector is converted into a feature matrix as input to the deep learning model.

[0118] Step 306: Setting the initial parameters of the model;

[0119] Set the relevant parameters of the deep learning model and perform model training.

[0120] Step 307: Train the model.

[0121] The model training is completed based on indicators such as recall rate and F1 value.

[0122] Step 308: Determine whether overfitting occurs;

[0123] If overfitting occurs, execute step 309; otherwise, execute step 310;

[0124] Step 309: Parameter tuning.

[0125] If overfitting occurs, continue training after tuning the model parameters.

[0126] Step 310: Save the model results.

[0127] Step 311: Determine whether the maximum number of training times has been reached.

[0128] If yes, execute step 312 , otherwise execute step 307 .

[0129] Step 312: Save the model results.

[0130] The model is saved and used to evaluate the software admission score.

[0131] The specific process description of a software access assessment method in a public cloud scenario based on automated inspection rule matching and deep learning models is as follows:

[0132] Step 1: Submit standard acceptance items according to the platform preset rules. Each acceptance item must include product, system, component, acceptance category, acceptance item name, acceptance script, expected results, and actual results;

[0133] Step 2: Use the automated acceptance platform to perform standard acceptance on the software to be evaluated, generate acceptance results, and divide them into multiple dimensions such as product baseline configuration, high availability, software architecture, performance, and security protection, and store them in a historical acceptance result set;

[0134] Step 3: Use the word vector model to extract features from each acceptance item and the corresponding acceptance result in the historical acceptance result data. The feature extraction formula is shown in formula (1);

[0135] Step 4: Convert the feature vector into a feature matrix for model input. The feature matrix formula is shown in formula (2).

[0136] Step 5: Build a deep learning model, set the neural network layer, activation function, convolution kernel, number of convolution layers, and Dropout value. For details, see Figure 2 As shown;

[0137] Step 6: Perform model training and set the maximum number of iterations. The embodiment of the present application selects the mean square error as the loss function for model training. The mean square error can effectively measure the gap between the true value and the model's estimated value. By adding the mean square error, the model can be effectively converged, the stability of the model can be improved, overfitting can be avoided, and local optimum can be jumped out;

[0138] Step 7: Evaluate the model performance based on the recall rate, F1 value and other indicators. If the training model does not achieve good results after reaching the maximum number of iterations, return to initialize the relevant parameters and retrain; if the training model reaches the relevant indicators, save the trained model to the system for software evaluation;

[0139] Step 8: As the historical standard database is updated, repeat steps 1-7 to iteratively optimize the model trained last time.

[0140] The technical solution of the embodiment of the present application establishes standard acceptance items through preset rules, and stipulates the fields contained in each acceptance item, which can improve the data validity in the subsequent feature extraction process and is more conducive to the performance improvement after model training; through the automated acceptance platform, the acceptance results are summarized into a historical acceptance result set, which is continuously updated, expanding the amount of information that can be used by the model and improving the sample diversity; through word vector technology, feature extraction is performed on each text in the historical acceptance result set, and it is converted into a feature matrix for model input, thereby improving the evaluation ability of the model and measuring the recall rate and F1 value of the model performance; the final output model is used to evaluate whether the software can meet the network access standards. In summary, the above improved methods provide a new solution for the software network access evaluation method in the public cloud scenario, which can complete the evaluation of software under standardized and automated conditions, greatly improving the stability of the public cloud existing network.

[0141] The key points of the embodiment of the present application are the establishment and conversion process of the acceptance standard database and the training set, as well as the implementation process of the software access evaluation method based on automated inspection rule matching and deep learning model. Compared with other schemes, the technical scheme of the embodiment of the present application has the stability of the existing network. With the rapid growth of the public cloud market, major public cloud vendors are also constantly launching new software products for demanders to choose from, and have higher requirements for software network access evaluation and existing network stability in the current public cloud scenario, from the original single-dimensional network access evaluation method relying on relevant auditors to relying on standardized acceptance items and acceptance systems for software network access evaluation. At present, there are not many known software network access evaluation methods, and most of the solutions still rely on the experience of relevant reviewers. Even if some methods have been used to evaluate the software through standardized and systematic platforms, manual access is still often required, and multi-dimensional, high-efficiency and relatively accurate evaluation capabilities cannot be achieved. At the same time, these methods often have lags in the face of standard updates, and cannot evaluate the software network access according to the latest standards, and may even lead to network access with illness, thereby affecting the stability of the existing network under the public cloud. The embodiment of the present application provides a software access assessment method in a public cloud scenario based on automated inspection rule matching and deep learning models. Taking into account the three aspects of standardization, assessment efficiency and assessment quality involved in software assessment, a solution combining automated inspection rule matching and deep learning models is selected. First, a standardized inspection set is established through standardized inspection items, which are summarized into an automated acceptance platform. At the same time, the acceptance results are updated in real time to the historical acceptance result set, which can achieve database standardization and is conducive to subsequent model training; secondly, the historical acceptance result data is extracted through a word vector model and converted into a feature matrix as the input of the model. Without losing features, the amount of data is greatly reduced, the evaluation ability of the model is improved, and the recall rate and F1 value of the model performance are measured; finally, the mean square error is selected as the loss function in the model training process, and the model parameters are tuned to meet the relevant indicators. Combining the above improved methods, a new solution can be provided in the public cloud scenario, that is, a software access assessment scenario with a large number and a large number of iterations.

[0142] Figure 4 Schematic diagram of the structure of the software access assessment device provided in the embodiment of the present application. Figure 4 As shown, the software access assessment device includes:

[0143] The creation unit 401 is used to create a historical acceptance result set, where the historical acceptance result set is a set of N acceptance results obtained by accepting N historical software to be accepted using the acceptance standard database, where N is a positive integer;

[0144] A determination unit 402, configured to determine a training data set based on a historical acceptance result set;

[0145] A training unit 403, configured to train a first network model using a training data set;

[0146] The evaluation unit 404 is used to evaluate whether the software meets the admission criteria based on the trained first network model.

[0147] In some embodiments, before creating a set of historical acceptance results, the device also includes: a processing unit 405; the processing unit 405 is used to receive a first request, the first request indicates a request to add a first software acceptance standard, and the first request includes the first software acceptance standard; determine a judgment result based on the first software acceptance standard and a first judgment rule; if the judgment result is that the first judgment rule is satisfied, determine first information of the first software acceptance standard according to the second judgment rule and the third judgment rule, the first information includes an acceptance category and an acceptance item; generate a first acceptance standard set according to the first information and the first software acceptance standard, and store the first acceptance standard set in an acceptance standard database.

[0148] In some embodiments, the creation unit 401 is used to receive N second requests, where the N second requests represent requests for N historical software to be accepted to enter the network; select M acceptance standard sets from an acceptance standard database based on the N second requests; where M is a positive integer, and M is greater than or equal to N; and determine N acceptance results based on the M acceptance standard sets and the N historical software to be accepted.

[0149] In some embodiments, the determination unit 402 is used to determine N feature vectors based on a set of historical acceptance results through a second network model, wherein each feature vector includes text information of each acceptance result and frequency information corresponding to the text information; and determine a training data set based on the N feature vectors.

[0150] In some embodiments, the training unit 403 is used to train the first network model according to the training data set and the first parameter set, wherein the training data set is the input of the first network model, and the first parameter set includes a loss function, a number of model iterations, and a learning rate.

[0151] In some embodiments, the loss function is: Where M is the number of acceptance results, It is expressed as the model prediction value of the i-th acceptance result, f i Represents the true value of the i-th acceptance result.

[0152] In some implementations, the determination unit 402 is used to determine an evaluation result based on a first indicator set and a first threshold set, where the first indicator set includes a recall rate and an F1 value; the evaluation result is used to evaluate the training status of the first network model.

[0153] In some implementations, the processing unit 405 is configured to retrain the first network model if the evaluation result indicates that the first network model is unavailable; and save the first network model if the evaluation result indicates that the first network model is available.

[0154] Those skilled in the art should understand that Figure 4 The implementation functions of each unit in the software access assessment device shown can be understood by referring to the relevant description of the aforementioned method. Figure 4 The functions of each unit in the software access assessment device shown can be implemented by a program running on a processor, or by a specific logic circuit.

[0155] Figure 5 It is a schematic structural diagram of a software access assessment device 500 provided in an embodiment of the present application. Figure 5 The software access assessment device 500 shown includes a processor 510, and the processor 510 can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0156] Alternatively, if Figure 5 As shown, the software access assessment device 500 may further include a memory 520. The processor 510 may call and run a computer program from the memory 520 to implement the method in the embodiment of the present application.

[0157] The memory 520 may be a separate device independent of the processor 510 , or may be integrated into the processor 510 .

[0158] Alternatively, if Figure 5 As shown, the software access evaluation device 500 may further include a transceiver 530, and the processor 510 may control the transceiver 530 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices.

[0159] The transceiver 530 may include a transmitter and a receiver. The transceiver 530 may further include an antenna, and the number of the antennas may be one or more.

[0160] The software access assessment device 500 can implement the corresponding processes implemented by the software access assessment apparatus in each method of the embodiments of the present application, and for the sake of brevity, they will not be described in detail here.

[0161] Figure 6It is a schematic structural diagram of the chip of an embodiment of the present application. Figure 6 The chip 600 shown includes a processor 610, which can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0162] Alternatively, if Figure 6 As shown, the chip 600 may further include a memory 620. The processor 610 may call and run a computer program from the memory 620 to implement the method in the embodiment of the present application.

[0163] The memory 620 may be a separate device independent of the processor 610 , or may be integrated into the processor 610 .

[0164] Optionally, the chip 600 may further include an input interface 630. The processor 610 may control the input interface 630 to communicate with other devices or chips, and specifically, may obtain information or data sent by other devices or chips.

[0165] Optionally, the chip 600 may further include an output interface 640. The processor 610 may control the output interface 640 to communicate with other devices or chips, and specifically, may output information or data to other devices or chips.

[0166] The chip can implement the corresponding processes implemented by the software access evaluation device in each method of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0167] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0168] It should be understood that the processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and performed. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0169] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0170] It should be understood that the above-mentioned memory is exemplary but not restrictive. For example, the memory in the embodiments of the present application may also be static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM), etc. That is to say, the memory in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.

[0171] An embodiment of the present application also provides a computer program product, including a computer program.

[0172] When the computer program is executed by the processor, it implements the corresponding processes implemented by the software access evaluation device in each method of the embodiments of the present application. For the sake of brevity, it will not be repeated here.

[0173] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0174] The computer program enables the computer to execute the corresponding processes implemented by the software access evaluation device in each method of the embodiments of the present application, which will not be described in detail here for the sake of brevity.

[0175] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0176] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0177] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0178] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0180] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0181] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A software access assessment method, characterized in that: The method comprises: Creating a historical acceptance result set, wherein the historical acceptance result set is a set of N acceptance results obtained by accepting N historical software to be accepted using the acceptance standard database, where N is a positive integer; Determine a training data set based on the historical acceptance result set, and use the training data set to train a first network model; Evaluate whether the software meets the admission criteria based on the trained first network model.

2. The method according to claim 1, characterized in that: Before creating the historical acceptance result set, the method further includes: receiving a first request, wherein the first request indicates a request to add a first software acceptance criterion, and the first request includes the first software acceptance criterion; Determine a judgment result based on the first software acceptance standard and the first judgment rule; If the judgment result satisfies the first judgment rule, determining first information of the first software acceptance standard according to the second judgment rule and the third judgment rule, the first information including the acceptance category and the acceptance item; A first acceptance criterion set is generated according to the first information and the first software acceptance criterion, and the first acceptance criterion set is stored in the acceptance criterion database.

3. The method according to claim 1, characterized in that The creating of the historical acceptance result set includes: Receiving N second requests, wherein the N second requests represent a request for the N historical software to be accepted to enter the network; Selecting M acceptance criterion sets from the acceptance criterion database according to the N second requests; wherein M is a positive integer, and M is greater than or equal to N; The N acceptance results are determined according to the M acceptance standard sets and the N historical software to be accepted.

4. The method according to claim 1, characterized in that: The determining of the training data set based on the historical acceptance result set comprises: Determine N feature vectors according to the historical acceptance result set by a second network model, wherein each feature vector includes text information of each acceptance result and frequency information corresponding to the text information; The training data set is determined according to the N feature vectors.

5. The method according to claim 4, characterized in that The step of training the first network model using the training data set includes: The first network model is trained according to the training data set and a first parameter set, wherein the training data set is an input of the first network model, and the first parameter set includes a loss function, a number of model iterations, and a learning rate.

6. The method according to claim 5, characterized in that The loss function is: Where M is the number of acceptance results, It is expressed as the model prediction value of the i-th acceptance result, f i Represents the true value of the i-th acceptance result.

7. The method according to claim 6, characterized in that The method further comprises: An evaluation result is determined according to a first indicator set and a first threshold set, wherein the first indicator set includes a recall rate and an F1 value; and the evaluation result is used to evaluate the training status of the first network model.

8. The method according to claim 7, characterized in that The method further comprises: If the evaluation result indicates that the first network model is unavailable, retraining the first network model; If the evaluation result indicates that the first network model is available, the first network model is saved.

9. A software access assessment device, characterized in that: The device comprises: A creation unit, used to create a historical acceptance result set, wherein the historical acceptance result set is a set of N acceptance results obtained by accepting N historical software to be accepted using an acceptance standard database, where N is a positive integer; A determination unit, configured to determine a training data set based on the historical acceptance result set; A training unit, configured to train a first network model using the training data set; An evaluation unit is used to evaluate whether the software meets the admission criteria based on the trained first network model.

10. A software access assessment device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 8.

11. A computer program product, characterized in that include: A computer program which, when executed by a processor, implements the method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program causes a computer to execute the method according to any one of claims 1 to 8.

Images