Model training method and system, prediction method and system, electronic equipment and medium

Abstract

According to the model training method and system, the prediction method and system, the electronic equipment and the medium provided by the invention, the defect generated under the specific development configuration data can be determined through the historical defect data, so that the corresponding historical development maturity is obtained; the development maturity prediction model is trained through the historical development maturity and the historical development configuration data, so that the trained development maturity prediction model can understand the relevance between the development configuration data and the development maturity; the development maturity can be predicted through the real-time development configuration data, and efficient identification and monitoring of defect anomalies in the software development process are achieved.

Description

Technical Field

[0001] The present invention relates to the field of software development, and in particular to a model training method, a prediction method, a system, an electronic device and a medium. Background Art

[0002] With the rapid development of the automotive industry towards intelligence and networking, the complexity and code volume of automotive software have increased dramatically, and software quality assurance has become a major challenge facing the automotive manufacturing industry. Therefore, how to efficiently identify and monitor software defects and anomalies during the development life cycle of vehicle software has become an issue that has received increasing attention. Summary of the invention

[0003] The main purpose of the present invention is to propose a model training method, a prediction method, a system, an electronic device and a medium, aiming to solve the problem of how to efficiently identify and monitor software defects and anomalies in the prior art.

[0004] To achieve the above object, the present invention provides a model training method, which comprises:

[0005] Acquire historical development configuration data and historical defect data corresponding to the historical development configuration data, wherein the historical defect data is defect data generated when development is performed using the historical development configuration data;

[0006] Identifying the severity of defects in the historical defect data to determine the historical development maturity corresponding to the historical defect data;

[0007] Acquire an initial development maturity prediction model, and train the initial development maturity prediction model using the historical development maturity and the historical development configuration data to obtain a trained development maturity prediction model;

[0008] Real-time development configuration data is acquired, and prediction is performed based on the real-time development configuration data using the trained development maturity prediction model to obtain predicted development maturity corresponding to the real-time development configuration data.

[0009] Optionally, the historical defect data includes a plurality of defect sub-data; and the step of identifying the defect severity of the historical defect data to determine the historical development maturity corresponding to the historical defect data includes:

[0010] Obtaining the degree of training completion level recognition model;

[0011] Using the trained severity level recognition model, the defect sub-data in the historical defect data are classified and identified by severity level to obtain a severity level array, wherein the severity level array includes the number of defect sub-data corresponding to different severity levels;

[0012] The historical development maturity is determined based on the severity level array.

[0013] Optionally, the performing severity classification and identification on the defect sub-data in the historical defect data by using the trained severity level identification model includes:

[0014] Obtain the trained defect recognition model of the same type;

[0015] Performing similar defect recognition on the defect sub-data by using the trained similar defect recognition model to determine similar sub-data with the same defect type;

[0016] Merging the sub-data of the same type with the same defect type in the historical defect data into one defect sub-data to obtain data without the same type of defects, wherein the defect types of any two of the defect sub-data in the data without the same type of defects are different;

[0017] The defect sub-data in the defect-free data are classified and identified by severity through the trained severity level identification model.

[0018] Optionally, the similar defect recognition model completed through the training performs similar defect recognition on the defect sub-data, and determining similar sub-data with the same defect type includes:

[0019] Obtaining the defect title in each defect sub-data;

[0020] Combining the defective titles in pairs to obtain multiple title combinations;

[0021] Performing similar defect identification on the defect titles in the title combination by using the trained similar defect identification model to determine whether the defect types of any two defect titles are the same;

[0022] The defect sub-data corresponding to the defect titles with the same defect type are clustered to obtain the same type of sub-data.

[0023] Optionally, the severity classification and identification of the defect sub-data in the defect data without the same type by the severity classification model trained includes:

[0024] Acquire manually annotated data corresponding to the historical defect data, wherein the manually annotated data includes an array of annotated sub-data of the same type and an array of annotated severity levels;

[0025] determining a first difference between the annotated similar sub-data and the similar sub-data, and a second difference between the annotated severity level array and the severity level array;

[0026] The first difference and the second difference are displayed.

[0027] Optionally, the training the initial development maturity prediction model by using the historical development maturity and the historical development configuration data includes:

[0028] Get the severity level array corresponding to the historical development maturity;

[0029] The historical development configuration data is used as a feature variable and the severity level array is used as a target variable to train the initial development maturity prediction model.

[0030] To achieve the above object, the present invention further provides a development maturity prediction method, which is applied to the model training method according to any one of claims 1 to 5; the development maturity prediction method comprises:

[0031] Obtain real-time development configuration data and the trained development maturity prediction model;

[0032] The trained development maturity prediction model performs prediction based on the real-time development configuration data to obtain the predicted development maturity corresponding to the real-time development configuration data.

[0033] Optionally, the development maturity prediction model completed through the training performs prediction based on the real-time development configuration data, and obtaining the predicted development maturity corresponding to the real-time development configuration data includes:

[0034] The trained development maturity prediction model performs prediction based on the real-time development configuration data to obtain a prediction severity level array, wherein the prediction severity level array includes a plurality of predicted defect quantities corresponding to different severity levels;

[0035] For each of the severity levels, determining a preset weight for the severity level;

[0036] The predicted development maturity corresponding to the real-time development configuration data is calculated based on the predicted number of defects corresponding to each severity and the preset weight.

[0037] To achieve the above object, the present invention further provides a development maturity prediction system, the development maturity prediction system comprising:

[0038] A first acquisition module, used to acquire historical development configuration data and historical defect data corresponding to the historical development configuration data, wherein the historical defect data is defect data generated when development is performed using the historical development configuration data;

[0039] A first identification module is used to identify the severity of defects in the historical defect data to determine the historical development maturity corresponding to the historical defect data;

[0040] A second acquisition module is used to acquire an initial development maturity prediction model, and train the initial development maturity prediction model through the historical development maturity and the historical development configuration data to obtain a trained development maturity prediction model;

[0041] The third acquisition module is used to acquire real-time development configuration data, and to perform prediction based on the real-time development configuration data through the trained development maturity prediction model to obtain the predicted development maturity corresponding to the real-time development configuration data.

[0042] To achieve the above-mentioned purpose, the present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the model training method or the development maturity prediction method as described above.

[0043] To achieve the above objectives, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the model training method or the development maturity prediction method as described above are implemented.

[0044] The present invention proposes a model training method, prediction method, system, electronic device and medium, which obtains historical development configuration data and historical defect data corresponding to the historical development configuration data, wherein the historical defect data is defect data generated when the historical development configuration data is used for development; identifies the defect severity of the historical defect data to determine the historical development maturity corresponding to the historical defect data; obtains an initial development maturity prediction model, and trains the initial development maturity prediction model through the historical development maturity and the historical development configuration data to obtain a trained development maturity prediction model. The defects generated under specific development configuration data can be determined through the historical defect data to obtain the corresponding historical development maturity, and the development maturity prediction model is trained through the historical development maturity and the historical development configuration data, so that the trained development maturity prediction model can understand the correlation between the development configuration data and the development maturity, so that in the software development process, the development maturity can be predicted through the real-time development configuration data, and the defect anomalies in the software development process can be efficiently identified and monitored. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0047] Figure 1 This is a flow chart of the first embodiment of the model training method of the present invention;

[0048] Figure 2 A schematic flow chart of a first embodiment of a maturity prediction method developed for the present invention;

[0049] Figure 3 It is a detailed flow chart of the model training method of the present invention;

[0050] Figure 4 It is a data display schematic diagram of the model training method of the present invention;

[0051] Figure 5 A module structure diagram of a maturity prediction system developed for the present invention;

[0052] Figure 6 It is a schematic diagram of the module structure of the electronic device of the present invention. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work should fall within the scope of protection of the present application.

[0054] The present invention provides a model training method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the model training method of the present invention, the method comprising the steps of:

[0055] Step S10, acquiring historical development configuration data and historical defect data corresponding to the historical development configuration data, wherein the historical defect data is defect data generated when development is performed using the historical development configuration data;

[0056] Historical development configuration data refers to the development configuration data set in completed software development; development configuration data is used to index specific project settings in software development, including but not limited to development time, release rhythm, number of testers, number of developers, number of test cases, number of test vehicles and other vehicle project information; development configuration data can be set in table form.

[0057] The historical defect data is the defect data generated when the development is carried out with the historical development configuration data; it is understandable that software defect problems will arise in the process of software development; specifically, different parameters in the development configuration data have a certain impact on the defect problems caused by software development, such as a short development time, which may lead to more defect problems, a fast release rhythm, which may lead to more defect problems, a small number of testers, which may lead to more defect problems, a large number of developers, which may lead to more defect problems, a large number of test cases, which may lead to fewer defect problems, and a large number of test vehicles, which may lead to fewer defect problems; it should be noted that the correspondence between the above-mentioned different parameters and the probability of occurrence of defect problems is only for explanation. In different software development scenarios, the influence of specific parameters on the number of defect problems caused varies; the historical defect data contains relevant information about the defect problems, including but not limited to defect title, defect description, and defect type.

[0058] The historical development configuration data and the corresponding historical defect data can be obtained through the completed software development logs, and can also be obtained through other channels.

[0059] Step S20, identifying the defect severity of the historical defect data to determine the historical development maturity corresponding to the historical defect data;

[0060] The DV (Design Verification) value is used to measure the degree of defects in software development. Specifically, defects can be divided into different categories based on severity, such as H, M, and L. The correspondence between severity and defect can be set based on actual needs. The DV value is determined by the number of defects in different severity levels, including:

[0061] DV=aH+bM+cL

[0062] Among them, H, M, and L are the number of defect problems included in the corresponding severity, a, b, and c are the preset weights of the corresponding severity, and the specific values ​​can be set based on actual needs.

[0063] It can be seen that the historical development maturity needs to be determined based on the type of defect problem. Therefore, in this embodiment, the defect severity of the historical defect data is identified to determine the number of defect problems of different severity levels, thereby determining the corresponding historical development maturity.

[0064] Step S30, obtaining an initial development maturity prediction model, and training the initial development maturity prediction model using the historical development maturity and the historical development configuration data to obtain a trained development maturity prediction model;

[0065] The initial development maturity prediction model is the development maturity prediction model before training with historical development maturity and historical development configuration data; it is understandable that when the development maturity prediction model is not trained, the trained development maturity prediction model is obtained by training with historical development maturity and historical development configuration data; when the initial development maturity prediction model has been trained, the initial development maturity prediction model is trained with historical development maturity and historical development configuration data to update the development maturity prediction model. It is understandable that after the software development is completed, the development configuration data and defect data generated by this software development can be used as historical development configuration data and historical defect data to update the development maturity prediction model.

[0066] Correspondingly, the present application also provides a development maturity prediction method, which is applied to the model training method as described above, see Figure 2 , the development maturity prediction method comprises:

[0067] Step S40, obtaining real-time development configuration data;

[0068] Step S50 , predicting based on the real-time development configuration data using the trained development maturity prediction model to obtain a predicted development maturity corresponding to the real-time development configuration data.

[0069] The real-time development configuration data is the real-time development configuration data of the current software development cycle; that is, the real-time development configuration data is the development configuration data of the currently ongoing software development application.

[0070] The development maturity prediction model is trained based on historical development configuration data and historical development maturity. Therefore, the trained development maturity prediction model can understand the correlation between development configuration data and development maturity. Therefore, the trained development maturity prediction model can predict development maturity based on real-time development configuration data to obtain predicted development maturity.

[0071] It can be understood that the predicted development maturity indicates the development maturity under the current development configuration data. Therefore, developers can specify goals for different development stages based on the predicted development maturity, as well as focus on related defects to reduce the occurrence of defect problems.

[0072] This embodiment can determine the defects generated under specific development configuration data through historical defect data to obtain the corresponding historical development maturity, and train the development maturity prediction model through the historical development maturity and historical development configuration data, so that the trained development maturity prediction model can understand the correlation between development configuration data and development maturity. Therefore, in the software development process, the development maturity can be predicted through real-time development configuration data, thereby realizing efficient identification and monitoring of defect anomalies in the software development process.

[0073] Further, see also Figure 3 In the second embodiment of the model training method of the present invention proposed based on the first embodiment of the present invention, the historical defect data includes a plurality of defect sub-data; the step S20 includes the steps of:

[0074] Step S21, obtaining a trained degree level recognition model;

[0075] Step S22, classifying and identifying the severity of the defect sub-data in the historical defect data by the trained severity level recognition model to obtain a severity level array, wherein the severity level array includes the number of defect sub-data corresponding to different severity levels;

[0076] Step S23, determining the historical development maturity according to the severity level array.

[0077] Defect sub-data is the smallest data unit used to indicate a defect problem; that is, one defect sub-data can reflect the complete information of a defect problem; specifically, the defect sub-data includes but is not limited to a defect title and a defect description.

[0078] The severity level identification model is used to determine the severity of defect sub-data.

[0079] The specific type and structure of the degree level recognition model can be set based on actual needs. Considering that most of the defect sub-data are text content, a natural language processing model can be selected as the degree level recognition model, such as BERT (Bidirectional Encoder Representations from Transformers).

[0080] In this embodiment, the BERT model is used as an example. The BERT model consists of multiple layers of Transformer encoders, and each layer of Transformer encoders contains the following two main submodules:

[0081] 1. Multi-Head Self-Attention Module: The Multi-Head Self-Attention Module includes multiple attention heads. Each attention head calculates the self-attention of the input sequence and merges its outputs.

[0082] For layer 1, the input is represented as H (l-1) , the output is expressed as H (l) ,have:

[0083] H (l) =LayerNorm(H (l-1) +MultiHeadAttention(H (l-1) , H (l-1) , H (l-1) ))

[0084] 2. FFN (Feed-Forward Neural Network): includes two linear transformations and an activation function; we can get:

[0085] H (l) =LayerNorm(H (l) +FFN(H (l) ))

[0086] After one layer of Transformer encoder, the output sequence is expressed as:

[0087] H={h1,h2 . . . , h n}

[0088] Then, for the classification task, the average pooled output of the entire sequence representation is used for classification, and then passed to a fully connected layer, and the probability distribution of the category is obtained through the softmax activation function. Specifically:

[0089] Average Pooling: The hidden state H = {h1, h2, ..., h n}Perform average pooling to obtain a vector representation of fixed dimension:

[0090]

[0091] Fully Connected Layer: The average pooled vector h mean Pass it to a fully connected layer to get the category score z, which is:

[0092] z=Wh mean +b

[0093] Among them, W is the weight matrix and b is the bias vector.

[0094] Softmax activation function: Softmax activation is performed on the output of the fully connected layer to obtain the category probability distribution P, that is:

[0095] P = softmax(z)

[0096] Where P is the probability of each category. Finally, the loss function of training is usually the cross loss function. Assuming the true label is y, the loss function is:

[0097] L = -log(p y )

[0098] where p y is the probability of the true class y predicted by the model; the true class is severity.

[0099] When the severity classification and identification of the defect sub-data in the historical defect data is performed through the trained severity level identification model, the defect title and defect description of each defect sub-data in the historical defect data are combined into a sentence to obtain X = {x1, x _2 , …, x n}, convert it into the corresponding input sequence I = {i1, i _2 ,…,i n}; Input the input serial number sequence into the trained severity level recognition model, and the probabilities of different severity levels corresponding to the defect sub-data can be obtained, and the severity level with the highest probability is taken as the severity level corresponding to the defect sub-data; and after the severity levels corresponding to all defect sub-data are determined, the number of defect sub-data corresponding to each severity level can be known, thereby obtaining a severity level array, such as the severity level array is {H, M, L}, where H is the number of defect sub-data with a heavier severity level, M is the number of defect sub-data with a medium severity level, and L is the number of defect sub-data with a lighter severity level.

[0100] After the severity level array is determined, the historical development maturity can be calculated based on the aforementioned development maturity calculation formula.

[0101] The specific training method of the degree level recognition model can be set based on actual needs.

[0102] In this embodiment, by setting a severity level recognition model, it is possible to determine the severity of defect sub-data, thereby accurately calculating the historical development maturity /

[0103] Furthermore, the step S22 comprises the steps of:

[0104] Step S221, obtaining a trained similar defect recognition model;

[0105] Step S222, performing similar defect recognition on the defect sub-data by using the trained similar defect recognition model to determine similar sub-data with the same defect type;

[0106] Step S223, merging the sub-data of the same type with the same defect type in the historical defect data into one defect sub-data to obtain defect-free data of the same type, wherein the defect types of any two defect sub-data in the defect-free data of the same type are different;

[0107] Step S224: classify and identify the severity of the defect sub-data in the defect-free data using the trained severity level recognition model.

[0108] The similar defect recognition model is used to determine the similarity between defect sub-data. The specific type and structure of the similar defect recognition model can be set based on actual needs. It is understandable that considering that most of the defect sub-data are text content, a natural language processing model can be selected as a degree level recognition model, such as a SimCSE model.

[0109] During the software development process, the same type of defect problem may generate multiple defect sub-data. If the corresponding multiple defect sub-data are independently used as a reflection of the defect problem, the impact of the defect problem on the development maturity will be increased, resulting in a deviation in the calculation of the development maturity. Therefore, in this embodiment, a similar defect recognition model is set to identify defect sub-data with the same defect type to determine similar sub-data, and then merge the similar sub-data so that one defect type corresponds to only one defect sub-data, thereby ensuring that the impact of each defect type on the development maturity will not be increased, thereby ensuring the accuracy of the development maturity calculation.

[0110] It can be understood that the same sub-data is defect sub-data corresponding to a plurality of defect sub-data corresponding to a corresponding defect type; that is, when a defect type corresponds to a plurality of defect sub-data, the corresponding defect sub-data are all the same sub-data; after determining the same sub-data, the same sub-data are merged into one defect sub-data, and at this time, the defect type corresponds to only one defect sub-data. After the same sub-data are merged, the same defect data is obtained, and it can be understood that the number of defect types contained in the same defect data is consistent with the historical defect data, but the number of defect sub-data is less than or equal to the historical defect data.

[0111] The severity classification and identification is carried out by using defect data without similar characteristics, so that the accuracy of development maturity calculation can be guaranteed.

[0112] Furthermore, the step S222 includes the steps of:

[0113] Step S2221, obtaining the defect title in each defect sub-data;

[0114] Step S2222, combining the defect titles in pairs to obtain multiple title combinations;

[0115] Step S2223, using the trained similar defect recognition model to perform similar defect recognition on the defect titles in the title combination to determine whether the defect types of any two defect titles are the same;

[0116] Step S2224, clustering the defect sub-data corresponding to the defect titles with the same defect type to obtain the same type of sub-data.

[0117] It is understandable that the defect sub-data includes a defect title and a defect description; since the defect title can basically reflect the defect type corresponding to the defect sub-data, in order to improve efficiency, in this embodiment, the defect title is used to determine whether the defect types of two defect sub-data are the same.

[0118] In this embodiment, whether the defect types of defect sub-data are the same is judged by comparing them pairwise; each title combination contains defect titles corresponding to two defect sub-data; the title combination is input into a trained defect recognition model of the same type, then it can be judged whether the defect types of the defect titles in the title combination are the same; specifically, the trained defect recognition model outputs the similarity of the defect sub-data corresponding to the two defect titles in the title combination; when the similarity is greater than or equal to a preset similarity threshold, it is considered that the defect types of the two defect sub-data are the same; when the similarity is less than the preset similarity threshold, it is considered that the defect types of the two defect sub-data are different.

[0119] After all title combinations are determined, for any defective sub-data, determine whether there is defective sub-data with the same defect type as the defective sub-data. If so, the defective sub-data and the defective sub-data with the same defect type are regarded as sub-data of the same type; if not, the defective sub-data is not regarded as sub-data of the same type.

[0120] The following uses the SimCSE model as an example to illustrate the training method of the same type of defect recognition model in this embodiment.

[0121] The core of the SimCSE model is to use BERT as the backbone model, encode sentences into vectors, and train the model through contrastive learning. Specifically:

[0122] 1. Encoder (BERT): First, combine the sentence S corresponding to the title i and S j Input into the BERT model for encoding; where S i and S j are respectively a defect title; the resulting sentence is represented as:

[0123] h i =BERT(S i )

[0124] h j =BERT(S j )

[0125] where h i and h j is the BERT model for sentence S i and S j The [CLS] tag output of the last hidden state of the layer.

[0126] 2. Contrastive learning: To train the unsupervised SimCSE model, the same sentence S can be used to generate positive and negative pairs. The goal of training is to make the representations of the same sentences (positive pairs) closer and the representations of different sentences (negative pairs) farther apart.

[0127] The loss function of contrastive learning is defined as:

[0128]

[0129] Among them, τ is the temperature parameter used to adjust the distribution of similarity; h i + It is the sentence S i positive sample; N is the total number of samples including one positive sample and multiple negative samples; sim(h i ,h j ) means h i and h j The similarity between them can be calculated using cosine similarity:

[0130]

[0131] 3. Similarity calculation: After the model training is completed, for any two defect titles S i and S j , we can calculate their corresponding h i and h j The similarity between the defect sub-data is evaluated by the similarity between them, which is:

[0132] sim(S i , S j )=sim(h i ,h j )

[0133] Furthermore, the step S22 includes the following steps:

[0134] Step S24, obtaining manually annotated data corresponding to the historical defect data, wherein the manually annotated data includes an array of annotated sub-data of the same type and an array of annotated severity levels;

[0135] Step S25, determining a first difference between the annotated similar sub-data and the similar sub-data, and a second difference between the annotated severity level array and the severity level array;

[0136] Step S26: displaying the first difference and the second difference.

[0137] It is understandable that after the software development is completed, the developer can manage the relevant data obtained, such as marking similar sub-data, determining the severity level array, etc.; however, manual management by the developer is prone to deviations; therefore, in this embodiment, the problems in the marked data are determined by comparing the relevant data marked by the developer with the relevant data determined by the model, so as to correct the data, and at the same time reduce the severity of the defect problem and the phenomenon of incorrect judgment of similar problems.

[0138] The marked sub-data of the same type are the sub-data of the same type marked by the developer; the standard severity level array is the severity level array marked by the developer.

[0139] When comparing labeled sub-data of the same type with sub-data of the same type, it is possible to compare whether the defect types of the labeled sub-data of the same type and the sub-data of the same type are the same, and whether the sub-data of the same type contained in each defect type is the same, so as to determine whether the labeled sub-data of the same type is consistent with the sub-data of the same type; when the labeled sub-data of the same type is inconsistent with the sub-data of the same type, determine the inconsistent content and obtain the first difference; if the labeled sub-data of the same type is consistent with the sub-data of the same type, the first difference is not generated.

[0140] When comparing the marked severity level array with the severity level array, the number of defect sub-data corresponding to each severity level in the marked severity level array and the severity level array can be compared to see whether they are the same, so as to determine whether the marked severity level array and the severity level array are consistent; when the marked severity level array and the severity level array are inconsistent, the inconsistent content is determined when the marked severity level array and the severity level array are compared to obtain a second difference; if the marked severity level array and the severity level array are consistent, no second difference is generated.

[0141] After determining the first difference and the second difference, the developer can be reminded by displaying the first difference and the second difference. The specific display method can be set based on actual needs, such as BI (Business Intelligence), see Figure 4 , such as FineBI; FineBI can conduct visual monitoring, which can better avoid anomalies, standardize the severity of questions raised by testers, merge similar questions by developers, and promote defects in various departments at different stages of the project to be kept within a controllable range, thereby ensuring the quality of delivered software; the first difference and the second difference are displayed through FineBI, so that the difference information can be visualized. It should be noted that in addition to the first difference and the second difference, the data generated during the software development process can also be visualized through FineBI, such as the severity level arrays of the past, present and future and the corresponding development maturity.

[0142] Further, in a third embodiment of the model training method of the present invention proposed based on the first embodiment of the present invention, step S30 includes the steps of:

[0143] Step S31, obtaining an array of severity levels corresponding to the historical development maturity;

[0144] Step S32, using the historical development configuration data as a feature variable and the severity level array as a target variable to train the initial development maturity prediction model.

[0145] It is understandable that, when the preset coefficient is determined, there is a direct correspondence between the development maturity and the severity level array; therefore, in order to more accurately predict the development maturity, the severity level array is used as the target variable to train the development maturity prediction model in this embodiment. It should be noted that in actual applications, the severity level array can be obtained based on historical defect data, and the development maturity prediction model can be trained without obtaining specific historical development maturity.

[0146] The development maturity prediction model predicts development maturity. Therefore, the historical development configuration data is used as the feature variable and the severity level array is used as the target variable. The feature variable is the input of the development maturity prediction model, and the target variable is the output of the development maturity prediction model.

[0147] The specific type and structure of the development maturity prediction model can be set based on actual needs. In this embodiment, the LightGBM model is used as an example for explanation; LightGBM is an efficient machine learning algorithm, which is an optimized version of GBDT (Gradient Boosting Decision Tree), and can provide faster training speed, lower memory consumption and higher accuracy. Compared with other tree model methods, it has more efficient training speed, lower memory consumption, better generalization ability, and supports parallel and distributed computing, GPU acceleration, so it is also very competitive in the current era of rapid development of deep learning.

[0148] The following uses the LightGBM model as an example to illustrate the training process of the development maturity prediction model.

[0149] 1. Set the feature variable to matrix X and the target variable to matrix Y; the feature variable X is an n×d matrix, where n is the number of samples and d is the number of parameters contained in the historical development configuration data; the feature vector of each sample is represented by X i .

[0150] The target variable Y is an n×3 matrix, where n is the number of samples and each row represents the three target values ​​of a sample [H i , M i , L i ], corresponding to the number of H, M, and L type problems respectively.

[0151] 2. Data preprocessing: Standardize or normalize the input data X and target data Y to improve the model training effect.

[0152] 3. Model training: In this example, LightGBM regression is used to predict the number of three target variables H, M and L at the same time. The target variable is y i =[H i , M i , L i ];

[0153] To build a regression model, the training process can be expressed as:

[0154]

[0155] in represents the prediction output of the development maturity prediction model, and LIGHTGBM(X, Y) indicates that the model is trained using the LightGBM algorithm.

[0156] 4. Loss function: Use the loss function of multi-objective regression, such as mean square error MSE, which is defined as:

[0157]

[0158] This loss function takes into account the prediction errors of the three target variables at the same time to guide model optimization.

[0159] 5. Model optimization: Through methods such as cross-validation and Bayesian search, adjust the hyperparameters of the LightGBM model, such as learning rate, maximum depth, sub-sample ratio, etc., to improve the predictive performance of the model.

[0160] 6. Prediction output: The trained LightGBM model can be used to predict new samples. For the new input feature matrix X new , use the trained model to predict the values ​​of H, M, and L at the same time:

[0161]

[0162] in, Contains the predicted values ​​of H, M, and L.

[0163] Further, in a fourth embodiment of the model training method of the present invention proposed based on the first embodiment of the present invention, step S50 includes the steps of:

[0164] Step S51, performing prediction based on the real-time development configuration data by the trained development maturity prediction model to obtain a predicted severity level array, wherein the predicted severity level array includes a plurality of predicted defect quantities corresponding to different severity levels;

[0165] Step S52, for each severity level, determining a preset weight of the severity level;

[0166] Step S53, calculating the predicted development maturity corresponding to the real-time development configuration data based on the predicted number of defects corresponding to each severity and the preset weight.

[0167] After obtaining the predicted severity level array, the specific values ​​of H, M, and L are obtained, based on the calculation formula of development maturity:

[0168] DV=aH+bM+cL

[0169] Among them, a, b, and c are preset coefficients corresponding to the severity, and the preset coefficients are set in advance based on actual needs; after the specific values ​​of H, M, and L are determined, they are substituted into the above formula to obtain the predicted development maturity.

[0170] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0171] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0172] The present application also provides a development maturity prediction system for implementing the above model training method, see Figure 5 , developing a maturity prediction system includes:

[0173] A first acquisition module, used to acquire historical development configuration data and historical defect data corresponding to the historical development configuration data, wherein the historical defect data is defect data generated when development is performed using the historical development configuration data;

[0174] A first identification module is used to identify the severity of defects in the historical defect data to determine the historical development maturity corresponding to the historical defect data;

[0175] A second acquisition module is used to acquire an initial development maturity prediction model, and train the initial development maturity prediction model through the historical development maturity and the historical development configuration data to obtain a trained development maturity prediction model;

[0176] The third acquisition module is used to acquire real-time development configuration data, and to perform prediction based on the real-time development configuration data through the trained development maturity prediction model to obtain the predicted development maturity corresponding to the real-time development configuration data.

[0177] This development maturity prediction system can determine the defects generated under specific development configuration data through historical defect data to obtain the corresponding historical development maturity. The development maturity prediction model is trained through historical development maturity and historical development configuration data, so that the trained development maturity prediction model can understand the correlation between development configuration data and development maturity. Therefore, in the software development process, the development maturity can be predicted through real-time development configuration data, thereby realizing efficient identification and monitoring of defect anomalies in the software development process.

[0178] It should be noted that the first acquisition module in this embodiment can be used to execute step S10 in the embodiment of the present application, the first identification module in this embodiment can be used to execute step S20 in the embodiment of the present application, the second acquisition module in this embodiment can be used to execute step S30 in the embodiment of the present application, and the third acquisition module in this embodiment can be used to execute step S40 in the embodiment of the present application.

[0179] Furthermore, the historical defect data includes a plurality of defect sub-data; and the first identification module includes:

[0180] A first acquisition submodule is used to acquire a trained degree level recognition model;

[0181] A first identification submodule is used to classify and identify the defect sub-data in the historical defect data by severity through the trained severity level identification model to obtain a severity level array, wherein the severity level array includes the number of defect sub-data corresponding to different severity levels;

[0182] The first determination submodule is used to determine the historical development maturity according to the severity level array.

[0183] Furthermore, the first identification submodule includes:

[0184] A first acquisition unit is used to acquire a trained defect recognition model of the same type;

[0185] A first recognition unit is used to perform similar defect recognition on the defect sub-data through the trained similar defect recognition model to determine similar sub-data with the same defect type;

[0186] A first merging unit is used to merge the sub-data of the same type with the same defect type in the historical defect data into one defect sub-data to obtain data without the same type of defects, wherein the defect types of any two of the defect sub-data in the data without the same type of defects are different;

[0187] The second recognition unit is used to classify and recognize the severity of the defect sub-data in the defect-free data using the trained severity level recognition model.

[0188] Furthermore, the first identification unit includes:

[0189] A first acquisition subunit is used to acquire a defect title in each defect sub-data;

[0190] A first combining subunit, used for combining the defective titles in pairs to obtain a plurality of title combinations;

[0191] A first identification subunit is used to identify the same type of defects for the defect titles in the title combination by using the trained same type defect identification model to determine whether the defect types of any two defect titles are the same;

[0192] The first clustering subunit is used to cluster the defect sub-data corresponding to the defect titles with the same defect type to obtain the same type of sub-data.

[0193] Furthermore, the first identification module further includes:

[0194] A second acquisition submodule is used to acquire manually annotated data corresponding to the historical defect data, wherein the manually annotated data includes annotated sub-data of the same type and an array of annotated severity levels;

[0195] A second determination submodule, configured to determine a first difference between the annotated sub-data of the same type and the sub-data of the same type, and a second difference between the annotated severity level array and the severity level array;

[0196] The first display submodule is configured to display the first difference and the second difference.

[0197] Furthermore, the second acquisition module includes:

[0198] A third acquisition submodule is used to obtain a severity level array corresponding to the historical development maturity;

[0199] The first training submodule is used to train the initial development maturity prediction model by taking the historical development configuration data as a feature variable and the severity level array as a target variable.

[0200] Furthermore, the third acquisition module includes:

[0201] A first prediction submodule is used to perform prediction based on the real-time development configuration data by using the trained development maturity prediction model to obtain a prediction severity level array, wherein the prediction severity level array includes a plurality of predicted defect quantities corresponding to different severities;

[0202] A second determination submodule is used to determine a preset weight of the severity level for each severity level;

[0203] The first calculation submodule is used to calculate the predicted development maturity corresponding to the real-time development configuration data based on the predicted number of defects corresponding to each severity and the preset weight.

[0204] Reference Figure 6 In terms of hardware structure, the electronic device may include components such as a communication module 10, a memory 20, and a processor 30. In the electronic device, the processor 30 is connected to the memory 20 and the communication module 10 respectively, and a computer program is stored in the memory 20. The computer program is executed by the processor 30 at the same time, and the steps of the above method embodiment are implemented when the computer program is executed.

[0205] The communication module 10 can be connected to an external communication device through a network. The communication module 10 can receive requests from an external communication device, and can also send requests, instructions and information to the external communication device, which can be other electronic devices, servers or Internet of Things devices, such as televisions, etc.

[0206] The memory 20 can be used to store software programs and various data. The memory 20 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function (such as obtaining historical development configuration data and historical defect data corresponding to the historical development configuration data), etc.; the data storage area can include a database, and the data storage area can store data or information created according to the use of the system, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0207] The processor 30 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 20, and calling data stored in the memory 20, so as to monitor the electronic device as a whole. The processor 30 may include one or more processing units; optionally, the processor 30 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 30.

[0208] although Figure 6 Although not shown, the electronic device may further include a circuit control module, which is used to connect to a power source to ensure the normal operation of other components. Figure 6 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0209] The present invention also provides a computer-readable storage medium on which a computer program is stored. The computer-readable storage medium may be Figure 6 The memory 20 in the electronic device may also be at least one of a ROM (Read-Only Memory) / RAM (Random Access Memory), a magnetic disk, and an optical disk. The computer-readable storage medium includes a number of instructions for enabling a terminal device with a processor (which may be a television, a car, a mobile phone, a computer, a server, a terminal, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0210] In the present invention, the terms "first", "second", "third", "fourth" and "fifth" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0211] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0212] Although the embodiments of the present invention have been shown and described above, the scope of protection of the present invention is not limited thereto. It is understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art can change, modify and replace the above embodiments within the scope of the present invention, and these changes, modifications and replacements should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A model training method, characterized in that: The model training method comprises: Acquire historical development configuration data and historical defect data corresponding to the historical development configuration data, wherein the historical defect data is defect data generated when development is performed using the historical development configuration data; Identifying the severity of defects in the historical defect data to determine the historical development maturity corresponding to the historical defect data; An initial development maturity prediction model is obtained, and the initial development maturity prediction model is trained using the historical development maturity and the historical development configuration data to obtain a trained development maturity prediction model.

2. The model training method according to claim 1, characterized in that: The historical defect data includes a plurality of defect sub-data; and the identifying the defect severity of the historical defect data to determine the historical development maturity corresponding to the historical defect data includes: Obtaining the degree of training completion level recognition model; Using the trained severity level recognition model, the defect sub-data in the historical defect data are classified and identified by severity level to obtain a severity level array, wherein the severity level array includes the number of defect sub-data corresponding to different severity levels; The historical development maturity is determined based on the severity level array.

3. The model training method according to claim 2, characterized in that: The severity classification and identification of the defect sub-data in the historical defect data by the trained severity level identification model includes: Obtain the trained defect recognition model of the same type; Performing similar defect recognition on the defect sub-data by using the trained similar defect recognition model to determine similar sub-data with the same defect type; Merging the sub-data of the same type with the same defect type in the historical defect data into one defect sub-data to obtain data without the same type of defects, wherein the defect types of any two of the defect sub-data in the data without the same type of defects are different; The defect sub-data in the defect-free data are classified and identified by severity through the trained severity level identification model.

4. The model training method according to claim 3, characterized in that: The similar defect recognition model completed through the training performs similar defect recognition on the defect sub-data, and determines that the similar sub-data with the same defect type include: Obtaining the defect title in each defect sub-data; Combining the defective titles in pairs to obtain multiple title combinations; Performing similar defect identification on the defect titles in the title combination by using the trained similar defect identification model to determine whether the defect types of any two defect titles are the same; The defect sub-data corresponding to the defect titles with the same defect type are clustered to obtain the same type of sub-data.

5. The model training method according to claim 1, characterized in that: The training of the initial development maturity prediction model by using the historical development maturity and the historical development configuration data includes: Get the severity level array corresponding to the historical development maturity; The historical development configuration data is used as a feature variable and the severity level array is used as a target variable to train the initial development maturity prediction model.

6. A development maturity prediction method, characterized in that: The development maturity prediction method is applied to the model training method as described in any one of claims 1 to 5; The development maturity prediction method comprises: Obtain real-time development configuration data and the trained development maturity prediction model; The trained development maturity prediction model performs prediction based on the real-time development configuration data to obtain the predicted development maturity corresponding to the real-time development configuration data.

7. The method for predicting development maturity according to claim 6, characterized in that: The development maturity prediction model completed through the training performs prediction based on the real-time development configuration data, and obtaining the predicted development maturity corresponding to the real-time development configuration data includes: The trained development maturity prediction model performs prediction based on the real-time development configuration data to obtain a prediction severity level array, wherein the prediction severity level array includes a plurality of predicted defect quantities corresponding to different severity levels; For each of the severity levels, determining a preset weight for the severity level; The predicted development maturity corresponding to the real-time development configuration data is calculated based on the predicted number of defects corresponding to each severity and the preset weight.

8. A development maturity prediction system, characterized in that: The development maturity prediction system comprises: A first acquisition module, used to acquire historical development configuration data and historical defect data corresponding to the historical development configuration data, wherein the historical defect data is defect data generated when development is performed using the historical development configuration data; A first identification module is used to identify the severity of defects in the historical defect data to determine the historical development maturity corresponding to the historical defect data; A second acquisition module is used to acquire an initial development maturity prediction model, and train the initial development maturity prediction model through the historical development maturity and the historical development configuration data to obtain a trained development maturity prediction model; The third acquisition module is used to acquire real-time development configuration data, and to perform prediction based on the real-time development configuration data through the trained development maturity prediction model to obtain the predicted development maturity corresponding to the real-time development configuration data.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the computer program implements the model training method as described in any one of claims 1 to 5, or the steps of the development maturity prediction method as described in claim 6 or 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the model training method according to any one of claims 1 to 5, or the development maturity prediction method according to claim 6 or 7 are implemented.

Images