Model training method, prediction method, system, electronic device, and medium

Abstract

A model training method, a prediction method, a system, an electronic device and a medium are provided, and a defect generated under specific development configuration data can be determined through historical defect data to obtain corresponding historical development maturity. The development maturity prediction model is trained through the historical development maturity and the historical development configuration data, so that the trained development maturity prediction model can understand the correlation between the development configuration data and the development maturity, thereby predicting the development maturity through real-time development configuration data in the software development process, and efficiently identifying and monitoring defects in the software development process.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of software development, and particularly relates to a model training method, a prediction method, a system, an electronic device and a medium. BACKGROUND

[0002] With the rapid development of the automobile industry towards intelligence and networking, the complexity and code volume of automobile software increase dramatically, and the guarantee of software quality has become an important challenge for the automobile manufacturing industry; therefore, how to efficiently identify and monitor software defect anomalies in the development life cycle of vehicle software has become a problem that is paid more and more attention to. SUMMARY

[0003] The main purpose of the present application is to provide a model training method, a prediction method, a system, an electronic device and a medium, which aims to solve the problem of how to efficiently identify and monitor software defect anomalies in the prior art.

[0004] To achieve the above purpose, the present application provides a model training method, which comprises:

[0005] obtaining historical development configuration data and historical defect data corresponding to the historical development configuration data, wherein the historical defect data is defect data generated when developing with the historical development configuration data;

[0006] performing defect severity identification on the historical defect data to determine the historical development maturity corresponding to the historical defect data;

[0007] obtaining an initial development maturity prediction model, and training the initial development maturity prediction model through the historical development maturity and the historical development configuration data to obtain a trained development maturity prediction model;

[0008] obtaining real-time development configuration data, and performing prediction based on the real-time development configuration data through the trained development maturity prediction model to obtain a predicted development maturity corresponding to the real-time development configuration data.

[0009] Optionally, the historical defect data comprises a plurality of defect sub-data; and the defect severity identification on the historical defect data to determine the historical development maturity corresponding to the historical defect data comprises:

[0010] obtaining a trained degree level identification model;

[0011] The defect sub-data in the historical defect data is classified and recognized in severity by the trained severity level recognition model, and a severity level array is obtained, wherein the severity level array contains the number of defect sub-data corresponding to different severity levels.

[0012] The historical development maturity is determined according to the severity level array.

[0013] Optionally, the classification and recognition of the defect sub-data in the historical defect data in severity by the trained severity level recognition model comprises:

[0014] A trained same-type defect recognition model is obtained.

[0015] Same-type sub-data with the same defect type are determined by performing same-type defect recognition on the defect sub-data by the trained same-type defect recognition model.

[0016] The same-type sub-data with the same defect type in the historical defect data are merged into one defect sub-data to obtain no same-type defect data, wherein the defect types of any two defect sub-data in the no same-type defect data are different.

[0017] The defect sub-data in the no same-type defect data is classified and recognized in severity by the trained severity level recognition model.

[0018] Optionally, the same-type defect recognition of the defect sub-data by the trained same-type defect recognition model to determine same-type sub-data with the same defect type comprises:

[0019] The defect titles in each defect sub-data are obtained.

[0020] The defect titles are combined in pairs to obtain a plurality of title combinations.

[0021] The defect titles in the title combinations are recognized in same-type defect by the trained same-type defect recognition model to determine whether the defect types of any two defect titles are the same.

[0022] The defect sub-data corresponding to the defect titles with the same defect type are clustered to obtain the same-type sub-data.

[0023] Optionally, the classification and recognition of the defect sub-data in the no same-type defect data in severity by the trained severity level recognition model comprises:

[0024] Artificial annotation data corresponding to the historical defect data is obtained, wherein the artificial annotation data includes annotation of same-type sub-data and annotation of a severity level array.

[0025] determining a first difference between the labeled sub-data and the sub-data, and a second difference between the labeled severity level array and the severity level array;

[0026] displaying the first difference and the second difference.

[0027] Optionally, the training of the initial development maturity prediction model by the historical development maturity and the historical development configuration data comprises:

[0028] obtaining a severity level array corresponding to the historical development maturity;

[0029] training the initial development maturity prediction model by taking the historical development configuration data as a feature variable and taking the severity level array as a target variable.

[0030] To achieve the above object, the present application further provides a development maturity prediction method, which is applied to a model training method; the development maturity prediction method comprises:

[0031] obtaining real-time development configuration data and a trained development maturity prediction model;

[0032] predicting by the trained development maturity prediction model based on the real-time development configuration data to obtain a predicted development maturity corresponding to the real-time development configuration data.

[0033] Optionally, the prediction by the trained development maturity prediction model based on the real-time development configuration data to obtain a predicted development maturity corresponding to the real-time development configuration data comprises:

[0034] predicting by the trained development maturity prediction model based on the real-time development configuration data to obtain a predicted severity level array, wherein the predicted severity level array comprises a plurality of predicted defect quantities corresponding to different severities;

[0035] determining a preset weight of each of the severities;

[0036] calculating the predicted development maturity corresponding to the real-time development configuration data based on the predicted defect quantity corresponding to each of the severities and the preset weight.

[0037] To achieve the above object, the present application further provides a development maturity prediction system, which comprises:

[0038] The first obtaining module is configured to obtain historical development configuration data and historical defect data corresponding to the historical development configuration data, wherein the historical defect data is defect data generated when development is performed using the historical development configuration data;

[0039] The first identifying module is configured to perform defect severity identification on the historical defect data to determine historical development maturity corresponding to the historical defect data;

[0040] The second obtaining module is configured to obtain an initial development maturity prediction model, and train the initial development maturity prediction model using the historical development maturity and the historical development configuration data to obtain a trained development maturity prediction model.

[0041] The third obtaining module is configured to obtain real-time development configuration data, and perform prediction based on the real-time development configuration data using the trained development maturity prediction model to obtain predicted development maturity corresponding to the real-time development configuration data.

[0042] To achieve the above object, the present application further provides an electronic device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program implements the steps of the model training method or the development maturity prediction method when executed by the processor.

[0043] To achieve the above object, the present application further provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the model training method or the development maturity prediction method when executed by a processor.

[0044] The model training method, the prediction method, the system, the electronic device and the medium provided by the application obtain historical development configuration data and historical defect data corresponding to the historical development configuration data, wherein the historical defect data is defect data generated when development is performed based on the historical development configuration data; defect severity of the historical defect data is identified to determine historical development maturity corresponding to the historical defect data; an initial development maturity prediction model is obtained, and the initial development maturity prediction model is trained based on the historical development maturity and the historical development configuration data to obtain a trained development maturity prediction model. The historical defect data can be used to determine defects generated under specific development configuration data to obtain corresponding historical development maturity. The development maturity prediction model is trained based on the historical development maturity and the historical development configuration data, so that the trained development maturity prediction model can understand the correlation between development configuration data and development maturity, thereby predicting development maturity based on real-time development configuration data in the software development process, and efficiently identifying and monitoring defects in the software development process. BRIEF DESCRIPTION OF DRAWINGS

[0045] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced as follows. Obviously, those skilled in the art can obtain other drawings from these drawings without creative labor.

[0047] Figure 1 A flowchart of a first embodiment of the model training method of the application;

[0048] Figure 2 A flowchart of a first embodiment of the development maturity prediction method of the application;

[0049] Figure 3 A detailed flowchart of the model training method of the application;

[0050] Figure 4 A data display schematic diagram of the model training method of the application;

[0051] Figure 5 A module structure diagram of the development maturity prediction system of the application;

[0052] Figure 6 A module structure schematic diagram of the electronic device of the application. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. In order to enable a person skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by a person skilled in the art without creative labor should be within the scope of protection of the present application.

[0054] The present application provides a model training method, referring to Figure 1 , Figure 1 The present application provides a model training method, referring to

[0055] Step S10, obtaining historical development configuration data and historical defect data corresponding to the historical development configuration data, wherein the historical defect data is defect data generated when development is performed with the historical development configuration data;

[0056] The historical development configuration data is development configuration data set in completed software development; the development configuration data is used to set specific projects in software development, including but not limited to development time, release rhythm, number of test personnel, number of developers, number of test cases, number of test vehicles, and vehicle project information; the development configuration data can be set in the form of a table.

[0057] The historical defect data is defect data generated when development is performed with the historical development configuration data; it can be understood that in the software development process, software defect problems will occur; specifically, different parameters in the development configuration data have a certain influence on the defect problems generated in software development, such as shorter development time, which can lead to more defect problems, faster release rhythm, which can lead to more defect problems, fewer test personnel, which can lead to more defect problems, more developers, which can lead to more defect problems, more test cases, which can lead to fewer defect problems, and more test vehicles, which can lead to fewer defect problems; it should be noted that the above correspondence between different parameters and the probability of occurrence of defect problems is only for illustration, and the influence of specific parameters on the number of defect problems varies in different software development scenarios; the historical defect data contains related information of defect problems, including but not limited to defect title, defect description, and defect type.

[0058] The historical development configuration data and the corresponding historical defect data can be obtained through the completed software development log, and can also be obtained through other channels.

[0059] Step S20, defect severity identification is performed on the historical defect data to determine the historical development maturity corresponding to the historical defect data;

[0060] A DV (Design Verification, development maturity) value is used to measure the defect level of software development; specifically, defect problems can be classified into different categories based on severity, such as H, M, and L from high to low severity; the specific correspondence between severity and defect problems can be set based on actual needs; and the DV value is determined by the number of defect problems in different severity, which has:

[0061]

[0062] Wherein, H, M, and L are the number of defect problems contained in the corresponding severity, and a, b, and c are the preset weights of the corresponding severity, and the specific values can be set based on actual needs.

[0063] Therefore, the historical development maturity needs to be determined based on the type of defect problem, so in this embodiment, the defect severity of the historical defect data is identified to determine the number of defect problems contained in different severity, thereby determining the corresponding historical development maturity.

[0064] Step S30, an initial development maturity prediction model is obtained, and the initial development maturity prediction model is trained based on the historical development maturity and the historical development configuration data to obtain a trained development maturity prediction model;

[0065] The initial development maturity prediction model is the development maturity prediction model before being trained based on the historical development maturity and the historical development configuration data; it can be understood that when the development maturity prediction model is not trained, the trained development maturity prediction model is obtained by training based on the historical development maturity and the historical development configuration data; when the initial development maturity prediction model has been trained, the initial development maturity prediction model is trained based on the historical development maturity and the historical development configuration data to update the development maturity prediction model. It can be understood that after the software development is completed, the development configuration data and defect data generated in this software development can be used as historical development configuration data and historical defect data to update the development maturity prediction model.

[0066] Correspondingly, the present application also provides a development maturity prediction method, which is applied to the model training method as described above, see Figure 2 , the development maturity prediction method comprises:

[0067] Step S40, obtaining real-time development configuration data;

[0068] Step S50, predicting by the trained development maturity prediction model based on the real-time development configuration data to obtain the predicted development maturity corresponding to the real-time development configuration data.

[0069] The real-time development configuration data is the real-time development configuration data of the current software development cycle, that is, the real-time development configuration data is the development configuration data of the software development application being performed.

[0070] The development maturity prediction model is trained based on the historical development configuration data and the historical development maturity, so the trained development maturity prediction model can understand the correlation between the development configuration data and the development maturity, and thus can predict the development maturity based on the real-time development configuration data by the trained development maturity prediction model to obtain the predicted development maturity.

[0071] It can be understood that the predicted development maturity indicates the development maturity under the current development configuration data, so the developer can specify the target of different development stages and pay attention to related defects to reduce the occurrence of defect problems based on the predicted development maturity.

[0072] The embodiment can determine the defects generated under the specific development configuration data by the historical defect data to obtain the corresponding historical development maturity, and train the development maturity prediction model by the historical development maturity and the historical development configuration data, so that the trained development maturity prediction model can understand the correlation between the development configuration data and the development maturity, thereby predicting the development maturity by the real-time development configuration data in the software development process to achieve efficient identification and monitoring of defect anomalies in the software development process.

[0073] Further, see Figure 3 In the second embodiment of the model training method of the present application based on the first embodiment of the present application, the historical defect data includes a plurality of defect sub-data; and the step S20 includes the steps of:

[0074] Step S21, obtaining a trained severity level recognition model;

[0075] Step S22, classifying and recognizing the severity of the defect sub-data in the historical defect data by the trained severity level recognition model to obtain a severity level array, wherein the severity level array contains the number of defect sub-data corresponding to different severity levels;

[0076] Step S23, determining the historical development maturity according to the severity level array.

[0077] The defect sub-data is the minimum data unit used to indicate the defect problem; that is, one defect sub-data can reflect the complete information of one defect problem; specifically, the defect sub-data includes but is not limited to defect title and defect description.

[0078] The severity level identification model is used to determine the severity of the defect sub-data.

[0079] The specific type and structure of the severity level identification model can be set based on actual needs. Considering that the defect sub-data is mostly text content, a natural language processing model such as BERT (Bidirectional Encoder Representations from Transformers) can be selected as the severity level identification model.

[0080] In this embodiment, the BERT model is taken as an example for illustration. The BERT model is composed of multiple layers of Transformer encoders, and each layer of Transformer encoder contains the following two main sub-modules:

[0081] 1. Multi-Head Self Attention module: The Multi-Head Self Attention module includes multiple attention heads, each of which calculates the self-attention of the input sequence and combines the outputs.

[0082] For the l-th layer, the input is represented as H (l-1) , the output is represented as H (l) , and has:

[0083]

[0084] 2. FFN (Feed-Forward Neural Network): includes two linear transformations and an activation function; available:

[0085]

[0086] After passing through the l-th layer of Transformer encoder, the output sequence representation is obtained as:

[0087]

[0088] Then, for the classification task, the average pooling output of the entire sequence representation is used for classification, and then passed to a fully connected layer to obtain the probability distribution of the class through the softmax activation function, specifically:

[0089] MeanPooling: average pooling on the hidden states H = {h1, h2, …, hn} for all positions, resulting in a fixed-dimensional vector representation: n} is obtained by averaging the hidden states H = {h1, h2, …, hn} for all positions, resulting in a fixed-dimensional vector representation:

[0090]

[0091] FullyConnectedLayer: the vector h mean after average pooling is passed to a fully connected layer to obtain class scores z, which has:

[0092]

[0093] where W is the weight matrix and b is the bias vector.

[0094] Softmax activation function: the output of the fully connected layer is activated by the softmax activation function to obtain the class probability distribution P, i.e.:

[0095]

[0096] where P is the probability of each class. Finally, the loss function of training is usually the cross-entropy loss function, assuming that the true label is y, then the loss function is:

[0097]

[0098] where p y is the probability of the model predicting the true class y; the true class is the severity.

[0099] When the severity classification and identification of the defect sub-data in the historical defect data are performed by the trained degree level identification model, the defect title and defect description of each defect sub-data in the historical defect data are combined to form a sentence X = {x1, x _2 , …, x n}, which is converted into a corresponding input sequence I = {i1, i _2 , …, i n}; the input sequence is input into the trained degree level identification model to obtain the probability of different severities corresponding to the defect sub-data, and the severity with the highest probability is taken as the severity corresponding to the defect sub-data; after the severities corresponding to all defect sub-data are determined, the number of defect sub-data corresponding to each severity is known, so that the severity level array is obtained, such as the severity level array {H, M, L}, where H is the number of defect sub-data with heavy severity, M is the number of defect sub-data with medium severity, and L is the number of defect sub-data with light severity.

[0100] After the severity level set is determined, the historical development maturity can be calculated based on the aforementioned calculation formula of development maturity.

[0101] The specific training manner of the severity level identification model can be set based on actual needs.

[0102] In the embodiment, the severity level identification model is set to determine the severity of the defect sub-data, thereby accurately calculating the historical development maturity.

[0103] Further, the step S22 comprises the steps of:

[0104] Step S221, obtaining the trained same-type defect identification model;

[0105] Step S222, identifying the same-type defect of the defect sub-data by using the trained same-type defect identification model, to determine the same-type sub-data of the same defect type;

[0106] Step S223, merging the same-type sub-data of the same defect type in the historical defect data into one defect sub-data, to obtain the same-type defect-free data, wherein the defect types of any two defect sub-data in the same-type defect-free data are different;

[0107] Step S224, classifying and identifying the severity of the defect sub-data in the same-type defect-free data by using the trained severity level identification model.

[0108] The same-type defect identification model is used to determine the similarity between the defect sub-data. The specific type and structure of the same-type defect identification model can be set based on actual needs. It can be understood that, considering that the defect sub-data is mostly text content, a natural language processing model can be selected as the severity level identification model, such as the SimCSE model.

[0109] In the software development process, multiple defect sub-data can be generated for the same type of defect problem. If the corresponding multiple defect sub-data are used as the reflection of the defect problem independently, the influence of the defect problem on the development maturity will be increased, and the calculation of the development maturity will be biased. Therefore, in the embodiment, the same-type defect identification model is set to identify the defect sub-data of the same defect type, to determine the same-type sub-data, and then to merge the same-type sub-data, so that one defect type corresponds to one defect sub-data, thereby ensuring that the influence of each defect type on the development maturity will not be increased, and ensuring the accuracy of the calculation of the development maturity.

[0110] It can be understood that the same type of sub-data is the defect sub-data corresponding to the defect type corresponding to a plurality of defect sub-data; that is, when the defect type corresponds to a plurality of defect sub-data, the corresponding defect sub-data of the defect type are all same type of sub-data; after determining the same type of sub-data, the same type of sub-data is merged into one defect sub-data, at this time, the defect type only corresponds to one defect sub-data. After the same type of sub-data is merged, the no-same-type defect data is obtained, and it can be understood that the number of defect types contained in the no-same-type defect data is consistent with the historical defect data, but the number of defect sub-data is less than or equal to the historical defect data.

[0111] The severity classification identification is performed through the no-same-type defect data, so that the accuracy of the development maturity calculation can be ensured.

[0112] Further, the step S222 includes the steps of:

[0113] Step S2221, obtaining the defect title in each of the defect sub-data;

[0114] Step S2222, combining the defect titles two by two to obtain a plurality of title combinations;

[0115] Step S2223, performing same defect identification on the defect titles in the title combination through the trained same defect identification model to determine whether the defect types of any two defect titles are the same;

[0116] Step S2224, clustering the defect sub-data corresponding to the defect titles with the same defect type to obtain the same type of sub-data.

[0117] It can be understood that the defect sub-data contains a defect title and a defect description; since the defect title can basically reflect the defect type corresponding to the defect sub-data, in order to improve efficiency, in the embodiment, the defect title is used to determine whether the defect types of two defect sub-data are the same.

[0118] In the embodiment, the same type of sub-data is determined by two-by-two comparison; each title combination contains the defect titles corresponding to two defect sub-data; the title combination is input into the trained same defect identification model, and then the defect types of the defect titles in the title combination can be determined; specifically, the trained same defect identification model outputs the similarity of the defect sub-data corresponding to the two defect titles in the title combination; when the similarity is greater than or equal to a preset similarity threshold, it is considered that the defect types of the two defect sub-data are the same; when the similarity is less than the preset similarity threshold, it is considered that the defect types of the two defect sub-data are different.

[0119] After all the title combinations are determined to be complete, for any defect sub-data, it is determined whether there is defect sub-data of the same defect type as the defect sub-data, if there is, the defect sub-data and the defect sub-data of the same defect type as the defect type are taken as the same type of sub-data, if not, the defect sub-data is not taken as the same type of sub-data.

[0120] The training method of the same defect identification model in the embodiment will be described below taking the SimCSE model as an example.

[0121] The core of the SimCSE model is to use BERT as the backbone model to encode the sentence into a vector, and train the model through contrastive learning. Specifically:

[0122] 1. Encoder (BERT): First, input the title combination corresponding sentence S i and S j into the BERT model for encoding; wherein S i and S j are a defect title; the obtained sentence representation is:

[0123]

[0124]

[0125] Where h i and h j are the [CLS] token outputs of the last hidden state of the BERT model for sentences S i and S j .

[0126] 2. Contrastive learning: In order to train an unsupervised SimCSE model, the same sentence S can be used to generate positive sample pairs and negative sample pairs. The goal of training is to make the representations of the same sentences (positive sample pairs) closer, and the representations of different sentences (negative sample pairs) further apart.

[0127] The loss function of contrastive learning is defined as:

[0128]

[0129] Where τ is the temperature parameter for adjusting the distribution of similarity; h i + is the positive sample of the sentence S i ; N is the total number of samples including one positive sample and multiple negative samples; sim(h i , h j ) represents the similarity between h i and h j , which can be calculated using cosine similarity:

[0130]

[0131] 3. Similarity calculation: after the model training is completed, for any two defect titles S i and S j , the similarity between the defect sub-data can be evaluated by calculating the similarity between their corresponding h i and h j , that is,

[0132]

[0133] Further, the step S22 comprises the following steps:

[0134] Step S24, obtaining the artificial annotation data corresponding to the historical defect data, wherein the artificial annotation data comprises annotation of the same type sub-data and annotation of the severity level array;

[0135] Step S25, determining the first difference between the annotation of the same type sub-data and the same type sub-data, and the second difference between the annotation of the severity level array and the severity level array;

[0136] Step S26, displaying the first difference and the second difference.

[0137] It can be understood that after the software development is completed, the developer can manage the obtained related data, such as annotating the same type sub-data, determining the severity level array, etc. However, the artificial management of the developer is prone to deviation. Therefore, in the embodiment, the problems in the annotation data are determined by comparing the related data annotated by the developer with the related data determined by the model, so as to correct the data, and at the same time, the phenomenon of misjudgment of the severity of the defect problem and the same type problem is alleviated.

[0138] The annotation of the same type sub-data is the same type sub-data annotated by the developer, and the standard severity level array is the severity level array annotated by the developer.

[0139] When comparing the annotation of the same type sub-data and the same type sub-data, whether the defect types of the annotation of the same type sub-data and the same type sub-data are the same, and whether the same type sub-data contained in each defect type is the same, can be compared, so as to determine whether the annotation of the same type sub-data and the same type sub-data are consistent. When the annotation of the same type sub-data and the same type sub-data are inconsistent, the inconsistent content is determined to obtain the first difference. If the annotation of the same type sub-data and the same type sub-data are consistent, the first difference is not generated.

[0140] In the comparison between the annotated severity level array and the severity level array, the number of defect sub-data corresponding to each severity in the comparison between the annotated severity level array and the severity level array can be compared to determine whether the annotated severity level array and the severity level array are consistent; when the annotated severity level array and the severity level array are inconsistent, the content of the annotated severity level array and the severity level array that is determined to be inconsistent is obtained as the second difference; if the annotated severity level array and the severity level array are consistent, the second difference is not generated.

[0141] After the first difference and the second difference are determined, the developer can be reminded by displaying the first difference and the second difference. The specific display manner can be set based on actual needs, such as BI (Business Intelligence, Business Intelligence), see Figure 4 , specifically, FineBI; FineBI can perform visual monitoring, can better avoid exceptions, and can keep the selected severity of the test personnel in proposing problems, the developer in merging similar problems, and the defect problems of each department in different stages of the project within a controllable range, thereby ensuring the quality of the delivered software; the first difference and the second difference are displayed through FineBI, so that the visualization of the difference information can be realized. It should be noted that in addition to the first difference and the second difference, data generated during software development can also be visualized through FineBI, such as past, present, and future severity level arrays and corresponding development maturity.

[0142] Further, in the third embodiment of the model training method of the present application based on the first embodiment of the present application, the step S30 comprises the steps of:

[0143] Step S31, obtaining a severity level array corresponding to the historical development maturity;

[0144] Step S32, taking the historical development configuration data as a feature variable and taking the severity level array as a target variable to train the initial development maturity prediction model.

[0145] It can be understood that in the case of a predetermined coefficient, the development maturity and the severity level array have a direct correspondence; therefore, in order to more accurately predict the development maturity, the severity level array is taken as a target variable to train the development maturity prediction model in this embodiment. It should be noted that in actual application, the severity level array can be obtained based on historical defect data, and the development maturity prediction model can be trained without obtaining specific historical development maturity.

[0146] The development maturity prediction model is used to predict the development maturity, and therefore, historical development configuration data is used as a characteristic variable, and a severity level array is used as a target variable; the characteristic variable is an input of the development maturity prediction model, and the target variable is an output of the development maturity prediction model.

[0147] The specific type and structure of the development maturity prediction model can be set based on actual needs, and in this embodiment, a LightGBM model is taken as an example for description; the LightGBM is a high-efficiency machine learning algorithm, which is an optimized version of the GBDT (Gradient Boosting Decision Tree), and can provide faster training speed, lower memory consumption, and higher accuracy. Compared with other tree model methods, the LightGBM has higher training speed, lower memory consumption, better generalization ability, and supports parallel and distributed computing, GPU acceleration, and therefore, has strong competitiveness in the current era of rapid development of deep learning.

[0148] The training process of the development maturity prediction model is described below by taking the LightGBM model as an example.

[0149] 1. The characteristic variable is set as a matrix X, and the target variable is set as a matrix Y; the characteristic variable X is an nxd matrix, where n is the number of samples, and d is the number of parameters contained in the historical development configuration data; the characteristic vector of each sample is represented as X i .

[0150] The target variable Y is an n×3 matrix, where n is the number of samples, and each row represents three target values [H i , M i , L i ] of a sample, which correspond to the numbers of H, M, and L class problems, respectively.

[0151] 2. Data preprocessing: standardization or normalization is performed on the input data X and the target data Y to improve the model training effect.

[0152] 3. Model training: in this embodiment, the regression of the LightGBM is used to predict the numbers of the three target variables H, M, and L, and the target variable is y i =[H i , M i , L i ];

[0153] A regression model is constructed, and the training process can be represented as:

[0154]

[0155] wherein represents the prediction output of the developed maturity prediction model, LIGHTGBM(X, Y) represents training the model using the LightGBM algorithm.

[0156] 4. Loss function: Use the loss function of multi-objective regression, such as mean square error MSE, defined as:

[0157]

[0158] This loss function will consider the prediction error of three target variables at the same time, guiding the model optimization.

[0159] 5. Model optimization: Through cross-validation and Bayesian search, etc., adjust the hyperparameters of the LightGBM model, such as learning rate, maximum depth, and subsample ratio, etc., to improve the prediction performance of the model.

[0160] 6. Prediction output, the trained LightGBM model can be used to predict new samples. For new input feature matrix X new , use the trained model to predict the values of H, M and L at the same time:

[0161]

[0162] Where, contains the predicted values of H, M and L.

[0163] Further, in the fourth embodiment of the model training method of the present application based on the first embodiment of the present application, the step S50 comprises the steps of:

[0164] Step S51, through the developed maturity prediction model based on the real-time development configuration data, the prediction severity level array is obtained, wherein the prediction severity level array includes a plurality of predicted defect quantities corresponding to different severity levels;

[0165] Step S52, for each severity, determine the preset weight of the severity;

[0166] Step S53, based on the predicted defect quantity corresponding to each severity and the preset weight, the predicted development maturity corresponding to the real-time development configuration data is calculated.

[0167] After obtaining the prediction severity level array, the specific values of H, M and L are obtained, based on the calculation formula of the development maturity:

[0168]

[0169] Wherein, a, b, c are preset coefficients corresponding to the severity, the preset coefficients are set in advance based on actual needs; after the specific values of H, M and L are determined, the above formula can be obtained by bringing them into the formula.

[0170] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0171] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, or network device, etc.) execute the method described in each embodiment of the present application.

[0172] The present application also provides a development maturity prediction system for implementing the above model training method, see Figure 5 The development maturity prediction system comprises:

[0173] A first acquisition module is configured to acquire historical development configuration data and historical defect data corresponding to the historical development configuration data, wherein the historical defect data is defect data generated when the historical development configuration data is developed;

[0174] A first identification module is configured to identify the severity of the historical defect data to determine the historical development maturity corresponding to the historical defect data;

[0175] A second acquisition module is configured to acquire an initial development maturity prediction model, and train the initial development maturity prediction model by using the historical development maturity and the historical development configuration data to obtain a trained development maturity prediction model;

[0176] A third acquisition module is configured to acquire real-time development configuration data, and predict the real-time development configuration data based on the trained development maturity prediction model to obtain a predicted development maturity corresponding to the real-time development configuration data.

[0177] The development maturity prediction system can determine defects generated under specific development configuration data through historical defect data to obtain corresponding historical development maturity, train a development maturity prediction model through historical development maturity and historical development configuration data, so that the trained development maturity prediction model can understand the correlation between development configuration data and development maturity, thereby predicting development maturity through real-time development configuration data in the software development process, and achieving efficient identification and monitoring of defect anomalies in the software development process.

[0178] It should be noted that the first acquisition module in this embodiment can be used to execute step S10 in the embodiments of the present application, the first identification module in this embodiment can be used to execute step S20 in the embodiments of the present application, the second acquisition module in this embodiment can be used to execute step S30 in the embodiments of the present application, and the third acquisition module in this embodiment can be used to execute step S40 in the embodiments of the present application.

[0179] Further, the historical defect data includes a plurality of defect sub-data; the first identification module includes:

[0180] A first acquisition sub-module is configured to acquire the trained degree level identification model.

[0181] A first identification sub-module is configured to perform severity classification and identification on the defect sub-data in the historical defect data through the trained degree level identification model to obtain a severity level array, wherein the severity level array contains the number of defect sub-data corresponding to different severity levels.

[0182] A first determination sub-module is configured to determine the historical development maturity according to the severity level array.

[0183] Further, the first identification sub-module includes:

[0184] A first acquisition unit is configured to acquire a trained same-type defect identification model.

[0185] A first identification unit is configured to perform same-type defect identification on the defect sub-data through the trained same-type defect identification model to determine same-type sub-data of the same defect type.

[0186] A first merging unit is configured to merge the same-type sub-data of the same defect type in the historical defect data into one defect sub-data to obtain no-same-type defect data, wherein the defect types of any two defect sub-data in the no-same-type defect data are different.

[0187] The second identification unit is configured to identify the severity of the defect sub-data in the same-class defect-free data by using the trained severity level identification model.

[0188] Further, the first identification unit comprises:

[0189] The first acquisition sub-unit is configured to acquire defect titles in each of the defect sub-data.

[0190] The first combination sub-unit is configured to combine the defect titles in pairs to obtain a plurality of title combinations.

[0191] The first identification sub-unit is configured to identify the same-class defects of the defect titles in the title combinations by using the trained same-class defect identification model, to determine whether the defect types of any two defect titles are the same.

[0192] The first clustering sub-unit is configured to cluster the defect sub-data corresponding to the defect titles with the same defect type to obtain the same-class sub-data.

[0193] Further, the first identification module further comprises:

[0194] The second acquisition sub-module is configured to acquire artificial annotation data corresponding to the historical defect data, wherein the artificial annotation data comprises annotated same-class sub-data and annotated severity level arrays.

[0195] The second determination sub-module is configured to determine a first difference between the annotated same-class sub-data and the same-class sub-data, and a second difference between the annotated severity level arrays and the severity level arrays.

[0196] The first display sub-module is configured to display the first difference and the second difference.

[0197] Further, the second acquisition module comprises:

[0198] The third acquisition sub-module is configured to acquire a severity level array corresponding to the historical development maturity.

[0199] The first training sub-module is configured to train the initial development maturity prediction model by taking the historical development configuration data as a feature variable and taking the severity level array as a target variable.

[0200] Further, the third acquisition module comprises:

[0201] The first prediction submodule is configured to perform prediction based on the real-time development configuration data by using the trained development maturity prediction model to obtain a predicted severity level array, wherein the predicted severity level array includes a plurality of predicted defect quantities corresponding to different severities.

[0202] The second determination submodule is configured to determine a preset weight of each of the severities.

[0203] The first calculation submodule is configured to calculate a predicted development maturity corresponding to the real-time development configuration data based on the predicted defect quantity corresponding to each of the severities and the preset weight.

[0204] Reference Figure 6 The electronic device can include a communication module 10, a memory 20, a processor 30, and the like in terms of hardware structure. In the electronic device, the processor 30 is connected with the memory 20 and the communication module 10, respectively. The memory 20 stores a computer program, and the computer program is executed by the processor 30. When the computer program is executed, the steps of the above method embodiments are implemented.

[0205] The communication module 10 can be connected with external communication devices through a network. The communication module 10 can receive a request sent by the external communication device, and can also send a request, an instruction, and information to the external communication device. The external communication device can be another electronic device, a server, or an Internet of Things device, such as a television, and the like.

[0206] The memory 20 can be used to store software programs and various data. The memory 20 can mainly include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as obtaining historical development configuration data and historical defect data corresponding to the historical development configuration data), and the like. The data storage area can include a database, and the data storage area can store data or information created according to the use of the system, and the like. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0207] The processor 30 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 20, and by calling data stored in the memory 20, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. The processor 30 may include one or more processing units; optionally, the processor 30 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 30.

[0208] although Figure 6 Not shown, but the above-described electronic device may further include a circuit control module for connecting to a power supply to ensure the normal operation of other components. Those skilled in the art will understand that... Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0209] The present invention also proposes a computer-readable storage medium having a computer program stored thereon. The computer-readable storage medium may be... Figure 6 The memory 20 in the electronic device may also be at least one of ROM (Read-Only Memory) / RAM (Random Access Memory), magnetic disk, optical disk, etc. The computer-readable storage medium includes a number of instructions to cause a terminal device with a processor (which may be a television, automobile, mobile phone, computer, server, terminal, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0210] In this invention, the terms "first," "second," "third," "fourth," and "fifth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0211] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, different embodiments or examples described in the specification and the features of different embodiments or examples can be combined and combined by those skilled in the art without contradiction.

[0212] Although the embodiments of the present application have been shown and described above, the scope of protection of the present application is not limited thereto, and it can be understood that the above-described embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications and replacements to the above-described embodiments within the scope of the present application, and these changes, modifications and replacements should be covered within the scope of protection of the present application. Therefore, the scope of protection of the present application should be subject to the scope of protection of the claims.

Claims

1. A model training method, characterized in that, The model training method comprises: obtaining historical development configuration data and historical defect data corresponding to the historical development configuration data, wherein the historical defect data is defect data generated when development is performed with the historical development configuration data; performing defect severity identification on the historical defect data to determine historical development maturity corresponding to the historical defect data; obtaining an initial development maturity prediction model, and training the initial development maturity prediction model through the historical development maturity and the historical development configuration data to obtain a trained development maturity prediction model; the historical defect data comprises a plurality of defect sub-data; the defect severity identification on the historical defect data to determine the historical development maturity corresponding to the historical defect data comprises: obtaining a trained degree level identification model; performing severity classification identification on the defect sub-data in the historical defect data through the trained degree level identification model to obtain a severity level array, wherein the severity level array contains the number of defect sub-data corresponding to different severity levels; determining the historical development maturity according to the severity level array; the training of the initial development maturity prediction model through the historical development maturity and the historical development configuration data comprises: obtaining a severity level array corresponding to the historical development maturity; training the initial development maturity prediction model with the historical development configuration data as a feature variable and the severity level array as a target variable. 2.The model training method of claim 1, wherein, the severity classification identification on the defect sub-data in the historical defect data through the trained degree level identification model comprises: obtaining a trained same-type defect identification model; performing same-type defect identification on the defect sub-data through the trained same-type defect identification model to determine same-type sub-data of the same defect type; merging the same-type sub-data of the same defect type in the historical defect data into one defect sub-data to obtain no-same-type defect data, wherein the defect types of any two defect sub-data in the no-same-type defect data are different; performing severity classification identification on the defect sub-data in the no-same-type defect data through the trained degree level identification model. 3.The model training method of claim 2, wherein, the same-type defect identification on the defect sub-data through the trained same-type defect identification model to determine same-type sub-data of the same defect type comprises: obtaining defect titles in each defect sub-data; combining the defect titles two by two to obtain a plurality of title combinations; performing same-type defect identification on the defect titles in the title combinations through the trained same-type defect identification model to determine whether the defect types of any two defect titles are the same; clustering the defect sub-data corresponding to the defect titles of the same defect type to obtain the same-type sub-data.

4. A method of developing a maturity prediction, characterized by, The development maturity prediction method is applied to the model training method in any one of claims 1-3; the development maturity prediction method comprises: Obtaining real-time development configuration data, and obtaining a trained development maturity prediction model; Performing prediction based on the real-time development configuration data by using the trained development maturity prediction model, to obtain a predicted development maturity corresponding to the real-time development configuration data.

5. The development maturity prediction method according to claim 4, characterized by, The prediction based on the real-time development configuration data by using the trained development maturity prediction model to obtain the predicted development maturity corresponding to the real-time development configuration data comprises: Performing prediction based on the real-time development configuration data by using the trained development maturity prediction model, to obtain a predicted severity level array, wherein the predicted severity level array comprises predicted defect quantities corresponding to different severities; Determining a preset weight of each of the severities; Calculating the predicted development maturity corresponding to the real-time development configuration data based on the predicted defect quantities corresponding to the severities and the preset weights.

6. A system for developing a maturity prediction system, characterized by, The development maturity prediction system comprises: A first obtaining module, configured to obtain historical development configuration data and historical defect data corresponding to the historical development configuration data, wherein the historical defect data is defect data generated when development is performed based on the historical development configuration data; A first identifying module, configured to perform defect severity identification on the historical defect data, to determine a historical development maturity corresponding to the historical defect data; A second obtaining module, configured to obtain an initial development maturity prediction model, and train the initial development maturity prediction model based on the historical development maturity and the historical development configuration data, to obtain a trained development maturity prediction model; A third obtaining module, configured to obtain real-time development configuration data, and perform prediction based on the real-time development configuration data by using the trained development maturity prediction model, to obtain a predicted development maturity corresponding to the real-time development configuration data; The historical defect data comprises a plurality of defect sub-data; and the first identifying module comprises: A first obtaining sub-module, configured to obtain a trained severity level identification model; A first identifying sub-module, configured to perform severity classification identification on the defect sub-data in the historical defect data by using the trained severity level identification model, to obtain a severity level array, wherein the severity level array comprises quantities of defect sub-data corresponding to different severity levels; A first determining sub-module, configured to determine the historical development maturity according to the severity level array; The second obtaining module comprises: A third obtaining sub-module, configured to obtain a severity level array corresponding to the historical development maturity; A first training sub-module, configured to train the initial development maturity prediction model by taking the historical development configuration data as a feature variable and taking the severity level array as a target variable.

7. An electronic device, comprising: The electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program, when executed by the processor, implements the steps of the model training method according to any one of claims 1 to 3, or the development maturity prediction method according to claim 4 or 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps of the model training method according to any one of claims 1 to 3, or the development maturity prediction method according to claim 4 or 5.

Images