Test result classification model training method, classification method and device

Abstract

The present disclosure provides a test result classification model training method, which is applied to the financial field, artificial intelligence field or other fields. The test result classification model training method includes: obtaining multiple log texts, wherein each log text includes a case where the automated test execution failed; generating a training sample data set based on the multiple log texts, wherein the training sample data set includes a word vector matrix and label information of the word vector matrix; and using the training sample data set to train the test result classification model to be trained to obtain a test result classification model for classifying automated test results. The present disclosure also provides a test result classification method, a test result classification model training device, a test result classification device, an electronic device, a computer-readable storage medium and a computer program product.

Description

Technical Field

[0001] The present disclosure relates to the fields of finance and artificial intelligence technology, and more specifically, to a test result classification model training method, classification method, device, electronic device, computer-readable storage medium and computer program product. Background Art

[0002] As the importance of automated testing increases, the corresponding execution frequency is also increasing. Each execution will generate a large number of results, among which failed execution results need to be manually classified by testers. In related technologies, some keywords and comparison rules of their categories are generally preset, and the key error logs of failed execution results are matched with the preset keyword rules. Classification is performed based on the matching results to achieve the purpose of analysis.

[0003] In the process of realizing the concept of the present disclosure, the inventors discovered that the related art has at least technical problems of limited classification, poor classification effect and high manual classification cost. Summary of the invention

[0004] In view of this, the present disclosure provides a test result classification model training method, classification method, device, electronic device, computer-readable storage medium and computer program product.

[0005] One aspect of the present disclosure provides a test result classification model training method, comprising:

[0006] Obtaining multiple log texts, wherein each of the log texts includes a case where the automated test execution fails;

[0007] Generating a training sample data set according to the plurality of log texts, wherein the training sample data set includes a word vector matrix and label information of the word vector matrix; and

[0008] The training sample data set is used to train the test result classification model to be trained, so as to obtain a test result classification model for classifying the automated test results.

[0009] According to an embodiment of the present disclosure, generating a training sample data set according to the log text includes:

[0010] Preprocessing the log text to delete interfering characters in the log text to obtain a deleted log text;

[0011] According to the number of words in the deleted log text, segment the deleted log text into at least one segmented log text;

[0012] Generate at least one word vector matrix corresponding to the segmented log text, wherein the number of the word vector matrices is the same as the number of the segmented log texts, a row of the word vector matrix represents a word in the segmented log text, and the training sample data set is composed of at least one word vector matrix.

[0013] According to an embodiment of the present disclosure, segmenting the abridged log text into at least one segmented log text according to the number of words in the abridged log text includes:

[0014] Determine the number of words in the above-mentioned abridged log text;

[0015] In a case where the number of words in the deleted log text is less than or equal to a preset threshold, determining the deleted log text as the segmented log text;

[0016] When the number of words in the deleted log text is greater than the preset threshold, the deleted log text is segmented into a plurality of segmented log texts using a word window with a preset sliding step size and a preset window length.

[0017] According to an embodiment of the present disclosure, the above-mentioned use of a word window with a preset sliding step size and a preset window length to segment the above-mentioned deleted log text into a plurality of the above-mentioned segmented log texts includes:

[0018] The word window is slid along the arrangement path of the words constituting the deleted log text at the preset sliding step length to determine one of the segmented log texts.

[0019] According to an embodiment of the present disclosure, the test result classification model to be trained includes a convolutional neural network model;

[0020] The above convolutional neural network model includes an input layer, a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a fully connected layer and an output layer which are cascaded in sequence.

[0021] According to an embodiment of the present disclosure, the method of using the training sample data set to train the test result classification model to be trained to obtain the test result classification model includes:

[0022] Divide the above training sample data set into a training set and a validation set;

[0023] Using the training set to train the test result classification model to be trained to obtain the test result classification model to be verified;

[0024] The above-mentioned verification set is used to verify the above-mentioned test result classification model to be verified, and a verification result used to characterize the classification accuracy is obtained. If the above-mentioned verification result does not meet the iteration stopping condition, the above-mentioned test result classification model to be verified is continued to be trained until the above-mentioned verification result meets the iteration stopping condition, and the above-mentioned test result classification model is output.

[0025] According to an embodiment of the present disclosure, the above-mentioned tag information includes: program problems, abnormalities in the tested environment, test script problems, and test framework problems.

[0026] According to an embodiment of the present disclosure, the above-mentioned interference characters include at least one of the following: punctuation marks, numbers, and preset target characters.

[0027] Another aspect of the present disclosure provides a method for categorizing automated test results, comprising:

[0028] Obtaining log text to be classified, where the log text to be classified corresponds to the automated test item that failed to execute;

[0029] Generate a target word vector matrix corresponding to the log text to be classified according to the log text to be classified; and

[0030] The target word vector matrix is ​​input into the test result classification model, and the classification result is output, wherein the test result classification model is trained by the test result classification model training method.

[0031] According to an embodiment of the present disclosure, in the case where a plurality of target word vector matrices corresponding to the log text to be classified are generated according to the log text to be classified, the target word vector matrix is ​​input into the test result classification model, and the output classification result includes:

[0032] The above-mentioned multiple word vector matrices are input into the above-mentioned test result classification model in sequence until the above-mentioned test result classification model outputs a classification result.

[0033] According to an embodiment of the present disclosure, the above classification result includes an erroneous classification result, and the above method further includes:

[0034] Generate an error sample data set according to the target word vector matrix corresponding to the erroneous classification result, wherein the error sample data set includes the target word vector matrix corresponding to the erroneous classification result and label information of the target word vector matrix;

[0035] The above-mentioned error sample data set is used to optimize the above-mentioned test result classification model to obtain an optimized test result classification model.

[0036] According to an embodiment of the present disclosure, the above-mentioned method of optimizing the above-mentioned test result classification model by using the above-mentioned error sample data set to obtain the optimized test result classification model includes:

[0037] Divide the above error sample data set into an optimized training set and an optimized validation set;

[0038] Using the above optimized training set to train the above test result classification model, to obtain the test result classification model to be optimized and verified;

[0039] The above-mentioned optimization verification set is used to verify the above-mentioned verification test result classification model to be optimized, and a verification result used to characterize the classification accuracy is obtained. If the above-mentioned verification result does not meet the iteration stopping condition, the above-mentioned verification test result classification model to be optimized is continued to be trained until the above-mentioned verification result meets the iteration stopping condition, and the above-mentioned optimized test result classification model is output.

[0040] Another aspect of the present disclosure provides a test result classification model training device, comprising:

[0041] A first acquisition module is used to acquire a plurality of log texts, wherein each of the log texts includes a case where the automated test execution fails;

[0042] A second acquisition module is used to generate a training sample data set according to the plurality of log texts, wherein the training sample data set includes a word vector matrix and label information of the word vector matrix;

[0043] The training module is used to train the test result classification model to be trained using the above training sample data set to obtain a test result classification model for classifying automated test results.

[0044] Another aspect of the present disclosure provides a test result classification device, comprising:

[0045] A third acquisition module is used to acquire log text to be classified, where the log text to be classified corresponds to the automated test item that failed to execute;

[0046] A fourth acquisition module is used to generate a target word vector matrix corresponding to the log text to be classified according to the log text to be classified;

[0047] A classification module is used to input the above-mentioned target word vector matrix into a test result classification model and output a classification result, wherein the above-mentioned test result classification model is trained by the above-mentioned test result classification model training method.

[0048] Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more instructions, wherein when the one or more instructions are executed by the one or more processors, the one or more processors implement the method as described above.

[0049] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0050] Another aspect of the present disclosure provides a computer program product, which includes computer executable instructions, and when the instructions are executed, are used to implement the method as described above.

[0051] The test result classification model training method provided by the embodiment of the present disclosure generates a word vector matrix based on the log text, and trains the test result classification model to be trained through the word vector matrix. The trained test result classification model has the characteristic of high classification accuracy for automated test results, thereby at least partially overcoming the technical problem of poor classification effect existing in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0053] Figure 1 Schematically illustrates an exemplary system architecture to which a test result classification model training method, a test result classification method, a test result classification model training device, and a test result classification device according to an embodiment of the present disclosure can be applied;

[0054] Figure 2 A flowchart of a test result classification model training method according to an embodiment of the present disclosure is schematically shown;

[0055] Figure 3 A flowchart of generating a training sample data set according to log text according to an embodiment of the present disclosure is schematically shown;

[0056] Figure 4 A flowchart schematically illustrates a method of segmenting a deleted log text into at least one segmented log text according to the number of words in the deleted log text according to an embodiment of the present disclosure;

[0057] Figure 5 A schematic diagram schematically shows a method of generating a training sample data set based on log text according to an embodiment of the present disclosure;

[0058] Figure 6 A schematic diagram schematically shows a test result classification model to be trained according to an embodiment of the present disclosure;

[0059] Figure 7 A flowchart of using a training sample data set to train a test result classification model to be trained to obtain a test result classification model according to an embodiment of the present disclosure is schematically shown;

[0060] Figure 8 A flowchart of an automated test result classification method according to an embodiment of the present disclosure is schematically shown;

[0061] Fig. 9 A flowchart of optimizing a test result classification model according to an embodiment of the present disclosure is schematically shown;

[0062] Fig.10 A flowchart of optimizing a test result classification model by using an error sample data set to obtain an optimized test result classification model according to an embodiment of the present disclosure is schematically shown;

[0063] Fig.11 A block diagram of a test result classification model training device according to an embodiment of the present disclosure is schematically shown;

[0064] Fig.12 A block diagram of a test result classification device according to an embodiment of the present disclosure is schematically shown; and

[0065] Fig.13 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0066] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0067] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0068] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0069] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0070] The present disclosure provides a test result classification model training method, which is applied to the financial field, artificial intelligence field or other fields. The test result classification model training method includes: obtaining multiple log texts, wherein each log text includes a case where the automated test execution failed; generating a training sample data set based on the multiple log texts, wherein the training sample data set includes a word vector matrix and label information of the word vector matrix; and using the training sample data set to train the test result classification model to be trained to obtain a test result classification model for classifying automated test results. The present disclosure also provides a test result classification method, a test result classification model training device, a test result classification device, an electronic device, a computer-readable storage medium and a computer program product.

[0071] Figure 1 The exemplary system architecture 100 to which the test result classification model training method, the test result classification method, the test result classification model training device and the test result classification device according to the embodiment of the present disclosure can be applied is schematically shown. It should be noted that, Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0072] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0073] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software, etc. (only as examples).

[0074] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0075] The server 105 may be a server that provides various services, such as a background management server (only an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0076] It should be noted that the test result classification model training method and the test result classification method of the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the test result classification model training device and the test result classification device provided in the embodiment of the present disclosure can generally be set in the server 105. The test result classification model training method and the test result classification method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal device 101, 102, 103 and / or the server 105. Correspondingly, the test result classification model training device and the test result classification device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal device 101, 102, 103 and / or the server 105. Alternatively, the test result classification model training method and the test result classification method provided in the embodiment of the present disclosure can also be executed by the terminal device 101, 102, or 103, or can also be executed by other terminal devices different from the terminal device 101, 102, or 103. Correspondingly, the test result classification model training device and the test result classification device provided in the embodiments of the present disclosure may also be arranged in the terminal device 101, 102, or 103, or in other terminal devices different from the terminal device 101, 102, or 103.

[0077] For example, the word vector matrix, the label information of the word vector matrix, and the target word vector matrix can be originally stored in any one of the terminal devices 101, 102, or 103 (for example, the terminal device 101, but not limited thereto), or stored in an external storage device and can be imported into the terminal device 101. Then, the terminal device 101 can locally execute the test result classification model training method and the test result classification method provided in the embodiment of the present disclosure, or send the word vector matrix, the label information of the word vector matrix, and the target word vector matrix to other terminal devices, servers, or server clusters, and the test result classification model training method and the test result classification method provided in the embodiment of the present disclosure are executed by other terminal devices, servers, or server clusters that receive the word vector matrix, the label information of the word vector matrix, and the target word vector matrix.

[0078] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0079] In the process of implementing the concept of the present disclosure, the inventors discovered that the analysis of the execution results of automated tests in the related art mainly involves presetting some keywords and comparison rules for the categories to which the execution results of automated tests belong, performing string matching on the key error logs of failed execution results and the preset keyword rules, and classifying them according to the matching results to achieve the purpose of analysis.

[0080] This automated test execution result analysis method in the related technology mainly relies on the information in the error log. Since the output form of the error log has no mandatory constraints and has a certain degree of uncertainty, the analysis method of preset keyword rules has a poor effect on handling such uncertain situations. It cannot classify execution failure results that exceed the preset rules, and the classification accuracy of execution failure results within the preset rules is not high. It does not have self-learning and generalization capabilities, and it also relies on manual experience to maintain rules, which increases the cost of automated result analysis and the efficiency needs to be improved.

[0081] In response to the above problems, an embodiment of the present disclosure provides a test result classification model training method.

[0082] Figure 2 A flowchart of a test result classification model training method according to an embodiment of the present disclosure is schematically shown.

[0083] like Figure 2 As shown, the method includes operations S201 to S203.

[0084] In operation S201, a plurality of log texts are obtained, wherein each log text includes a case where the automated test execution fails.

[0085] According to an embodiment of the present disclosure, multiple log texts may be obtained by saving log texts of historical automated test execution failure cases.

[0086] In operation S202, a training sample data set is generated according to a plurality of log texts, wherein the training sample data set includes a word vector matrix and label information of the word vector matrix.

[0087] According to an embodiment of the present disclosure, words in a log text may be converted into a word vector matrix, so as to use the word vector matrix to train a test result classification model to be trained.

[0088] According to an embodiment of the present disclosure, while generating a word vector matrix, a label file with the same name as the word vector matrix but a different suffix may also be generated, that is, label information corresponding to the word vector matrix.

[0089] According to an embodiment of the present disclosure, label information can be used to characterize the reason why the automated test execution corresponding to the word vector matrix fails.

[0090] In operation S203, the test result classification model to be trained is trained using the training sample data set to obtain a test result classification model for classifying automated test results.

[0091] According to an embodiment of the present disclosure, the test result classification model to be trained may be a pre-built neural network model.

[0092] The test result classification model training method provided by the embodiment of the present disclosure generates a word vector matrix based on the log text, and trains the test result classification model to be trained through the word vector matrix. The trained test result classification model has the characteristic of high classification accuracy for automated test results, thereby at least partially overcoming the technical problem of poor classification effect existing in the related technology.

[0093] Reference below Figure 3 to Figure 9 , combined with specific embodiments Figure 2 The method shown is further explained.

[0094] It should be noted that, unless it is explicitly stated that there is a sequence of execution between different operations shown in the flowchart in the embodiments of the present disclosure, or there is a sequence of execution between different operations in technical implementation, otherwise, the execution order of multiple operations may not be prioritized, and multiple operations may also be executed simultaneously.

[0095] Figure 3 The flowchart of generating a training sample data set according to log text according to an embodiment of the present disclosure is schematically shown.

[0096] like Figure 3As shown, the method includes operations S301 to S303.

[0097] In operation S301, the log text is preprocessed to delete interference characters in the log text to obtain a reduced log text.

[0098] According to an embodiment of the present disclosure, the interference characters include at least one of the following: punctuation marks, numbers, and preset target characters.

[0099] According to an embodiment of the present disclosure, the preset target characters may include, for example, modal particles, articles, and pronouns, but are not limited thereto. The preset target characters may also include characters that are not highly associated with the actual meaning of the log text.

[0100] According to the embodiments of the present disclosure, those skilled in the art may set the preset target characters according to the requirements of actual applications, and the embodiments of the present disclosure do not specifically limit the preset target characters.

[0101] In operation S302, the abridged log text is segmented into at least one segmented log text according to the number of words in the abridged log text.

[0102] In operation S303, at least one word vector matrix corresponding to the segmented log text is generated, wherein the number of word vector matrices is the same as the number of segmented log texts, a row of the word vector matrix represents a word in the segmented log text, and the training sample data set is composed of at least one word vector matrix.

[0103] According to an embodiment of the present disclosure, for example, based on the number of words in the deleted log text, the deleted log text is segmented into a first segmented log text, a second segmented log text, and a third segmented log text, and then a first word vector matrix can be generated based on the first segmented log text, a second word vector matrix can be generated based on the second segmented log text, and a third word vector matrix can be generated based on the third segmented log text.

[0104] According to an embodiment of the present disclosure, when multiple word vector matrices are generated based on a log text, the same label information can be annotated on the multiple word vector matrices.

[0105] Figure 4 The flowchart of segmenting a shortened log text into at least one segmented log text according to the number of words in the shortened log text according to an embodiment of the present disclosure is schematically shown.

[0106] like Figure 4 As shown, the method includes operations S401 to S404.

[0107] In operation S401, the number of words in the deleted log text is determined.

[0108] According to an embodiment of the present disclosure, the number of words in the deleted log text may be determined by traversing the words in the deleted log text and counting them.

[0109] In operation S402, it is determined whether the number of words in the deleted log text is less than or equal to a preset threshold.

[0110] When the number of words in the deleted log text is less than or equal to the preset threshold, operation S403 is performed; otherwise, operation S404 is performed.

[0111] In operation S403, the abridged log text is determined as a segmented log text.

[0112] According to an embodiment of the present disclosure, when the number of words in the abridged log text is less than a preset threshold, the abridged log may be padded so that the number of words in the padded abridged log is equal to the preset threshold.

[0113] According to an embodiment of the present disclosure, the deleted log can be padded with zeros. For example, if the number of words in the deleted log text is 5 less than a preset threshold, 5 zeros can be padded into the deleted log so that the number of words in the padded deleted log is equal to the preset threshold.

[0114] In operation S404, the deleted log text is segmented into a plurality of segmented log texts using a word window with a preset sliding step size and a preset window length.

[0115] According to an embodiment of the present disclosure, the preset window length may be equal to the preset threshold.

[0116] According to an embodiment of the present disclosure, by setting the preset window length to be equal to the pre-threshold value, the number of words in a plurality of segmented log texts generated by using the word window can be made the same.

[0117] According to an embodiment of the present disclosure, using a word window with a preset sliding step size and a preset window length to segment the deleted log text into multiple segmented log texts includes: sliding the word window with a preset sliding step size along the arrangement path of the words constituting the deleted log text each time, determining a segmented log text.

[0118] According to an embodiment of the present disclosure, a word window may be slid along the arrangement path of words constituting the deleted log text with a preset sliding step length each time to determine a segmented log text until the number of words in the segmented log text determined by sliding the word window is less than a preset threshold.

[0119] According to an embodiment of the present disclosure, for a segmented log text whose word count determined by sliding the word window is less than a preset threshold, the segmented log text may be padded so that the word count of the segmented log text is equal to the preset threshold.

[0120] Figure 5The schematic diagram schematically shows a schematic diagram of generating a training sample data set according to log text according to an embodiment of the present disclosure.

[0121] Reference Figure 5 String1 can represent the log text. In addition to words, the log text also includes interference characters, such as numbers, punctuation marks, and other contents that are not highly related to the actual meaning of the log text. These contents may cause data pollution during the model training process and affect the efficiency of model training. Therefore, String1 can be preprocessed to delete the interference characters in String1 to obtain the deleted log text String2. It should be noted that Figure 5 The English characters in are the specific content of the log text of the embodiment of the present disclosure, and the English characters are only used as an example of the log text of the embodiment of the present disclosure.

[0122] After preprocessing String1, the number of words in String2 can be determined by traversing the words in String2 and counting them. In this example, the preset threshold can be 20, for example. Since the number of words in String2 is 28, which is greater than the preset threshold, String2 needs to be segmented using a word window.

[0123] In this example, the preset step size of the word window may be, for example, 5, and the preset window length of the word window may be, for example, 20. The word window starts to slide from the starting end corresponding to the word at position 1 in String2 and the ending end corresponding to the word at position 20 in String2, and determines the first segmented log text String21, which includes 20 words, and the 20 words correspond to the words at positions 1 to 20 in String2 respectively; then, the starting end of the word window slides from position 1 in String2 with a step size of 5, and slides to the starting end of the word window corresponding to the word at position 6 in String2 and the ending end corresponding to the word at position 25 in String2, and determines the second segmented log text String22, which includes 20 words, which correspond to the words at positions 6 to 25 in String2 respectively; finally, the word window slides from the beginning corresponding to the word at position 6 in String2 with a step size of 5, and the beginning of the word window slides to the word corresponding to position 11 in String2. At this time, since the number of words in String2 is 28, the end of the word window can only slide to the word corresponding to position 28 in String2. The word window slides this time to determine the third segmented log text String23, which includes 18 words, which correspond to the words at positions 11 to 28 in String2 respectively. Since the number of words in String23 determined by the sliding of the word window this time is less than the preset threshold value 20, the sliding of the word window ends, and String2 is segmented into 3 segmented log texts using the word window with the preset sliding step size and the preset window length.

[0124] For String23, since the number of words in String23 is less than 20, two zeros are padded in String23 so that the number of words in String23 is equal to the preset threshold.

[0125] Since String21, String22 and String23 are generated according to String2, the first word vector matrix, the second word vector matrix and the third word vector matrix can be generated according to String21, String22 and String23 respectively, wherein the label information of the first word vector matrix, the second word vector matrix and the third word vector matrix are the same.

[0126] For the first word vector matrix generated according to String21, the parameters [0.1, 0.4, 0.5, 0.8] of the first row in the first word vector matrix are used to represent the first word Exception in String21.

[0127] So far, combined with Figure 5, an exemplary description is given of the process of generating a training sample data set based on log text. It should be noted that the above examples are only used to help those skilled in the art understand the present disclosure and should not make any improper limitations on the present disclosure.

[0128] According to an embodiment of the present disclosure, the test result classification model to be trained includes a convolutional neural network model.

[0129] According to the embodiments of the present disclosure, since the test result classification model to be trained is a model built based on a convolutional neural network, when the trained test result classification model is used to classify the automated test results, log texts in various output forms can be classified, thereby achieving the effect of improving the generalization ability of automated test result classification; and, by using the trained test result classification model to classify the automated test results, there is no need to rely on manual experience to maintain the rules, thereby reducing labor costs. Since the test result classification model is built based on a neural network, the test result classification model also has self-learning capabilities.

[0130] Figure 6 A schematic diagram of a test result classification model to be trained according to an embodiment of the present disclosure is schematically shown.

[0131] Reference Figure 6 , the convolutional neural network model includes an input layer, a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a fully connected layer and an output layer, which are cascaded in sequence.

[0132] According to an embodiment of the present disclosure, the word vector matrix is ​​input into the test result classification model through the input layer, and is sequentially input into the first convolution layer, the first pooling layer, the second convolution layer, and the second pooling layer. The first convolution layer, the first pooling layer, the second convolution layer, and the second pooling layer output the first feature map, the second feature map, the third feature map, and the fourth feature map, respectively.

[0133] Then the fourth feature map is input into the fully connected layer, which may include multiple output neurons, each of which corresponds to the automated test execution failure reason represented by the label information. The multiple output neurons calculate the fourth feature map respectively and output the calculation results respectively, and finally the largest calculation result among the multiple calculation results is input into the output layer, and the output layer outputs the automated test execution failure reason corresponding to the largest calculation result.

[0134] For example, the fully connected layer includes four output neurons, output neurons 1 to output neurons 4, where the reasons for the failure of the automated test execution represented by output neurons 1 to output neurons 4 can be program problems, abnormalities in the tested environment, test script problems, and test framework problems. After neurons 1 to output neurons 4 calculate the fourth feature map respectively, the calculation result output by output neuron 3 is the largest, and the calculation result output by output neuron 3 is input to the output layer, and the output layer outputs the reason for the failure of the automated test execution corresponding to output neuron 3: test script problem.

[0135] According to an embodiment of the present disclosure, each of the multiple output neurons in the fully connected layer may include a softmax activation function.

[0136] According to an embodiment of the present disclosure, the number of input neurons of the input layer may be the same as the dimension of the word vector matrix.

[0137] According to an embodiment of the present disclosure, the first convolution layer and / or the second convolution layer may include a convolution kernel of a size of 4x4.

[0138] According to an embodiment of the present disclosure, the calculation process of the first convolutional layer and / or the second convolutional layer can be expressed by formula (1).

[0139] S(i, j)=(I*K)(i, j)=∑ m ∑ n I(i+m,j+n)K(m,n); (1)

[0140] Among them, S represents the calculation result of the current convolutional layer, I represents the input data of the previous layer of the current convolutional layer, K represents the convolution kernel, m and n represent the height and width of the convolution kernel, and i and j represent the element positions in the word vector matrix.

[0141] For a word vector matrix [Y1, Y2, Y3…Y19, Y20] T , the process of calculating the word vector matrix using the first convolution layer and / or the second convolution layer includes formula (2), formula (3) and formula (4).

[0142]

[0143] c i =f(w·y i：i+h-1 +b); (3)

[0144] c=[c1,c2,…,c n-h+1 ]; (4)

[0145] Among them, Y i Represents the word vector at position i in the word vector matrix, represents row concatenation, f represents the nonlinear activation function, w represents the convolution kernel, h represents the convolution kernel size, b represents the random bias value, and C represents the feature map.

[0146] According to an embodiment of the present disclosure, the first pooling layer and / or the second pooling layer may include a maximum pooling layer.

[0147] According to an embodiment of the present disclosure, since the first pooling layer and / or the second pooling layer includes a maximum pooling layer, important features in the first feature map and / or the third feature map can be extracted while ignoring local features, thereby simplifying the amount of calculation.

[0148] Figure 7 The flowchart of using a training sample data set to train a test result classification model to be trained to obtain a test result classification model according to an embodiment of the present disclosure is schematically shown.

[0149] like Figure 7 As shown, the method includes operations S701 to S703.

[0150] In operation S701, a training sample data set is divided into a training set and a validation set.

[0151] In operation S702, the test result classification model to be trained is trained using the training set to obtain the test result classification model to be verified.

[0152] In operation S703, the verification set is used to verify the classification model of the test result to be verified, and a verification result used to characterize the classification accuracy is obtained. If the verification result does not meet the iteration stopping condition, the classification model of the test result to be verified continues to be trained until the verification result meets the iteration stopping condition, and the test result classification model is output.

[0153] According to an embodiment of the present disclosure, the verification result may be the classification accuracy of the test result classification model to be verified for the word vector matrix in the verification set.

[0154] According to an embodiment of the present disclosure, by dividing the training sample data set into a training set and a verification set, and using the training set and the verification set to train and verify the test result classification model to be trained and the test result classification model to be verified, respectively, overfitting of the test result classification model can be avoided and the training efficiency of the model can be improved.

[0155] According to an embodiment of the present disclosure, in the process of iterative training of the test result classification model, the network parameters of the test result classification model can be iteratively adjusted based on the output value and label information of the test result classification model using a back propagation algorithm until the error between the output value and label information of the test result classification model meets preset conditions.

[0156] According to an embodiment of the present disclosure, during the back propagation process, the error term of the lth layer of the test result classification model can be calculated by formula (5).

[0157] δ (l) =((W (l) ) T δ (l+1) +b (l) )·f′(z l ): (5)

[0158] Where (W, b) represents the network parameters of the test result classification model, δ (1+1) represents the error term of the (l+1)th layer, represents the derivative of the activation function.

[0159] According to an embodiment of the present disclosure, the tag information includes: program problems, abnormalities in the tested environment, test script problems, and test framework problems.

[0160] Another aspect of the disclosed embodiments provides a method for automatic test result classification.

[0161] Figure 8 The flowchart of the method for automatic test result classification according to an embodiment of the present disclosure is schematically shown.

[0162] like Figure 8 As shown, the method includes operations S801 to S803.

[0163] In operation S801, a log text to be classified is obtained, where the log text to be classified corresponds to an automated test item that has failed to execute.

[0164] In operation S802, a target word vector matrix corresponding to the log text to be classified is generated according to the log text to be classified.

[0165] In operation S803, the target word vector matrix is ​​input into a test result classification model, and a classification result is output, wherein the test result classification model is trained by the test result classification model training method provided by an embodiment of the present disclosure.

[0166] According to an embodiment of the present disclosure, the specific process of generating a target word vector matrix corresponding to the log text to be classified based on the log text to be classified can refer to the test result classification model training method provided in the embodiment of the present disclosure, which will not be repeated here.

[0167] According to an embodiment of the present disclosure, when multiple target word vector matrices corresponding to a log text to be classified are generated based on a log text to be classified, the target word vector matrix is ​​input into a test result classification model, and outputting the classification result includes the following operations: inputting multiple word vector matrices into the test result classification model in sequence until the test result classification model outputs the classification result.

[0168] According to an embodiment of the present disclosure, when three target word vector matrices are generated based on a log text to be classified, the first target word vector matrix can be first input into the test result classification model. If the test result classification model does not output a classification result, the second target word vector matrix can be input into the test result classification model. At this time, the test result classification model outputs a classification result corresponding to the second target word vector matrix, for example, a test script problem. It can be determined that the log text to be classified can be classified as a test script problem.

[0169] According to an embodiment of the present disclosure, in operation S803, the classification result includes an erroneous classification result.

[0170] According to an embodiment of the present disclosure, since not all types of log texts can be taken into account during the training of the test result classification model, the test result classification model may not be able to correctly classify all log texts to be classified, resulting in erroneous classification results.

[0171] Fig. 9 A flowchart of optimizing a test result classification model according to an embodiment of the present disclosure is schematically shown.

[0172] like Fig. 9 As shown, the method includes operations S901 to S902.

[0173] In operation S901, an error sample data set is generated according to a target word vector matrix corresponding to an erroneous classification result, wherein the error sample data set includes a target word vector matrix corresponding to an erroneous classification result and label information of the target word vector matrix.

[0174] In operation S902, the test result classification model is optimized using the error sample data set to obtain an optimized test result classification model.

[0175] Fig.10 The flowchart of optimizing the test result classification model by using the error sample data set according to the embodiment of the present disclosure to obtain the optimized test result classification model is schematically shown.

[0176] like Fig.10 As shown, the method includes operations S1001 to S1003.

[0177] In operation S1001, the error sample data set is divided into an optimized training set and an optimized verification set.

[0178] In operation S1002, the test result classification model is trained using the optimized training set to obtain a test result classification model to be optimized and verified.

[0179] In operation S1003, the optimized verification set is used to verify the classification model of the verification test result to be optimized, and a verification result used to characterize the classification accuracy is obtained. If the verification result does not meet the iteration stopping condition, the classification model of the verification test result to be optimized continues to be trained until the verification result meets the iteration stopping condition, and the optimized test result classification model is output.

[0180] Fig.11 A block diagram of a test result classification model training device according to an embodiment of the present disclosure is schematically shown.

[0181] like Fig.11 As shown, the test result classification model training device 1100 includes a first acquisition module 1101, a second acquisition module 1102 and a training module 1103.

[0182] The first acquisition module 1101 is used to acquire multiple log texts, wherein each log text includes a case where the automated test execution fails.

[0183] The second acquisition module 1102 is used to generate a training sample data set according to a plurality of log texts, wherein the training sample data set includes a word vector matrix and label information of the word vector matrix.

[0184] The training module 1103 is used to train the test result classification model to be trained using the training sample data set to obtain a test result classification model for classifying the automated test results.

[0185] According to an embodiment of the present disclosure, the second acquisition module 1102 includes a preprocessing submodule, a segmentation submodule and a generation submodule.

[0186] The preprocessing submodule is used to preprocess the log text to delete the interfering characters in the log text to obtain the abridged log text.

[0187] The segmentation submodule is used to segment the abridged log text into at least one segmented log text according to the number of words in the abridged log text.

[0188] A generation submodule is used to generate at least one word vector matrix corresponding to the segmented log text, wherein the number of word vector matrices is the same as the number of segmented log texts, a row of the word vector matrix represents a word in the segmented log text, and the training sample data set is composed of at least one word vector matrix.

[0189] According to an embodiment of the present disclosure, the segmentation unit includes a determination unit, a deletion unit, and a sliding unit.

[0190] A determination unit is used to determine the number of words in the deleted log text.

[0191] The deletion unit is used to determine the deleted log text as the segmented log text when the number of words in the deleted log text is less than or equal to a preset threshold.

[0192] The sliding unit is used to divide the deleted log text into a plurality of segmented log texts by using a word window with a preset sliding step and a preset window length when the number of words in the deleted log text is greater than a preset threshold.

[0193] According to an embodiment of the present disclosure, the sliding unit includes a determining subunit.

[0194] The determination subunit is used to slide the word window along the arrangement path of the words constituting the deleted log text at a preset sliding step length each time, thereby determining a segmented log text.

[0195] According to an embodiment of the present disclosure, the test result classification model to be trained includes a convolutional neural network model; the convolutional neural network model includes an input layer, a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a fully connected layer and an output layer cascaded in sequence.

[0196] According to an embodiment of the present disclosure, the training module 1203 includes a dividing unit, a training unit, and a verification unit.

[0197] The partitioning unit is used to partition the training sample data set into a training set and a validation set.

[0198] The training unit is used to train the classification model of the test result to be trained using the training set to obtain the classification model of the test result to be verified.

[0199] The verification unit is used to verify the classification model of the test result to be verified using the verification set to obtain a verification result used to characterize the classification accuracy. If the verification result does not meet the iteration stopping condition, the classification model of the test result to be verified continues to be trained until the verification result meets the iteration stopping condition, and the test result classification model is output.

[0200] According to an embodiment of the present disclosure, the tag information includes: program problems, abnormalities in the tested environment, test script problems, and test framework problems.

[0201] According to an embodiment of the present disclosure, the interference characters include at least one of the following: punctuation marks, numbers, and preset target characters.

[0202] Fig.12A block diagram of a test result classification device according to an embodiment of the present disclosure is schematically shown.

[0203] like Fig.12 As shown, the test result classification model training device 1200 includes a third acquisition module 1201 , a fourth acquisition module 1202 and a classification module 1203 .

[0204] The third acquisition module 1201 is used to acquire the log text to be classified, where the log text to be classified corresponds to the automated test item that failed to execute.

[0205] The fourth acquisition module 1202 is used to generate a target word vector matrix corresponding to the log text to be classified according to the log text to be classified.

[0206] The classification module 1203 is used to input the target word vector matrix into the test result classification model and output the classification result, wherein the test result classification model is trained by the test result classification model training method provided by the embodiment of the present disclosure.

[0207] According to an embodiment of the present disclosure, in the case of generating a plurality of target word vector matrices corresponding to a log text to be classified according to a log text to be classified, the classification module 1203 includes a classification unit.

[0208] The classification unit is used to input multiple word vector matrices into the test result classification model in sequence until the test result classification model outputs a classification result.

[0209] According to an embodiment of the present disclosure, the classification results include erroneous classification results, and the test result classification model training device 1200 also includes an erroneous sample data set generation module and an optimization module.

[0210] The error sample data set generation module is used to generate an error sample data set according to the target word vector matrix corresponding to the incorrect classification result, wherein the error sample data set includes the target word vector matrix corresponding to the incorrect classification result and the label information of the target word vector matrix.

[0211] The optimization module is used to optimize the test result classification model using the error sample data set to obtain an optimized test result classification model.

[0212] According to an embodiment of the present disclosure, the optimization module includes a second partitioning unit, an optimization training unit, and an optimization verification unit.

[0213] The second division unit is used to divide the error sample data set into an optimized training set and an optimized verification set.

[0214] The optimization training unit is used to train the test result classification model using the optimized training set to obtain the test result classification model to be optimized and verified.

[0215] The optimization verification unit is used to verify the classification model of the verification test result to be optimized using the optimization verification set to obtain a verification result used to characterize the classification accuracy. If the verification result does not meet the iteration stopping condition, the classification model of the verification test result to be optimized continues to be trained until the verification result meets the iteration stopping condition, and the optimization test result classification model is output.

[0216] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0217] For example, any multiple of the first acquisition module 1101, the second acquisition module 1102, the training module 1103, the third acquisition module 1201, the fourth acquisition module 1202, and the classification module 1203 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the first acquisition module 1101, the second acquisition module 1102, the training module 1103, the third acquisition module 1201, the fourth acquisition module 1202, and the classification module 1203 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by an appropriate combination of any of them. Alternatively, at least one of the first acquisition module 1101, the second acquisition module 1102, the training module 1103, the third acquisition module 1201, the fourth acquisition module 1202, and the classification module 1203 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0218] Fig.13 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. Fig.13 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0219] like Fig.13 As shown, the electronic device 1300 according to an embodiment of the present disclosure includes a processor 1301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a storage part 1308 into a random access memory (RAM) 1303. The processor 1301 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1301 may also include an onboard memory for caching purposes. The processor 1301 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0220] In RAM1303, various programs and data required for the operation of electronic device 1300 are stored. Processor 1301, ROM1302 and RAM1303 are connected to each other through bus 1304. Processor 1301 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1302 and / or RAM1303. It should be noted that the program can also be stored in one or more memories other than ROM 1302 and RAM 1303. Processor 1301 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0221] According to an embodiment of the present disclosure, the electronic device 1300 may further include an input / output (I / O) interface 1305, which is also connected to the bus 1304. The electronic device 1300 may further include one or more of the following components connected to the I / O interface 1305: an input portion 1306 including a keyboard, a mouse, etc.; an output portion 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1308 including a hard disk, etc.; and a communication portion 1309 including a network interface card such as a LAN card, a modem, etc. The communication portion 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1310 as needed, so that a computer program read therefrom is installed into the storage portion 1308 as needed.

[0222] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1309, and / or installed from the removable medium 1311. When the computer program is executed by the processor 1301, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0223] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0224] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0225] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 1302 and / or the RAM 1303 described above and / or one or more memories other than the ROM 1302 and the RAM 1303 .

[0226] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the character recognition model training method and character recognition method provided by the embodiment of the present disclosure.

[0227] When the computer program is executed by the processor 1301, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0228] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1309, and / or installed from the removable medium 1311. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0229] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0230] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments and / or claims of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

Claims

1. A test result classification model training method, comprising: Acquire multiple log texts, wherein each of the log texts includes a case where the automated test execution fails; Generating a training sample data set according to the plurality of log texts, wherein the training sample data set includes a word vector matrix and label information of the word vector matrix; and Using the training sample data set to train the test result classification model to be trained to obtain a test result classification model for classifying automated test results; The step of generating a training sample data set according to the plurality of log texts comprises: Preprocessing the log text to delete interfering characters in the log text to obtain a deleted log text; According to the number of words in the abridged log text, segmenting the abridged log text into at least one segmented log text; Generate at least one word vector matrix corresponding to the segmented log text, wherein the number of the word vector matrices is the same as the number of the segmented log texts, a row of the word vector matrix represents a word in the segmented log text, and the training sample data set is composed of at least one of the word vector matrices; The step of segmenting the abridged log text into at least one segmented log text according to the number of words in the abridged log text comprises: determining the number of words in the abridged log text; When the number of words in the deleted log text is less than or equal to a preset threshold, determining the deleted log text as the segmented log text; When the number of words in the abridged log text is greater than the preset threshold, the abridged log text is segmented into a plurality of segmented log texts using a word window with a preset sliding step size and a preset window length.

2. The method according to claim 1, wherein: The step of segmenting the deleted log text into a plurality of segmented log texts by using a word window with a preset sliding step size and a preset window length comprises: The word window is slid along the arrangement path of the words constituting the deleted log text at the preset sliding step length to determine one of the segmented log texts.

3. The method according to claim 1, wherein: The test result classification model to be trained includes a convolutional neural network model; The convolutional neural network model includes an input layer, a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a fully connected layer and an output layer which are cascaded in sequence.

4. The method according to claim 1, wherein: The method of using the training sample data set to train the test result classification model to be trained to obtain the test result classification model comprises: Dividing the training sample data set into a training set and a validation set; Using the training set to train the test result classification model to be trained to obtain the test result classification model to be verified; The verification set is used to verify the classification model of the test result to be verified to obtain a verification result used to characterize the classification accuracy. If the verification result does not meet the iteration stopping condition, the classification model of the test result to be verified continues to be trained until the verification result meets the iteration stopping condition, and the test result classification model is output.

5. The method according to claim 1, wherein: The tag information includes: program problems, tested environment anomalies, test script problems, and test framework problems.

6. The method according to claim 1, wherein: The interference characters include at least one of the following: punctuation marks, numbers, and preset target characters.

7. A method for categorizing automated test results, comprising: Obtaining a log text to be classified, wherein the log text to be classified corresponds to an automated test item that failed to execute; Generate a target word vector matrix corresponding to the log text to be classified according to the log text to be classified; as well as The target word vector matrix is ​​input into a test result classification model, and a classification result is output, wherein the test result classification model is trained by the test result classification model training method described in any one of claims 1 to 6.

8. The method according to claim 7, In the case where a plurality of target word vector matrices corresponding to the log text to be classified are generated according to the log text to be classified, the inputting the target word vector matrix into the test result classification model and outputting the classification result comprises: The multiple word vector matrices are sequentially input into the test result classification model until the test result classification model outputs a classification result.

9. The method according to claim 7, wherein: The classification results include erroneous classification results, and the method further includes: Generate an error sample data set according to the target word vector matrix corresponding to the erroneous classification result, wherein the error sample data set includes the target word vector matrix corresponding to the erroneous classification result and label information of the target word vector matrix; The test result classification model is optimized using the error sample data set to obtain an optimized test result classification model.

10. The method according to claim 9, wherein: The optimizing the test result classification model by using the error sample data set to obtain the optimized test result classification model comprises: Dividing the error sample data set into an optimized training set and an optimized verification set; Using the optimized training set to train the test result classification model to obtain a test result classification model to be optimized and verified; The optimization verification set is used to verify the verification test result classification model to be optimized to obtain a verification result used to characterize the classification accuracy. If the verification result does not meet the iteration stopping condition, the verification test result classification model to be optimized is continued to be trained until the verification result meets the iteration stopping condition, and the optimized test result classification model is output.

11. A test result classification model training device, comprising: A first acquisition module is used to acquire a plurality of log texts, wherein each of the log texts includes a case where the automated test execution fails; A second acquisition module is used to generate a training sample data set according to the plurality of log texts, wherein the training sample data set includes a word vector matrix and label information of the word vector matrix; A training module, used to train the test result classification model to be trained using the training sample data set to obtain a test result classification model for classifying automated test results; Wherein, the second acquisition module includes: A preprocessing submodule, used for preprocessing the log text to delete interfering characters in the log text to obtain a deleted log text; a segmentation submodule, configured to segment the abridged log text into at least one segmented log text according to the number of words in the abridged log text; A generating submodule, configured to generate at least one word vector matrix corresponding to the segmented log text, wherein the number of the word vector matrices is the same as the number of the segmented log texts, a row of the word vector matrix represents a word in the segmented log text, and the training sample data set is composed of at least one of the word vector matrices; The segmentation submodule includes: A determination unit, used to determine the number of words in the deleted log text; a deletion unit, configured to determine the deleted log text as the segmented log text when the number of words in the deleted log text is less than or equal to a preset threshold; The sliding unit is used to divide the abridged log text into a plurality of divided log texts by using a word window with a preset sliding step and a preset window length when the number of words in the abridged log text is greater than the preset threshold.

12. A test result classification device, comprising: A third acquisition module is used to acquire the log text to be classified, where the log text to be classified corresponds to the automated test item that failed to execute; A fourth acquisition module is used to generate a target word vector matrix corresponding to the log text to be classified according to the log text to be classified; A classification module is used to input the target word vector matrix into a test result classification model and output a classification result, wherein the test result classification model is trained by the test result classification model training method described in any one of claims 1 to 6.

13. An electronic device comprising: one or more processors; a memory for storing one or more instructions, Wherein, when the one or more instructions are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 10.

14. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 10.

15. A computer program product, comprising computer executable instructions, which when executed are used to implement the method of any one of claims 1 to 10.

Images