Test case evaluation method, device and storage medium

Abstract

The disclosed embodiment provides a test case evaluation method, device and storage medium, which obtains a test case to be evaluated; constructs a target instruction for evaluating the test case to be evaluated according to the test case to be evaluated and a preset test case evaluation instruction template; uses the target instruction as input, calls a test case evaluation model, and outputs the evaluation result of the test case to be evaluated. The disclosed embodiment utilizes the understanding ability of the test case evaluation model and the supervision training of the test case evaluation task to realize the automatic evaluation of the test case, provides a basis for users to modify and optimize the test case, effectively improves the quality of the test case, and has high efficiency, saving the cost of manual evaluation.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer and network communication technology, and more particularly to a test case evaluation method, device, and storage medium. Background Art

[0002] In the software development process, the testing process is indispensable. Test case is an important product in the testing process. It is a manually written description of the test task, which reflects the tester's steps, methods, techniques, overall plan and strategy.

[0003] In order to ensure the test effect of the test case and the quality of the final product, the test case can be evaluated so as to modify and optimize the test case according to the evaluation result. However, the evaluation of the test case is usually done manually, which has high labor cost and low efficiency. Summary of the invention

[0004] The embodiments of the present disclosure provide a test case evaluation method, device, and storage medium to implement automated evaluation of test cases.

[0005] In a first aspect, an embodiment of the present disclosure provides a test case evaluation method, comprising:

[0006] Get the test cases to be evaluated;

[0007] Constructing a target instruction for evaluating the test case to be evaluated according to the test case to be evaluated and a preset test case evaluation instruction template;

[0008] Taking the target instruction as input, calling the test case evaluation model, and outputting the evaluation result of the test case to be evaluated, wherein the test case evaluation model is obtained by training based on training samples, and the training samples include instructions for evaluating the test case samples and preset evaluation results of the test case samples.

[0009] In a second aspect, an embodiment of the present disclosure provides a test case evaluation device, including:

[0010] An acquisition unit, used for acquiring test cases to be evaluated;

[0011] An instruction generating unit, configured to construct a target instruction for evaluating the test case to be evaluated according to the test case to be evaluated and a preset test case evaluation instruction template;

[0012] An evaluation unit is used to call a test case evaluation model with the target instruction as input, and output an evaluation result of the test case to be evaluated, wherein the test case evaluation model is obtained by training based on training samples, and the training samples include instructions for evaluating the test case samples and preset evaluation results of the test case samples.

[0013] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor and a memory;

[0014] The memory stores computer-executable instructions;

[0015] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the test case evaluation method described in the first aspect and various possible designs of the first aspect.

[0016] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the test case evaluation method described in the first aspect and various possible designs of the first aspect is implemented.

[0017] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including computer execution instructions. When a processor executes the computer execution instructions, it implements the test case evaluation method described in the first aspect and various possible designs of the first aspect.

[0018] The test case evaluation method, device and storage medium provided in the embodiments of the present disclosure obtain the test case to be evaluated; construct a target instruction for evaluating the test case to be evaluated according to the test case to be evaluated and a preset test case evaluation instruction template; use the target instruction as input, call the test case evaluation model, and output the evaluation result of the test case to be evaluated, wherein the test case evaluation model is obtained by training based on training samples, and the training samples include instructions for evaluating the test case samples and preset evaluation results of the test case samples. In the embodiments of the present disclosure, the understanding ability of the test case evaluation model is used to realize automatic evaluation of the test case, provide a basis for users to modify and optimize the test case, effectively improve the quality of the test case, and have high efficiency, saving the cost of manual evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0020] Figure 1 A scenario example diagram of a test case evaluation method provided by an embodiment of the present disclosure;

[0021] Figure 2 A flow chart of a test case evaluation method provided by an embodiment of the present disclosure;

[0022] Figure 3 A flow chart of a test case evaluation method provided by another embodiment of the present disclosure;

[0023] Figure 4 A schematic diagram of test case evaluation in a test case writing interface provided in an embodiment of the present disclosure;

[0024] Figure 5 A schematic diagram of test case evaluation in a test case writing interface provided by another embodiment of the present disclosure;

[0025] Figure 6 A flow chart of a test case evaluation method provided by another embodiment of the present disclosure;

[0026] Figure 7 A schematic diagram of a test case set evaluation report provided in an embodiment of the present disclosure;

[0027] Figure 8 A schematic diagram of a test case periodic evaluation report provided by an embodiment of the present disclosure;

[0028] Fig. 9 A schematic diagram of a test case periodic evaluation report provided by another embodiment of the present disclosure;

[0029] Fig.10 A structural block diagram of a test case evaluation device provided in an embodiment of the present disclosure;

[0030] Fig.11 A schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0032] In the prior art, in order to ensure the test effect of the test case and the quality of the final product, the test case can be evaluated so as to modify and optimize the test case according to the evaluation result. However, the evaluation of the test case is usually carried out manually, which has high labor cost and low efficiency.

[0033] In order to solve the above technical problems, the present disclosure provides a test case evaluation method. Considering that the language model has a strong natural language understanding ability and performs close to or even beyond the level of humans in multiple natural language tasks, the present disclosure can use the understanding ability of the language model to realize the automatic evaluation of the test case. Specifically, the test case to be evaluated can be obtained; according to the test case to be evaluated and the preset test case evaluation instruction template, a target instruction for evaluating the test case to be evaluated is constructed; with the target instruction as input, the test case evaluation model is called, and the evaluation result of the test case to be evaluated is output, wherein the test case evaluation model is obtained by training according to the training sample, and the training sample includes the instruction for evaluating the test case sample and the preset evaluation result of the test case sample. In the present disclosure, the understanding ability of the test case evaluation model is used to realize the automatic evaluation of the test case, provide a basis for users to modify and optimize the test case, effectively improve the quality of the test case, and have high efficiency, saving the cost of manual evaluation.

[0034] The test case evaluation method disclosed in the present invention can be applied to electronic devices such as terminal devices or servers. Taking the server as an example, Figure 1 As shown, the server can obtain the test case to be evaluated, for example, from the user terminal device; construct a target instruction for evaluating the test case to be evaluated based on the test case to be evaluated and a preset test case evaluation instruction template; use the target instruction as input, call the test case evaluation model, output the evaluation result of the test case to be evaluated, and further send the evaluation result to the user terminal device.

[0035] It should be noted that the user information and data involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0036] The test case evaluation method disclosed in the present invention will be described in detail below in conjunction with specific embodiments.

[0037] refer to Figure 2 , Figure 2 A flow chart of a test case evaluation method provided by an embodiment of the present disclosure. The method of this embodiment can be applied to electronic devices such as terminal devices or servers, and the test case evaluation method includes:

[0038] S201. Obtain test cases to be evaluated.

[0039] In this embodiment, the test case to be evaluated is a single test case, which can be input by the user, or the test case completed by the user can be obtained from the test case management platform in real time as the test case to be evaluated when the user writes the test case in the test case management platform, or a test case set can be obtained, and one or more test cases can be taken out from the test case set as the test case to be evaluated. Of course, it can also be a test case to be evaluated obtained through any other means, and there is no limitation here.

[0040] Among them, a single test case is a basic component unit of a test case. A standard single test case describes the case title, preconditions, execution steps and expected results for a test point in natural language text form. A test case set is a collection of single test cases, which is a concrete manifestation of test cases.

[0041] S202: Construct a target instruction for evaluating the test case to be evaluated according to the test case to be evaluated and a preset test case evaluation instruction template.

[0042] In this embodiment, the test cases to be evaluated can be evaluated by a test case evaluation model, wherein the test case evaluation model can be implemented using an existing machine learning model architecture, or can be implemented based on a large language model architecture. In this embodiment, the model structure is not specifically limited.

[0043] Optionally, taking the large language model architecture as an example, the test case evaluation model specifically used in this embodiment can be any large language model (LLM). The large language model is a new type of language model built based on deep learning in the field of natural language. It is characterized by a very large parameter scale (usually more than 1 billion) and is trained with a very large natural language corpus. Thanks to these characteristics, the large language model has a very powerful natural language understanding ability, and performs at a level close to or even beyond that of humans in many natural language tasks.

[0044] In this embodiment, the input of the test case evaluation model can be an instruction for evaluating the test case. The test case evaluation model can perform semantic understanding and test case evaluation based on the instruction for evaluating the test case, and output the evaluation result of the test case. The evaluation result of the test case may include the target evaluation indicator of the test case, which is not limited to the test case score and the reason for the score. In addition, it may also include modification opinions, etc.

[0045] The test case model can be pre-trained so that the test case evaluation model can give the test case an evaluation result that meets the user's expectations. Specifically, multiple training samples can be obtained, wherein the training samples include instructions for evaluating the test case samples and preset evaluation results of the test case samples. The preset evaluation results specifically include the target evaluation indicators of the test case samples, which are not limited to the test case scores and the reasons for the scores, and may also include modification opinions, etc. The specific process of obtaining training samples and model training is as follows:

[0046] You can first obtain multiple test case samples and obtain preset evaluation results of the test case samples, where the preset evaluation results can be obtained by manually evaluating the test case samples or by other means, where the scores of the test case samples can be obtained by following specific test case evaluation standards, for example:

[0047] 1) The test case lacks preconditions: -2 points, corresponding modification suggestions: please add preconditions;

[0048] 2) Test case steps are vague: -3 points, corresponding modification suggestions: Please improve the test case steps;

[0049] 3) Test case checkpoint fuzziness: -3 points, corresponding modification suggestion: please improve the checkpoint;

[0050] 4) Test cases are suspected to be abnormal or incomplete, 0 points, corresponding modification suggestions: please rewrite.

[0051] In addition, a template for the input instruction of the test case evaluation model can be pre-designed, that is, an instruction template for test case evaluation. For a large language model, the instruction template for test case evaluation is a prompt template. A prompt is a structured text used as the input of a large language model. The prompt template specifies a fixed format of the prompt, which contains some variable parts with replaceable content. As an example, the preset test case evaluation instruction template can be as follows:

[0052] "You are a professional test engineer. Please rate the following use case in terms of standardization (the full score is 10 and the lowest score is 0). The use case content is in [ ]. If it is not full marks, please give a reason for the score of no more than 30 words: [test case text], and reply to me in the format of "Standardization score: x, reason for score: x"."

[0053] Among them, the preset test case evaluation instruction template includes the description content of the test case evaluation operation, the replaceable variables used to represent the test case, and the target evaluation indicators. In the above example, "You are a professional test engineer... Please give a scoring reason of no more than 30 words" and other contents are the description content of the test case evaluation operation, which is used for the test case evaluation model to understand that this target instruction is a test case evaluation operation, and the normative score and scoring reason are the target evaluation indicators. "

Test Case Text

[0054] Furthermore, a training sample may be constructed based on the test case sample and the test case evaluation instruction template, wherein the training sample includes an instruction for evaluating the test case sample and a preset evaluation result of the test case sample. The following is an example of a training sample:

[0055] Example 1:

[0056] {

[0057] "instruction":"You are a professional test engineer. Please rate the following use case based on its standardization (the full score is 10 points and the minimum score is 0 points). The use case content is in []. If it is not full marks, please give a rating reason of no more than 30 words: [Professional Selection Assistant-Built-Business Line 1-click_button-Click in the evaluation result page-button_type-major_brief:Professional Introduction] Reply to me in the format of "Standardization score: x, Rating reason: x",

[0058] "input":"",

[0059] "output":"Normativity score: 0, Rating reason: Use case suspected to be abnormal or incomplete"

[0060] }

[0061] Example 2:

[0062] {

[0063] "instruction":"You are a professional test engineer. Please rate the following use case based on its standardization (the full score is 10 points and the minimum score is 0 points). The use case content is in []. If it is not full marks, please give a rating reason of no more than 30 words: [Real-person Q&A Management Backend-Function-New Function Scenario-Q&A Management Information Management Backend-Part 2-List-Header-Field 1->Field 2] Reply to me in the format of "Standardization Rating: x, Rating Reason: x",

[0064] "input":"",

[0065] "output":"Normativity score: 7, Rating reason: Use case checkpoints are vague"

[0066] }

[0067] Example 3:

[0068] {

[0069] "instruction":"You are a professional test engineer. Please rate the following use case based on its standardization (the full score is 10 points and the minimum score is 0 points). The use case content is in []. If it is not full marks, please give a rating reason of no more than 30 words: [Learning core expression-Learning core expression page-Function point-Dialogue-Follow-reading-Follow-reading button-Long press to start recording-If there is no microphone permission, the permission request will be raised] Reply to me in the format of "Standardization score: x, rating reason: x",

[0070] "input":"",

[0071] "output":"Normativity score: 10, Rating reason: None"

[0072] }

[0073] In the above example, "instruction" is an instruction for evaluating the test case sample, using the test case evaluation instruction template, the square brackets "[ ]" are the test case sample; and "output" is the preset evaluation result of the test case sample.

[0074] The test case evaluation model is trained based on the above training samples. It should be noted that the execution subject of the training process can be the same as the execution subject of the test case evaluation method of this embodiment, or it can be different. After the training is completed, the test case evaluation model is deployed to the execution subject of the test case evaluation method of this embodiment. Optionally, the test case evaluation model can be deployed to a CPU, a CPU cluster, a GPU, or a GPU cluster.

[0075] In this embodiment, when the test case evaluation model is needed to evaluate the test case to be evaluated, it is necessary to construct the input of the test case evaluation model, that is, to instruct the test case evaluation model to evaluate the target instruction to be evaluated. Among them, a preset test case evaluation instruction template that is the same as (or substantially the same as) the training sample can be used to construct the target instruction to evaluate the test case to be evaluated.

[0076] Among them, the preset test case evaluation instruction template includes the description content of the test case evaluation operation, the replaceable variables used to represent the test case, and the target evaluation index, such as the above example. When constructing the target instruction for evaluating the test case to be evaluated, the replaceable variables in the test case evaluation instruction template can be replaced with the test case to be evaluated, thereby obtaining the target instruction.

[0077] S203: Taking the target instruction as input, calling the test case evaluation model, and outputting the evaluation result of the test case to be evaluated.

[0078] In this embodiment, after the target instruction is constructed, the test case evaluation model may be called, the target instruction may be input into the test case evaluation model, and the test case evaluation model may generate an evaluation result of the test case to be evaluated.

[0079] Optionally, the target instruction is input into the test case evaluation model. With the help of the powerful natural language understanding ability of the language model, the description content of the test case evaluation operation in the target instruction can be semantically understood and the test case can be evaluated, and the target evaluation index of the test case to be evaluated can be obtained, such as the above-mentioned test case score and scoring reasons. Finally, the target evaluation index of the test case to be evaluated is determined as the evaluation result of the test case to be evaluated. The test case score provides a digital measurement of the standardization of the test case, and the scoring reasons can help users intuitively recognize the standardization of the test case and make defect modifications.

[0080] Optionally, the target instruction also includes the output format of the target evaluation indicator, such as "reply to me in the format of "normative score: x, score reason: x" in the above example", so when the test case evaluation model outputs the evaluation result of the test case to be evaluated, it outputs it in the output format of the target evaluation indicator.

[0081] The test case evaluation method provided in this embodiment obtains the test case to be evaluated; constructs a target instruction for evaluating the test case to be evaluated according to the test case to be evaluated and a preset test case evaluation instruction template; takes the target instruction as input, calls the test case evaluation model, and outputs the evaluation result of the test case to be evaluated. In this embodiment, the understanding ability of the test case evaluation model is used to realize automatic evaluation of the test case, provide a basis for users to modify and optimize the test case, effectively improve the quality of the test case, and have high efficiency, saving the cost of manual evaluation.

[0082] Based on any of the above embodiments, after outputting the evaluation result of the test case to be evaluated, the method further includes:

[0083] Obtain modification suggestions for the test case to be evaluated based on the scoring reasons; and / or

[0084] According to the scoring reasons, the test case to be evaluated is automatically modified.

[0085] In this embodiment, after the test case evaluation model obtains the test case score and scoring reason of the test case to be evaluated, modification opinions on the test case to be evaluated can be obtained based on the scoring reason. The modification opinions can be pre-set or dynamically determined according to the test case to be evaluated.

[0086] In an optional embodiment, the modification opinions are pre-set. For example, in the test case evaluation criteria in the above embodiment, different reasons for deduction correspond to different preset modification opinions, such as the test case lacks a prerequisite: -2 points, and the corresponding modification opinions: please add the prerequisite, etc., which are not listed here. Based on this, when the test case evaluation model obtains the test case score of the test case to be evaluated, it can be determined which reasons for deduction exist, and these reasons for deduction are included in the scoring reasons. Then, based on the scoring reasons and the test case evaluation criteria, the corresponding preset modification opinions can be determined.

[0087] In another optional embodiment, on the basis of the above embodiment, on the basis of determining the preset modification opinions, the actual modification opinions for the test cases to be evaluated can be further determined in combination with the test cases to be evaluated. For example, if the preset modification opinions are determined to be: please add preconditions, then it is also necessary to determine what specific preconditions need to be added in combination with the test cases to be evaluated; if the preset modification opinions are determined to be: please improve the checkpoints, then it is also necessary to determine what specific checkpoints need to be improved in combination with the test cases to be evaluated, and so on.

[0088] Through any of the above optional embodiments, modification suggestions for the test case to be evaluated can be provided, providing a reference and basis for the user to modify the test case, so as to improve the quality of the test case.

[0089] In another optional embodiment, the test case to be evaluated can also be automatically modified directly. For example, on the basis of the actual modification opinions for the test case to be evaluated determined above, the test case to be evaluated can be automatically modified based on the actual modification opinions, thereby achieving automatic modification of the test case and improving the quality of the test case. For example, the reason for the deduction is that the test case lacks a prerequisite: -2 points, and the corresponding modification opinion is: please add the prerequisite, then the prerequisite is automatically added in the test case accordingly to complete the automatic modification.

[0090] Based on any of the above embodiments, Figure 3 As shown, the above test case evaluation method can be applied to the real-time writing process of test cases, as follows:

[0091] S301, when it is detected that a user has completed writing any test case on the test case management platform, the test case is obtained, and the test case is determined as the test case to be evaluated;

[0092] In this embodiment, users can write test cases on the test case management platform. The test case management platform is a user platform that provides test case management, which can realize functions such as test case writing, maintenance, and review. Some actual functions are embedded in the company's internal test case management platform. The technical solution proposed in this embodiment is also applicable to other test case management platforms.

[0093] For example, the test case management platform can provide a test case writing interface, such as Figure 4 As shown, the user writes test cases in the test case writing interface; optionally, the writing of test cases can also be implemented in the form of mind mapping, such as Figure 5 As shown, the test path can be determined according to the mind map to obtain the test case.

[0094] S302: construct a target instruction for evaluating the test case to be evaluated according to the test case to be evaluated and a preset test case evaluation instruction template;

[0095] S303, taking the target instruction as input, calling the test case evaluation model, and outputting the evaluation result of the test case to be evaluated;

[0096] S304, displaying the evaluation results of the test case to be evaluated on the test case management platform;

[0097] In this embodiment, the test case management platform can display the evaluation results of the test cases to be evaluated in real time, such as Figure 4 As shown, the evaluation results can be displayed in a text box or the like at a preset position near each test case; or Figure 5As shown, the evaluation results are displayed in the form of a small window or the like.

[0098] Optionally, the evaluation results of the test case to be evaluated can be displayed in real time in the test case writing interface, where the evaluation results may be positive evaluation results (such as a full score) or negative evaluation results (such as a score that is not a full score); the negative evaluation results can also give the test case score and the reason for the score (including reasons for deducting points), and of course, modification suggestions can also be given, so that the user can modify the test case according to the evaluation results. Optionally, after completing the modification of the test case, the above process can be repeated until a positive evaluation result (such as a full score) is obtained.

[0099] S305, receiving feedback information from the user on the evaluation result of the test case to be evaluated;

[0100] In this embodiment, after viewing the evaluation results, the user can determine whether the evaluation results are reasonable, that is, feedback information on the evaluation results. Optionally, a feedback control can be set in the display interface of the evaluation results, such as a selection box for judging whether it is reasonable. If the judgment is unreasonable, a selection can be made in the selection box.

[0101] S306: Obtain feedback information of the evaluation result to determine unreasonable test cases and optimize the test case evaluation model.

[0102] In this embodiment, feedback information of the evaluation results can be collected to judge unreasonable test cases. Training samples can be constructed in the manner of constructing training samples in the above embodiment, and the test case evaluation model can be optimized to improve the accuracy of the test case evaluation model in evaluating the test cases.

[0103] Based on any of the above embodiments, Figure 6 As shown, the above test case evaluation method can also be applied to offline scenarios, as follows:

[0104] S601, obtaining a test case set from a test case management platform;

[0105] S602, splitting the test case set into multiple test cases, traversing the multiple test cases, and determining each test case as the test case to be evaluated;

[0106] In this embodiment, in an offline scenario, a test case set can be obtained from a test case management platform or other channels, and the test case set can be split into individual test cases, and the test cases can be evaluated in sequence by traversal; of course, in an offline scenario, only one test case can be obtained instead of a test case set. This situation is similar to the test case set, except that the test case set splitting process is not required, which will not be repeated here.

[0107] S603, constructing a target instruction for evaluating each of the test cases to be evaluated according to each of the test cases to be evaluated and a preset test case evaluation instruction template;

[0108] S604, taking each of the target instructions as input, calling the test case evaluation model, and outputting the evaluation result of each of the test cases to be evaluated;

[0109] S605, generating an evaluation report for the test case set according to the evaluation result of each of the test cases to be evaluated;

[0110] In this embodiment, a statistical analysis may be performed on the evaluation result of each test case in the test case set to generate an evaluation report for the test case set.

[0111] Optionally, at least one of the following information may be obtained based on the evaluation result of each test case in the test case set, including but not limited to: the average score of the test cases in the test case set, the proportion of test cases in different score ranges, the test cases with scores below a preset threshold (or not full marks) and the reasons for the scores, and then an evaluation report of the test case set may be generated based on the above information, and the evaluation report includes the above information, such as Figure 7 shown.

[0112] S606: Send the evaluation report of the test case set to the user, and / or display the evaluation report of the test case set;

[0113] In this embodiment, the evaluation report of the test case set can be sent to the user through any communication method, such as email, and can also be sent to the user directly through the test case management platform. The user can display and view the evaluation report on the test case management platform, and then the user can modify the test case according to the evaluation results.

[0114] S607, receiving feedback information from the user on the evaluation result of any test case in the evaluation report of the test case set;

[0115] In this embodiment, when viewing the evaluation report, the user can determine whether the evaluation result of any test case in the evaluation report is reasonable, that is, feedback information on the evaluation result. Optionally, a feedback control can be set for the evaluation result of each test case in the display interface of the evaluation report, such as a selection box for judging whether it is reasonable. If the user determines that the evaluation result of any test case is unreasonable, the user can make a selection in the selection box.

[0116] S608: Obtain feedback information of the evaluation result to determine unreasonable test cases and optimize the test case evaluation model.

[0117] In this embodiment, feedback information of the evaluation results can be collected to judge unreasonable test cases. Training samples can be constructed in the manner of constructing training samples in the above embodiment, and the test case evaluation model can be optimized to improve the accuracy of the test case evaluation model in evaluating the test cases.

[0118] Based on any of the above embodiments, after generating the evaluation result of the test case to be evaluated, the following steps may also be included:

[0119] Based on the evaluation results of different test cases of the target business line within a preset period, a test case periodic evaluation report of the target business line is generated, and the test case periodic evaluation report is sent to the user, and / or the test case periodic evaluation report is displayed.

[0120] In this embodiment, the evaluation results of different test cases of the target business can be statistically analyzed periodically to generate a test case periodic evaluation report for the target business line, wherein the target business can be one or more, and the period can be one week or one month, etc., which can be set according to actual needs; for any target business line, the evaluation results of different test cases within a period can be obtained, for example, the evaluation results of multiple test cases corresponding to different requirements can be obtained respectively, or the evaluation results of multiple test cases corresponding to different users can be obtained respectively, or the evaluation results of multiple test cases in different test case sets can be obtained respectively, wherein the acquisition process can adopt a sampling method or other methods, and further, statistical analysis is performed based on the evaluation results of the obtained multiple test cases, for example, the average score of test cases for different requirements within a period, the proportion of test cases in different scoring intervals, etc. are obtained through statistical analysis, and the average score of test cases for different users within a period, the proportion of test cases in different scoring intervals, etc. are obtained through statistical analysis, and the average score of different test cases within a period, the proportion of test cases in different scoring intervals, etc. are obtained through statistical analysis, and finally a test case periodic evaluation report for the target business line is generated, such as Figure 8 or Fig. 9 shown.

[0121] Furthermore, the periodic evaluation report of the test cases of the target business line may be sent to the user and / or displayed, where the user may be the person in charge, etc., so that the user can perceive the quality of the test cases within the target business line cycle and perform appropriate management actions when necessary.

[0122] Corresponding to the test case evaluation method of the above embodiment, Fig.10 A structural block diagram of a test case evaluation device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Fig.10The test case evaluation device 1000 includes: an acquisition unit 1001, an instruction generation unit 1002, and an evaluation unit 1003.

[0123] Wherein, the acquisition unit 1001 is used to acquire the test case to be evaluated;

[0124] The instruction generating unit 1002 is used to construct a target instruction for evaluating the test case to be evaluated according to the test case to be evaluated and a preset test case evaluation instruction template;

[0125] The evaluation unit 1003 is used to call the test case evaluation model with the target instruction as input, and output the evaluation result of the test case to be evaluated, wherein the test case evaluation model is obtained by training based on training samples, and the training samples include instructions for evaluating the test case samples and preset evaluation results of the test case samples.

[0126] In one or more embodiments of the present disclosure, the test case evaluation instruction template includes replaceable variables for representing the test case;

[0127] When constructing a target instruction for evaluating the test case to be evaluated according to the test case to be evaluated and a preset test case evaluation instruction template, the instruction generating unit 1002 is used to:

[0128] The replaceable variables in the test case evaluation instruction template are replaced with the test case to be evaluated to obtain the target instruction.

[0129] In one or more embodiments of the present disclosure, the test case evaluation instruction template includes a description of the test case evaluation operation and a target evaluation indicator; when the evaluation unit 1003 takes the target instruction as input, calls the test case evaluation model, and outputs the evaluation result of the test case to be evaluated, it is used to:

[0130] The target instruction is input into the test case evaluation model, the description content of the test case evaluation operation is semantically understood and the test case is evaluated through the test case evaluation model, the target evaluation index of the test case to be evaluated is obtained, and the target evaluation index of the test case to be evaluated is determined as the evaluation result of the test case to be evaluated.

[0131] In one or more embodiments of the present disclosure, the target instruction also includes an output format of a target evaluation indicator; when the evaluation unit 1003 determines the target evaluation indicator of the test case to be evaluated as the evaluation result of the test case to be evaluated, it is used to:

[0132] The target evaluation index of the test case to be evaluated is used to generate the evaluation result of the test case to be evaluated according to the output format.

[0133] In one or more embodiments of the present disclosure, the target evaluation index of the test case to be evaluated includes a test case score and a reason for the score.

[0134] In one or more embodiments of the present disclosure, after outputting the evaluation result of the test case to be evaluated, the evaluation unit 1003 is further configured to:

[0135] Obtain modification suggestions for the test case to be evaluated based on the scoring reasons; and / or

[0136] According to the scoring reasons, the test case to be evaluated is automatically modified.

[0137] In one or more embodiments of the present disclosure, when acquiring the test case to be evaluated, the acquiring unit 1001 is used to:

[0138] When it is detected that the user has completed writing any test case on the test case management platform, the test case is obtained and determined as the test case to be evaluated.

[0139] In one or more embodiments of the present disclosure, the device further includes an output unit 1004, configured to:

[0140] The evaluation result of the test case to be evaluated is displayed on the test case management platform to prompt the user to modify the test case to be evaluated according to the evaluation result of the test case to be evaluated, and / or to provide feedback on the evaluation result of the test case to be evaluated.

[0141] In one or more embodiments of the present disclosure, when acquiring the test case to be evaluated, the acquiring unit 1001 is used to:

[0142] Get the test case set from the test case management platform;

[0143] The test case set is split into multiple test cases, the multiple test cases are traversed, and each test case is determined as the test case to be evaluated.

[0144] In one or more embodiments of the present disclosure, the output unit 1004 is further configured to:

[0145] According to the evaluation result of each test case in the test case set, an evaluation report of the test case set is generated, and the evaluation report of the test case set is sent to a user, and / or the evaluation report of the test case set is displayed.

[0146] In one or more embodiments of the present disclosure, when the output unit 1004 generates an evaluation report of the test case set according to the evaluation result of each test case in the test case set, it is used to:

[0147] According to the evaluation result of each test case in the test case set, at least one of the following information is obtained: an average score of the test cases in the test case set, a proportion of test cases in different score ranges, test cases with scores below a preset threshold, and reasons for the scores;

[0148] An evaluation report of the test case set is generated according to the information.

[0149] In one or more embodiments of the present disclosure, the output unit 1004 is further configured to:

[0150] Based on the evaluation results of different test cases of the target business line within a preset period, a test case periodic evaluation report of the target business line is generated, and the test case periodic evaluation report is sent to the user, and / or the test case periodic evaluation report is displayed.

[0151] In one or more embodiments of the present disclosure, the test case evaluation model is trained by the following process:

[0152] Obtaining multiple test case samples and obtaining preset evaluation results of the test case samples;

[0153] Constructing a training sample according to the test case sample and a preset test case evaluation instruction template, wherein the training sample includes an instruction for evaluating the test case sample and a preset evaluation result of the test case sample;

[0154] The test case evaluation model is trained according to the training samples.

[0155] In one or more embodiments of the present disclosure, the evaluation unit 1003 is further configured to:

[0156] Obtaining feedback information from users on evaluation results of the test case to be evaluated;

[0157] If the feedback information indicates that the evaluation result of the test case to be evaluated is unreasonable, obtaining the true evaluation result of the test case to be evaluated;

[0158] Constructing a training sample according to the target instruction and the actual evaluation result of the test case to be evaluated;

[0159] The test case evaluation model is optimized according to the training samples.

[0160] The device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0161] refer to Fig.11 , which shows a schematic diagram of the structure of an electronic device 1100 suitable for implementing the embodiment of the present disclosure, and the electronic device 1100 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.11 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0162] like Fig.11 As shown, the electronic device 1100 may include a processing device (e.g., a central processing unit CPU, a graphics processing unit GPU, etc.) 1101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (Read Only Memory, referred to as ROM) 1102 or a program loaded from a storage device 1108 to a random access memory (Random Access Memory, referred to as RAM) 1103. In RAM 1103, various programs and data required for the operation of the electronic device 1100 are also stored. The processing device 1101, ROM 1102, and RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0163] Typically, the following devices may be connected to the I / O interface 1105: input devices 1106 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 1107 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1108 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1109. The communication devices 1109 may allow the electronic device 1100 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.11 The electronic device 1100 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0164] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 1109, or installed from a storage device 1108, or installed from a ROM 1102. When the computer program is executed by the processing device 1101, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0165] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0166] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0167] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0168] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0169] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0170] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0171] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: CPU or CPU cluster, GPU or GPU cluster, field programmable gate array (FPGA), application specific integrated circuit (ASIC), application specific standard product (ASSP), system on chip (SOC), complex programmable logic device (CPLD), etc.

[0172] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0173] In a first aspect, according to one or more embodiments of the present disclosure, a test case evaluation method is provided, comprising:

[0174] Get the test cases to be evaluated;

[0175] According to the test case to be evaluated and a preset test case evaluation instruction template, construct a target instruction for evaluating the test case to be evaluated;

[0176] Taking the target instruction as input, calling the test case evaluation model, and outputting the evaluation result of the test case to be evaluated, wherein the test case evaluation model is obtained by training based on training samples, and the training samples include instructions for evaluating the test case samples and preset evaluation results of the test case samples.

[0177] According to one or more embodiments of the present disclosure, the test case evaluation instruction template includes replaceable variables for representing the test case;

[0178] The step of constructing a target instruction for evaluating the test case to be evaluated according to the test case to be evaluated and a preset test case evaluation instruction template includes:

[0179] The replaceable variables in the test case evaluation instruction template are replaced with the test case to be evaluated to obtain the target instruction.

[0180] According to one or more embodiments of the present disclosure, the test case evaluation instruction template includes a description of the test case evaluation operation and a target evaluation indicator; the target instruction is used as input, the test case evaluation model is called, and the evaluation result of the test case to be evaluated is output, including:

[0181] The target instruction is input into the test case evaluation model, the description content of the test case evaluation operation is semantically understood and the test case is evaluated through the test case evaluation model, the target evaluation index of the test case to be evaluated is obtained, and the target evaluation index of the test case to be evaluated is determined as the evaluation result of the test case to be evaluated.

[0182] According to one or more embodiments of the present disclosure, the target instruction also includes an output format of a target evaluation indicator; and determining the target evaluation indicator of the test case to be evaluated as the evaluation result of the test case to be evaluated includes:

[0183] The target evaluation index of the test case to be evaluated is used to generate the evaluation result of the test case to be evaluated according to the output format.

[0184] According to one or more embodiments of the present disclosure, the target evaluation index of the test case to be evaluated includes a test case score and a reason for the score.

[0185] According to one or more embodiments of the present disclosure, after outputting the evaluation result of the test case to be evaluated, the method further includes:

[0186] Obtain modification suggestions for the test case to be evaluated based on the scoring reasons; and / or

[0187] According to the scoring reasons, the test case to be evaluated is automatically modified.

[0188] According to one or more embodiments of the present disclosure, obtaining a test case to be evaluated includes:

[0189] When it is detected that the user has completed writing any test case on the test case management platform, the test case is obtained and determined as the test case to be evaluated.

[0190] According to one or more embodiments of the present disclosure, after outputting the evaluation result of the test case to be evaluated, the method further includes:

[0191] The evaluation result of the test case to be evaluated is displayed on the test case management platform to prompt the user to modify the test case to be evaluated according to the evaluation result of the test case to be evaluated, and / or to provide feedback on the evaluation result of the test case to be evaluated.

[0192] According to one or more embodiments of the present disclosure, obtaining a test case to be evaluated includes:

[0193] Get the test case set from the test case management platform;

[0194] The test case set is split into multiple test cases, the multiple test cases are traversed, and each test case is determined as the test case to be evaluated.

[0195] According to one or more embodiments of the present disclosure, after outputting the evaluation result of the test case to be evaluated, the method further includes:

[0196] According to the evaluation result of each test case in the test case set, an evaluation report of the test case set is generated, and the evaluation report of the test case set is sent to a user, and / or the evaluation report of the test case set is displayed.

[0197] According to one or more embodiments of the present disclosure, generating an evaluation report for the test case set according to the evaluation result of each test case in the test case set includes:

[0198] According to the evaluation result of each test case in the test case set, at least one of the following information is obtained: an average score of the test cases in the test case set, a proportion of test cases in different score ranges, test cases with scores below a preset threshold, and reasons for the scores;

[0199] An evaluation report of the test case set is generated according to the information.

[0200] According to one or more embodiments of the present disclosure, after outputting the evaluation result of the test case to be evaluated, the method further includes:

[0201] Based on the evaluation results of different test cases of the target business line within a preset period, a test case periodic evaluation report of the target business line is generated, and the test case periodic evaluation report is sent to the user, and / or the test case periodic evaluation report is displayed.

[0202] According to one or more embodiments of the present disclosure, the test case evaluation model is trained through the following process:

[0203] Obtaining multiple test case samples and obtaining preset evaluation results of the test case samples;

[0204] Constructing a training sample according to the test case sample and a preset test case evaluation instruction template, wherein the training sample includes an instruction for evaluating the test case sample and a preset evaluation result of the test case sample;

[0205] The test case evaluation model is trained according to the training samples.

[0206] According to one or more embodiments of the present disclosure, the method further includes:

[0207] Obtaining feedback information from users on evaluation results of the test case to be evaluated;

[0208] If the feedback information indicates that the evaluation result of the test case to be evaluated is unreasonable, obtaining the true evaluation result of the test case to be evaluated;

[0209] Constructing a training sample according to the target instruction and the actual evaluation result of the test case to be evaluated;

[0210] The test case evaluation model is optimized according to the training samples.

[0211] In a second aspect, according to one or more embodiments of the present disclosure, a test case evaluation device is provided, comprising:

[0212] An acquisition unit, used for acquiring test cases to be evaluated;

[0213] An instruction generating unit, configured to construct a target instruction for evaluating the test case to be evaluated according to the test case to be evaluated and a preset test case evaluation instruction template;

[0214] An evaluation unit is used to call a test case evaluation model with the target instruction as input, and output an evaluation result of the test case to be evaluated, wherein the test case evaluation model is obtained by training based on training samples, and the training samples include instructions for evaluating the test case samples and preset evaluation results of the test case samples.

[0215] According to one or more embodiments of the present disclosure, the test case evaluation instruction template includes replaceable variables for representing the test case;

[0216] When the instruction generation unit constructs a target instruction for evaluating the test case to be evaluated according to the test case to be evaluated and a preset test case evaluation instruction template, it is used to:

[0217] The replaceable variables in the test case evaluation instruction template are replaced with the test case to be evaluated to obtain the target instruction.

[0218] According to one or more embodiments of the present disclosure, the test case evaluation instruction template includes a description of the test case evaluation operation and a target evaluation indicator; when the evaluation unit takes the target instruction as input, calls the test case evaluation model, and outputs the evaluation result of the test case to be evaluated, the evaluation unit is used to:

[0219] The target instruction is input into the test case evaluation model, the description content of the test case evaluation operation is semantically understood and the test case is evaluated through the test case evaluation model, the target evaluation index of the test case to be evaluated is obtained, and the target evaluation index of the test case to be evaluated is determined as the evaluation result of the test case to be evaluated.

[0220] According to one or more embodiments of the present disclosure, the target instruction also includes an output format of a target evaluation indicator; when the evaluation unit determines the target evaluation indicator of the test case to be evaluated as the evaluation result of the test case to be evaluated, it is used to:

[0221] The target evaluation index of the test case to be evaluated is used to generate the evaluation result of the test case to be evaluated according to the output format.

[0222] According to one or more embodiments of the present disclosure, the target evaluation index of the test case to be evaluated includes a test case score and a reason for the score.

[0223] According to one or more embodiments of the present disclosure, after outputting the evaluation result of the test case to be evaluated, the evaluation unit is further used to:

[0224] Obtain modification suggestions for the test case to be evaluated based on the scoring reasons; and / or

[0225] According to the scoring reasons, the test case to be evaluated is automatically modified.

[0226] According to one or more embodiments of the present disclosure, when acquiring the test case to be evaluated, the acquisition unit is used to:

[0227] When it is detected that the user has completed writing any test case on the test case management platform, the test case is obtained and determined as the test case to be evaluated.

[0228] According to one or more embodiments of the present disclosure, the device further includes an output unit, configured to:

[0229] The evaluation result of the test case to be evaluated is displayed on the test case management platform to prompt the user to modify the test case to be evaluated according to the evaluation result of the test case to be evaluated, and / or to provide feedback on the evaluation result of the test case to be evaluated.

[0230] According to one or more embodiments of the present disclosure, when acquiring the test case to be evaluated, the acquisition unit is used to:

[0231] Get the test case set from the test case management platform;

[0232] The test case set is split into multiple test cases, the multiple test cases are traversed, and each test case is determined as the test case to be evaluated.

[0233] According to one or more embodiments of the present disclosure, the output unit is further configured to:

[0234] According to the evaluation result of each test case in the test case set, an evaluation report of the test case set is generated, and the evaluation report of the test case set is sent to a user, and / or the evaluation report of the test case set is displayed.

[0235] According to one or more embodiments of the present disclosure, when the output unit generates an evaluation report of the test case set according to the evaluation result of each test case in the test case set, it is used to:

[0236] According to the evaluation result of each test case in the test case set, at least one of the following information is obtained: an average score of the test cases in the test case set, a proportion of test cases in different score ranges, test cases with scores below a preset threshold, and reasons for the scores;

[0237] An evaluation report of the test case set is generated according to the information.

[0238] According to one or more embodiments of the present disclosure, the output unit is further configured to:

[0239] Based on the evaluation results of different test cases of the target business line within a preset period, a test case periodic evaluation report of the target business line is generated, and the test case periodic evaluation report is sent to the user, and / or the test case periodic evaluation report is displayed.

[0240] According to one or more embodiments of the present disclosure, the test case evaluation model is trained through the following process:

[0241] Obtaining multiple test case samples and obtaining preset evaluation results of the test case samples;

[0242] Constructing a training sample according to the test case sample and a preset test case evaluation instruction template, wherein the training sample includes an instruction for evaluating the test case sample and a preset evaluation result of the test case sample;

[0243] The test case evaluation model is trained according to the training samples.

[0244] According to one or more embodiments of the present disclosure, the evaluation unit is further used for:

[0245] Obtaining feedback information from users on evaluation results of the test case to be evaluated;

[0246] If the feedback information indicates that the evaluation result of the test case to be evaluated is unreasonable, obtaining the true evaluation result of the test case to be evaluated;

[0247] Constructing a training sample according to the target instruction and the actual evaluation result of the test case to be evaluated;

[0248] The test case evaluation model is optimized according to the training samples.

[0249] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0250] The memory stores computer-executable instructions;

[0251] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the test case evaluation method described in the first aspect and various possible designs of the first aspect.

[0252] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the test case evaluation method described in the first aspect and various possible designs of the first aspect is implemented.

[0253] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, comprising computer execution instructions. When a processor executes the computer execution instructions, the test case evaluation method as described in the first aspect and various possible designs of the first aspect is implemented.

[0254] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0255] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0256] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A test case evaluation method, characterized in that: include: Get the test cases to be evaluated; According to the test case to be evaluated and a preset test case evaluation instruction template, construct a target instruction for evaluating the test case to be evaluated; Taking the target instruction as input, calling the test case evaluation model, and outputting the evaluation result of the test case to be evaluated, wherein the test case evaluation model is obtained by training based on training samples, and the training samples include instructions for evaluating the test case samples and preset evaluation results of the test case samples.

2. The method according to claim 1, characterized in that The test case evaluation instruction template includes replaceable variables for representing the test case; The step of constructing a target instruction for evaluating the test case to be evaluated according to the test case to be evaluated and a preset test case evaluation instruction template includes: The replaceable variables in the test case evaluation instruction template are replaced with the test case to be evaluated to obtain the target instruction.

3. The method according to claim 2, characterized in that The test case evaluation instruction template includes the description content of the test case evaluation operation and the target evaluation index; the test case evaluation model is called with the target instruction as input, and the evaluation result of the test case to be evaluated is output, including: The target instruction is input into the test case evaluation model, the description content of the test case evaluation operation is semantically understood and the test case is evaluated through the test case evaluation model, the target evaluation index of the test case to be evaluated is obtained, and the target evaluation index of the test case to be evaluated is determined as the evaluation result of the test case to be evaluated.

4. The method according to claim 3, characterized in that The target instruction also includes an output format of a target evaluation indicator; and determining the target evaluation indicator of the test case to be evaluated as the evaluation result of the test case to be evaluated includes: The target evaluation index of the test case to be evaluated is used to generate the evaluation result of the test case to be evaluated according to the output format.

5. The method according to any one of claims 1 to 4, characterized in that: The target evaluation index of the test case to be evaluated includes the test case score and the reason for the score.

6. The method according to claim 5, characterized in that After outputting the evaluation result of the test case to be evaluated, the method further includes: Obtain modification suggestions for the test case to be evaluated based on the scoring reasons; and / or According to the scoring reasons, the test case to be evaluated is automatically modified.

7. The method according to any one of claims 1 to 4, characterized in that: The obtaining of the test case to be evaluated includes: When it is detected that the user has completed writing any test case on the test case management platform, the test case is obtained and determined as the test case to be evaluated.

8. The method according to claim 7, characterized in that After outputting the evaluation result of the test case to be evaluated, the method further includes: The evaluation result of the test case to be evaluated is displayed on the test case management platform to prompt the user to modify the test case to be evaluated according to the evaluation result of the test case to be evaluated, and / or to provide feedback on the evaluation result of the test case to be evaluated.

9. The method according to any one of claims 1 to 4, characterized in that: The obtaining of the test case to be evaluated includes: Get the test case set from the test case management platform; The test case set is split into multiple test cases, the multiple test cases are traversed, and each test case is determined as the test case to be evaluated.

10. The method according to claim 9, characterized in that After outputting the evaluation result of the test case to be evaluated, the method further includes: According to the evaluation result of each test case in the test case set, an evaluation report of the test case set is generated, and the evaluation report of the test case set is sent to a user, and / or the evaluation report of the test case set is displayed.

11. The method according to claim 9, characterized in that The step of generating an evaluation report for the test case set according to the evaluation result of each test case in the test case set includes: According to the evaluation result of each test case in the test case set, at least one of the following information is obtained: an average score of the test cases in the test case set, a proportion of test cases in different score ranges, test cases with scores below a preset threshold, and reasons for the scores; An evaluation report of the test case set is generated according to the information.

12. The method according to any one of claims 1 to 4, characterized in that: After outputting the evaluation result of the test case to be evaluated, the method further includes: Based on the evaluation results of different test cases of the target business line within a preset period, a test case periodic evaluation report of the target business line is generated, and the test case periodic evaluation report is sent to the user, and / or the test case periodic evaluation report is displayed.

13. The method according to any one of claims 1 to 4, characterized in that: The test case evaluation model is trained through the following process: Obtaining multiple test case samples and obtaining preset evaluation results of the test case samples; Constructing a training sample according to the test case sample and a preset test case evaluation instruction template, wherein the training sample includes an instruction for evaluating the test case sample and a preset evaluation result of the test case sample; The test case evaluation model is trained according to the training samples.

14. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Obtaining feedback information from users on evaluation results of the test case to be evaluated; If the feedback information indicates that the evaluation result of the test case to be evaluated is unreasonable, obtaining the true evaluation result of the test case to be evaluated; Constructing a training sample according to the target instruction and the actual evaluation result of the test case to be evaluated; The test case evaluation model is optimized according to the training samples.

15. A test case evaluation device, characterized in that: include: An acquisition unit, used for acquiring test cases to be evaluated; An instruction generating unit, configured to construct a target instruction for evaluating the test case to be evaluated according to the test case to be evaluated and a preset test case evaluation instruction template; An evaluation unit is used to call a test case evaluation model with the target instruction as input, and output an evaluation result of the test case to be evaluated, wherein the test case evaluation model is obtained by training based on training samples, and the training samples include instructions for evaluating the test case samples and preset evaluation results of the test case samples.

16. An electronic device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method according to any one of claims 1 to 14 is implemented.

18. A computer program product, characterized in that The method comprises computer-executable instructions, and when a processor executes the computer-executable instructions, the method according to any one of claims 1 to 14 is implemented.

Images