Verification Method, Device, Medium and Electronic Device for Test Cases

Abstract

The present disclosure provides a method, apparatus, medium, and electronic device for verifying test cases. The method includes: obtaining at least two case strings corresponding to a test case, and determining a common string of the at least two case strings; calculating the similarity between every two of the at least two case strings to obtain a similarity set; obtaining a string to be verified obtained by executing the test case, and verifying the string to be verified by using the common string and the similarity set to obtain a verification result of the test case. On the one hand, determining the common string and the similarity set of at least two case strings is a self-learning case verification method, and the running process is automated and intelligent, without the need to write code, which improves the verification efficiency of the strings of test cases. On the other hand, verifying the string to be verified by using the common string and the similarity set does not require manual participation, which improves the reliability and accuracy of string verification.

Description

Technical Field

[0001] The present disclosure relates to the technical field of text processing, and in particular, to a method for verifying test cases, an apparatus for verifying test cases, a computer-readable medium, and an electronic device. Background Art

[0002] There are various verification methods for strings of types such as text, for example, testers write assertions or matching codes in various forms.

[0003] However, the efficiency of writing code for verification is too low, and manual judgment and maintenance are also required in different scenarios. Therefore, this method has low reliability and is not automated and intelligent enough.

[0004] In view of this, there is an urgent need in the art to develop a new method and apparatus for verifying test cases.

[0005] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the technical background of the present application, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0006] The purpose of the present disclosure is to provide a method for verifying test cases, an apparatus for verifying test cases, a computer-readable medium, and an electronic device, so as to at least overcome to some extent the technical problems of too low efficiency and reliability of the string verification method.

[0007] Other features and advantages of the present disclosure will become apparent through the following detailed description, or will be partially learned through the practice of the present disclosure.

[0008] According to one aspect of the embodiments of the present disclosure, a method for verifying test cases is provided. The method includes: obtaining at least two use case strings corresponding to a test case, and determining a common string of the at least two use case strings;

[0009] Calculating the similarity between every two of the at least two use case strings to obtain a similarity set;

[0010] Obtaining a to-be-verified string obtained by executing the test case, and verifying the to-be-verified string by using the common string and the similarity set to obtain a verification result of the test case.

[0011] According to one aspect of the embodiments of the present disclosure, an apparatus for verifying test cases is provided. The apparatus includes: a common part module configured to obtain at least two use case strings corresponding to a test case, and determine a common string of the at least two use case strings;

[0012] A difference calculation module, configured to calculate the similarity between each pair of the at least two use case strings to obtain a similarity set;

[0013] A verification execution module, configured to obtain a to-be-verified string obtained by executing the test case, and use the common string and the similarity set to verify the to-be-verified string to obtain the verification result of the test case.

[0014] In some embodiments of the present disclosure, based on the above technical solution, the verification execution module includes: a number determination sub-module, configured to obtain the number of strings of the common string and obtain the to-be-verified number of the to-be-verified string;

[0015] A set determination sub-module, configured to determine a to-be-verified similarity set corresponding to the to-be-verified string if the to-be-verified string includes the common string and the number of strings is less than the to-be-verified number;

[0016] A set verification sub-module, configured to use the similarity set to verify the to-be-verified similarity set to obtain the verification result of the test case.

[0017] In some embodiments of the present disclosure, based on the above technical solution, the set verification sub-module includes: a verification success unit, configured to determine that the verification result of the test case is verification success if the similarity set includes the to-be-verified similarity set;

[0018] A verification failure unit, configured to determine that the verification result of the test case is verification failure if the similarity set does not include the to-be-verified similarity set.

[0019] In some embodiments of the present disclosure, based on the above technical solution, the set verification sub-module includes: a parameter acquisition unit, configured to acquire a fluctuation parameter corresponding to the similarity set;

[0020] A random verification unit, configured to use the similarity set and the fluctuation parameter to verify the to-be-verified similarity set to obtain a verification result.

[0021] In some embodiments of the present disclosure, based on the above technical solution, the verification device for the test case further includes: a number determination module, configured to obtain the number of strings of the common string and obtain the to-be-verified number of the to-be-verified string; [[ID=2)]

[0022] An identical verification module, configured to determine that the verification result of the test case is verification success if the to-be-verified string includes the common string and the to-be-verified number is equal to the number of strings.

[0023] In some embodiments of the present disclosure, based on the above technical solutions, the verification device for test cases further includes: a verification update module configured to obtain a verification string corresponding to the at least two use case strings;

[0024] a set update module configured to update the common string and the similarity set by using the verification string.

[0025] In some embodiments of the present disclosure, based on the above technical solutions, the set update module includes: a set determination sub-module configured to determine a verification common string and a verification similarity set of the at least two use case strings and the verification string;

[0026] a set comparison sub-module configured to compare the common string with the verification common string to obtain a common comparison result, and compare the similarity set with the verification similarity set to obtain a similarity comparison result;

[0027] a comparison result sub-module configured to update the common string and the similarity set according to the common comparison result and the similarity comparison result.

[0028] In some embodiments of the present disclosure, based on the above technical solutions, the comparison result sub-module includes: a common update unit configured to update the common string if the common comparison result shows a difference between the common string and the verification common string; and / or

[0029] a difference update unit configured to update the similarity set if the similarity comparison result shows a difference between the similarity set and the verification similarity set.

[0030] In some embodiments of the present disclosure, based on the above technical solutions, the common part module includes: an operation determination sub-module configured to determine operation substrings between the at least two operation strings pairwise;

[0031] an operation aggregation sub-module configured to perform an aggregation process on the operation substrings to obtain an operation aggregation string;

[0032] an operation duplicate removal sub-module configured to perform a duplicate removal process on the operation aggregation string to obtain a common string.

[0033] In some embodiments of the present disclosure, based on the above technical solutions, the difference calculation module includes: an operation similarity sub-module configured to calculate the similarity between the at least two operation strings pairwise by using a similarity algorithm to obtain an operation similarity;

[0034] an operation set sub-module configured to perform an aggregation process on the operation similarity to obtain a similarity set.

[0035] In some embodiments of the present disclosure, based on the above technical solutions, the common part module includes: a network determination sub-module configured to determine network substrings between every two of the at least two network strings;

[0036] a network aggregation sub-module configured to perform an aggregation process on the network substrings to obtain a network aggregation string;

[0037] a network deduplication sub-module configured to perform a deduplication process on the network aggregation string to obtain a common string.

[0038] In some embodiments of the present disclosure, based on the above technical solutions, the difference calculation module includes: a network similarity sub-module configured to calculate the similarity between every two of the at least two network strings by using a similarity algorithm to obtain a network similarity;

[0039] a network set sub-module configured to perform an aggregation process on the network similarity to obtain a similarity set.

[0040] According to one aspect of the embodiments of the present disclosure, there is provided a computer-readable medium having a computer program stored thereon, and when the computer program is executed by a processor, it implements the verification method of the test case in the above technical solutions.

[0041] According to one aspect of the embodiments of the present disclosure, there is provided an electronic device, which includes: a processor; and a memory for storing executable instructions of the processor; wherein, the processor is configured to execute the verification method of the test case in the above technical solutions by executing the executable instructions.

[0042] In the technical solutions provided by the embodiments of the present disclosure, on the one hand, determining the common string and the similarity set of at least two use case strings is a self-learning use case verification method, and the running process is automated and intelligent, without the need to write code, which improves the verification efficiency of the strings of the test cases; on the other hand, using the common string and the similarity set to verify the strings to be verified does not require manual participation, which improves the reliability and accuracy of the string verification.

[0043] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. In the drawings:

[0045] Figure 1 Schematically shows the architecture diagram of an exemplary system applying the technical solution of the present disclosure;

[0046] Figure 2 Schematically shows the flowchart of the steps of a verification method for a test case in some embodiments of the present disclosure;

[0047] Figure 3 Schematically shows the flowchart of the steps of a method for obtaining a common string by operating on strings in some embodiments of the present disclosure;

[0048] Figure 4 Schematically shows the flowchart of the steps of a method for obtaining a common one from network strings in some embodiments of the present disclosure;

[0049] Figure 5 Schematically shows the flowchart of the steps of a method for obtaining a similarity set by calculating the similarity between operation strings in some embodiments of the present disclosure;

[0050] Figure 6 Schematically shows the flowchart of the steps of a method for obtaining a similarity set by calculating the similarity between network strings in some embodiments of the present disclosure;

[0051] Figure 7 Schematically shows the flowchart of the steps of a method for updating the common string and the similarity set in some embodiments of the present disclosure;

[0052] Figure 8 Schematically shows the flowchart of the steps of a method for updating the common string and the similarity set according to the comparison result in some embodiments of the present disclosure;

[0053] Figure 9 Schematically shows the flowchart of the steps of a method for further updating the common string and the similarity set according to the comparison result in some embodiments of the present disclosure;

[0054] Figure 10 Schematically shows the flowchart of the steps of a method for performing verification by simultaneously using the common string and the similarity set in some embodiments of the present disclosure;

[0055] Figure 11A flowchart showing the steps of a method for obtaining a verification result in some embodiments of the present disclosure is schematically shown;

[0056] Figure 12 A flowchart showing the steps of a method for verification using fluctuation parameters in some embodiments of the present disclosure is schematically shown;

[0057] Figure 13 A flowchart showing the steps of a method for verification using a common string in some embodiments of the present disclosure is schematically shown;

[0058] Figure 14 A flowchart showing the steps of a method for verifying test cases in an application scenario in some embodiments of the present disclosure is schematically shown;

[0059] Figure 15 A schematic diagram of an interface for setting a recording scenario in an application scenario in some embodiments of the present disclosure is schematically shown;

[0060] Figure 16 A schematic diagram of an interface for a learning task in an application scenario in some embodiments of the present disclosure is schematically shown;

[0061] Figure 17 A schematic diagram of an interface for at least two strings in an application scenario in some embodiments of the present disclosure is schematically shown;

[0062] Figure 18 A schematic diagram of an interface for another at least two strings in an application scenario in some embodiments of the present disclosure is schematically shown;

[0063] Figure 19 A schematic diagram of an interface for storing in text mode in an application scenario in some embodiments of the present disclosure is schematically shown;

[0064] Figure 20 A schematic diagram of a scenario that can be verified by a method for verifying test cases in some embodiments of the present disclosure is schematically shown;

[0065] Figure 21 A schematic diagram of an interface for a record of verification of a string to be verified in an application scenario in some embodiments of the present disclosure is schematically shown;

[0066] Figure 22 A schematic diagram of an interface for all verification results in an application scenario in some embodiments of the present disclosure is schematically shown;

[0067] Figure 23 A schematic diagram of an interface for an example of a test method for applying a test case in some embodiments of the present disclosure is schematically shown;

[0068] Figure 24A structural block diagram of a verification device for a test case in some embodiments of the present disclosure is schematically shown;

[0069] Figure 25 A schematic structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present disclosure is schematically shown. Detailed implementation manners

[0070] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art.

[0071] In addition, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present disclosure. However, those skilled in the art will realize that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be used. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present disclosure.

[0072] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0073] The flowcharts shown in the drawings are only illustrative and do not necessarily include all the contents and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to the actual situation.

[0074] In the related art of the present field, there are many methods for string verification. For example, writing various forms of assertions and matches. Among them, assertions are a commonly used debugging method in software development and are supported by many programming languages. Assertions are used to ensure the most basic and critical correctness of the program.

[0075] Especially in the testing field, testers need to write a large amount of code for string verification. And, in different scenarios, testers also need to make judgments and maintain.

[0076] This way of testers writing code to verify strings makes the verification results vary from person to person, with low reliability and large variations. Moreover, for different scenarios, developers have to participate in writing code for verification, and the verification efficiency is too low.

[0077] Based on the problems existing in the above solutions, the present disclosure provides a verification method for test cases, a verification device for test cases, a computer-readable medium, and an electronic device based on artificial intelligence.

[0078] Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science. It attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines, enabling the machines to have the functions of perception, reasoning, and decision-making.

[0079] Artificial intelligence technology is an interdisciplinary subject with a wide range of fields, including both hardware-level technologies and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0080] Among them, Machine Learning (ML) is an interdisciplinary subject that involves multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications cover all fields of artificial intelligence. Machine learning and deep learning usually include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rote learning.

[0081] The verification method for test cases using machine learning technology in artificial intelligence does not require manual participation, improves the reliability of the test case verification results, and also does not require writing code, thus improving the verification efficiency of test cases.

[0082] Figure 1 Shows an exemplary system architecture diagram applying the technical solution of the present disclosure.

[0083] Such asFigure 1 As shown, the system architecture 100 may include a terminal 110, a network 120, and a server side 130. Among them, the terminal 110 and the server side 130 are connected through the network 120.

[0084] The terminal 110 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The network 120 may be a communication medium of various connection types capable of providing a communication link between the terminal 110 and the server side 130. For example, it may be a wired communication link, a wireless communication link, or an optical fiber cable, etc. This application does not make any restrictions here. The server 130 may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers. It may also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0085] Specifically, the server side 130 obtains at least two use case strings corresponding to the test case, determines the common string of the at least two use case strings, and calculates the similarity between every two of the at least two use case strings to obtain a similarity set. Further, a to-be-verified string obtained by executing the test case is obtained, so as to verify the to-be-verified string by using the common string and the similarity set to obtain the verification result of the test case.

[0086] In addition, the test case verification method in the embodiments of the present disclosure may be applied to the terminal or the server side. The present disclosure does not make any special limitations here. The embodiments of the present disclosure mainly take the application of the test case verification method to the server side 130 as an example for illustration.

[0087] The following will make a detailed description of the test case verification method, the test case verification device, the computer-readable medium, and the electronic device provided by the present disclosure in combination with specific embodiments.

[0088] Figure 2 Schematically shows the step flow chart of the test case verification method in some embodiments of the present disclosure. As Figure 2 shown, the test case verification method mainly may include the following steps:

[0089] Step S210. Obtain at least two use case strings corresponding to the test case, and determine the common string of the at least two use case strings.

[0090] Step S220. Calculate the similarity between every two of the at least two use case strings to obtain a similarity set.

[0091] Step S230. Obtain the string to be verified obtained by executing the test case, and verify the string to be verified by using the common string and the similarity set to obtain the verification result of the test case.

[0092] In an exemplary embodiment of the present disclosure, on the one hand, determining the common string and the similarity set of at least two use case strings is a self-learning use case verification method, and the running process is automated and intelligent, without writing code, which improves the verification efficiency of the strings of the test case; on the other hand, using the common string and the similarity set to verify the string to be verified does not require manual participation, which improves the reliability and accuracy of the string verification.

[0093] The following will explain each step of the test case verification method in detail.

[0094] In step S210, obtain at least two use case strings corresponding to the test case, and determine the common string of the at least two use case strings.

[0095] In an exemplary embodiment of the present disclosure, the test case can be a recorded use case, and can be recorded by a recording engine.

[0096] It should be noted that the recorded test case can be regarded as a series of operation flows and network flows. Among them, the operation flow can be the various operations of the user, and the network flow can be the network requests generated by the user in each operation.

[0097] The at least two use case strings corresponding to the test case can be the strings generated during the playback of the recorded test case.

[0098] For example, when the test case is a use case for testing the function of a user clicking on a search engine to search for data, the string corresponding to the operation flow can be the name of the search engine or the value of other elements, and the network flow can be the identification string corresponding to the search data, etc. This exemplary embodiment does not make special limitations on this.

[0099] Therefore, the at least two use case strings can include two categories: the string corresponding to the operation flow and the string corresponding to the network flow, so as to determine the common string under different categories.

[0100] Figure 3 and Figure 4 respectively show the step flowcharts of the methods for determining the common string under the two categories. Among them, Figure 3 shows the step flowchart of the method for determining the common string of the operation strings corresponding to the operation flow, Figure 4 shows the step flowchart of the method for determining the common string of the network strings corresponding to the network flow.

[0101] In an alternative embodiment, at least two use case strings include at least two operation strings. Figure 3 The flowchart of the steps of a method for obtaining a common string from operation strings is shown, as Figure 3 shown. The method at least includes the following steps: In step S310, determine the operation substrings between every two of the at least two operation strings.

[0102] The at least two operation strings are strings corresponding to an operation flow, and this string can be in text form or in digital form. This exemplary embodiment does not make special limitations on this.

[0103] The way to determine the identical strings between every two of the at least two strings can be to use the standard algorithm of the difflib module in the standard library Python. The difflib module does not need to be installed, and the function of the provided algorithm is to compare the differences between strings such as texts, and it also supports outputting a document with relatively strong readability.

[0104] Therefore, the standard algorithm of the difflib model can pairwise compare the operation substrings between the operation strings. This substring can be the identical string between every two operation strings.

[0105] Specifically, the standard algorithm of the difflib model can first store all the strings included in the two operation strings into the corresponding sets respectively to obtain two string sets; then, find the intersection of the two string sets to obtain the operation substring of these two operation strings. When the number of operation strings is greater than 2, the way to determine the operation substrings between other pairwise operation strings is the same and will not be elaborated.

[0106] For example, when the string set of the first operation string is {element 1, element 2, element 3}, the string set of the second operation string is {element 1, element 2, element 4}, and the string set of the third operation string is {element 2, element 5}, find the intersection of the first operation string and the second operation string to obtain the operation substring between the first operation string and the second operation string as {element 1, element 2}, find the intersection of the first operation string and the third operation string to obtain the operation substring between the first operation string and the third operation string as {element 2}, and find the intersection of the second operation string and the third operation string to obtain the operation substring between the second operation string and the third operation string as {element 2}. Therefore, the operation substrings of these three operation strings can be obtained as {element 1, element 2}, {element 2}, and {element 2} respectively.

[0107] In step S320, perform an aggregation process on the operation substrings to obtain an operation aggregation string.

[0108] After determining the identical operation substrings between pairwise operation strings, the operation substrings can be aggregated to obtain an operation aggregation string, which can include at least one operation substring.

[0109] For example, when the operation substrings of three operation strings are {element 1, element 2}, {element 2}, and {element 2} respectively, aggregating these three operation substrings can obtain an aggregation string {element 1, element 2, element 2, element 2}.

[0110] In step S330, the operation aggregation string is de-duplicated to obtain a common string.

[0111] After obtaining the operation aggregation string, the operation aggregation string can be further de-duplicated to obtain a common string, which can include at least one different operation substring.

[0112] For example, when the operation aggregation string is {element 1, element 2, element 2, element 2}, de-duplicating {element 1, element 2, element 2, element 2} can obtain a common string {element 1, element 2}.

[0113] In this exemplary embodiment, a method for determining a common string when at least two operation strings are given is provided. The logically rigorous determination method ensures the accuracy of the common string and improves the determination efficiency of the common string.

[0114] In an alternative embodiment, at least two use case strings include at least two network strings. Figure 4 The flowchart of the steps of the method for obtaining a common string from network strings is shown, as Figure 4 shown, the method at least includes the following steps: In step S410, determine the network substrings between pairwise at least two network strings.

[0115] The at least two network strings are strings corresponding to network flows, and the strings can be in text form or digital form, and this exemplary embodiment does not make special limitations thereon.

[0116] The method for determining the identical strings between at least two strings pairwise can also be to use the standard algorithm of the difflib module in the standard library Python.

[0117] Therefore, using the standard algorithm of the difflib model can pairwise compare the network substrings between network strings. The network substring can be the identical string between pairwise network strings.

[0118] For example, when the string sets of the first network string are {1, 2, 3}, the string sets of the second network string are {1, 2, 4}, and the string sets of the third network string are {2, 5}, taking the intersection of the first network string and the second network string gives the network substring between the first network string and the second network string as {1, 2}, taking the intersection of the first network string and the third network string gives the network substring between the first network string and the third network string as {2}, and taking the intersection of the second network string and the third network string gives the network substring between the second network string and the third network string as {2}. Therefore, the network substrings of these three network strings are {1, 2}, {2}, and {2} respectively.

[0119] In step S420, the network substrings are aggregated to obtain a network aggregation string.

[0120] After determining the identical network substrings between pairwise network strings, the network strings can be aggregated to obtain a network aggregation string, which includes multiple network strings.

[0121] For example, when the network substrings of three network strings are {1, 2}, {2}, and {2} respectively, aggregating these three network substrings can obtain the network aggregation string as {1, 2, 2, 2}.

[0122] In step S430, the network aggregation string is de-duplicated to obtain a common string.

[0123] After obtaining the network aggregation string, the network aggregation string can be further de-duplicated to obtain a common string, which can include at least one different network aggregation string.

[0124] For example, when the network aggregation string is {1, 2, 2, 2}, de-duplicating {1, 2, 2, 2} can obtain the common string as {1, 2}.

[0125] In this exemplary embodiment, a method for determining a common string when there are at least two network strings is given. The logically rigorous determination method ensures the accuracy of the common string and improves the determination efficiency of the common string.

[0126] It should be noted that whether it is to determine the corresponding common string for the operation string or for the network string, other algorithms or methods can also be used to directly obtain the non-repeated common string according to the actual situation. The determination method of the common string in this exemplary embodiment is not specially limited.

[0127] In step S220, the similarity between at least two use case strings is calculated pairwise to obtain a similarity set.

[0128] In an exemplary embodiment of the present disclosure, in addition to determining the common string between at least two use case strings, the similarity between at least two use case strings can be further calculated.

[0129] Figure 5 and Figure 6 respectively show the flowchart of the steps of a method for calculating the similarity between an operation string and a network string to obtain a similarity set. Among them, Figure 5 shows the flowchart of the steps of a method for calculating the similarity between operation strings to obtain a similarity set, Figure 6 shows the flowchart of the steps of a method for calculating the similarity between network strings to obtain a similarity set.

[0130] In an alternative embodiment, Figure 5 shows the flowchart of the steps of a method for obtaining a similarity set of operation strings. As Figure 5 shown, the method at least includes the following steps: In step S510, the similarity between at least two operation strings is calculated pairwise using a similarity algorithm to obtain an operation similarity.

[0131] The similarity algorithm can also be the standard algorithm of the difflib model, or other algorithms for calculating similarity. This exemplary embodiment does not make special limitations on this.

[0132] Using the similarity algorithm to calculate the similarity of pairwise operation strings can respectively obtain the operation similarity between pairwise operation strings.

[0133] Specifically, the standard algorithm of the difflib model can first perform word segmentation on two operation strings respectively to obtain segmented strings, and then calculate the cosine value between the segmented strings as the corresponding operation similarity.

[0134] The ways of performing word segmentation on two operation strings respectively include rule-based word segmentation and statistic-based word segmentation.

[0135] Among them, rule-based word segmentation is to perform word segmentation and splitting according to the matching method by pre-constructing a dictionary. The dictionary can be a multi-gram N-gram (Chinese language model) dictionary, and the operation string is matched with the pre-constructed multi-gram N-gram dictionary according to the word segmentation strategy to obtain the possible segmentation results of each string, and then the shortest path method based on the multi-gram N-gram dictionary is used to calculate the final segmented string.

[0136] Statistical word segmentation uses a classifier trained and constructed using annotated corpora for word segmentation. The classifier can be trained and constructed using machine learning or deep learning algorithms. Such algorithms can adopt Hidden Markov Model (HMM for short), conditional random field algorithm (CRF for short), and deep learning, etc.

[0137] In addition, it is also possible to directly use multiple different word segmentation tools as the word segmentation model, and separately call different types of word segmentation tools to perform word segmentation on the operation string to obtain the segmented string. This operation string can also be called the roughly segmented corpus. Call different types of word segmentation tools to perform preliminary word segmentation on the operation string to obtain multiple initial segmentations corresponding to different word segmentation tools, and merge the multiple initial segmentations into an initial segmentation set. At this time, the initial segmentation set contains a large amount of initial segmentation data, and the number of initial segmentations can be reduced by voting on each initial segmentation. Among them, the voting process can be statistically obtained according to the word segmentation tools. For example, for a certain initial segmentation, all three word segmentation tools can segment this initial segmentation from the original text. At this time, this initial segmentation is used as the segmented string. If the segmentation results of the three word segmentation tools for this initial segmentation are inconsistent, directly discard this initial segmentation; if the segmentation results of two word segmentation tools for this initial segmentation are consistent and the segmentation result of the other word segmentation tool is inconsistent, it can be determined that this initial segmentation is the segmented string.

[0138] When using multiple word segmentation tools for word segmentation, the voting method can initially reduce the number of initial segmentations in the initial segmentation set and ensure the effectiveness of the segmented string.

[0139] Among them, open-source Chinese word segmentation tools can be used as the word segmentation tools, such as Jieba Chinese word segmentation, Hanlp tokenizer, Harbin Institute of Technology Language Technology Platform (LTP for short), a set of Chinese lexical analysis toolkit developed by the Natural Language Processing and Social Humanities Computing Laboratory of Tsinghua University (THU Lexial Analyzer for Chinese, THULAC for short), Stanford tokenizer, Natural Language Processing and Information Retrieval Sharing Platform NLPIR, etc. These word segmentation tools each have their own word segmentation characteristics. For example, the multiple word segmentation tools called can be LTP, THULAV, and NLPIR, three word segmentation tools, to perform word segmentation on the operation string.

[0140] The word segmentation module of LTP is trained and decoded based on the CRF model. It models the target sequence based on the observation sequence, and the data source used is the data from People's Daily from January to June 1998. It is initialized through the file path word segmentation interface, and the word segmentation interface is called to perform word segmentation on the operation string to obtain the segmented string.

[0141] The THULAC toolkit comes with the original corpus for model training, but authorization is required. The Chinese word segmentation and part-of-speech tagging functions of the THULAC toolkit are characterized by strong capabilities and high accuracy. It can perform word segmentation on the operation string by configuring the interface parameters and calling the word segmentation sentence to obtain the segmented string.

[0142] The NLPIR tool is a full-chain analysis tool that can perform word segmentation on the operation string. In the specific word segmentation process, it is necessary to import a pre-built dictionary and perform the initial segmentation by calling the dictionary to obtain the segmentation result. Further, probability statistics methods and simple rules are used to eliminate ambiguous words, and word frequency information is used to identify out-of-vocabulary words. After eliminating ambiguity and identifying out-of-vocabulary words, the segmented string is obtained.

[0143] After obtaining the segmented string, the cosine value between the segmented strings can be calculated as the operation similarity.

[0144] Cosine similarity measures the similarity between two segmented strings by measuring the cosine value of the angle between the vectors of the two segmented strings. The cosine value of a 0-degree angle is 1, and the cosine value of any other angle is not greater than 1, and its minimum value is -1. Thus, the cosine value of the angle between two vectors determines whether the two vectors generally point in the same direction. When the two vectors have the same direction, the cosine similarity value is 1; when the angle between the two vectors is 90°, the cosine similarity value is 0; when the two vectors point in exactly opposite directions, the cosine similarity value is -1. This result is independent of the length of the vectors and is only related to the direction of the vectors. Cosine similarity is usually used in positive space, so the value given is between 0 and 1.

[0145] The cosine similarity between two segmented strings can be obtained through the Euclidean dot product formula:

[0146] a·b=‖a‖‖b‖cosθ (1)

[0147] When given two vectors A and B of strings, the cosine similarity θ is obtained from the dot product and the vector lengths:

[0148]

[0149] where A i and B i represent the corresponding components of vector A and vector B respectively.

[0150] Therefore, the operation similarity between the first operation string and the second operation string is 20%, the operation similarity between the first operation string and the third operation string is 31%, and the operation similarity between the second operation string and the third operation string is 50%.

[0151] It should be noted that the form of the operation similarity can be in the form of a percentage, a decimal form, or other numerical forms, and this exemplary embodiment does not make special limitations on this.

[0152] In step S520, the operation similarities are aggregated to obtain a similarity set.

[0153] After obtaining the operation similarities between pairwise operation strings, further aggregation processing can be performed to obtain a similarity set, and the similarity set includes at least one operation similarity.

[0154] For example, the operation similarity between the first operation string and the second operation string is 20%, the operation similarity between the first operation string and the third operation string is 31%, and the operation similarity between the second operation string and the third operation string is 50%. Further, aggregating these three operation similarities can obtain a similarity set as {20%, 31%, 50%}.

[0155] It should be noted that the similarity set can also be in the form of a list or a sequence. When the similarity set is in the form of a sequence, it can be sorted according to the magnitude of the operation similarity to verify whether the learning is completed in the subsequent process or to be quickly used during the verification process.

[0156] In this exemplary embodiment, by calculating the operation similarities between pairwise operation strings respectively, the corresponding similarity set can be obtained, and the calculation method is simple and accurate, which improves the overall learning efficiency from the aspect of the similarity set.

[0157] Figure 6 shows a flowchart of the steps of a method for calculating the similarity between network strings to obtain a similarity set, as Figure 6 shown, the method at least includes the following steps: In step S610, a similarity algorithm is used to calculate the similarity between at least two network strings pairwise to obtain a network similarity.

[0158] The similarity algorithm can also be the standard algorithm of the difflib model, or other algorithms for calculating similarity, and this exemplary embodiment does not make special limitations on this.

[0159] By using the similarity algorithm to calculate the similarity between pairwise network strings, the network similarity between pairwise network strings can be obtained respectively. The specific calculation method is the same as that in step S510 and will not be elaborated here.

[0160] For example, the network similarity between the first network string and the second network string is 80%, the network similarity between the first network string and the third network string is 71%, and the network similarity between the second network string and the third network string is 77%.

[0161] It should be noted that the form of the network similarity can be in the form of a percentage, a decimal, or other numerical forms, and this exemplary embodiment does not make special limitations on this.

[0162] In step S620, the network similarities are aggregated to obtain a similarity set.

[0163] After obtaining the network similarities between pairwise network strings, they can be further aggregated to obtain a similarity set.

[0164] For example, the network similarity between the first network string and the second network string is 80%, the network similarity between the first network string and the third network string is 71%, and the network similarity between the second network string and the third network string is 77%. Further, aggregating these three network similarities can obtain a similarity set of {80%, 71%, 77%}.

[0165] It should be noted that the similarity set can also be in the form of a list or a sequence. When the similarity set is in the form of a sequence, it can be sorted according to the magnitude of the network similarity for subsequent verification of whether the learning is completed or for quick use during the verification process.

[0166] In this exemplary embodiment, by calculating the network similarities between pairwise network strings respectively, the corresponding similarity set can be obtained. The calculation method is simple and accurate, and the overall learning efficiency is improved from the aspect of the similarity set.

[0167] It should be noted that whether it is to calculate the operation similarity between operation strings to obtain a similarity set or to calculate the network similarity between network strings to obtain a similarity set, other algorithms or methods can also be used to obtain the similarity set according to the actual situation. This exemplary embodiment does not make special limitations on the calculation method of the similarity set.

[0168] After determining the common strings and similarity sets of at least two use case strings, the learned common strings and similarity sets can be further verified at this time. When the verification result is that both the common strings and the similarity sets no longer change, it is determined that the learning ends at this time and there is no need to update the common strings and the similarity sets; when the verification result is that the common strings and the similarity sets are still unstable, it is necessary to update the common strings and / or the similarity sets.

[0169] In an alternative embodiment, Figure 7 A flowchart of the steps of a method for updating the common strings and similarity sets is shown, as Figure 7 shown. The method at least includes the following steps: In step S710, obtain verification strings corresponding to at least two use case strings.

[0170] The verification strings can be obtained by copying one of the at least two use case strings, or can be strings corresponding to the operation flow or network flow generated by re-playing the test case again. The exemplary embodiment does not make special limitations on the way of obtaining the verification strings.

[0171] It is worth noting that when verifying the common strings or similarity sets obtained according to the operation strings, the verification strings are the copied strings or the strings obtained according to the operation flow of the test case; when verifying the common strings or similarity sets obtained according to the network strings, the verification strings are the copied strings or the strings obtained according to the network flow of the test case.

[0172] In step S720, update the common strings and similarity sets using the verification strings.

[0173] In an alternative embodiment, Figure 8 A flowchart of the steps of a method for updating the common strings and similarity sets according to the comparison result is shown, as Figure 8 shown. The method at least includes the following steps: In step S810, determine the verified common strings and verified similarity sets of at least two use case strings and the verification strings.

[0174] When determining the verification strings, the strings for determining the verified common strings include, in addition to at least two use case strings, an additional verification string. The way of determining the verified common strings is the same as the way of determining the common strings in step S210 and will not be elaborated here.

[0175] Correspondingly, in addition to at least two use case strings, a verification string is newly added to the string for determining the verification similarity. The method for calculating the verification similarity set by calculating the similarity between at least two use case strings and the verification string pairwise is the same as the method for obtaining the similarity set in step S220, and will not be elaborated here.

[0176] In step S820, compare the common comparison result obtained by comparing the common string with the verification common string, and compare the similarity set with the verification similarity set to obtain the similarity comparison result.

[0177] The comparison method for the common string and the verification common string can be to compare whether there are differences between the two to obtain the common comparison result. The ways of having differences include, but are not limited to, whether the number of common strings is the same as the number of verification common strings, whether the common string is the same as the string in the verification common string, etc.

[0178] The comparison method between the similarity set and the verification similarity set can also be to compare whether there are differences between the two to obtain the similarity comparison result. The ways of having differences include, but are not limited to, whether the number of similarities in the similarity set is the same as the number of similarities in the verification similarity, whether the similarities in the similarity set are the same as those in the verification similarity set, etc.

[0179] In step S830, update the common string and the similarity set according to the common comparison result and the similarity comparison result.

[0180] In an alternative embodiment, Figure 9 The flowchart of the steps showing the method for further updating the common string and the similarity set according to the comparison result is as Figure 9 shown. The method at least includes the following steps: In step S910, if the common comparison result is that there are differences between the common string and the verification common string, update the common string.

[0181] If the common comparison result of the common string and the verification common string is that there are differences between the two, the common string can be updated. The update method can be to replace the common string with the verification common string as the new common string, or to add the strings in the verification common string that are not in the common string to the common string, or other update methods. This exemplary embodiment does not make special limitations on the update method.

[0182] In step S920, if the similarity comparison result is that there are differences between the similarity set and the verification similarity set, update the similarity set.

[0183] If the similarity comparison result between the similarity set and the verification similarity set shows a difference between the two, the similarity set can be updated. The update method can be to replace the similarity set with the verification similarity set as the new similarity set, or to add the similarities in the verification similarity set that are not in the similarity set to the similarity set, or other update methods. This exemplary embodiment does not make special limitations on this.

[0184] It should be noted that when the public comparison result shows a difference between the public string and the verification public string, it can be to only update the public string, or to update both the public string and the similarity set at the same time; when the similarity comparison result shows a difference between the similarity set and the verification similarity set, it can be to only update the similarity set, or to update both the similarity set and the public string at the same time.

[0185] After updating the public string and / or the similarity set, to determine whether the updated public string and similarity set are stable, the string to be verified can be continuously obtained for a new round of verification and update until after the verification times are satisfied, and the public string and the similarity set do not change in each round of verification, indicating that the learning is over.

[0186] Or when the public comparison result shows no difference between the public string and the verification public string, and the similarity comparison result shows no difference between the similarity set and the verification similarity set, a new string to be verified can be continuously obtained for a new round of verification until after the verification times are satisfied, and the public string and the similarity set do not change in each round of verification, determining that the learning is over.

[0187] In this exemplary embodiment, after determining the public string and the similarity set, the public string and the similarity set can be updated to make the public string and the similarity set stable, providing a data basis for the subsequent verification process and ensuring the accuracy of the verification result.

[0188] In step S230, the string to be verified obtained by executing the test case is acquired, and the verification result of the test case is obtained by verifying the string to be verified using the public string and the similarity set.

[0189] In the exemplary embodiment of the present disclosure, the string to be verified can be the string corresponding to the operation flow or network flow obtained by executing the test case when verifying whether the function to be tested by the test case has changed or is available.

[0190] After obtaining the string to be verified, the string to be verified can be verified using the public string and the similarity set to implement the verification of the test case.

[0191] In an alternative embodiment, Figure 10The flowchart of the steps of a method for performing verification by simultaneously using a common string and a similarity set is shown. As Figure 10 shown, the method at least includes the following steps: In step S1010, obtain the number of strings of the common string and obtain the number of strings to be verified of the string to be verified.

[0192] The number of strings can be the number of non-repeating strings included in the common string.

[0193] For example, when the common string is obtained from at least two operation strings and the common string is {element 1, element 2, element 3}, the number of strings can be counted as 3 at this time; when the common string is obtained from at least two network strings and the common string is {1, 2, 3, 4}, the number of strings can be counted as 4 at this time.

[0194] The number of strings to be verified is the number of non-repeating strings included in the string to be verified. For example, when the string to be verified is {element 1, element 2}, the number of strings to be verified is 2; when the string to be verified is {1, 2, 3}, the number of strings to be verified is 3.

[0195] In addition, the number of strings to be verified can also be, according to the requirements of the actual situation, the number of all strings in the string to be verified. That is, in this statistical process, whether the strings in the string to be verified are repeated is not distinguished, and they are all included in the statistical result of the number of strings to be verified.

[0196] In step S1020, if the string to be verified includes the common string and the number of strings is less than the number of strings to be verified, determine the similarity set to be verified corresponding to the string to be verified.

[0197] Match the common string and the string to be verified, that is, determine whether each string in the common string also exists in the string to be verified. When it is determined that each string in the common string exists in the string to be verified, it can be determined that the string to be verified includes the common string.

[0198] Furthermore, compare the number of strings and the number of strings to be verified. When the number of strings is less than the number of strings to be verified, it indicates that there are common strings and different strings between the common string and the string to be verified.

[0199] To further verify whether the different strings can still implement the function of the test case, the similarity set to be verified corresponding to the string to be verified can be further determined.

[0200] Among them, the similarity set to be verified can be obtained by calculating the similarity between the string to be verified and each of at least two case strings. The calculation method is the same as that for calculating the similarity set in step S220, and will not be elaborated here.

[0201] In step S1030, the similarity set is used to verify the similarity set to be verified to obtain the verification result of the test case.

[0202] In an alternative embodiment, Figure 11 The flowchart of the steps of the method for obtaining the verification result is shown, as Figure 11 shown. The method at least includes the following steps: In step S1110, if the similarity set includes the similarity set to be verified, determine that the verification result of the test case is verification successful.

[0203] When the similarity set includes each similarity in the similarity set to be verified, it is determined that the similarity set includes the similarity set to be verified. At this time, the verification result of the test case can be determined to be verification successful by verifying the string to be verified.

[0204] In step S1120, if the similarity set does not include the similarity set to be verified, determine that the verification result of the test case is verification failed.

[0205] When the similarity set does not include each similarity in the similarity set to be verified, it is determined that the similarity set does not include the similarity set to be verified. At this time, the verification result of the test case is determined to be verification failed by verifying the string to be verified.

[0206] In this exemplary embodiment, through the verification result between the similarity set and the similarity set to be verified, the verification result of the test case can be determined. The verification method has high accuracy and improves the verification efficiency.

[0207] When the string rule of the different part between the common string and the string to be verified changes, only the common string and the similarity set can be used for verification. When the string of the different part between the common string and the similarity to be verified changes randomly, the fluctuation parameter can also be obtained for joint verification.

[0208] In an alternative embodiment, Figure 12 The flowchart of the steps of the method for verification using the fluctuation parameter is shown, as Figure 12 shown. The method at least includes the following steps: In step S1210, obtain the fluctuation parameter corresponding to the similarity set.

[0209] The fluctuation parameter is a threshold that allows the similarity to fluctuate up and down. It can be preset, calculated based on the common string and the string to be verified, or determined by other means. This exemplary embodiment does not make special limitations on this.

[0210] In step S1220, the similarity set and the fluctuation parameter are used to verify the similarity set to be verified to obtain a verification result.

[0211] When verifying the similarity to be verified using the similarity and the fluctuation parameter, it can be first determined whether the similarity set to be verified is included in the similarity set according to the Figure 11 method.

[0212] If each similarity in the similarity set to be verified is not included in the similarity set, the difference can be calculated between the similarities in the similarity set and the similarities in each similarity set to be verified, and it is determined whether the difference is less than the fluctuation parameter.

[0213] If all the differences are less than the fluctuation parameter, the verification result of the test case is determined to be verification successful; if there are individual or all means that are not less than the fluctuation parameter, the verification result of the test case can be determined to be verification failed.

[0214] In this exemplary embodiment, the common string, the similarity set, and the fluctuation parameter are used to verify the string to be verified to determine the verification result of the test case, which improves the accuracy and efficiency of verification and enriches the application scenarios of the verification method.

[0215] In addition to verifying using the common string and the similarity set simultaneously, or verifying using the common string, the similarity set, and the fluctuation parameter simultaneously, it is also possible to verify only based on the common string.

[0216] In an alternative embodiment, Figure 13 shows a flowchart of the steps of a method for verifying using a common string, as Figure 13 shown. This method at least includes the following steps: In step S1310, the number of strings of the common string is obtained, and the number of strings to be verified of the string to be verified is obtained.

[0217] The number of strings can be the number of common strings representing non-repeating strings.

[0218] In step S1320, if the string to be verified includes the common string and the number of strings to be verified is equal to the number of strings, the verification result of the test case is determined to be verification successful.

[0219] Match the common string with the string to be verified, that is, determine whether each string in the common string also exists in the string to be verified. When it is determined that each string in the common string exists in the string to be verified, it can be determined that the string to be verified includes the common string.

[0220] Further, compare the number of strings with the number of strings to be verified. When the number of strings is equal to the number of strings to be verified, it indicates that the common string and the string to be verified are exactly the same, and then it can be further determined that the test case verification is successful.

[0221] In this exemplary embodiment, using the common string to verify the string to be verified to determine the verification result of the test case improves the accuracy and efficiency of verification and enriches the application scenarios of the verification method.

[0222] The following makes a detailed description of the verification method of the test case provided in the embodiments of the present disclosure in combination with a specific application scenario.

[0223] Figure 14 The flowchart of the steps of the verification method of the test case in the application scenario is shown. As Figure 14 shown, in step S1401, a playback learning task is issued.

[0224] Figure 15 The schematic diagram of the interface for setting the recording scenario in the application scenario is shown. As Figure 15 shown, when the tester sets the recorded scenario to ONLINE (online), the server will issue a learning task to the task list.

[0225] Figure 16 The schematic diagram of the learning task in the application scenario is shown. As Figure 16 shown, the name of the learning task is Scene Auto Learn Task (scene automatic learning task), and the scene identifier (scene list) is 41, and the product identifier (product id) is 4.

[0226] Further, the playback engine can obtain the current product identifier as 4 from the server according to the name of the learning task Scene Auto Learn Task. Then, through the product identifier 4, the task information of the currently unexecuted task can be pulled. The task information may include the scene identifier and information indicating that the task is a learning (Learn) task or a verification (Verify) task, and may also include other task information. This exemplary embodiment does not make special limitations on this.

[0227] Among them, the time interval for the playback engine to obtain the current product identifier is 1 second, and other durations can also be set. This exemplary embodiment does not make special limitations on this. The playback engine can support two categories of tasks, namely the Learn task and the Verify task. Each time a task is executed, the operation flow and network flow in that scenario can be pulled from the server through the scenario identifier and accurately played back.

[0228] In step S1402, when the playback engine executes, it reports the text to be verified.

[0229] During the process of the playback engine executing the operation flow and network flow, text corresponding to the operation flow and text corresponding to the network flow can be generated, that is, at least two use case strings. And at least two use case strings are reported to the server.

[0230] Figure 17 Shows a schematic diagram of an interface with at least two strings in an application scenario, such as Figure 17 As shown, the at least two strings are operation strings of the operation flow. The operation flow string is the content corresponding to text, which are 6, 4, 3, 2, SMB customer list, customer management, customer operation, and INPUT respectively.

[0231] Figure 18 Shows a schematic diagram of another interface with at least two strings in an application scenario, such as Figure 18 As shown, the at least two strings are network strings corresponding to the network flow. The network string is the content corresponding to text, which are {"table_name":"client_ka_smb",……}, {"table_name":"label_router",……}, {}, {"table_name":"label_client",……}, and {"table_name":"supplier",……} respectively.

[0232] In step S1403, learn the common part of the text.

[0233] In the application scenario, when the number of at least two use case strings exceeds 3, the execution of the learning task can be triggered.

[0234] The learning method of the common part can be to pairwise compare the at least two use case strings using the difflib module to find the substring between them. When the at least two use case strings are at least two operation strings, the substring is the operation substring; when the at least two use case strings are at least two network strings, the substring is the network substring.

[0235] In step S1404, partial learning of text difference similarity.

[0236] The learning of the difference similarity part can also be a similarity algorithm such as the standard algorithm of the difflib model. By using the similarity algorithm to calculate the similarity of pairwise operation strings, the operation similarities between pairwise operation strings can be obtained respectively. Correspondingly, by using the similarity algorithm to calculate the similarity of pairwise network strings, the network similarities between pairwise network strings can be obtained respectively.

[0237] In step S1405, aggregation of text common substrings.

[0238] After obtaining the operation substrings, the operation substrings can be aggregated to obtain an operation aggregation string; after obtaining the network substrings, the network substrings can be aggregated to obtain a network aggregation string.

[0239] If there are duplicate parts in the operation aggregation string or the network aggregation string, the duplicate parts can be removed to obtain a common string.

[0240] In step S1406, aggregation of the text difference similarity part.

[0241] After obtaining the operation similarities, aggregation processing can be performed to obtain a similarity set. Correspondingly, after obtaining the network similarities, aggregation processing can be performed to obtain a similarity set.

[0242] In step S1407, judgment of the common substring pattern.

[0243] After obtaining the common string, a verification string can be added to judge whether the common string will change. If it changes, the common string can be updated and a new round of verification can be initiated until the common string does not change for 3 consecutive times.

[0244] In step S1408, judgment of the difference part pattern.

[0245] Similarly, the newly added verification string can also be used to judge whether the similarity list will change. If it changes, the similarity list can be updated and a new round of verification can be initiated until the similarity list does not change for 3 consecutive times.

[0246] In step S1409, storage of the text pattern.

[0247] When the common string and the similarity list do not change for 3 consecutive times at the same time, it indicates that the learning task is completed, and the common string and the similarity set can be stored in a fixed format, that is, the text pattern, for example, stored in a database.

[0248] Figure 19 It shows a schematic diagram of the interface stored in text mode in an application scenario. For example, Figure 19 as shown, at least two learned operation strings are saved in text mode.

[0249] For example, the specific format of this text mode can be:

[0250] {

[0251] "now_time":"2020-09-28 19:52:36",

[0252] "text_list":["6"],

[0253] "result":

[0254] ["6"], []

[0256] ,

[0257] "wave_num":5 / /

[0258] }

[0259] Among them, now_time is the time when learning ends, text_list is the case string used during learning, result includes the common string and the similarity in the similarity set, and wave_num is the fluctuation parameter corresponding to this test case. When wave_num is 5, it means that the values within the range of plus or minus 5 in similarity are all determined to pass the verification, otherwise the verification fails.

[0260] Furthermore, when the playback engine receives a Verify task, it can import the string to be verified into the verification table in the server. When the text verification engine in the server discovers that the string to be verified exists in the verification table, it can execute the algorithm for the verification task.

[0261] In step S1410, the string to be verified obtained by executing the test case is acquired, that is, the text generated by the network request flow. Among them, the text flow is the determined common string and the similarity set.

[0262] Figure 20 It shows a schematic diagram of the scenarios that can be verified by the verification method of the test case. For example, Figure 20 as shown, the scenarios that can be verified first include the exactly same scenario. The verification of the exactly same scenario can be performed only through the common string.

[0263] In step S1411, all the common substring verifications exist.

[0264] Obtain the number of strings of the common string, and obtain the number of strings to be verified of the string to be verified. Then, if the string to be verified includes the common string and the number of strings to be verified is equal to the number of strings, determine that the verification result of the test case is successful verification.

[0265] However, when the string to be verified includes the common string and the number of strings is less than the number of strings to be verified, further verification is performed using the similarity set.

[0266] Therefore, Figure 20 The verifiable scenarios in also include partial difference scenarios. When the partial difference scenario is that the different parts change regularly, further verification can be performed only using the similarity set.

[0267] In step S1412, similarity difference calculation.

[0268] Determine the similarity set to be verified corresponding to the string to be verified. Among them, the similarity set to be verified can be obtained by calculating the similarity between the string to be verified and at least two case strings pairwise.

[0269] When the similarity set includes each similarity in the similarity set to be verified, determine that the similarity set includes the similarity set to be verified. At this time, the verification result of the test case can be determined to be successful verification by verifying the string to be verified; when the similarity set does not include each similarity in the similarity set to be verified, determine that the similarity set does not include the similarity set to be verified. At this time, the verification result of the test case is determined to be failed verification by verifying the string to be verified.

[0270] In addition, Figure 20 The verifiable scenarios in also include partial difference scenarios where the different parts change randomly, and the fluctuation parameter can be obtained for verification.

[0271] In step S1413, similarity fluctuation matching verification.

[0272] When verifying the similarity to be verified using the similarity and the fluctuation parameter, it can be first determined whether the similarity set includes the similarity set to be verified in the Figure 11 way.

[0273] If the similarity set does not include each similarity in the similarity set to be verified, the differences between the similarities in the similarity set and the similarities in the similarity set to be verified can be calculated pairwise, and it is determined whether the differences are less than the fluctuation parameter.

[0274] If all the differences are less than the fluctuation parameter, determine that the verification result of the test case is successful verification; if there are individual or all means not less than the fluctuation parameter, it can be determined that the verification result of the test case is failed verification.

[0275] In step S1414, the single-operation text results are aggregated.

[0276] If there are multiple completely identical scenarios, they can be verified one by one according to the verification method for completely identical scenarios, and the multiple single verification results are aggregated to obtain the verification results for multiple completely identical scenarios.

[0277] In step S1415, the single-operation network request results are aggregated.

[0278] When there are multiple partial difference scenarios, they can be verified one by one according to the verification methods for the two types of partial difference scenarios, and the multiple single verification results are aggregated to obtain the verification results for multiple partial difference scenarios.

[0279] In step S1416, the single-scenario results are aggregated.

[0280] The verification results of the completely identical scenarios and the partial difference scenarios are aggregated to obtain the verification results of all the verification scenarios finally.

[0281] It can be seen that steps S1401 - S1409 are the learning part for the new playback scenario to be stored in the database, and steps S1410 - S1416 are the verification part for the new scenario to execute the test.

[0282] Figure 21 Shows a schematic diagram of the interface for recording the verification of the string to be verified in the application scenario, as Figure 21 shown, state_text is the verification status, and text is the string to be verified.

[0283] Figure 22 Shows a schematic diagram of the interface for all verification results in the application scenario, as Figure 22 shown, state_text being SUCCESS indicates successful verification, and state_text being FALL indicates failed verification.

[0284] Figure 23 Shows a step result diagram of an example of the verification method for application test cases, as Figure 23As shown, at least two use case strings include 4 use case strings, namely, ‘{"platform":"pc","reqid":"bc319c38-7009-4d2b-ad75-b2266037eeee","conferenceld":"5ed10d0372101c6d5a4b130c"}’, ‘{"platform":"pc","reqid":"cfc151aa-a886-4184-bfa2-10fd9709f263","conferenceld":"5ed10d0372101c6d5a4b130c"}’, ‘{"platform":"pc","reqid":"8a986dc5-d40c-47cb-869d-3e8bd6411e03","conferenceld":"5ed10d0372101c6d5a4b130c"}’ and ‘{"platform":"pc","reqid":"39689567-f4be-458d-a187-9a7fa140b6d2","conferenceld":"5ed10d0372101c6d5a4b130c"}’.

[0285] It is determined that the common string of the four use case strings is {"platform":"pc","reqid":"conferenceld":"5ed10d0372101c6d5a4b130c"}, and the similarity set is {74,77,70,74,74,76}.

[0286] The string to be verified obtained by executing the test case is ‘{"platform":"pc","reqid":"fjd5md74-fqce-deac-b983-3c5a98c4f3ca","conferenceld":"5ed10d0372101c6d5a4b130c"}’.

[0287] The common string can be used to verify the string to be verified, and it is obtained that the string to be verified includes the common string. The similarity set to be verified is determined by using the similarity set as {77,76,78,73}. Moreover, the fluctuation parameter c = 1 can be set, then it can be ensured that the similarity set to be verified corresponding to the string to be verified belongs to the similarity set, that is, to ensure that the verification result of the test case is successful verification.

[0288] Based on the above application scenarios, the test case verification method provided by the embodiments of the present disclosure, on the one hand, determines the common string and similarity set of at least two use case strings, which is a self-learning use case verification method. The running process is automated and intelligent, without the need to write code, improving the verification efficiency of the test case strings; on the other hand, using the common string and similarity set to verify the string to be verified, without manual participation, improving the reliability and accuracy of the string verification.

[0289] It should be noted that although the steps of the methods in the present disclosure are described in a specific order in the drawings, this does not require or imply that these steps must be executed in that specific order, or that all the steps shown must be executed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc.

[0290] The following introduces the device embodiments of the present disclosure, which can be used to execute the test case verification method in the above embodiments of the present disclosure. For the details not disclosed in the device embodiments of the present disclosure, please refer to the embodiments of the test case verification method in the above of the present disclosure.

[0291] Figure 24 Schematically shows a structural block diagram of a test case verification device in some embodiments of the present disclosure, as Figure 24 shown, the test case verification device 2400 mainly may include: a common part module 2410, a difference calculation module 2420, and a verification execution module 2430.

[0292] The common part module 2410 is configured to obtain at least two use case strings corresponding to the test case and determine the common string of the at least two use case strings; the difference calculation module 2420 is configured to calculate the similarity between every two of the at least two use case strings to obtain a similarity set; the verification execution module 2430 is configured to obtain the string to be verified obtained by executing the test case, and use the common string and the similarity set to verify the string to be verified to obtain the verification result of the test case.

[0293] In some embodiments of the present disclosure, the verification execution module includes: a number determination sub-module, configured to obtain the number of characters of the common string and obtain the number of characters to be verified of the string to be verified;

[0294] a set determination sub-module, configured to, if the string to be verified includes the common string and the number of characters is less than the number of characters to be verified, determine the similarity set to be verified corresponding to the string to be verified;

[0295] The set verification sub-module is configured to verify the similarity set to be verified by using the similarity set to obtain the verification result of the test case.

[0296] In some embodiments of the present disclosure, the set verification sub-module includes: a verification success unit configured to determine that the verification result of the test case is verification success if the similarity set includes the similarity set to be verified;

[0297] A verification failure unit configured to determine that the verification result of the test case is verification failure if the similarity set does not include the similarity set to be verified.

[0298] In some embodiments of the present disclosure, the set verification sub-module includes: a parameter acquisition unit configured to acquire a fluctuation parameter corresponding to the similarity set;

[0299] A random verification unit configured to verify the similarity set to be verified by using the similarity set and the fluctuation parameter to obtain a verification result.

[0300] In some embodiments of the present disclosure, the verification device for the test case further includes: a number determination module configured to acquire the number of strings of the common string and acquire the number to be verified of the string to be verified;

[0301] An identical verification module configured to determine that the verification result of the test case is verification success if the string to be verified includes the common string and the number to be verified is equal to the number of strings.

[0302] In some embodiments of the present disclosure, the verification device for the test case further includes: a verification update module configured to acquire a verification string corresponding to at least two case strings;

[0303] A set update module configured to update the common string and the similarity set by using the verification string.

[0304] In some embodiments of the present disclosure, based on the above technical solution, the set update module includes: a set determination sub-module configured to determine a verification common string and a verification similarity set of at least two case strings and the verification string;

[0305] A set comparison sub-module configured to compare the common string with the verification common string to obtain a common comparison result, and compare the similarity set with the verification similarity set to obtain a similarity comparison result;

[0306] A comparison result sub-module configured to update the common string and the similarity set according to the common comparison result and the similarity comparison result.

[0307] In some embodiments of the present disclosure, the comparison result sub-module includes: a common update unit configured to update the common string if there is a difference between the common comparison result and the verified common string; and / or

[0308] a difference update unit configured to update the similarity set if there is a difference between the similarity comparison result and the verified similarity set.

[0309] In some embodiments of the present disclosure, the common part module includes: an operation determination sub-module configured to determine the operation substrings between at least two operation strings pairwise;

[0310] an operation aggregation sub-module configured to perform an aggregation process on the operation substrings to obtain an operation aggregation string;

[0311] an operation duplicate removal sub-module configured to perform a duplicate removal process on the operation aggregation string to obtain a common string.

[0312] In some embodiments of the present disclosure, the difference calculation module includes: an operation similarity sub-module configured to calculate the similarity between at least two operation strings pairwise using a similarity algorithm to obtain an operation similarity;

[0313] an operation set sub-module configured to perform an aggregation process on the operation similarity to obtain a similarity set.

[0314] In some embodiments of the present disclosure, the common part module includes: a network determination sub-module configured to determine the network substrings between at least two network strings pairwise;

[0315] a network aggregation sub-module configured to perform an aggregation process on the network substrings to obtain a network aggregation string;

[0316] a network duplicate removal sub-module configured to perform a duplicate removal process on the network aggregation string to obtain a common string.

[0317] In some embodiments of the present disclosure, the difference calculation module includes: a network similarity sub-module configured to calculate the similarity between at least two network strings pairwise using a similarity algorithm to obtain a network similarity;

[0318] a network set sub-module configured to perform an aggregation process on the network similarity to obtain a similarity set.

[0319] The specific details of the test case verification device provided in each embodiment of the present disclosure have been described in detail in the corresponding method embodiments, and thus will not be elaborated here.

[0320] Figure 25The figure shows a schematic structural diagram of a computer system of an electronic device suitable for implementing embodiments of the present disclosure.

[0321] It should be noted that Figure 25 The computer system 2500 of the shown electronic device is only an example, and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0322] As Figure 25 shown, the computer system 2500 includes a central processing unit (CPU) 2501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 2502 or the program loaded from the storage section 2508 into the random access memory (RAM) 2503. In the RAM 2503, various programs and data required for system operation are also stored. The CPU 2501, ROM 2502, and RAM 2503 are connected to each other via a bus 2504. The input / output (I / O) interface 2505 is also connected to the bus 2504.

[0323] The following components are connected to the I / O interface 2505: an input section 2506 including a keyboard, a mouse, etc.; an output section 2507 including such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 2508 including a hard disk, etc.; and a communication section 2509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 2509 performs communication processing via a network such as the Internet. A drive 2510 is also connected to the I / O interface 2505 as required. A removable medium 2511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 2510 as required, so that the computer program read from it can be installed into the storage section 2508 as required.

[0324] In particular, according to embodiments of the present disclosure, the processes described in each method flow chart may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes program code for performing the methods shown in the flow charts. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 2509, and / or installed from the removable medium 2511. When the computer program is executed by the central processing unit (CPU) 2501, various functions defined in the system of the present application are performed.

[0325] It should be noted that the computer-readable medium shown in the embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted by any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0326] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code, and the above-mentioned module, segment of a program, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0327] It should be noted that although several modules or units of devices for action execution are mentioned in the above detailed description, such a division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0328] From the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or can be implemented by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0329] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed in the present disclosure.

[0330] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A verification method for test cases, characterized in that, The method includes: Obtaining at least two use case strings corresponding to a test case, and determining a common string of the at least two use case strings; Calculating the similarity between every two of the at least two use case strings to obtain a similarity set; Obtaining a string to be verified obtained by executing the test case, and verifying the string to be verified by using the common string and the similarity set to obtain a verification result of the test case; Wherein, the verifying the string to be verified by using the common string and the similarity set to obtain a verification result of the test case includes: Obtaining the number of strings of the common string, and obtaining the number of strings to be verified of the string to be verified; If the string to be verified includes the common string and the number of strings is less than the number of strings to be verified, determining a similarity set to be verified corresponding to the string to be verified; Verifying the similarity set to be verified by using the similarity set to obtain a verification result of the test case.

2. The verification method of the test case according to claim 1, wherein The verifying the similarity set to be verified by using the similarity set to obtain a verification result of the test case includes: If the similarity set includes the similarity set to be verified, determining that the verification result of the test case is verification successful; If the similarity set does not include the similarity set to be verified, determining that the verification result of the test case is verification failed.

3. The verification method of the test case according to claim 1, characterized in that, The verifying the similarity set to be verified by using the similarity set to obtain a verification result of the test case includes: Obtaining a fluctuation parameter corresponding to the similarity set; Verifying the similarity set to be verified by using the similarity set and the fluctuation parameter to obtain a verification result.

4. The verification method of the test case according to claim 1, wherein After obtaining the string to be verified obtained by executing the test case, the method further includes: Obtaining the number of strings of the common string, and obtaining the number of strings to be verified of the string to be verified; If the string to be verified includes the common string and the number of strings to be verified is equal to the number of strings, determining that the verification result of the test case is verification successful.

5. The verification method of the test case according to claim 1, characterized in that, After calculating the similarity between every two of the at least two use case strings to obtain a similarity set, the method further includes: Obtaining a verification string corresponding to the at least two use case strings; Updating the common string and the similarity set by using the verification string.

6. The verification method of the test case according to claim 5, characterized in that, The updating the common string and the similarity set by using the verification string includes: Determining a verification common string and a verification similarity set of the at least two use case strings and the verification string; Comparing the common string with the verification common string to obtain a common comparison result, and comparing the similarity set with the verification similarity set to obtain a similarity comparison result; Updating the common string and the similarity set according to the common comparison result and the similarity comparison result.

7. The verification method of the test case according to claim 6, characterized in that, The updating the common string and the similarity set according to the common comparison result and the similarity comparison result includes: If the public comparison result indicates a difference between the public string and the verified public string, update the public string; and / or If the similarity comparison result indicates a difference between the similarity set and the verified similarity set, update the similarity set.

8. The verification method of the test case according to claim 1, wherein, The at least two use case strings include at least two operation strings; Determining the public string of the at least two use case strings includes: Determining operation substrings between every two of the at least two operation strings; Performing an aggregation process on the operation substrings to obtain an operation aggregation string; Performing a deduplication process on the operation aggregation string to obtain a public string.

9. The verification method for test cases according to claim 8, characterized in that Calculating the similarity between every two of the at least two use case strings to obtain a similarity set includes: Calculating the similarity between every two of the at least two operation strings using a similarity algorithm to obtain an operation similarity; Performing an aggregation process on the operation similarity to obtain a similarity set.

10. The verification method of the test case according to claim 1, characterized in that, The at least two use case strings include at least two network strings; Determining the public string of the at least two use case strings includes: Determining network substrings between every two of the at least two network strings; Performing an aggregation process on the network substrings to obtain a network aggregation string; Performing a deduplication process on the network aggregation string to obtain a public string.

11. The verification method for test cases according to claim 10, characterized in that, Calculating the similarity between every two of the at least two use case strings to obtain a similarity set includes: Calculating the similarity between every two of the at least two network strings using a similarity algorithm to obtain a network similarity; Performing an aggregation process on the network similarity to obtain a similarity set.

12. A verification device for test cases, characterized in that The apparatus includes: A public part module configured to obtain at least two use case strings corresponding to a test case and determine the public string of the at least two use case strings; A difference calculation module configured to calculate the similarity between every two of the at least two use case strings to obtain a similarity set; A verification execution module configured to obtain a string to be verified obtained by executing the test case, and use the public string and the similarity set to verify the string to be verified to obtain a verification result of the test case; Wherein, the verification execution module is configured to: obtain the number of characters of the public string, and obtain the number of characters to be verified of the string to be verified; if the string to be verified includes the public string and the number of characters is less than the number of characters to be verified, determine a similarity set to be verified corresponding to the string to be verified; use the similarity set to verify the similarity set to be verified to obtain a verification result of the test case.

13. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the test case verification method according to any one of claims 1 to 11.

14. An electronic device, characterized in that, Including: A processor; And A memory for storing executable instructions of the processor; Wherein, the processor is configured to execute the test case verification method according to any one of claims 1 to 11 by executing the executable instructions.

15. A computer program product, characterized in that, The computer program product includes a computer program which is stored in a computer-readable storage medium. A processor of a computer device reads and executes the computer program, so that the computer device executes the verification method of the test case according to any one of claims 1 to 11.

Images