System testing method, system testing device, equipment, storage medium and product

Abstract

The invention provides a system testing method, a system testing device, equipment, a storage medium and a product, which can be applied to the fields of big data and financial science and technology. The method comprises the following steps: acquiring a character information set for a tested system; the character information set is processed through an active learning algorithm so as to determine whether the operation behavior of the user can be responded by the tested system or not, an observation table containing multiple sets of observation data is generated, the observation data are used for recording response results, and the response results comprise responsibility and non-responsibility; constructing an intermediate state diagram based on the target observation data whose response result is respondable in the observation table; carrying out crossover and mutation on the character information set by utilizing a genetic algorithm to obtain multiple groups of target character sets; determining a counter-example character set from the multiple groups of target character sets by utilizing an active learning algorithm; the intermediate state diagram is corrected based on the counter-example character set, a target state diagram is generated, and the target state diagram is used for representing the responsible function of the tested system.

Description

Technical Field

[0001] The present disclosure relates to the fields of big data and financial technology, and specifically to a system testing method, a system testing device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] As the complexity of software systems continues to increase, traditional manual testing methods are unable to cope with rapidly changing requirements. As an important means to improve testing efficiency, automated testing has become an essential tool in the software development process. Especially in state-driven systems, how to accurately identify system behavior through test cases and ensure its correctness becomes a challenge. In automated testing, active learning algorithms can gradually build an accurate model by querying the test system. However, the use of active learning algorithms will cause the problem of low query efficiency. In order to improve the efficiency of the algorithm, solving this problem has become the key. Summary of the invention

[0003] In view of the above problems, the present disclosure provides a system testing method, a system testing device, an electronic device, a computer-readable storage medium, and a computer program product.

[0004] According to the first aspect of the present disclosure, a system testing method is provided, including: operating behavior of a system under test; processing a character information set using an active learning algorithm to determine whether the user's operating behavior can be responded to by the system under test, and generating an observation table containing multiple groups of observation data, wherein the observation data is used to record response results, and the response results include responsive and unresponsive; constructing an intermediate state diagram based on target observation data whose response results are responsive in the observation table; performing crossover and mutation on the character information set using a genetic algorithm to obtain multiple groups of target character sets; determining a counterexample character set from the multiple groups of target character sets using an active learning algorithm; and correcting the intermediate state diagram based on the counterexample character set to generate a target state diagram, which is used to characterize the functions to which the system under test can respond.

[0005] A second aspect of the present disclosure provides a system testing device, comprising:

[0006] An acquisition module, used to acquire a character information set for the system under test, wherein the character information set is used to represent the user's operation behavior on the system under test;

[0007] An observation table generation module is used to process the character information set using an active learning algorithm to determine whether the user's operation behavior can be responded to by the tested system, and to generate an observation table containing multiple groups of observation data, wherein the observation data is used to record the response results, and the response results include responsive and unresponsive;

[0008] An intermediate state diagram construction module is used to construct an intermediate state diagram for responding target observation data based on the response results in the observation table;

[0009] The target character set obtaining module is used to perform crossover and mutation on the character information set by using a genetic algorithm to obtain multiple groups of target character sets;

[0010] A counter-example character set determination module, used for determining a counter-example character set from multiple target character sets using an active learning algorithm;

[0011] The target state diagram generation module is used to modify the intermediate state diagram based on the counterexample character set to generate a target state diagram, and the target state diagram is used to characterize the functions that the system under test can respond to.

[0012] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0013] The fourth aspect of the present disclosure further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the above computer program or instructions are executed by a processor.

[0014] The fifth aspect of the present disclosure further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when the above computer program or instructions are executed by a processor.

[0015] According to an embodiment of the present disclosure, an active learning algorithm is used to process a character information set to determine whether the user's operating behavior can be responded to by the tested system, and an observation table containing multiple groups of observation data is generated. An intermediate state diagram can be constructed based on the response results in the observation table as the responsive target observation data, and then a genetic algorithm is used to cross and mutate the character information set to obtain multiple groups of target character sets. An active learning algorithm can be used to determine a counter-example character set from the multiple groups of target character sets, and the intermediate state diagram is corrected based on the counter-example character set. A target state diagram for characterizing the responsive function of the tested system can be generated. A genetic algorithm is used to cross and mutate the character information set, and the multiple groups of target character sets obtained are used to accelerate the discovery of counter-example character sets, thereby reducing the time cost of discovering counter-example character sets, solving the problem of inefficiency caused by the active learning algorithm processing the character information set, and improving the test efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above contents and other purposes, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0017] Figure 1 The application scenario diagram of the system testing method according to the embodiment of the present disclosure is schematically shown;

[0018] Figure 2 A flowchart of a system testing method according to an embodiment of the present disclosure is schematically shown;

[0019] Figure 3 Schematically shows a structural diagram of an observation table according to an embodiment of the present disclosure;

[0020] Figure 4 A schematic diagram schematically illustrates an intermediate state diagram according to an embodiment of the present disclosure;

[0021] Figure 5 A flowchart of a method for generating a target character set according to an embodiment of the present disclosure is schematically shown;

[0022] Figure 6 A structural block diagram of a system testing device according to an embodiment of the present disclosure is schematically shown; and

[0023] Figure 7 A block diagram of an electronic device suitable for implementing a system testing method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0024] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0025] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0026] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0027] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0028] As the complexity of software systems continues to increase, traditional manual testing methods are unable to cope with rapidly changing requirements. As an important means to improve testing efficiency, automated testing has become an essential tool in the software development process. Especially in state-driven systems, how to accurately identify system behavior through test cases and ensure its correctness becomes a challenge.

[0029] In automated testing, active learning algorithms can be used to gradually build an accurate model by querying the test system. However, the use of active learning algorithms will cause the problem of low query efficiency. In order to improve the efficiency of the algorithm, solving this problem has become the key.

[0030] In view of this, the present disclosure provides a system testing method, system testing device, equipment, storage medium and product. The method includes: operating behavior of the system under test; processing the character information set using an active learning algorithm to determine whether the user's operating behavior can be responded to by the system under test, and generating an observation table containing multiple groups of observation data, wherein the observation data is used to record the response results, and the response results include responsive and unresponsive; constructing an intermediate state diagram based on the target observation data whose response results are responsive in the observation table; crossover and mutation of the character information set using a genetic algorithm to obtain multiple groups of target character sets; determining a counter-example character set from the multiple groups of target character sets using an active learning algorithm; modifying the intermediate state diagram based on the counter-example character set to generate a target state diagram, and the target state diagram is used to characterize the functions that the system under test can respond to.

[0031] It should be noted that the system testing method and system testing device provided in the present disclosure can be used in the field of financial technology, such as banks and other financial institutions, and can also be used in any field other than the field of financial technology, such as the field of big data, etc. Therefore, the application field of the system testing method and system testing device provided in the present disclosure is not limited.

[0032] In the technical solution of the present disclosure, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0033] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present disclosure provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating a person's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0034] Figure 1 The application scenario diagram of the system testing method according to an embodiment of the present disclosure is schematically shown.

[0035] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a recommendation of a bank financial product. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0036] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0037] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0038] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0039] It should be noted that the system testing method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the system testing device provided in the embodiment of the present disclosure can generally be set in the server 105. The system testing method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the system testing device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0040] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0041] Figure 2 The flowchart of the system testing method according to the embodiment of the present disclosure is schematically shown.

[0042] like Figure 2 As shown, the system testing method of this embodiment includes operations S210 to S260, and the system testing method can be executed by an electronic device.

[0043] In operation S210, a character information set for the system under test is obtained.

[0044] In operation S220, the character information set is processed using an active learning algorithm to determine whether the user's operation behavior can be responded to by the tested system, and an observation table containing multiple groups of observation data is generated.

[0045] In operation S230, an intermediate state diagram is constructed for the target observation data that can be responded to based on the response results in the observation table.

[0046] In operation S240, a genetic algorithm is used to perform crossover and mutation on the character information set to obtain multiple groups of target character sets.

[0047] In operation S250, an active learning algorithm is used to determine a counter-example character set from the plurality of target character sets.

[0048] In operation S260, the intermediate state diagram is modified based on the counterexample character set to generate a target state diagram.

[0049] According to an embodiment of the present disclosure, a character information set may be used to characterize a user's operation behavior on the system under test. The character information set may refer to a limited set of all input characters that the system under test can accept, usually represented by the symbol Representation. In a software system, the character information set can be an abstract representation of all possible commands, operations, button clicks, etc. provided by the user. By abstractly modeling the behavior of the system under test, the size of the character information set is limited to an appropriate range. If the behavior space of the system under test is large, the size of the character information set can be reduced through static analysis or domain knowledge to avoid too many unnecessary characters from participating in the learning process. For example, in a user login system, the character information set can be: , where a can represent the input of a user name, b can represent the input of a password, and c can represent clicking a login button, etc.

[0050] According to an embodiment of the present disclosure, an observation table (Occupational Therapy, OT) may refer to a key data structure in an active learning algorithm, and observation data may be used to record response results, which may include responsiveness and unresponsiveness. An active learning algorithm may be used to process a character information set to determine whether a user's operation behavior can be responded to by the system under test, and to generate an observation table containing multiple sets of observation data. An active learning algorithm may be used to construct a deterministic finite state automaton.

[0051] According to an embodiment of the present disclosure, a target state diagram (which may also be represented as a target automaton) may be gradually constructed through a series of membership queries and equivalence queries. A membership query may be a query directed to the system under test, the purpose of which is to determine whether a certain input character sequence is acceptable to the system under test. An equivalence query may be an operation used to verify whether the current intermediate state diagram is completely consistent with the system under test.

[0052] According to the embodiments of the present disclosure, an intermediate state diagram (which can also be expressed as a hypothetical automaton) can be constructed based on the target observation data that can be responded to based on the response results in the observation table. A query can be performed in the system under test based on the character information set, and the intermediate state diagram can be gradually improved based on the query execution results. Each query of a new character information set will add a new state (i.e., node) or state transition (i.e., the relationship between nodes). The state transition function can be expressed as ,state and the next state ,have , q is the initial character, is the character after q makes a state transition, and Q is the state set. The accepting state is a set of states in the automaton, indicating that the input character sequence is successfully processed, usually In the automaton, the accepting state indicates that the input sequence can successfully pass the system verification, that is, the response result can be responded to by the system under test (for example, the user logs in successfully).

[0053] According to an embodiment of the present disclosure, an intermediate state diagram can be constructed for the target observation data that can be responded to based on the response results in the observation table. Specifically, it can include checking whether the OT table meets the consistency and closure conditions. If the OT table meets the consistency and closure conditions, the intermediate state diagram can be constructed, and if not, it is expanded to ensure query efficiency.

[0054] According to an embodiment of the present disclosure, consistency may refer to that if the prefix set in the observation table Two states in and Have equal rows, i.e. , then add any input symbol to these two states After that, the new state generated and It should be consistent in the OT table, as shown in formula (1).

[0055] (1)

[0056] According to an embodiment of the present disclosure, closure may mean that for each state (ie status Input symbol set The state obtained by the transfer), there is a , so that That is, for each state With an input symbol The new state generated ,Right now , you should be able to find a row that is identical to it in the existing state set In this case, the responses between states are ensured to be consistent, as shown in formula (2).

[0057] (2)

[0058] According to the embodiments of the present disclosure, the hypothetical automaton is an intermediate model in the automaton learning process, which is gradually constructed and optimized through continuous member queries and equivalence queries, and finally converges to the correct automaton model (i.e., the target state diagram). The hypothetical automaton does not directly represent the real behavior of the system under test, but is an approximate model that is continuously adjusted according to the query results. When the OT table meets consistency and closure, the hypothetical automaton can be constructed using an active learning algorithm.

[0059] According to an embodiment of the present disclosure, a genetic algorithm can be used to optimize the efficiency of equivalent query. The equivalent query process can be expressed as is the current intermediate state diagram (i.e., the hypothetical automaton), is the system under test, is a string ( ), Represents a hypothetical automaton Accepted language (i.e. The set of all strings accepted by Represents the system under test accepted language. The goal of an equivalence query is to check Whether are exactly equal. If , then the equivalent query will return a counterexample w. Genetic algorithms can be used to perform crossover and mutation on character information sets to obtain multiple sets of target character sets.

[0060] According to the embodiments of the present disclosure, an active learning algorithm can be used to determine a counter-example character set from multiple target character sets. When a counter-example character set exists, the hypothesis automaton is adjusted immediately, otherwise, iteration continues. Finally, the intermediate state diagram can be modified based on the counter-example character set to generate a target state diagram, which can be used to characterize the functions to which the system under test can respond.

[0061] According to an embodiment of the present disclosure, an active learning algorithm is used to process a character information set to determine whether the user's operating behavior can be responded to by the tested system, and an observation table containing multiple groups of observation data is generated. An intermediate state diagram can be constructed based on the response results in the observation table as the responsive target observation data, and then a genetic algorithm is used to cross and mutate the character information set to obtain multiple groups of target character sets. An active learning algorithm can be used to determine a counter-example character set from the multiple groups of target character sets, and the intermediate state diagram is corrected based on the counter-example character set. A target state diagram for characterizing the responsive function of the tested system can be generated. A genetic algorithm is used to cross and mutate the character information set, and the multiple groups of target character sets obtained are used to accelerate the discovery of counter-example character sets, thereby reducing the time cost of discovering counter-example character sets, solving the problem of inefficiency caused by the active learning algorithm processing the character information set, and improving the test efficiency.

[0062] According to an embodiment of the present disclosure, the observation table may include a prefix set of a character information set, a suffix set of a character information set, and observation data.

[0063] Figure 3 The structure of the observation table according to the embodiment of the present disclosure is schematically shown.

[0064] like Figure 3 As shown, the OT table can use triples Represents, where: the state set (i.e., the prefix set of the character information set), representing a non-empty finite prefix closed set It contains all possible prefix sets, which represent the behavior of the system after receiving a series of input symbols. The prefix here represents the state of the system after a number of symbols from the initial state. Suffix set , represents a non-empty finite suffix closed set It contains all possible input suffixes, which are sequences of input symbols that further affect state transitions. is a definition , as shown in formula (3).

[0065] = (3)

[0066] The accepting state can be represented as the state s responding to the input suffix as being responsive, and the rejecting state can be represented as the state s responding to the input suffix as being non-responsive.

[0067] According to an embodiment of the present disclosure, the rows and columns of the OT table are respectively and Marking Character Information Set Candidate states for state generation, Represents the set of all possible suffixes. Each cell in the table Indicates that the state is for the suffix sequence When the OT table is initialized, Whenever a new symbol is queried, the active learning algorithm can update the state and fill in the transition relations based on the current content in the OT table.

[0068] According to an embodiment of the present disclosure, the intermediate state diagram may include multiple groups of state transition result legends, and the state transition result legends may display the response results in a graphical manner.

[0069] Figure 4 A schematic diagram of an intermediate state diagram according to an embodiment of the present disclosure is schematically shown.

[0070] like Figure 4 As shown, the intermediate state diagram may include multiple groups of state transition result legends, for example, the state 1 is converted to the state 2, the state 2 is converted to the state 3, etc. In order to determine whether the character information set can be implemented in the intermediate state diagram, for example, the character information set is aab, first the intermediate state diagram must be executed from the starting position. If aab is input, the traversal starts from the string head, the a symbol is converted to 2 at position 1, a (the second a) is converted to 3 at position 2, b is converted to 2 at position 3, and then the string traversal ends. At this point, it can be found that the final state of the intermediate state diagram stops at state 2, and the intermediate state diagram also stops, so aab can be recognized by the intermediate state diagram.

[0071] Figure 5 The flowchart of the target character set generation method according to the embodiment of the present disclosure is schematically shown.

[0072] like Figure 5 As shown, the target character set generation method of this embodiment includes operations S510 to S550.

[0073] In operation S510, the character information set is expanded to obtain multiple groups of expanded character sets.

[0074] In operation S520, a discrimination value of the extended character set is calculated.

[0075] In operation S530, when the discrimination value satisfies a predetermined numerical condition, the adaptability of the extended character set is determined according to the character length and similarity measure of the extended character set.

[0076] In operation S540, when the fitness is greater than a preset fitness threshold, the extended character set is determined as the preferred character set.

[0077] In operation S550, crossover and mutation are performed on the multiple groups of preferred character sets to obtain multiple groups of target character sets.

[0078] According to an embodiment of the present disclosure, multiple groups of random extended character sets can be generated based on the character information set, and the extended character set can be used to characterize the user's extended operation behavior on the system under test. , the extended character set can be expressed as .

[0079] According to an embodiment of the present disclosure, the discrimination value of the extended character set can be calculated. According to the extended character set, assuming that the automaton and the system under test The corresponding behaviors are recorded as , , where q can represent a string in the extended character set. If the string in the extended character set can distinguish the hypothetical automaton from the system under test, then the discrimination value corresponding to the string is 1. If the string in the extended character set cannot distinguish the hypothetical automaton from the system under test, then the discrimination value corresponding to the string is 0.

[0080] According to an embodiment of the present disclosure, when the discrimination value is 0, the fitness is zero. When the discrimination value is 1, the fitness of the extended character set can be determined according to the character length and similarity measurement of the extended character set, as shown in formula (4).

[0081] (4)

[0082] Among them, Fitness (q) is the fitness of the extended character set, is the string length, α is a constant, is a similarity measure.

[0083] According to an embodiment of the present disclosure, the range of string length is usually determined by the length of the longest string. Assuming the maximum length is , then the range of length is ; Similarity measure To measure the difference between string q and other strings P, methods such as Hamming distance or edit distance can be used. The range of similarity is , where 0 means completely similar and 1 means completely different; the constant α is a parameter for adjusting the impact of diversity, and a small constant value (such as α=0.1) is usually selected.

[0084] According to an embodiment of the present disclosure, when the fitness is greater than a preset fitness threshold, the extended character set can be determined as the preferred character set. A portion of high-quality individuals can also be selected for crossover and mutation using a roulette wheel based on the obtained fitness value. Multiple groups of preferred character sets can be crossovered and mutated to obtain multiple groups of target character sets.

[0085] According to the embodiments of the present disclosure, a genetic algorithm is used to perform crossover and mutation on a character information set to obtain multiple groups of target character sets, which can improve the test efficiency and reduce the time cost of finding counterexample character sets.

[0086] According to an embodiment of the present disclosure, multiple groups of preferred character sets are crossed and mutated to obtain multiple groups of target character sets, including: combining multiple groups of preferred character sets to obtain a crossed preferred character set; changing N characters in the preferred character set to obtain a mutated preferred character set, wherein N is an integer greater than 0; combining the crossed preferred character set and the mutated preferred character set to obtain multiple groups of target character sets.

[0087] According to the embodiments of the present disclosure, multiple groups of preferred character sets can be combined to obtain a cross preferred character set. For example, a cross preferred character set can be generated by combining two preferred character sets, for example, if the preferred character sets are abc and bcd, the cross preferred character set can include abcbcd, abcd, abcbd, etc., but is not limited thereto. The embodiments of the present disclosure do not limit the generation of the cross preferred character set.

[0088] According to the embodiments of the present disclosure, N characters in the preferred character set can be changed to obtain a variant preferred character set. For example, if the preferred character set is abc, the variant preferred character set can be ac, abcd, etc., but is not limited thereto. The embodiments of the present disclosure do not limit the generation of the variant preferred character set.

[0089] According to an embodiment of the present disclosure, the cross-preferred character set and the variant preferred character set may be combined to obtain multiple groups of target character sets.

[0090] According to an embodiment of the present disclosure, an active learning algorithm is used to determine a counter-example character set from multiple groups of target character sets, including: using an active learning algorithm to process the multiple groups of target character sets to determine whether the user's extended operation behavior can respond to the extended processing results of the tested system; determining a counter-example character set from the multiple groups of target character sets based on the extended processing results.

[0091] According to the embodiments of the present disclosure, multiple target character sets can be processed using an active learning algorithm to determine whether the user's extended operation behavior can be responded to by the tested system. According to the extended processing result, a counterexample character set can be determined from the multiple target character sets.

[0092] According to the embodiments of the present disclosure, after several generations of iterations in the genetic algorithm, if no counterexamples are found from multiple target character sets using the active learning algorithm, it can be decided whether to stop according to the set stop conditions. The stop conditions may include reaching the maximum number of generations, fitness reaching a certain threshold, set error accuracy or a predetermined learning goal, etc., to ensure that the algorithm completes the test task within a reasonable time, while avoiding unnecessary iterations and calculations, and improving test efficiency.

[0093] According to an embodiment of the present disclosure, an observation table includes a prefix set of a character information set, a suffix set of a character information set and observation data; wherein the system testing method further includes: based on the target observation data whose response result in the observation table is unresponsive, performing information expansion on the prefix set of the character information set to obtain a new prefix expansion set; using an active learning algorithm to process the first part of the new prefix expansion set in the new prefix expansion set; determining a prefix string related to the first part of the new prefix expansion set; determining a second part of the new prefix expansion set that does not contain the prefix string in the new prefix expansion set; using an active learning algorithm to process the second part of the new prefix expansion set until the response results of the new prefix expansion set are all responded to.

[0094] According to an embodiment of the present disclosure, based on the target observation data whose response result in the observation table is unresponsive, the prefix set of the character information set is expanded to obtain a new prefix expansion set. Specifically, it may include: if the OT table does not meet the closed property, that is, , the algorithm will convert the new state Add to state collection In the example above, we expand the OT table and expand the domain of the T function to , then perform a member query to fill in the new status information to ensure that the OT table meets the closed condition.

[0095] According to an embodiment of the present disclosure, an active learning algorithm can be used to process the first part of the new prefix extension set in the new prefix extension set to determine the prefix string related to the first part of the new prefix extension set. For example, the first part of the new prefix extension set includes the string , string Accepted (i.e. ,in is a language accepted by the automaton), then for any prefix satisfy When .in, Represents the prefix string associated with the first part of the new prefix expansion set, that is, for a string (in ), whose prefix set is:

[0096]

[0097] According to an embodiment of the present disclosure, a second part of the new prefix extension set that does not include the prefix string is determined, and the second part of the new prefix extension set is processed using an active learning algorithm until all response results of the new prefix extension set are responded to.

[0098] According to an embodiment of the present disclosure, a data structure such as a hash table or a prefix tree can be used to store prefix strings related to the first part of the new prefix extension set. The corresponding operation behavior is responded by the system under test, then all The operation behavior corresponding to the prefix string should also be responded to by the system under test. There is no need to perform member query on the prefix string, avoiding redundant queries and optimizing algorithm efficiency.

[0099] According to an embodiment of the present disclosure, the system testing method also includes: based on the target observation data whose response results are unresponsive in the observation table, expanding the information of the suffix set of the character information set to obtain a new suffix expansion set; using an active learning algorithm to process the new suffix expansion set until all response results of the new suffix expansion set are responded to.

[0100] According to an embodiment of the present disclosure, based on the target observation data whose response result in the observation table is unresponsive, the suffix set of the character information set can be expanded to obtain a new suffix expansion set. Specifically, if the OT table does not meet the consistency, it means that the combination of the current state and the suffix cannot distinguish two equal states, so it needs to be added to the query set to clarify the difference between them. Specifically, if there is satisfy , but for a certain ,have , you need to expand the suffix set for The new suffix expansion set is processed using an active learning algorithm until all the response results of the new suffix expansion set are responded to, thereby ensuring the consistency of the observation table.

[0101] According to an embodiment of the present disclosure, the response result associated with the counter-example character set may include a first response result and a second response result, the first response result may be that the response result of the counter-example character set in the system under test is unresponsive and the response result in the intermediate state diagram is responsive, and the second response result may be that the response result of the counter-example character set in the system under test is responsive and the response result in the intermediate state diagram is unresponsive. The intermediate state diagram is corrected based on the counter-example character set to generate a target state diagram, including: when the response result associated with the counter-example character set is the first response result, the state transition result legend in the intermediate state diagram is corrected using the counter-example character set to obtain the target state diagram; when the response result associated with the counter-example character set is the second response result, the observation table is expanded according to the counter-example character set to obtain the target observation table; based on the target observation table, the target state diagram is constructed.

[0102] According to an embodiment of the present disclosure, when the response result associated with the counterexample character set is the first response result, the state transition result legend in the intermediate state diagram can be corrected using the counterexample character set to obtain the target state diagram. Specifically, when the response result associated with the counterexample character set is the first response result, it can be indicated that some states or state transitions in the hypothetical automaton are redundant or erroneous, and the state transition function (i.e., the state transition result legend in the intermediate state diagram) needs to be corrected. Adjusting the state transition function can mean that if the counterexample indicates that the transition function of a certain state is wrong (for example, the state transition caused by a certain input symbol is incorrect), the state transition function needs to be modified. , correcting the erroneous transfer to the correct transfer.

[0103] According to an embodiment of the present disclosure, when the response result associated with the counter-example character set is the second response result, the observation table can be expanded according to the counter-example character set to obtain the target observation table, and the target state diagram can be constructed based on the target observation table. Specifically, when the response result associated with the counter-example character set is the second response result, it can be indicated that some states or state transitions of the hypothetical automaton are incorrect, and the state space needs to be expanded; expanding the state space can mean that if the counter-example finds that the current state behaves inconsistently when processing a certain string, the state space needs to be expanded. ,in is a newly added state. By adding new states, the automaton can more accurately simulate the behavior of the system under test. Whenever the state space is expanded or the state transition is adjusted, the data in the OT table also needs to be updated. The updated states and state transitions must be verified through member queries and counterexamples to ensure the correctness of the OT table and meet the consistency and closure conditions of the observation table.

[0104] According to the embodiments of the present disclosure, the active learning algorithm makes the construction process of the target state diagram fully automated without manual intervention. Through a series of queries, an accurate target state diagram is gradually constructed to ensure that the test cases are consistent with the behavior of the system under test, and the behavior space of the system under test can be fully covered, eliminating blind spots and omissions in manual testing. The construction process of the target state diagram can greatly reduce the workload of manual design and debugging, while improving the accuracy and completeness of the model.

[0105] Figure 6 The structural block diagram of a system testing device according to an embodiment of the present disclosure is schematically shown.

[0106] like Figure 6 As shown, the system testing device 600 of this embodiment includes an acquisition module 610, an observation table generation module 620, an intermediate state diagram construction module 630, a target character set acquisition module 640, a counterexample character set determination module 650 and a target state diagram generation module 660.

[0107] The acquisition module 610 is used to acquire a character information set for the system under test, wherein the character information set is used to represent the user's operation behavior on the system under test.

[0108] The observation table generation module 620 is used to process the character information set using an active learning algorithm to determine whether the user's operation behavior can be responded to by the tested system, and generate an observation table containing multiple groups of observation data, wherein the observation data is used to record the response results, and the response results include responsive and unresponsive.

[0109] The intermediate state diagram construction module 630 is used to construct an intermediate state diagram based on the target observation data that can be responded to based on the response results in the observation table.

[0110] The target character set obtaining module 640 is used to perform crossover and mutation on the character information set using a genetic algorithm to obtain multiple groups of target character sets.

[0111] The counter-example character set determination module 650 is used to determine a counter-example character set from multiple groups of target character sets using an active learning algorithm.

[0112] The target state diagram generation module 660 is used to modify the intermediate state diagram based on the counterexample character set to generate a target state diagram, and the target state diagram is used to characterize the functions that the system under test can respond to.

[0113] According to an embodiment of the present disclosure, an active learning algorithm is used to process a character information set to determine whether the user's operating behavior can be responded to by the tested system, and an observation table containing multiple groups of observation data is generated. An intermediate state diagram can be constructed based on the response results in the observation table as the responsive target observation data, and then a genetic algorithm is used to cross and mutate the character information set to obtain multiple groups of target character sets. An active learning algorithm can be used to determine a counter-example character set from the multiple groups of target character sets, and the intermediate state diagram is corrected based on the counter-example character set. A target state diagram for characterizing the responsive function of the tested system can be generated. A genetic algorithm is used to cross and mutate the character information set, and the multiple groups of target character sets obtained are used to accelerate the discovery of counter-example character sets, thereby reducing the time cost of discovering counter-example character sets, solving the problem of inefficiency caused by the active learning algorithm processing the character information set, and improving the test efficiency.

[0114] According to an embodiment of the present disclosure, the intermediate state diagram includes multiple groups of state transition result legends, and the state transition result legends display the response results in a graphical manner.

[0115] According to an embodiment of the present disclosure, the target character set obtaining module includes: an extended character set obtaining unit, a discrimination value calculating unit, a fitness determining unit, a preferred character set determining unit and a target character set obtaining unit.

[0116] The extended character set obtaining unit is used to expand the character information set to obtain multiple groups of extended character sets, and the extended character sets are used to represent the user's extended operation behavior on the system under test.

[0117] The discrimination value calculation unit is used to calculate the discrimination value of the extended character set.

[0118] The fitness determination unit is used to determine the fitness of the extended character set according to the character length and similarity measurement of the extended character set when the discrimination value meets the predetermined numerical condition, wherein the similarity measurement is used to measure the similarity between the extended character set and other extended character sets.

[0119] The preferred character set determining unit is used to determine the extended character set as the preferred character set when the fitness is greater than a preset fitness threshold.

[0120] The target character set obtaining unit is used to perform crossover and mutation on multiple groups of preferred character sets to obtain multiple groups of target character sets.

[0121] According to an embodiment of the present disclosure, the target character set obtaining unit includes: a cross-preferred character set obtaining sub-unit, a variant preferred character set obtaining sub-unit and a target character set obtaining sub-unit.

[0122] The cross-preferred character set obtains a subunit, which is used to combine multiple groups of preferred character sets to obtain a cross-preferred character set.

[0123] The variant preferred character set obtains a subunit, which is used to change N characters in the preferred character set to obtain a variant preferred character set, wherein N is an integer greater than 0.

[0124] The target character set obtaining subunit is used to combine the cross-preferred character set and the variation-preferred character set to obtain multiple groups of target character sets.

[0125] According to an embodiment of the present disclosure, the counter-example character set determination module includes: an expansion processing result determination unit and a counter-example character set determination unit.

[0126] The extended processing result determination unit is used to process multiple groups of target character sets using an active learning algorithm to determine whether the user's extended operation behavior can generate an extended processing result responded by the tested system.

[0127] The counter-example character set determination unit is used to determine the counter-example character set from multiple groups of target character sets according to the expansion processing result.

[0128] According to an embodiment of the present disclosure, the target observation data includes a prefix set of a character information set and a suffix set of the character information set.

[0129] According to an embodiment of the present disclosure, the system testing device 600 further includes: a new prefix extension set obtaining module, a first processing module, a prefix string determining module, a second part new prefix extension set determining module, and a first responding module.

[0130] The new prefix extension set obtaining module is used to expand the prefix set of the character information set based on the target observation data whose response result in the observation table is unresponsive, so as to obtain a new prefix extension set.

[0131] The first processing module is used to process a first part of the new prefix extension set in the new prefix extension set by using an active learning algorithm.

[0132] The prefix string determination module is used to determine the prefix string related to the first part of the new prefix extension set.

[0133] The second part new prefix extension set determination module is used to determine the second part new prefix extension set that does not contain the prefix character string in the new prefix extension set.

[0134] The first response module is used to process the second part of the new prefix extension set by using an active learning algorithm until all response results of the new prefix extension set are responded to.

[0135] According to an embodiment of the present disclosure, the system testing device 600 further includes: a new suffix extension set obtaining module and a second response module.

[0136] The module for obtaining a new suffix extension set is used to expand the suffix set of the character information set based on the target observation data whose response result in the observation table is unresponsive, so as to obtain a new suffix extension set.

[0137] The second response module is used to process the new suffix extension set by using an active learning algorithm until all response results of the new suffix extension set are responded to.

[0138] According to an embodiment of the present disclosure, the response result associated with the counterexample character set includes a first response result and a second response result, the first response result being that the response result of the counterexample character set in the system under test is unresponsive and the response result in the intermediate state diagram is responsive, and the second response result being that the response result of the counterexample character set in the system under test is responsive and the response result in the intermediate state diagram is unresponsive.

[0139] According to an embodiment of the present disclosure, a target state diagram generation module includes: a correction module, a target observation table acquisition module and a target state diagram construction module.

[0140] The correction module is used to correct the state transition result legend in the intermediate state diagram using the counter-example character set when the response result associated with the counter-example character set is the first response result, so as to obtain the target state diagram.

[0141] The target observation table obtaining module is used to expand the observation table according to the counter-example character set to obtain the target observation table when the response result associated with the counter-example character set is the second response result.

[0142] The target state diagram construction module is used to construct the target state diagram based on the target observation table.

[0143] According to an embodiment of the present disclosure, any multiple modules among the acquisition module 610, the observation table generation module 620, the intermediate state diagram construction module 630, the target character set acquisition module 640, the counter-example character set determination module 650 and the target state diagram generation module 660 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 610, the observation table generation module 620, the intermediate state diagram construction module 630, the target character set acquisition module 640, the counter-example character set determination module 650 and the target state diagram generation module 660 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 610, the observation table generation module 620, the intermediate state diagram construction module 630, the target character set acquisition module 640, the counter-example character set determination module 650 and the target state diagram generation module 660 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0144] Figure 7 A block diagram of an electronic device suitable for implementing a system testing method according to an embodiment of the present disclosure is schematically shown.

[0145] like Figure 7As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0146] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0147] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0148] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0149] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 702 and / or RAM 703 described above and / or one or more memories other than ROM 702 and RAM 703.

[0150] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the system testing method provided by the embodiment of the present disclosure.

[0151] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 701. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0152] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 709, and / or installed from the removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0153] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0154] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0155] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0156] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0157] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A system testing method, characterized in that: include: Acquire a character information set for the system under test, wherein the character information set is used to represent the user's operation behavior on the system under test; The character information set is processed by using an active learning algorithm to determine whether the user's operation behavior can be responded to by the tested system, and an observation table containing multiple groups of observation data is generated, wherein the observation data is used to record the response results, and the response results include responsive and unresponsive; Constructing an intermediate state diagram based on the target observation data that can be responded to based on the response results in the observation table; Using a genetic algorithm to perform crossover and mutation on the character information set to obtain multiple groups of target character sets; Determine a counterexample character set from the multiple groups of target character sets using the active learning algorithm; The intermediate state diagram is modified based on the counterexample character set to generate a target state diagram, wherein the target state diagram is used to characterize the functions to which the system under test can respond.

2. The method according to claim 1, characterized in that The intermediate state diagram includes a plurality of groups of state transition result legends, and the state transition result legends display the response results in a graphical manner; The method of using a genetic algorithm to perform crossover and mutation on the character information set to obtain multiple groups of target character sets includes: Expanding the character information set to obtain multiple groups of extended character sets, where the extended character sets are used to represent the user's extended operation behavior on the system under test; Calculating the discrimination value of the extended character set; In the case where the discrimination value satisfies a predetermined numerical condition, determining the fitness of the extended character set according to the character length and similarity metric of the extended character set, wherein the similarity metric is used to measure the similarity between the extended character set and other extended character sets; In a case where the fitness is greater than a preset fitness threshold, determining the extended character set as a preferred character set; Crossover and mutation are performed on the multiple groups of the preferred character sets to obtain multiple groups of target character sets.

3. The method according to claim 2, characterized in that The step of performing crossover and mutation on the plurality of groups of the preferred character sets to obtain a plurality of groups of target character sets includes: Combining multiple groups of the preferred character sets to obtain a cross-preferred character set; Changing N characters in the preferred character set to obtain a variant preferred character set, where N is an integer greater than 0; The cross-preferred character set and the variant preferred character set are combined to obtain the multiple groups of target character sets.

4. The method according to claim 2, characterized in that: The step of using the active learning algorithm to determine a counterexample character set from the multiple target character sets includes: Using an active learning algorithm to process the multiple target character sets to determine whether the user's extended operation behavior can be responded to by the tested system; A counter-example character set is determined from the multiple groups of target character sets according to the expansion processing result.

5. The method according to claim 1, characterized in that The observation table includes a prefix set of a character information set, a suffix set of a character information set, and observation data; Wherein, the method further comprises: Based on the target observation data in the observation table whose response result is unresponsive, performing information expansion on the prefix set of the character information set to obtain a new prefix expansion set; Using the active learning algorithm to process a first part of the new prefix extension set in the new prefix extension set; Determine a prefix string associated with the first part of the new prefix extension set; Determine a second part of the new prefix extension set that does not include the prefix string in the new prefix extension set; The active learning algorithm is used to process the second part of the new prefix extension set until all response results of the new prefix extension set are responded to.

6. The method according to claim 5, characterized in that The method further comprises: Based on the target observation data in the observation table whose response result is unresponsive, expanding the suffix set of the character information set to obtain a new suffix expansion set; The new suffix extension set is processed using the active learning algorithm until all response results of the new suffix extension set are responded to.

7. The method according to claim 1, characterized in that The response result associated with the counter-example character set includes a first response result and a second response result, the first response result being that the response result of the counter-example character set in the system under test is not responsive and the response result in the intermediate state diagram is responsive, and the second response result being that the response result of the counter-example character set in the system under test is responsive and the response result in the intermediate state diagram is not responsive; The step of modifying the intermediate state diagram based on the counterexample character set to generate a target state diagram includes: When the response result associated with the counter-example character set is the first response result, the state transition result legend in the intermediate state diagram is modified by using the counter-example character set to obtain a target state diagram; When the response result associated with the counter-example character set is the second response result, expanding the observation table according to the counter-example character set to obtain a target observation table; Based on the target observation table, a target state diagram is constructed.

8. A system testing device, characterized in that: include: An acquisition module, used to acquire a character information set for the system under test, wherein the character information set is used to represent the user's operation behavior on the system under test; An observation table generation module is used to process the character information set using an active learning algorithm to determine whether the user's operation behavior can be responded to by the tested system, and generate an observation table containing multiple groups of observation data, wherein the observation data is used to record the response results, and the response results include responsive and unresponsive; An intermediate state diagram construction module, used for constructing an intermediate state diagram based on the target observation data that can be responded to based on the response results in the observation table; A target character set obtaining module is used to perform crossover and mutation on the character information set using a genetic algorithm to obtain multiple groups of target character sets; a counter-example character set determination module, configured to determine a counter-example character set from the plurality of target character sets using the active learning algorithm; A target state diagram generating module is used to modify the intermediate state diagram based on the counterexample character set to generate a target state diagram, wherein the target state diagram is used to characterize the functions to which the system under test can respond.

9. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images