Methods, systems, and corresponding computer programs for determining a sequence of actions to be performed by an implementation of a computer program

Abstract

This disclosure relates to a demand automaton (A) comprising multiple states (s0, s1, ..., sn). R (10) determines the sequence of actions (SeqA) to be executed by the implementation (I) of the computer program to achieve a predetermined result (Obj), which is determined by the demand automaton (A). R The method, implemented by an electronic device, represents a set of states in a tree-like data structure (T1) and includes at least one iteration of the following steps: selecting (S01) action subsequences (SSeqA) within the tree-like data structure (T1); using a demand automaton (A... R (S02) Determine (S03) the current action (SSeqA) to be executed after the action subsequence (SSeqA); execute (S04) the action subsequence (SSeqA) and the current action (S05) through implementation (I) to obtain the current action result (S06); execute (S07) the complementary action subsequence (CSSeqA) according to the current action result (S06) and according to the action optimization data structure (AODS) through implementation (I) to obtain the reward value (r) associated with the current action (S06); add (S05) the current action (S06) and its reward value (r) to the tree data structure (T1) and determine whether the predetermined result (Obj) has been reached to obtain the action sequence (SeqA).

Description

Technical Field

[0001] This disclosure relates to the field of computer technology. More specifically, this disclosure relates to the field of determining the behavior of a computer program. More specifically, this disclosure relates to determining the behavior of a black-box computer program. Priority is claimed to European Patent Application No. EP23307081.2, filed November 29, 2023, the contents of which are incorporated herein by reference. Background Technology

[0002] In many cases, it is necessary to deal with black-box computer programs or systems with cyclic behavior, where in each loop the system reads an assigned input variable and outputs an assigned output variable. Such programs can be found, for example, in real-time computer systems. For such computer programs, the problem of concern is finding the input sequence that guides the system in a specific state (e.g., obtaining a specific output assignment).

[0003] Finding the input sequence that guides such a program or system under specific conditions has several applications. A first example is in online black-box program testing, where the goal is to find execution that meets coverage criteria by interacting with the program solely through input feeds / output reads. A second example is in artificial intelligence planning, where it is useful for computing plans that satisfy a given objective.

[0004] Because the program's functionality is a black box, the system's source code is inaccessible, and the interaction depends solely on the order of input and output variables. The difficulty lies in the fact that the user providing these inputs does not know in advance what outputs the system will provide; these can only be observed during the interaction. This formal system appears in several fields, such as control programs (e.g., programmable logic controllers for factory automation systems); and also in artificial intelligence planning problems, where people interact with the system cyclically by sending actions (inputs) and observing their states (outputs), with the aim of guiding the system to a specific state.

[0005] However, in these cases, the attributes to be performed can be specified using a set of requirements given in the form of at least one automaton (called a demand automaton). The transitions of this automaton are protected, for example, by Boolean combinations of input and output assignments, such that they recognize sequences of assignments to input (action) variables and output (result) variables. This automaton accepts sequences of input-output assignments, and reaches a final state in the automaton if a sequence satisfies the attribute being sought (coverage criterion in testing or goal in planning).

[0006] Demand automata can be implemented as a consistency test, where the goal is to check if the implementation under test (IUT) conforms to its specification, and more specifically, to check the correctness of online synthesized test cases that verify that consistency. "Online" indicates that a portion of the synthesis is performed during execution.

[0007] The problem to be solved is to use this demand automaton at hand to find the sequence of actions (inputs) that will drive the black-box implementation of the computer program to expose a given behavior (i.e., the expected goal).

[0008] To address this issue, for example in the context of testing, interaction with a black-box computer program can be viewed as a competition between the tester and the computer system that reacts to the tester's actions. In such a competition, the tester wants to drive the computer system to a specific state or exhibit certain specific behaviors suspected of being inconsistent. These behaviors are typically described by test objectives. They can be derived from coverage criteria (e.g., state or transition coverage) or written from requirements. The ways in which these test objectives are designed are beyond the scope of this work. In this case, the tester's control over the system is limited: it can choose what inputs to give the system (which are therefore controllable) rather than the outputs (which are uncontrollable). The system can then be considered the adversary, and if the tester manages to drive the computer system to the test objective, the tester wins the competition.

[0009] One solution to this problem could be to use a randomized test automaton to accept all possible inputs until a sequence of inputs leads to the objective (i.e., the final state) exposed in the demand automaton. While this solution might allow finding a suitable sequence of inputs (actions), it does not provide any guarantee about the time taken to arrive at the solution, nor does it guarantee that a suitable solution will be found efficiently.

[0010] Therefore, it is necessary to provide a solution for efficiently finding action (input) sequences from demand automata that allows a black-box implementation of a computer program to expose the expected behavior embodied by the final state of the demand automaton. Summary of the Invention

[0011] This disclosure aims to improve the situation. In particular, this disclosure aims to overcome at least some of the limitations of the prior art described above by proposing a solution that can efficiently and more quickly determine the sequence of actions used to achieve a predetermined behavior or goal.

[0012] This is achieved through a method for determining a sequence of actions from a demand automaton comprising multiple states, to be executed by an implementation of a computer program to achieve a predetermined result, which is determined by a set in the demand automaton. This method is implemented by an electronic device. According to this disclosure, the method includes at least one iteration of the following steps: - Select action subsequences within a tree-structured data structure - Use the demand automaton to determine the current action to be performed after the action subsequence. - By executing the action subsequence and the current action as described above, the result of the current action is obtained. - By implementing the above method, complementary action subsequences are executed based on the current action result and the action optimization data structure to obtain a reward value associated with the current action. - Add the current action and its reward value to the tree-like data structure, and determine whether the predetermined result has been achieved, in order to obtain the action sequence.

[0013] By implementing this method, the sequence of actions executed by the computer program to achieve a predetermined result is obtained in a more efficient and faster manner, with less resource consumption.

[0014] Based on the characteristics, when it is determined that the result of the current action provides a set from the current state in the demand automaton. Stop iterating when the transformation occurs.

[0015] Based on the characteristics, when it is determined that the execution of complementary action subsequences provides a sequence from the current state to the set in the demand automaton. Stop iterating when the transformation occurs.

[0016] According to the features, performing the complementary action subsequence includes: - Obtain the current state from the demand automaton, the current state being determined from the result of the current action. - Identify action sequences within the action optimization data structure that may allow the predetermined result to be achieved from the current state. - Select a predetermined number of actions from the actions in the action sequence to obtain the complementary action subsequence. - By sequentially executing each action of the complementary action subsequence, each result of each action allows for a transition from a state to an intermediate state in the demand automaton. - The reward value of the complementary action subsequence is determined based on the intermediate state reached in the demand automaton at the end of the execution of the complementary action subsequence.

[0017] Based on the characteristics, the reward value associated with the current state and the current action is the sum of the final state of the demand automaton and the set. A function of the distance between the two sides.

[0018] Based on features, based on the current state and set The distance is determined by the remaining intermediate states between them.

[0019] Based on the characteristics, the maximum number of actions in the complementary subsequences of actions is bounded.

[0020] Based on its characteristics, the action optimization data structure is constructed using a demand automaton, and the action optimization data structure is derived from a set Begin recursively identifying what needs to be done to transition between states of the demand automaton in order to reach a set. Some possible action sequences.

[0021] Based on features, select within a tree-like data structure, and execute action subsequences based on the action-optimized data structure.

[0022] Based on the features, the action subsequence is selected to reach a predetermined number of iterations according to the action optimization data structure.

[0023] Based on the characteristics, once the predetermined number of iterations is reached while the predetermined result remains unachieved, an action subsequence is selected based on the reward value stored in the tree-like data structure.

[0024] The bounds for the predetermined number of iterations are applied to each node of the tree data structure. This allows selection to be applied in several directions.

[0025] According to another aspect, the present invention also relates to a system for determining a sequence of actions to be executed by an implementation of a computer program to achieve a predetermined result from a set of demand automata comprising multiple states. The system includes at least one processor and at least one memory. The system is configured such that at least one processor performs the method described above.

[0026] This disclosure also relates to a computer program that includes instructions that, when executed by a processor, cause the methods defined above to be implemented. Attached Figure Description

[0027] This disclosure will be better understood after reading the following description, which is given by way of example and is by no means limiting, and is carried out with reference to the accompanying drawings, which illustrate: - Figure 1 This method uses an example of a demand automaton. - Figure 2 : A diagram illustrating the main steps of an exemplary implementation of a method for determining a sequence of actions to be performed by an implementation of a computer program. - Figure 3This illustrates the action optimization data structure. - Figure 4 Example of a device designed to generate action sequences according to the method of this disclosure is shown.

[0028] In these figures, the same reference numerals denote the same or similar elements from one figure to another. For clarity, unless otherwise explicitly stated, the elements shown are not drawn to scale. Furthermore, unless required by the language of the specification, the order of steps shown in these figures is for illustrative purposes only and is not intended to limit this disclosure, which can be applied to the same steps performed in a different order. Detailed Implementation

[0029] As disclosed, this disclosure relates to methods, apparatus, and systems for computing execution that satisfies the requirements of an automaton in a black-box computer program or system. A black-box program is essentially the only possible interaction involving sending inputs and observing outputs. For clarity, a requirement automaton is defined as a set of requirements associated with the implementation of a program to be tested or executed, formatted as one or more automata (which are deterministic finite automata) that define a sequence of actions to be performed and their links. Thus, according to this disclosure, computing execution that satisfies a requirement automaton means starting from the initial state of the automaton and reaching the final or error state of the automaton through a given sequence of actions input to the black-box computer program, each action resulting in an assignment output of the computer program, each assignment output driving a transition to the state of the automaton, and so on, until the final or error state is reached. Examples of such automata are described below.

[0030] The disclosed method can be used for online testing of computer programs and systems, but also for artificial intelligence planning, and allows for limiting the resource consumption used to achieve requirements. Generally, the method involves reinforcement learning algorithms (e.g., Monte Carlo tree search), whose performance is significantly improved by leveraging specific reward attribution techniques applied due to the demand automaton. About Figure 1 Demand automata (A) are described R Examples of ) are provided. For readability, input and output letters are used instead of atomic propositions. Circles represent states (here, "s0", "s1", "t", and "o"). "s0" is the initial state, and "o" is the final state. Boxes represent the execution of the implementation under test. Furthermore, {a, b} are input symbols (feeded to the program's implementation), and {0, 1} are output symbols (provided by the program's implementation). For example, in Figure 1In the automaton, the initial state allows inputs "a", "b", or "c" to the implementation. As shown, regardless of the result produced by the implementation after this input, inputting "b" or "c" will cause a return to the initial state. However, if the result provided by the implementation is "0", then inputting "a" will allow a transition to state s1. If the result is "1", then the terminal state "t" will be reached. From state "s1", regardless of the result of the implementation, inputting "a" will cycle back to state "s1". Conversely, if the result of the implementation is "1", then inputting "b" or "c" can allow reaching the final state "o" (i.e., the predetermined result Obj or the goal). In other cases (where the result of the implementation is "0"), it cycles back to the initial state "s0". In this example, in A R In this example, a theoretically covered trajectory (path) exists because the input sequence "ab" can generate a trajectory (a0b1) in the "correct" implementation, which will lead to "from the initial state "s0" to the final state "o". This example of the automaton is deterministic; furthermore, letting "t" be an error state makes it a safe automaton. This can be accomplished by adding cyclic input-output transitions (similar to the transition to the lower left of "s0") on "t" and "o". While performing various iterations of the method according to this disclosure, in the context of this disclosure, an implementation of such an automaton is implemented in a convenient form (i.e., a parsed data structure), which is described below. This data structure can take any convenient form, such as AIGER format. Of course, this example of the automaton is purely illustrative.

[0031] Figure 2 The main steps of method 10, which uses at least one automaton to define requirements, are illustrated in the illustration of the use of at least one automaton to determine the sequence of actions to be performed on the implementation of a computer program to achieve a target (which may be a test target or an overall action target, such as achieving a predetermined result).

[0032] Method 10 is determined by a computer system or device (described below). Figure 4 (Presented in the middle) Execution.

[0033] More specifically, while the implementation I itself is a black box, it is assumed that the implementation I also behaves as a finite (unknown) automaton; it is input-complete, meaning that any action (input) can be set by the test environment, and it is output-deterministic, meaning that any output state (outcome) has exactly one transition. Finally, it is assumed that from any input state of I, I can be reset to its unique initial state at any time. These properties ensure that if a sequence of inputs is fed to the implementation from its initial state, the system produces a unique sequence of outputs.

[0034] like Figure 2As shown, a method 10 is disclosed, which is used to process data from multiple states (s0, s1, ... s2). n Demand automaton A R Determine the sequence of actions SeqA to be executed by the implementation I of the computer program to achieve a predetermined result Obj, which is determined by the demand automaton A. R state set (The final state as a function of the situation) (and / or error status) are reflected. According to this disclosure, the method includes at least one iteration of the following steps: - Select the S01 action subsequence SSeqA within the tree data structure T1; - Using demand automaton A R Determine the current action v that S02 will execute after the action subsequence SSeqA. in ; - The implementation of I executes the S03 action subsequence SSeqA and the current action v in Give the result v of the current action. out This step also allows obtaining the current state s in the demand automaton (AR). i To the subsequent state s i+1 The conversion; - Implemented by I based on the current action result v out Furthermore, based on the Action Optimization Data Structure (AODS), the S04 complementary action subsequence (CSSeqA) is executed, which provides the result of the current action v. in The associated reward value r. This step also allows for the use of demand automata A. R Obtain from subsequent state s i+1 To the intermediate state s i+j At least one transformation; - Add the current action v of S05 to the tree data structure T1 in It also determines its reward value r and whether the predetermined result Obj has been achieved, in order to obtain the action sequence SeqA.

[0035] The tree-like data structure T1 is constructed starting from an empty data structure and expanded according to a so-called tree strategy. As described below, the tree strategy selects the branch to expand based on the input state of the demand automaton AR. In the first iteration, since no tree exists, the first action (assignment) is directly determined in step S02 (for...). Figure 1 For example, it would be "a", "b" or "c" (since the tree data structure is empty, the action subsequence SSeqA is also empty).

[0036] Therefore, the proposed method allows for input to the current action v in Provides reward value (current action v) inDemand automaton A R Provided based on previous actions processed in previous iterations and registered in the tree data structure T1, not only based on the current action v in The result of its own execution, and also based on the implementation of I in the current action v in The subsequent execution of a specific complementary action subsequence CSSeqA is then performed. Unlike other possible methods in the prior art, the specific complementary action subsequence CSSeqA is selected based on a promising strategy for achieving the predetermined result Obj. This promising strategy is arbitrarily represented as a "greedy strategy," and it is selected within the action optimization data structure AODS. In other words, after the current action v has been executed... in Next, regarding demand automaton A R Perform another specific series of subsequent actions. In the example implementation, this is attributed to the current action v. in The reward of the result (whose value can be included in a given interval, for example, between 0 and 1 or other suitable boundaries, as described below) is calculated as the state to be reached after a specific complementary action subsequence CSSeqA (in the demand automaton A). R The distance between (in the middle) and the predetermined result Obj is a function.

[0037] The process iterates through the disclosed method until the predetermined result Obj is reached, i.e., until the current action v. in A predetermined result Obj is allowed to be reached, or until a specific complementary action subsequence CSSeqA allows the predetermined result Obj to be reached. According to this disclosure, this allows for fast convergence of the solution for achieving the predetermined result Obj compared to existing uniform, naive, and blind solutions.

[0038] According to this disclosure, the specific complementary column action sub-sequence CSSeqA is selected from the action optimization data structure AODS, which includes methods for retrieving data from the demand automaton A. R Given a state, there is at least one possible path to reach the predetermined result Obj. The action optimization data structure AODS is pre-calculated (pre-processed) based on the required automaton. In other words, once the current action v is obtained... in In at least one embodiment, the method includes the step of determining a specific complementary action subsequence CSSeqA from an Action Optimization Data Structure (AODS). According to this disclosure, the length of the specific complementary action subsequence CSSeqA is determined in the current action v. in The number of actions to be performed subsequently. According to this disclosure, this length is parameterized and limited to a given number of actions. Once the actions of a specific complementary action subsequence CSSeqA have been performed, the demand automaton A is obtained. RThe current (reached) state (CS) is used to calculate the distance to the predetermined result Obj, thereby calculating the distance provided to the current action v. in The overall reward for the results.

[0039] According to this disclosure, method 10 can be implemented as a Monte Carlo tree search algorithm, modified to use the strategy of the Action Optimized Data Structure (AODS). According to this disclosure, the use of the strategy of the AODS can be completed during the rollout phase (i.e., for obtaining the complementary action subsequence CSSeqA). Furthermore, a randomization algorithm can be added to select the strategy from the AODS, or the input can be selected randomly and uniformly from the available inputs.

[0040] According to this disclosure, the action optimization data structure AODS can also be used during the selection of action subsequences SSeqA within the tree data structure T1 in order to select the most promising starting point when determining the action sequence that should be allowed to reach the predetermined result (target) Obj.

[0041] One aspect of this disclosure relates to a demand-based automaton A. R The implementation specification preprocesses the action optimization data structure AODS. This is accomplished via a recursive algorithm that starts with a predetermined result Obj and recursively loops until it reaches the demand automaton A. R The initial state is s0. Recall that the predetermined result Obj can be a demand automaton A. R The final state of o and / or error (e.g., when the purpose of the action is to test the implementation of a computer program to determine whether it can be driven to implement an execution error).

[0042] about Figure 3 A diagram of the action optimization data structure is provided, and the construction of the diagram and data structure is explained below.

[0043] This paper provides a detailed formal explanation of the creation process of the Action Optimization Data Structure (AODS). From a certain perspective, finding the desired outcome... A R The trajectory can be viewed as occurring between the following two players in the demand automaton A. R The game is a round-robin contest between the tester and the system (i.e., the implementation of the computer program). At each step, the tester provides input assignments. (Action), and the system assigns the output value. (The result of this action, as the function that assigns the output value) is responded to, and the game moves to the demand automaton A. RThis is a new state in the process. In this case, the online testing algorithm defines a strategy implemented by the tester, while the system executes a fixed strategy determined by the implementation. However, this fixed strategy is a black box and therefore unknown to the tester.

[0044] For the purpose of completeness, this paper provides a demand automaton A. R And a formal explanation of the related algorithms. Assume a is specified as having... and test target Complete deterministic safe automata ,in: - It refers to the state of the input state and the state of the output state. A finite set of unions, - ∈ It is the initial state. - AP is a set of atomic propositions, divided into , - It is a finite set of transformations. - It is a set of accepting states. - AP is a set of atomic propositions, divided into , - The assignment of AP is the set of 2 atomic propositions that are determined to be true. AP element, - (correspondingly, v) out ) is v in AP in (resp.AP out The projection onto ) makes , - Bool(AP) represents the set of Boolean combinations of atomic propositions in AP.

[0045] Consider a reaction system as a series of atomic movements: at each step, the tester first sets the input assignments. Then implement I to immediately set the output assignment. Therefore, the assignment observed at the point of atomic movement is The system's trajectory is a sequence of atomic movements. The implementation I can use internal variables to calculate the output based on the input and some internal state, but these internal variables cannot be observed by the environment (tester) because the implementation is a black box.

[0046] Next, we will mainly focus on the status. : It is transient. For state s inand assignment , let Post AR (s in , ) indicates that makes state s ′in The set, and Pre AR (s in , ) indicates that makes state s ′in The set. Note that for deterministic holistic automata, Post AR (s in , ) is a single-point set.

[0047] For sets The set of direct predecessors of B is And its direct successor set is From these sets, the set of states that B can reach from it (i.e., those that are otherwise reachable from B) can be defined as... And define the set of states reachable from B as , where lfp represents the minimal fixed-point operator. Definition The fixed point is equivalent to: .

[0048] This corresponds to the sequence (V) i Iterative computation of i∈N, from Start (this is why we obtain the minimum number of fixed points) and make Through inductive observation regarding i, from V i For any state s, there exists a path to B within i steps. (Sequence) It is non-decreasing, and its constraint is the set of all states that can be reached from B: from each state There exists a finite trajectory σ such that by reading σ from s, the trajectory ends at B. Similarly, for each state... Existence state And the trajectory σ, such that by reading σ from s, the trajectory ends at s'.

[0049] These theoretical elements were introduced, and one considered the competition between the tester, who plays the role of the environment, and the system (implementation I). The tester's goal is to achieve Or reveal inconsistencies (i.e., reaching) The tester controls the input assignment. And observe the selection made by the system. The output is assigned.

[0050] The concept of turn-based gameplay is based on the concept of controllable advances. The directly controllable front-wheel drive is the following set:

[0051] Therefore, from each state Testers can choose some assignments. This ensures that regardless of the system's output... Whatever it is, it guarantees entering state B or exhibiting inconsistency. In other words, the tester can ensure entering B or revealing inconsistency within a step (regardless of the system's strategy).

[0052] The controllable predecessor set of subset B is defined as Using the same reasoning as before, this minimum fixed point defines the set of all states from which a tester can either be guaranteed to enter B within a finite number of steps or reveal inconsistencies regardless of the system's policy.

[0053] For example, in Figure 3 middle, and .

[0054] The strategy for this competition is a function. This associates each trajectory with a subset of input assignments that can be applied in the next step. As long as for Any two trajectories that reach the same state and Strategy Keep Then the strategy can be considered This refers to what is meant by memoryless. If a state sequence exists... Then the trajectory (path) With the strategy starting from state s Compatibility. Write it down. to indicate and from The set of all compatible trajectories, also known as from The result.

[0055] If all results eventually reach s The tester's strategy From state B wins. If the test subject has the ability to... Winning strategy, then state yes The set of winning states can be used To calculate: for all , strategy This makes it possible to obtain all states ,all Finally achieved .

[0056] You can consider doing it directly. We search for winning strategies to use as test algorithms. However, in most operational cases (i.e., for complex demand automata), Since it does not include an initial state, this strategy is practically unapplicable. In game theory terms, this means that the tester has no winning strategy from the initial state.

[0057] Therefore, alternative constructions for action-optimized data structures, namely collaborative strategies, should be considered.

[0058] For a given set ,consider , it is The set of direct predecessors: from these states, there exists a successor state. A pair of assignments in From this point onward, the tester cannot always guarantee achieving the desired result within a single step. However, it can choose to move the system to... Input assignment In fact, when trying to reach If the current state is not In the middle, choose this. It's a good choice.

[0059] If for all and all ,exist Make Then strategy f is called cooperation.

[0060] The construction of the action optimization data structure combines winning and cooperative strategies to define a greedy strategy, which can be viewed as a best-effort strategy that can be adopted when there is no winning strategy from the initial state. These guarantees only apply to some system strategies in A. R Central Progress. Then, in the context, a greedy strategy stored in the action optimization data structure can be used to select the optimal trajectory (path) to attempt to reach the goal. However, the use of these strategies can be mitigated to ensure parameterization diversity when selecting the strategy to use, as described below.

[0061] Let us consider It is non-empty because it contains .make The winning strategy is O as described above. If belong ,but Ensure that all system policies are implemented from arrive In this situation, you can stop and return. As a greedy strategy.

[0062] The most common situation is when In this case, the increasing sequence of the state set is defined inductively. And corresponding strategies, making It is the set of all reachable states.

[0063] Let us consider And assume that the sequence has been defined up to the index. .make It is beyond the state that has already been seen. The set of direct predecessors. Defined as W i -Cooperation strategy. Then, let And consider the corresponding winning strategies. .

[0064] Let us use Representation strategy and The pointwise union: for all ,if ,but ;if ,but ; and otherwise, any definition .

[0065] if only The sequence stops when it becomes empty. Note that... and sequence It is increasing. Therefore, the recursive construction builds an increasing hierarchy of sets (inserted into the action optimization data structure) that are getting larger and larger. In each level In state Collaboration is needed to get closer within this hierarchy. .because All states belong to a certain In particular, if Available from Arrival (i.e., If ), then there exists a certain index. , making exist In the middle, the rank of s will be called and will make Labeled as rank(s).

[0066] It is by The defined strategy; and it is arbitrarily defined for other states.

[0067] Note the composition. Winning strategy or None of them contain loops. However, because It also consists of collaborative strategies, therefore, due to achieving The output of a higher-rank state in the hierarchy may lead to a loop. In other words, The algorithm does not guarantee that it will achieve the desired result. Furthermore, it may cause infinite loops for some system policies.

[0068] This structure is in Figure 3 The image shows the tester's state. The states in the diagram are represented by circles, and the system states are located within these circles. The state is represented by a square. .exist The state in ,Strategy Indicated by thick gray arrows. Black arrows represent sets reaching higher rank. The state or reached Those outputs. For example, in The bold input wins because all subsequent outputs reach the desired result. However, the other input did not win because a possible output looped back to the previous input. Therefore, the loop returns to... And another returns to a higher rank. Different situations are caused by This demonstrates that, after some inputs in a collaborative strategy, an output can achieve... And another output reaches The state in.

[0069] Using this algorithm only involves constructing a strategy based on the demand automaton. ,Strategy It is inserted into the action optimization data structure and requires minimal cooperation from the system to achieve [the desired result]. or In practice, deterministic implementations can be consistent with requirements but are not very cooperative: for a given input in a state, even if consistent, the proposed output is not necessarily among those outputs with reduced rank. While they do not guarantee progress toward the goal at each step, those greedy (pre-computed) strategies are a best-effort approach in that direction when the behavior of the implementation is unknown.

[0070] Greedy strategies can be used to define random testing algorithms, similar to uniformly and randomly selecting input actions from the available inputs.

[0071] Let's review the event trajectory After that, automata In state In a uniform (standard) strategy, then... Select input evenly The simple modification lies in using... The uniform selection in replaces the selection, thus restricting the domain to a transition chosen only by a greedy policy. If for some i, exist In the middle, following this strategy will inevitably lead to (if and Or, if the implementation of I is inconsistent, it may lead to Once in Testers used Because there is entry The possibility; however, even if consistent, implementation is not forced to cooperate, and it is possible to revert to having Some (That is, back off). Therefore, this so-called pure greedy algorithm is incomplete.

[0072] However, it can be achieved by being uniform and Randomization is used between the given information to obtain the complete algorithm: ,algorithm Included in each state With probability use ; and with probability Use uniform selection. Then complete the result. algorithm.

[0073] As mentioned above, these strategies, which can be obtained from the action optimization data structure, are used not only to select complementary action subsequences (CSSeqA), but also to select subsequences of actions to be performed previously (SSeqA).

[0074] More specifically, for example, when the disclosed method is implemented using the Monte Carlo tree search algorithm (or an equivalent algorithm), The algorithm was used as an exit strategy, rather than a pure "uniform" algorithm.

[0075] Additionally, a tree strategy (i.e., the selection of action subsequences SSeqA) can also be used. The algorithm. More specifically, the modification lies in using a greedy algorithm within the tree strategy. For example, this is accomplished by fixing the boundary M > 0. During the expansion phase, in its... The projection of a state s onto a given node in the tree is such that, during the first M visits to that node (i.e., in the first M iterations), the conventional policy (e.g., the upper confidence bound (UCT) policy applied to the tree) is restricted to the greedy policy. Given the input; after the first M visits to the tree node, the algorithm backs back to the regular strategy without restriction, which covers the entire set of input assignments. .

[0076] Both implementations of this method (e.g., by modifying the standard MCTS) do not alter its integrity properties. They aim to improve convergence speed by introducing biases into probabilistic choices. These biases are based on additional knowledge about the system, or more precisely, expectations (because they use offline computation of the requirements), rather than on the unknown implementation itself.

[0077] Other possibilities would involve modifying the reward shaping by choosing a rank to represent the reward value. Furthermore, instead of using uniform probabilities among actions in f(·), in cooperative states, the choice of input can be biased based on the expected cooperation of the system after the input (i.e., the percentage of the output that reduces the rank).

[0078] Another aspect of this disclosure relates to calculating reward values ​​to accelerate the convergence of sequences that reach a predetermined result Obj (objective).

[0079] For example, if the trajectory is in A R If a trajectory is induced to become very close to the target Obj, then that trajectory may be given a better (i.e., lower) reward value than another trajectory that travels further away from the target. According to this disclosure, A R It is only used to calculate the reward value of the trajectory, while the actual testing is carried out by the proposed testing method. In other words, the current action ( The execution steps (S03, S04) of the complementary action subsequence CSSeqA (obtained from the action optimization data structure AODS) will be used by the demand automaton A. R The current state s is set, and the distance between this state and the target is used to determine the current action performed in step S03. The associated reward value.

[0080] make And targeting ,definition Then, having .

[0081] In fact, each Is with The set of states that are a distance i from a certain state in (in Included in (In the sense of a travel path of length i). According to this disclosure, two ways of allocating rewards to simulated trajectories are considered.

[0082] For the first one, let In order to make The maximum value. Let's define... That is, all states that are unreachable. If The path to it is at A. R In and in s R Each trajectory σ terminated in the middle is assigned a reward. Note that this is explicitly defined because of sets. They are paired and non-overlapping, and they cover all states. Therefore, the closer the trajectory σ is to the target... The smaller the reward, the better. A reward of 0 means that the state obtained from the trajectory satisfies the objective. .

[0083] The second reward allocation considers not only the final reward, but also all rewards seen during the simulation, as described below. Let This represents the sequence of rewards encountered during the simulation (these are the rewards prefixed to the trajectory σ). If the simulation length is less than K, the last reward is simply repeated to extend the sequence to size K. The simulation reward value is then given by:

[0084] Where γ∈(0,1) is the discount factor. Note that the discount factor is true if and only if... At that time, the value is 0 and the minimum. Furthermore, although... The first factor is the most important, but the second factor means that simulations with larger initial reward values ​​are preferred. This can actually be considered as... The weighted version, where if the simulation has a small reward value in the first step, the weight is smaller.

[0085] The proposed method 10 is fully automated. Its implementation reduces software development and verification time. It should be emphasized that this disclosure is not limited to the exemplary embodiments described above. Variations of the exemplary embodiments described above are also within the scope of this disclosure.

[0086] like Figure 4As shown, the device DV for determining the sequence of actions SeqA to be executed by the implementation I of the computer program to achieve a predetermined result Obj, as described above, may include an antenna system AS, for example, coupled via an interface IN to processing circuitry including a processor PROC and a memory MEM. Alternatively or supplementarily, or as a variant of the antenna system AS, necessary data (e.g., a demand automaton, the implementation of the computer program to achieve the predetermined result) may also be received via a communication link CL, allowing the device to determine the sequence of actions of this disclosure. The memory stores at least the instructions of the computer program according to this disclosure. By performing this type of processing, and particularly by implementing the data structure (T1, AODS) according to the method of this disclosure, the processor of the device has been shown to be able to search thousands of possibilities and combinations of various possible sequences of actions within a short time period to achieve the predetermined result. This effect is obviously impossible by attempting to manually or mentally calculate or obtain these sequences of actions, as it would be too time-consuming and completely meaningless given the complexity of the problem.

[0087] In a preferred embodiment, the computer system or device includes one or more processors (which may belong to the same computer or different computers) and one or more memories (magnetic hard disk, optical disk, electronic memory, or any computer-readable storage medium), wherein a computer program product is stored in the form of a set of program code instructions that will be executed to implement all or part of the steps of determining method 10. Alternatively, or in combination thereof, the computer system may include one or more programmable logic circuits (FPGA, PLD, etc.) and / or one or more application-specific integrated circuits (ASICs), etc., adapted to implement all or part of the steps of determining method 10. In other words, the computer system includes a set of means configured by software (specific computer program product) and / or hardware (processor, FPGA, PLD, ASIC, etc.) to implement the steps of determining method 10.

Claims

1. Used to process data from multiple states (s0, s1, ... s... n Demand automata (A) R (10) determines the sequence of actions (SeqA) to be executed by the implementation (I) of the computer program to achieve a predetermined result (Obj), which is determined by the demand automaton (A). R The set of states in ) The method is implemented by an electronic device and includes at least one iteration of the following steps: - Select (S01) action subsequences (SSeqA) within the tree-structured data (T1); - Using the demand automaton (A R Determine (S02) the current action to be executed after the action subsequence (SSeqA). ); - By implementing (I) the action subsequence (SSeqA) and the current action (S03) are executed (S03). ), to obtain the result of the current action ( ); - By implementing (I) based on the result of the current action ( And execute (S04) complementary action subsequence (CSSeqA) according to the Action Optimization Data Structure (AODS) to obtain the current action ( The associated reward value (r); - The current action ( ) and the current action ( The reward value (r) is added (S05) into the tree data structure (T1), and it is determined whether the predetermined result (Obj) has been reached, in order to obtain the action sequence (SeqA).

2. The method (10) according to claim 1, wherein, When the result of the current action is determined ( ) in the demand automaton (A R Provides information from the current state (s) i ) to the set The iteration stops when the state changes.

3. The method (10) according to any one of claims 1 or 2, wherein, When it is determined that the complementary action subsequence (CSSeqA) to be executed (S04) is in the demand automaton (A) R Provides information from the current state (s) i ) to the set The iteration stops when the state changes.

4. The method (10) according to any one of claims 1 to 3, wherein, The steps for executing (S04) complementary action subsequence (CSSeqA) include the following: - In the demand automaton (A R Obtain the current state (s) i The current state (s) i ) is from the result of the current action ( It is certain; - Identify possible actions that could be taken from the current state (s) within the Action Optimization Data Structure (AODS). i The sequence of actions to achieve the predetermined result (Obj); - Select a predetermined number of actions from the actions in the action sequence to obtain the complementary action subsequence (CSSeqA); - By sequentially executing each action of the complementary action subsequence (CSSeqA) through the implementation (I), each result of each action allows the demand automaton (A) to... R From state (s) k The transition from the intermediate state (s'k); - Based on the completion of the execution of the complementary action subsequence (CSSeqA) in the demand automaton (A) R The intermediate state (s) reached in ) l The reward value (r) of the complementary action subsequence (CSSeqA) is determined.

5. The method (10) according to any one of claims 1 to 4, characterized in that, With the current state (s) i ) and the current action ( The associated reward value (r) is the reward value associated with the demand automaton (A). R The final state (s) l ) and sets A function of the distance between the two sides.

6. The method (10) according to claim 5, wherein, According to the current state (s) i ) and the set The distance (d) is determined by the remaining intermediate states between them.

7. The method (10) according to any one of claims 1 to 6, wherein, The maximum number of actions in the complementary action subsequence (CSSeqA) is bounded.

8. The method (10) according to any one of claims 1 to 7, wherein, The Action Optimization Data Structure (AODS) is based on the Demand Automaton (A). R The structure is constructed from the set, and the Action Optimization Data Structure (AODS) is derived from the set. One of the states begins to recursively identify the action to be performed in the demand automaton (A). R The transitions between states are necessary to achieve the set. One of the possible action sequences in one of the states.

9. The method (10) according to any one of claims 1 to 8, wherein, Within the tree-like data structure (T1), the action subsequence (SSeqA) is selected (S01) and executed according to the action optimization data structure (AODS).

10. The method (10) according to claim 9, wherein, The action subsequence (SSeqA) is selected according to the Action Optimization Data Structure (AODS) for a predetermined number of iterations (M).

11. The method (10) according to claim 10, wherein, Once the predetermined number of iterations (M) is reached while the predetermined result (Obj) remains unachieved, the action subsequence (SSeqA) is selected based on the reward value stored in the tree data structure (T1).

12. A demand automaton (A) for processing multiple states (s0, s1, ..., sn). R A system that determines the sequence of actions (SeqA) to be executed by the implementation (I) of a computer program to achieve a predetermined result (Obj), the predetermined result (Obj) being determined by the demand automaton (A). R The set of states The system comprises at least one processor and at least one memory, wherein, The at least one processor is configured to perform the method (10) according to any one of claims 1 to 11.

13. A computer program product comprising instructions that, when executed by at least one processor, configure the at least one processor to perform the method (10) according to any one of claims 1 to 11.

Images