Industrial automation test case dynamic optimization method based on time weighted Bayesian model

Abstract

The invention discloses an industrial automation test case dynamic optimization method based on a time-weighted Bayesian model, which comprises the following steps: calculating a posterior probability through the following formula: the formula is used for calculating whether a test case still has the posterior probability of an execution value, and then judging whether to execute the test case according to whether the posterior probability exceeds a threshold value. If the test case exceeds the threshold value, executing the test case, and if the test case is lower than the threshold value, not executing the test case; according to the method, the time decay factor is introduced into the historical execution result of the test case, the weighted historical observation data is constructed and used as the input of the Bayesian inference model, and the probability of discovering the defect by the test case is dynamically evaluated, so that the adaptive sorting of the execution sequence of the test case is realized, and the test efficiency and the defect discovering capability are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of industrial automation testing, and particularly relates to an industrial automation test case dynamic optimization method based on a time-weighted Bayesian model. BACKGROUND

[0002] With the continuous improvement of intelligent manufacturing and industrial automation level, various industrial products increasingly rely on functional testing in the design and production process. Especially in complex system integration, multi-module collaborative work and continuous iteration development mode, how to efficiently and accurately perform product functional testing has become a key link to ensure product quality and shorten the development cycle. In the early days, industrial automation testing mainly relied on human experience, which was inefficient and prone to errors. Later, automation testing technology gradually emerged, greatly improving testing efficiency, but the optimization of test cases has always been a challenge for the industry.

[0003] In the traditional product functional testing process, testers usually develop test case execution strategies based on experience or historical defect data, or arrange test order in a fixed priority order. However, these methods have the following problems in actual application:

[0004] 1. Low utilization of testing resources: In the testing process, a large number of testing resources are evenly distributed to each test case, without reasonable allocation according to the importance and potential value of the test cases, resulting in insufficient resources for some critical test cases, while non-critical test cases occupy too many resources.

[0005] 2. Lack of modeling of test result timeliness: Some test cases were effective in discovering defects in the past, but may be ineffective after the product function is updated; while some originally insensitive test cases may become particularly important in the new version. The traditional method often ignores the timeliness of test cases, and cannot adjust the execution order and priority of test cases in time for new problems and potential risks after system updates, so that the test results cannot reflect the latest state of the system in time.

[0006] 3. Failure to fully utilize the value of historical test data: The results of each test execution (whether a problem is found, the type of problem, the frequency of occurrence, etc.) contain rich information. Existing methods often only focus on the test results themselves, ignoring the time distribution characteristics and statistical laws of historical data, and cannot optimize and adjust test cases based on historical data.

[0007] 4. Lack of adaptive ability: Most test scheduling strategies are statically configured and cannot automatically adjust the execution order or priority of test cases based on test feedback, resulting in rigid testing processes that are difficult to adapt to the fast-paced product development rhythm.

[0008] On the other hand, Bayesian inference methods can dynamically estimate the probability of test case validity by integrating prior knowledge and observational data. However, existing Bayesian models often use equal weighting to process historical data, failing to consider the varying importance of test results at different points in time to the current judgment, thus limiting the model's accuracy and practicality. Therefore, there is an urgent need for a dynamic test case optimization method that combines time decay factors with Bayesian inference mechanisms to improve the intelligence level of product functional testing, increase defect discovery efficiency, and reduce testing costs. Summary of the Invention

[0009] To address the problems raised in the background, the purpose of this invention is to provide a dynamic optimization method for industrial automation test cases based on a time-weighted Bayesian model. The core of this method lies in introducing a time decay factor into the historical execution results of test cases to construct weighted historical observation data, which is then used as input to a Bayesian inference model to dynamically evaluate the probability of test cases discovering defects. This enables adaptive sorting of the execution order of test cases, improving testing efficiency and defect detection capabilities.

[0010] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0011] A dynamic optimization method for industrial automation test cases based on a time-weighted Bayesian model includes calculating the posterior probability using the following formula:

[0012]

[0013] The above formula is used to calculate the posterior probability of whether a test case still has execution value. Then, it is determined whether to execute the test case based on whether the posterior probability exceeds a threshold. If it exceeds the threshold, the test case is executed; if it is below the threshold, the test case is not executed.

[0014] Among them, V i =1 indicates that test case i is still necessary to execute, representing an initial estimate of whether the test case is still necessary to execute before specific historical execution records are available; P(V i =1) is the prior probability, representing the initial judgment on whether the test case is still worth executing without reviewing historical data; P(H i |V i =1) is the likelihood function, which represents how likely it is that the current historical result will occur, assuming that the test case is still necessary to execute.

[0015] Further specifying, the P(H) i The specific calculation formula is as follows:

[0016] P(H i )=P(Hi |V i =1)·P(V i =1)+P(H i |V i =0)·P(V i =0)

[0017] Wherein, the H i It is the historical execution record of the i-th test case, wherein P(H) i ) is a normalization factor that ensures that the posterior probability value is within the interval [0,1].

[0018] Further, based on a preset threshold θ, it is used to determine whether a test case is worth executing. If P(V i =1|H i If P(V) < θ, then the test case is considered unnecessary to execute and is marked as "skipped"; otherwise, if P(V) < θ, then the test case is marked as "skipped". i =1|H i If )>θ, then the test case will be executed.

[0019] Further specifying, the likelihood function assumes that each executed test case is an independent Bernoulli event, as shown in the following formula:

[0020]

[0021] Wherein, the k w n represents the weighted number of test failures. w This represents the total weighted number of times.

[0022] Further specifying, the p weighted Calculated using the following formula:

[0023]

[0024] Wherein, p weighted It is the constructed weighted test failure rate, which is the weighted proportion of test failure events that occurred in the history of this test case.

[0025] Further specifying, the weighted number k of the test failures w Based on the following weighting coefficient function:

[0026] k w =∑w j (t,v)·F ij

[0027] The total weighting number n w Based on the following weighting coefficient function:

[0028] n w=∑w j (t,v)

[0029] The design goal of the two weighting coefficient functions mentioned above is to assign higher weights to more recent test results and test results that occurred in new versions.

[0030] Further specifying, the weighted number k of the test failures w The total weighted number n w w in j The specific calculation of (t,v) includes the following formulas:

[0031]

[0032] Where t0 is the current time, t j λ is the execution time of the j-th test case, λ1 is the time decay coefficient, and λ1 is used to control the weight of whether the current test case is executed if the execution time is longer. Δv j It is the difference between the version number executed in the j-th test case and the version number of the previous execution, and α is used to adjust the weighting coefficient of the version change.

[0033] Further specifying, the F ij The specific calculations include the following formulas:

[0034]

[0035] Wherein, the F ij This expresses the result of the j-th execution of test case i.

[0036] The beneficial effects of this invention are:

[0037] 1. Evaluation objectives that are more closely aligned with engineering practices, improving the efficiency of test resource utilization.

[0038] This invention proposes using "whether test cases still have execution value" as the modeling objective, rather than "whether defects are found" commonly used in traditional methods. This objective is more in line with the actual testing scenario of long-term operation and continuous iteration of industrial products, and helps to identify redundant test cases that have become stable and no longer reveal new problems, thereby effectively reducing the waste of resources caused by repeated testing and improving overall testing efficiency.

[0039] 2. Introduce a dual weighting mechanism of time and version to enhance the model's adaptability to system evolution.

[0040] In the historical data processing stage, this invention employs a weighted approach combining a time decay factor and a version number change factor, giving higher weight to recent test results and test records occurring in newer versions during the evaluation. This mechanism more accurately reflects the test behavior characteristics under the current system state, improves the model's adaptability to product updates and iterations, and avoids misjudgments caused by interference from older version data.

[0041] 3. A dynamic evaluation model based on Bayesian inference enables quantitative decision support.

[0042] This invention achieves quantitative decision-making by constructing a Bayesian inference framework to calculate the posterior probability of test cases. Test cases are then ranked and filtered based on their posterior probabilities, avoiding the uncertainty of subjective judgment. This model possesses a strong statistical foundation and mathematical interpretability, enabling test scheduling strategies to shift from experience-driven to data-driven approaches, thus improving the scientific rigor and stability of decision-making.

[0043] 4. An adaptive scheduling of the test process is achieved through a skip mechanism based on posterior probability.

[0044] The adaptive scheduling mechanism of this invention can adjust the execution order of test cases in real time based on test results. During the testing process, test cases that have proven to have low risk are automatically downgraded in priority, reducing redundant testing, and a threshold is set to determine whether to skip test cases that are not necessary to test. This mechanism reduces manual intervention and improves the automation level and response speed of the testing process. Attached Figure Description

[0045] The present invention can be further illustrated by the non-limiting embodiments given in the accompanying drawings;

[0046] Figure 1 This is an execution logic diagram of the automatic test process in an embodiment of the dynamic optimization method for industrial automation test cases based on a time-weighted Bayesian model of the present invention.

[0047] Figure 2 This is a diagram of the test scheme planning parameter configuration interface for an embodiment of the dynamic optimization method for industrial automation test cases based on the time-weighted Bayesian model of the present invention.

[0048] Figure 3 This is a JSON format file diagram of test cases from an embodiment of the dynamic optimization method for industrial automation test cases based on a time-weighted Bayesian model according to the present invention.

[0049] Figure 4 This is a historical test record table of test cases for an embodiment of the dynamic optimization method for industrial automation test cases based on the time-weighted Bayesian model of the present invention. Detailed Implementation

[0050] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments. The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0051] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0052] Furthermore, the use of terms such as "first" and "second" in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0053] The present invention provides a dynamic optimization method for industrial automation test cases based on a time-weighted Bayesian model, which includes calculating the posterior probability using the following formula:

[0054]

[0055] The above formula is used to calculate the posterior probability of whether a test case still has execution value. Then, it is determined whether to execute the test case based on whether the posterior probability exceeds a threshold. If it exceeds the threshold, the test case is executed; if it is below the threshold, the test case is not executed.

[0056] Among them, V i =1 indicates that test case i is still necessary to execute, representing an initial estimate of whether the test case is still necessary to execute before specific historical execution records are available; P(V i =1) is the prior probability, representing the initial judgment on whether the test case is still worth executing without reviewing historical data; P(H i |V i=1) is the likelihood function, representing the probability of the current historical result occurring, assuming the test case is still necessary to execute. Where V i It can be set to a uniform value, or it can be set according to factors such as its test case coverage, the frequency of changes to the module it belongs to, and its performance in the initial version.

[0057] In the practical application of this embodiment, the P(H) i The specific calculation formula (7) is as follows:

[0058] P(H i )=P(H i |V i =1)·P(V i =1)+P(H i |V i =0)·P(V i =0) Formula (7)

[0059] Wherein, the H i It is the historical execution record of the i-th test case, wherein P(H) i ) is a normalization factor that ensures that the posterior probability value is within the interval [0,1].

[0060] In the practical application of this embodiment, a preset threshold is used to determine whether a test case is worth executing. If P(V) i =1|H i If P(V) < θ, then the test case is considered unnecessary to execute and is marked as "skipped"; otherwise, if P(V) < θ, then the test case is marked as "skipped". i =1|H i If )>θ, then the test case will be executed.

[0061] In the practical application of this embodiment, the likelihood function assumes that each executed test case is an independent Bernoulli event, as shown in the following formula (2):

[0062]

[0063] Wherein, the k w n represents the weighted number of test failures. w This represents the total weighted number of times.

[0064] In the practical application of this embodiment, the p weighted Calculated using the following formula:

[0065]

[0066] Wherein, p weightedIt is the constructed weighted test failure rate, which is the weighted proportion of test failure events that occurred in the history of this test case.

[0067] In the practical application of this embodiment, formula (3) represents the weighted number of test failures, k. w Based on the following weighting coefficient function:

[0068] k w =∑w j (t,v)·F ij Formula (3)

[0069] Formula (4) represents the total weighted number n. w Based on the following weighting coefficient function:

[0070] n w =∑w j Formula (4) (t,v)

[0071] The design goal of the two weighting coefficient functions mentioned above is to assign higher weights to more recent test results and test results that occurred in new versions.

[0072] In the practical application of this embodiment, the weighted number of test failures k w The total weighted number n w w in j The specific calculation of (t,v) includes the following formula (5):

[0073]

[0074] Where t0 is the current time, t j λ is the execution time of the j-th test case, λ1 is the time decay coefficient, and λ1 is used to control the weight of whether the current test case is executed if the execution time is longer. Δv j It is the difference between the version number executed in the j-th test case and the version number of the previous execution, and α is used to adjust the weighting coefficient of the version change.

[0075] In the practical application of this embodiment, the F ij The specific calculation includes the following formula (6):

[0076]

[0077] Wherein, the F ij This expresses the result of the j-th execution of test case i.

[0078] The core of this invention lies in providing a dynamic optimization method for test cases for functional testing of industrial products, the key technical features of which are as follows:

[0079] 1. Propose using "whether the test cases are still necessary to execute" as the evaluation objective.

[0080] Unlike traditional methods that use "whether a defect is found" as the core indicator, this invention proposes for the first time to use "whether the test case still has execution value" as the modeling objective, which is closer to the actual test resource management needs and is suitable for industrial product testing scenarios that run for a long time and are frequently updated.

[0081] 2. Introduce a time and version weighting mechanism to process historical data.

[0082] When analyzing the historical execution records of test cases, this invention introduces a dual weighting factor: for the time decay factor, the older the test result, the smaller the weight; on the other hand, for the version change factor, test results that occur in new versions are given higher weight. This mechanism ensures that the model can more accurately reflect the behavior of test cases in the current system state.

[0083] 3. Construct a dynamic evaluation model based on Bayesian inference.

[0084] This invention utilizes a Bayesian model to calculate the posterior probability of test cases, achieving quantitative decision-making. Test cases are then ranked and filtered based on their posterior probabilities, avoiding the uncertainty of subjective judgment. In testing multiple industrial automation projects, it can more accurately select high-risk test cases, thereby improving the accuracy of defect detection.

[0085] 4. Design a scheduling mechanism based on posterior probability ranking and threshold judgment.

[0086] The present invention further proposes the following operation process: First, a threshold is set to determine whether a test case should be skipped; second, test cases with low testing necessity are skipped to reduce redundant execution. This mechanism realizes the automation and intelligent scheduling of the testing process.

[0087] The embodiments of the present invention are as follows:

[0088] This invention takes the automated testing of industrial touch screens as an example and describes the implementation of the invention with reference to the accompanying drawings. The industrial touch screen product has undergone several rounds of product iterations and has accumulated a large number of test cases and basic data on whether each test case passed. Before implementing this invention, the test cases and the test pass rate have been statistically analyzed, and the success or failure of each test case in each test has been recorded. At the same time, the OS version number corresponding to this test has also been recorded.

[0089] Appendix Figure 1This is the execution logic diagram of the automatic testing process of the present invention. The entire testing process starts from the "Start" node. When a new device under test is placed and ready for automated testing, the testing system loads and traverses all test cases and allows the user to select which test cases do not need to be executed based on the engineer's experience through the human-computer interaction interface. These test cases are set to p=0. In addition, the human-computer interaction interface also allows the user to select which test cases must be executed based on the engineer's experience. These test cases are set to p=1. The remaining test cases are executed one by one according to formula (1), that is... Calculations are performed to obtain the posterior probability p of this test case. After calculating all test cases, p for each test case is compared with a preset threshold θ. If p > θ, the test case is added to the set of test cases to be executed; otherwise, if p ≤ θ, the test case is ignored. After completing the threshold comparison of all test cases, a new set of test cases to be tested is formed. Next, the system executes the automated testing process according to this set of test cases. After completing the automated testing process, according to formula (3), i.e., k w =∑w j (t,v)·F ij Define and update the data F indicating whether the test case passed. ij Finally, the process reaches the "end" node, marking the official completion of this testing cycle.

[0090] Appendix Figure 2 This is the calculation parameter configuration interface involved in this invention. Users can set the threshold for determining whether a test passes or fails on this interface. They can also set whether each test case must be executed or must not be executed. The symbol "√" to the left of a test case indicates that the test case must be executed, and the symbol "×" to the left of a test case indicates that the test case must not be executed① (the corresponding numbers are shown in the attached figures). Other test cases are indicated in the attached figures. Figure 1 In the "uncertain" state ② described in the flowchart, each test case can have its prior probability ③ set according to the test engineer's subjective judgment of that test case. The default value is 0.5, representing no subjective bias. To reduce the workload of test engineers, see the attached... Figure 2 The interface also provides a function to import historical solutions (④). This function imports a set of historically stored test cases that definitely need to be tested, test cases that definitely won't be tested, and test cases whose testing is uncertain, allowing engineers to quickly formulate the current solution. (Appendix) Figure 2 The interface also provides a function to export the current solution (⑤). This function exports a set of test cases that definitely need to be tested, test cases that definitely will not be tested, and test cases whose need for testing is uncertain. (Appendix) Figure 2The "Calculate" button function allows the software to automatically calculate the posterior probability p of all test cases in the "uncertain" set after the test engineer clicks it⑥, and update the value in the JSON file of each test case, as shown in the attached figure. Figure 3 As shown.

[0091] Appendix Figure 3 In addition to the basic parameters for test case execution, the JSON format also includes the number of times the test case has been executed in the past①, the number of times the test failed in the past②, and the posterior probability p of the current test case③.

[0092] Appendix Figure 4 This is a record table documenting whether a particular test case's historical tests were executed, and if so, whether they passed. The table includes the results of each test's success or failure, the test time, the version number of the tested component, the corresponding historical data for that test, the posterior probability, and whether the status was manually adjusted to "√" or "×". The purpose of this data table is to support formula (3), i.e., k w =∑w j (t,v)·F ij And formula (4), i.e., n w =∑w j The calculation of (t,v) provides historical data.

[0093] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A dynamic optimization method for industrial automation test cases based on a time-weighted Bayesian model, characterized in that, This includes calculating the posterior probability using the following formula: The above formula is used to calculate the posterior probability of whether a test case still has execution value. Then, it is determined whether to execute the test case based on whether the posterior probability exceeds a threshold. If it exceeds the threshold, the test case is executed; if it is below the threshold, the test case is not executed. Among them, V i =1 indicates that test case i is still necessary to execute, representing an initial estimate of whether the test case is still necessary to execute before specific historical execution records are available; P(V i =1) is the prior probability, representing the initial judgment on whether the test case is still worth executing without reviewing historical data; P(H i |V i =1) is the likelihood function, which represents how likely it is that the current historical result will occur, assuming that the test case is still necessary to execute.

2. The method for dynamic optimization of industrial automation test cases based on a time-weighted Bayesian model according to claim 1, characterized in that, The P(H) i The specific calculation formula is as follows: P(H i )=P(H i |V i =1)·P(V i =1)+P(H i |V i =0)·P(V i =0) Wherein, the H i It is the historical execution record of the i-th test case, wherein P(H) i ) is a normalization factor that ensures that the posterior probability value is within the interval [0,1].

3. The method for dynamic optimization of industrial automation test cases based on a time-weighted Bayesian model according to claim 1, characterized in that, It also includes comparing the posterior probability with a preset threshold: based on a preset threshold θ, it is used to determine whether a test case is worth executing; if P(V) i =1|H i If P(V) < θ, then the test case is considered unnecessary to execute and is marked as "skipped"; otherwise, if P(V) < θ, then the test case is marked as "skipped". i =1|H i If )>θ, then the test case will be executed.

4. The method for dynamic optimization of industrial automation test cases based on a time-weighted Bayesian model according to claim 1, characterized in that: The likelihood function assumes that each executed test case is an independent Bernoulli event, as shown in the following formula: Wherein, the k w n represents the weighted number of test failures. w This represents the total weighted number of times.

5. The method for dynamic optimization of industrial automation test cases based on a time-weighted Bayesian model according to claim 4, characterized in that, The p weighted Calculated using the following formula: Wherein, p weighted It is the constructed weighted test failure rate, which is the weighted proportion of test failure events that occurred in the history of this test case.

6. The method for dynamic optimization of industrial automation test cases based on a time-weighted Bayesian model according to claim 4, characterized in that, The weighted number k of test failures w Based on the following weighting coefficient function: k w =∑w j (t,v)·F ij The total weighting number n w Based on the following weighting coefficient function: n w =∑w j (t,v) The design goal of the two weighting coefficient functions mentioned above is to assign higher weights to more recent test results and test results that occurred in new versions.

7. The method for dynamic optimization of industrial automation test cases based on a time-weighted Bayesian model according to claim 6, characterized in that: The weighted number k of test failures w The total weighted number n w w in j The specific calculation of (t,v) includes the following formulas: Where t0 is the current time, t j λ is the execution time of the j-th test case, λ1 is the time decay coefficient, and λ1 is used to control the weight of whether the current test case is executed if the execution time is longer. Δv j It is the difference between the version number executed in the j-th test case and the version number of the previous execution, and α is used to adjust the weighting coefficient of the version change.

8. The method for dynamic optimization of industrial automation test cases based on a time-weighted Bayesian model according to claim 6, characterized in that: The F ij The specific calculations include the following formulas: Wherein, the F ij This expresses the result of the j-th execution of test case i.

Images