Security transaction system automatic test method based on result feedback

Abstract

The invention discloses a security transaction system automatic test system based on result feedback. The system comprises a use case generation module, a use case execution and feedback module, a use case iterative optimization module and an abnormality diagnosis module. The case generation module reads and analyzes a drive data file input by a tester, extracts features in the drive data file, generates a plurality of test case sets covering different feature combinations, and performs priority ranking on all test cases; the use case execution and feedback module executes the test use cases according to the priority sequence and generates feedback data according to execution results; the case iterative optimization module identifies an uncovered scene based on the feedback data, generates a new test case, and performs priority ranking on the newly generated test case; the abnormality diagnosis module collects various indexes during execution of the test cases, sets baseline values of the indexes, marks abnormal cases, classifies abnormal conditions, feeds classified abnormal information back to the case iterative optimization module, and improves a generation strategy of the cases.

Description

Technical Field

[0001] The present invention relates to the technical field of automated testing, and in particular to an automated testing method for a securities trading system based on result feedback. Background Art

[0002] User interface (UI) testing of securities trading systems is a complex process that requires comprehensive consideration of multiple dimensions such as interface diversity, real-time data updates, and user interaction to ensure the robustness, security, and user-friendliness of the application. These factors work together to ensure the reliability and efficiency of the securities trading system in providing services.

[0003] Currently, the UI automation testing of securities trading systems mainly relies on setting specific parameters for each test case and recording the corresponding scripts. Specifically, it includes the setting of parameters such as fund account, password, stock code, transaction price, and the recording of scripts such as login verification, order operation, and result verification. Test execution is run by issuing commands and collecting test results to evaluate the performance and stability of the application.

[0004] Although the existing UI automation testing method can meet the testing needs to a certain extent, it has obvious deficiencies in intelligence and flexibility. Traditional testing platforms are based on fixed operation paths and behavior patterns, making it difficult to respond to changes in the securities market environment in real time, and unable to dynamically generate test scenarios and use cases that adapt to market changes. This is particularly evident when facing a complex and dynamic market environment. In addition, the current analysis of test results mainly relies on test reports and manual inspections, which makes it difficult to obtain in-depth analysis from the result feedback and difficult to use to optimize subsequent test processes. Summary of the invention

[0005] The present invention proposes an automated testing method for a securities trading system based on result feedback, which solves the problem that the existing testing method cannot be dynamically adjusted according to market changes.

[0006] In order to solve the above technical problems, the present invention provides a securities trading system automated testing method based on result feedback, comprising the following steps: Step S1: taking the account type, market type, transaction type and price range in the securities transaction request as transaction features, performing equivalence class division and boundary value analysis on all the transaction features, combining the equivalence classes and boundary values ​​to obtain a number of transaction feature combinations, and generating test cases covering all the transaction feature combinations to obtain a test case set; Step S2: performing abnormal identification on the execution results of the test case set, clustering the identified abnormal cases, and obtaining an abnormal scenario set; Step S3: According to the whole process of securities trading, the scenarios of price boundary values ​​not covered, market types not covered, transaction types not covered, and test paths not covered by the test case set are identified to obtain an uncovered scenario set; Step S4: Generate a new test case set based on the uncovered scenario set and the abnormal scenario set; Step S5: Repeat steps S2 to S4 until the set termination condition is reached, summarize the test results, and generate a test report.

[0007] Preferably, after the test case set is generated in step S1, all test cases in the test case set are scored and prioritized, including the following steps: Step S11: The risk level, business importance, fault impact and historical failure rate of the test case are used as evaluation indicators to calculate the evaluation index values ​​of all test cases; the risk level is the risk level of the scenario, the business importance is the importance of the scenario to the transaction business, the fault impact is the impact on the user experience or the transaction system when the test case fails, and the historical failure rate is the failure probability of the test case in the historical records; Step S12: adopting the hierarchical analysis method, taking the priority ranking of test cases as the target layer, taking the evaluation index as the criterion layer, taking the set of test cases to be evaluated as the solution layer, and constructing a hierarchical structure model; Step S13: construct a pairwise comparison matrix of evaluation indicators, calculate the relative weights of all evaluation indicators according to the pairwise comparison matrix, and perform consistency check on all relative weights; Step S14: Calculate the comprehensive score of each test case according to the relative weight of the evaluation index and the evaluation index value of the test case; Step S15: Prioritize all test cases in descending order of the comprehensive scores.

[0008] Preferably, in step S2, abnormal use cases are identified based on the execution results of the test cases, which include functional indicators, performance indicators, error indicators and behavioral indicators, and the means of all indicators are calculated respectively, and the positive and negative three times standard deviation of the indicator mean are used as the baseline value of the indicator, and the test cases whose indicator values ​​exceed the baseline value are regarded as abnormal use cases.

[0009] Preferably, the functional indicators include the execution status of the test case, the reason for execution failure and the test coverage; the performance indicators include the response time, resource utilization and throughput of the system when executing the test case; the error indicators include the exception type and crash frequency of the system when executing the test case; and the behavioral indicators include the number of repeated executions and execution time of the test case.

[0010] Preferably, step S4 comprises the following steps: Step S41: performing data cleaning and standardization on the execution results of all test cases; Step S42: converting uncovered scenes and abnormal scenes into feature vectors, randomly generating a number of noise vectors as a training set for a generator of a generative adversarial network (GAN), wherein the generator outputs a number of virtual test cases corresponding to the noise vectors; Step S43: using the plurality of virtual test cases and the feature vectors as a training set for a GAN discriminator, wherein the discriminator outputs a probability that the input test case is a real sample; Step S44: updating the parameters of the generator according to the discrimination result of the discriminator, and generating new test cases using the trained generator; Step S45: perform deduplication operation on the newly generated test cases, and score and prioritize the deduplicated test cases.

[0011] Preferably, the generator of the generative adversarial network GAN includes several hidden layers and an output layer, the randomly generated noise vector is input into the several hidden layers for linear transformation, the activation output of the hidden layer is calculated using the activation function, the activation output is passed to the output layer, and the test case is generated; the discriminator of the generative adversarial network GAN includes two hidden layers and an output layer, the test cases generated by the generator and the real test cases are input into the two hidden layers for linear transformation, the activation output of the hidden layer is calculated using the activation function, the activation output is passed to the output layer, and the probability that the input test case is a real test case is obtained.

[0012] Preferably, in step S44, a reinforcement learning algorithm or an adaptive generation algorithm is used to optimize the generation strategy of the GAN generator, wherein the reinforcement learning algorithm rewards test cases with high coverage or high success rate, so that the generator gives priority to generating test cases with high coverage and high success rate; and the adaptive generation algorithm adjusts the generation strategy according to the failure rate of the test cases, giving priority to generating boundary scenario cases with high coverage and high failure rate.

[0013] Preferably, in step S45, by calculating the similarity between the newly generated test cases and the existing test cases, the test cases in the newly generated test cases whose similarity is greater than a set similarity threshold are deleted, thereby achieving deduplication of the newly generated test cases.

[0014] Preferably, the expression for scoring the test cases after deduplication in step S45 is: ; In the formula, Score the test cases; coverage is the scene coverage; boundary is the boundary value; failure is the historical failure rate; The scene coverage 、 Boundary value, weight of historical failure rate.

[0015] Preferably, in step S1, an orthogonal experimental design method is used to combine equivalence classes and boundary values ​​to obtain several transaction feature combinations.

[0016] The benefits of the present invention include at least: 1. Analyzing boundary values ​​helps capture situations that may cause errors on the boundaries of the input domain. Equivalence class partitioning ensures that each input domain is represented and tested, increasing the possibility of discovering defects. 2. By continuously identifying uncovered and abnormal scenarios and generating new test case sets based on these scenarios, the testing process can improve itself and gradually improve the coverage and quality of the test; 3. Updating test cases based on the latest test results at each iteration can ensure that the test cases are always consistent with the latest status of the system and can adapt to real-time changes in the securities trading system. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a method flow chart of an embodiment of the present invention; Figure 2 4 is a system framework diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0019] like Figure 1 As shown, an embodiment of the present invention provides a method for automated testing of a securities trading system based on result feedback, comprising the following steps: Step S1: taking the account type, market type, transaction type and price range in the securities transaction request as transaction features, performing equivalence class division and boundary value analysis on all transaction features, combining equivalence classes and boundary values ​​to obtain several transaction feature combinations, and generating test cases covering all transaction feature combinations to obtain a test case set.

[0020] Specifically, the driver data file input by the tester contains the core features of each test case, such as the scenario name, use case type, i.e. positive example or negative example, market type, transaction category, verification level, expected results, etc. The file format is JSON.

[0021] The analysis of driver data is crucial in the entire solution, as it determines the coverage of test cases and the rationality of the test strategy. First, read the driver data file, and query the corresponding use cases from the counter based on these feature data, such as fund account, maximum buy and sell volume, current price, price limit, etc., perform structured analysis on them, and then extract them as feature data to create a specific scenario description for each test case. For example, if the scenario name is "Limited Price Trading in City A", the feature data extracted after analysis may include: Account permissions: ordinary user, VIP user, administrator.

[0022] Stock code: multiple A-share stock codes.

[0023] Price range: high, low, middle.

[0024] Asset types: cash, margin trading, etc.

[0025] The parsed feature data is then preprocessed for standardization to ensure that it meets the input requirements of the model. The standardized feature data is mapped to the use case generation model to form a basic data set, which will serve as the initial input for generating use cases.

[0026] Based on the feature data obtained through analysis, different algorithms are used to generate different test cases to cover various possible scenarios. Through the permutations and combinations of different features, test cases containing different feature combinations and covering different scenarios are generated. For example, the price limit trading scenario in City A may include the following combinations: Account permissions: ordinary account, VIP account, institutional account.

[0027] Stock code: Select stocks from different sectors, such as main board and science and technology innovation board stocks.

[0028] Price: different price ranges, such as high price, low price and limit price, etc.

[0029] Assets: Different asset mixes, including high-asset and low-asset users.

[0030] The embodiment of the present invention adopts full combination, orthogonal experimental design and equivalence class division method to generate different test cases.

[0031] A. Full Combination If the number of features is small, use the Cartesian product to generate all combinations to ensure comprehensive coverage of use cases. Cartesian product means that for multiple sets, each element in each set is combined with each element in other sets to form a new set. Suppose there is a set A , B and C , which contain the elements: ; The Cartesian product All combinations of are: .

[0032] Assume the following feature data: Account type: Normal user (Normal), VIP user (VIP).

[0033] Market type: A shares (A), B shares (B).

[0034] Transaction type: Buy, Sell.

[0035] First, define all features as separate lists. Then use the itertools.product function to generate all possible combinations of account type, market type, and transaction type, ensuring that every case is covered. There are 2 account types, 2 market types, and 2 transaction types, for a total of combinations. The output is: ; ; ; ; ; ; ; .

[0036] The value of each combination is mapped to the input parameter in the test case to generate a specific test case.

[0037] B. Orthogonal Experimental Design For multiple feature data, such as more than 5, in order to avoid the "combination explosion" problem caused by the full combination of all features, orthogonal design is used to control the number of tests and cover the main combinations. Orthogonal experimental design uses an orthogonal array to arrange experiments. Each row represents a test case, each column represents a test factor, that is, a feature, and the value in each cell represents the level of the factor under a specific combination, that is, the value.

[0038] First, list all the factors to be tested and determine the number of levels for each factor. Then, according to the number of factors and levels, select the corresponding orthogonal array. The representation method of the orthogonal array is ,in Indicates that the table contains N OK, that isN Test cases; s Indicates the number of levels of each factor; k Indicates the number of factors.

[0039] For example: Represents an orthogonal array with 9 rows, 3 factors, and each factor has 3 levels.

[0040] Map each column in the orthogonal table to the test factor, and fill the value of each factor into the corresponding position in the orthogonal table. Finally, generate the test case. Each row in the orthogonal table represents a test case, and the combination of all rows is the set of test cases that need to be executed.

[0041] For example, the use case set to be generated has the following three factors: (1) Account type: Normal, VIP, Institutional; (2) Market type: A shares (A), B shares (B), ChiNext (C); (3) Transaction type: Buy, Sell, Cancel.

[0042] Each factor has 3 levels, so choose one An orthogonal table is used to generate the combination of main scenes.

[0043]

[0044] Map the orthogonal table to specific factor values ​​and assign specific values ​​to each factor: (1) Factor 1 (account type): 1=Normal, 2=VIP, 3=Institutional; (2) Factor 2 (market type): 1=A, 2=B, 3=C; (3) Factor 3 (transaction type): 1=Buy, 2=Sell, 3=Cancel.

[0045] According to the above mapping relationship, the orthogonal table is converted into a specific test combination:

[0046] The orthogonal experimental design avoids generating all 3×3×3=27 combinations and instead covers the main interaction situations with 9 test cases.

[0047] C. Equivalence Classification The input data domain is divided into several equivalence classes. The data in each equivalence class is considered to be equivalent to the test effect, so one or more representative values ​​can be selected.

[0048] First, determine the scope of the test objectives and input data. For example, the use case set has the following input conditions: (1) Transaction amount: ranges from 1 to 1,000,000, in RMB; (2) Stock code: must be 6 digits; (3) Account type: Normal account, VIP account, or Institutional account.

[0049] Divide the input data domain into equivalence classes to find valid and invalid equivalence classes. At the same time, identify the boundaries of each equivalence class and add more boundary test cases through boundary value analysis. For example: Equivalence class division: (1) Transaction amount: Valid equivalence classes: [1,1,000,000]; Invalid equivalence classes: values ​​less than 1, such as 0, -1, and values ​​greater than 1,000,000, such as 1,000,001.

[0050] (2) Stock Code: Valid equivalence class: 6-digit numbers, such as 000001, 123456; Invalid equivalence class: non-6-digit numbers, such as 12345, 1234567, containing letters or special characters, such as AB1234, 123#56.

[0051] (3) Account type: Valid equivalence classes: Normal, VIP, Institutional; Invalid equivalence class: non-specified type, such as Guest, Admin.

[0052] Boundary Value Analysis: (1) Transaction amount: Minimum value = 1, inside the boundary: 1, outside the boundary: 0; Maximum value = 1,000,000, inside boundary: 1,000,000, outside boundary: 1,000,001.

[0053] (2) Stock Code: Minimum value: 000001, outside the boundary: 5 digits, such as 12345; Maximum value: 999999, outside the boundary: 7 digits, such as 1234567.

[0054] Then the orthogonal test method is used to generate test cases:

[0055] In test scenarios with clear input ranges, all valid and invalid scenarios can be covered and problems can be effectively captured by reasonably dividing equivalence classes and analyzing boundary values.

[0056] After the test case set is generated, all the test cases in the test case set are scored and prioritized.

[0057] Specifically, the embodiment of the present invention adopts the Analytic Hierarchy Process (AHP) to construct a hierarchical model and comprehensively score each test case according to factors such as the risk, business importance, fault impact, and historical failure rate of the test case, so as to perform the test according to the priority. The following steps are included: Step S11: define evaluation indicators.

[0058] The factors influencing the test case prioritization are determined and broken down into the following indicators, which constitute the first-level indicators in the hierarchical analysis method: Risk (R): The risk level of the scenario, determined through business analysis and assessment; Business Importance (I): The core importance of the scenario to the system business. Scenarios that affect key business processes have a higher weight. Failure impact (F): the impact of test failure on user experience or the overall system; Historical failure rate (H): The probability of failure of a test case in historical records.

[0059] Step S12: construct a hierarchical structure model.

[0060] Construct the hierarchical structure of AHP and divide the problems into three levels: Target layer: Prioritization of test cases; Criteria layer: risk level, business importance, failure impact, historical failure rate; Scenario layer: the set of test cases to be evaluated.

[0061] Step S13: Determine the weight of each indicator.

[0062] Through expert scoring and data analysis, a pairwise comparison matrix (PairwiseComparison Matrix) between criterion-level indicators is constructed.

[0063] For example, there are four indicators: risk (R), business importance (I), failure impact (F), and historical failure rate (H):

[0064] Constructing a pairwise comparison matrix A : .

[0065] Divide each column's data by the total sum of that column, and add A Each element of is standardized, and the calculation formula is: ; in: It is a standardized element; is a matrix A Elements in For the matrix A Middle j Column No. k elements.

[0066] Then the matrix A Sum each column of : .

[0067] Normalize the matrix by its column sums: .

[0068] Calculate the average value of each normalized row to get the weight of each indicator. The calculation formula is: ; in: are the elements in the normalized matrix; n is the dimension of the matrix, i.e. the number of indicators.

[0069] According to the normalized matrix , calculate the mean for each row:

[0070] For example: Average of row 1: ; Average of row 2: .

[0071] And so on, we finally get the weight vector: , that is, risk level (R): 40%; business importance (I): 30%; failure impact (F): 20%; historical failure rate (H): 10%.

[0072] Perform consistency check on all weights to verify the rationality of the weights: ; ; ; In the above formula, is the maximum eigenvalue of the matrix; is the element in matrix A; is the calculated weight vector; is the consistency indicator; is the consistency ratio; is a random consistency indicator.

[0073] The random consistency index can be selected according to the following table. n Different dimensions correspond to different values:

[0074] if , the weights are considered to meet the consistency requirements.

[0075] Step S14: Calculate the comprehensive score of each test case according to the relative weight of the evaluation index and the evaluation index value of the test case.

[0076] Specifically, through the above steps, the weight vector of each indicator can be obtained: , and ensure its logical consistency. The weight vector will be used to comprehensively score the test cases and guide the test priority sorting. Based on the weight vector and the actual indicator value of the test case, the comprehensive score of each test case can be calculated.

[0077] For example, the indicator values ​​of the test cases to be evaluated are as follows:

[0078] First, normalize all indicator values ​​to the range of 0-1, assuming that the range of all indicators is 0-10: ; In the formula, is the normalized index value; is the original indicator value; =10, =0.

[0079] The calculation results of the evaluation indicators of each test case are:

[0080] The weight vector W=[0.40, 0.30, 0.20, 0.10], and the comprehensive score formula is used to calculate the scores of each evaluation indicator:

[0081] Step S15: Prioritize all test cases in descending order of the comprehensive scores.

[0082] Specifically, all test cases are prioritized according to the comprehensive scores. The higher the score, the higher the priority. The ranking result is used as the initial execution plan. Test cases are assigned to idle devices for execution. Test case 3, test case 2, and test case 1 are executed in sequence. Each time a test case is executed, the remaining test cases in the set of test cases are adjusted according to the result feedback of the executed test cases until the condition for terminating execution is triggered.

[0083] During the test execution, the monitored indicators can be divided into the following categories: A. Functional indicators By analyzing the running log, the functional execution status of the test case is obtained, including: (1) Execution status: whether it is successful or not, including success or failure; (2) Cause of failure: such as assertion failure or exception thrown; (3) Test coverage: whether the current test case covers a specific functional module or path in the system.

[0084] B. Performance indicators The system response time, throughput, resource utilization, etc. are obtained through performance monitoring tools. Performance indicators reflect the performance of the system during the execution of test cases, including: (1) Response time: The time it takes for the system to respond when processing a request, such as API call time and page loading time; (2) Resource usage: CPU, memory, disk, network and other resource usage when the system executes the test; (3) Throughput: the number of test tasks processed per unit time.

[0085] C. Error indicators The automated testing tool Airtest is used to obtain operation errors, crashes, assertion failures, and test case execution results, which reflect the stability and error conditions of the system or test cases, including: (1) Runtime error: Capture the exception type when the system executes the test; (2) Crash frequency: the number of times the system or module crashes during test execution; (3) Assertion failure: The expected value of the use case fails to match the actual value.

[0086] D. Test behavior indicators The automated testing tool Airtest is used to obtain the number of test case executions and execution time, which reflects the behavioral characteristics of the test cases, including: (1) Number of repeated executions: the number of times the use case is repeatedly executed; (2) Use case execution time: The overall execution time of a single use case, that is, the total time from the start to the end of executing a single use case.

[0087] The monitored indicators need to be preprocessed, feature extracted and standardized before they can be used as input for the optimization model. The feature extraction process is as follows: A. Functional indicators (1) Execution status: Convert the execution status into binary features: success = 1, failure = 0; Add classification features for failure reasons: assertion failure, timeout, invalid input, etc.

[0088] (2) Test coverage: Establish a mapping relationship between functional requirements and test cases, clarify the functional requirements covered by each test case, and quickly identify the blank areas of the test cases by mapping the uncovered functional requirements in the table. After each round of testing, count the change in coverage, where the calculation expression for functional coverage is: .

[0089] B. Performance indicators All performance indicators are normalized, and indicators such as response time and resource utilization are standardized to [0,1]. The normalized expression is: ; In the formula, is the normalized index value; is the original indicator value; is the maximum value among the original indicator values.

[0090] C. Error indicators Perform text feature extraction on error logs and use keyword matching to extract common error categories.

[0091] D. Test behavior indicators (1) Number of repeated executions: counted by use case number; (2) Use case execution time: Normalization processing to reduce numerical differences.

[0092] The data standardization process is as follows: Use the Min-Max normalization method to perform data standardization. Transform the data linearly to the interval [0,1], where the minimum value Mapped to 0, the maximum value The values ​​in between are scaled to be between 0 and 1.

[0093] The data processed through the above steps can be input into the use case iteration optimization module as feedback.

[0094] Step S2: Identify anomalies in the execution results of the test case set, cluster the identified abnormal cases, and obtain an abnormal scenario set. This includes the following steps: Collect various indicators during the execution of test cases, set the baseline values ​​of the indicators, use the anomaly detection algorithm to mark abnormal cases, classify abnormal situations through the rule engine, and obtain the abnormal scenario set.

[0095] Specifically, it automatically detects and diagnoses anomalies during the execution of test cases and generates a failure mode analysis report. By monitoring various performance indicators, such as response time, memory usage, network latency, etc., and analyzing abnormal patterns, it helps testers quickly locate problems, thereby improving test efficiency. It includes the following steps: Step S21: Data collection and preprocessing.

[0096] Collect various performance indicators during the execution of the use case, including response time, memory usage, CPU usage, network latency, etc. Preprocess this data into normalized values ​​for further analysis.

[0097] Step S22: Baseline setting and anomaly detection.

[0098] The normal range of performance indicators, i.e., baseline values, is set. Based on the assumption of normal distribution, data points outside the mean ± 3 times the standard deviation are considered abnormal.

[0099] Anomaly detection algorithms are used to identify indicators that exceed the normal range. In the embodiment of the present invention, the The score is calculated and the threshold is set to 3, and the data points exceeding the set threshold are marked as abnormal. The score is calculated as: ; in, is the current value; is the baseline mean; is the standard deviation.

[0100] Step S23: Fault classification and pattern analysis.

[0101] Analyze the failed use cases, extract the core reasons for the failure, such as illegal input parameters, logical errors, etc., and classify the error types, such as boundary value problems, null pointer exceptions, etc. Use the rule engine to classify the abnormal situations and classify them into specific fault types such as response timeout, memory leak, network failure, etc. For example: if the response time continues to exceed the standard, it is judged as "response timeout". If the memory usage rate gradually increases, it may be "memory leak".

[0102] The abnormal modes are then analyzed to generate a failure mode report containing the following information: Involved modules: Specific modules involved in the exception, such as database, network modules, etc.

[0103] Possible causes: Speculate possible causes based on the type and pattern of anomalies. For example, network delays may be due to insufficient bandwidth or slow server response.

[0104] Abnormal impact: Analyze the possible impact of abnormalities on the system or users and assess the risk level.

[0105] Step S3: According to the whole process of securities trading, identify the scenarios where the price boundary value, market type, transaction type and test path of the test case set are not covered, and obtain the uncovered scenario set. The following steps are included: Step S31: Processing feedback data.

[0106] (1) Data cleaning: Remove outliers: For example, remove invalid response times, i.e. negative values, or missing execution results; Missing value filling: Fill missing performance indicators with average values, default values, or historical data.

[0107] (2) Feature extraction: Extract key features from feedback data: Use success / failure status as categorical features, response time as continuous features, and error type as textual features.

[0108] For example:

[0109] (3) Data standardization: Normalize features of different dimensions, such as coverage and response time, to the range of [0,1].

[0110] Step S32: Analyze the coverage of existing test cases, find the functional modules or parameter spaces not covered by the existing test cases, and extract the uncovered areas based on the code coverage tool and the requirement matrix analysis.

[0111] For example, the feature vector could be: Use case parameter characteristics: input range, type, default value.

[0112] Execution result characteristics: whether it passed, failure point, and log keywords.

[0113] The uncovered area is defined as: (1) Parameter space is not covered: For example, the boundary values ​​of the price range or some special stock codes are not covered. Assume that the price range is [90,110], the tested prices are [91,95,105], and the uncovered prices are: lower limit 90 and upper limit 110; (2) Functional scenarios not covered: scenarios where functional modules are not tested, for example, in the order cancellation module, the failure scenario of limit order cancellation is not covered. Assume that the demand matrix is ​​as follows:

[0114] (3) Path not covered: Some branches in the code execution path are not triggered, for example: the transaction timeout processing logic caused by network anomalies is not tested. For example, the code snippet is as follows: if (price>upperLimit) { throw new IllegalArgumentException("Price exceeds upper limit"); } else if (price <lowerLimit) { throw new IllegalArgumentException("Price below lower limit"); }else { executeTrade(price, quantity); } Collecting code coverage using a code coverage tool or a requirements-use case mapping matrix can extract uncovered areas as follows: Branch: price<lowerLimit.

[0115] Scenario: The test price is lower than the lower limit, for example, 89 yuan.

[0116] Based on the above-set uncovered area, the uncovered area is identified, and the uncovered area identification result is obtained as follows:

[0117] Step S4: Generate a new test case set based on the uncovered scenario set and the abnormal scenario set.

[0118] Specifically, the uncovered areas, failure scenarios, etc. of existing test cases are mined based on feedback data, and new test cases with high testing value are generated through specific algorithms and training processes.

[0119] The input of this process is: {feedback data, i.e., processed and standardized data; existing use case set, including parameter range, input type, and scenario combination of use cases; coverage, i.e., uncovered functional modules, parameter space, and scenario combination}.

[0120] The output is: {a newly generated set of test cases; new test scenarios, i.e., combinations of test cases for uncovered scenarios and failed scenarios}.

[0121] Select a generation strategy based on the types of uncovered scenarios and failed scenarios. The generation strategies include: Orthogonal experimental design: used to generate multi-parameter combinations that can cover most scenarios.

[0122] Boundary value analysis: Generate boundary values ​​for scenarios that fail or are not covered in the feedback.

[0123] Machine learning generation: Use feedback data to train models to generate more complex use cases.

[0124] The core of machine learning to generate complex use cases is as follows: Generate new test cases using a generative adversarial network (GAN). GAN learns the distribution of use case data and generates new use cases through adversarial training of the generator G and the discriminator D. In a generative adversarial network (GAN), the generator G and the discriminator D are two core components. The generator G is responsible for generating new test cases, while the discriminator D is responsible for distinguishing between these generated use cases and real use cases.

[0125] Generator G: Generator G is a neural network model that generates a random noise vector according to the input Generate new use cases. Its goal is to make the generated use cases as close to the real use cases as possible, making it difficult for the discriminator D to distinguish between the real and generated use cases.

[0126] The input of the generator G is a random noise vector , the random noise vector is usually drawn from a standard normal distribution N (0,1), or from a uniform distribution. The random noise vector provides diversity, allowing the generator to generate a variety of different use cases. For example: The input of the generator is: the length of the random noise vector is 3, and the value is z=[0.1,-0.3,0.5], corresponding to the price, quantity and transaction type respectively.

[0127] The weight matrix of the hidden layer and the bias vector for: ; ; The weight matrix of the output layer and the bias vector for: ; ; Given an input noise vector After that, it is transformed through the linear transformation of the hidden layer to obtain the output of the hidden layer that is not activated , including the following steps: The generator includes multiple hidden layers corresponding to the dimensions of the noise vector, and the linear combination of neurons in each hidden layer is calculated separately: The first hidden layer neuron: .

[0128] Second hidden layer neurons: .

[0129] The third hidden layer neurons: .

[0130] The linear combination output of the hidden layer is calculated as: .

[0131] Using the ReLU activation function Calculate the activation output of the hidden layer : .

[0132] The output of the hidden layer Passed to the output layer, the linear combination formula of the output layer is: .

[0133] After getting all the neurons in the hidden layer, calculate the output parameters of the generator: , rounded to 92.

[0134] , rounded to 1000.

[0135] Set 0 to buy at market price and 1 to buy at limit price: , rounded to get the transaction type of 1, which means limit buy.

[0136] That is, the generator output is: price = 92, quantity = 1000, transaction type = limit buy.

[0137] Discriminator D: Discriminator D is a binary classification model used to distinguish whether the input sample is a real case or a generated case. The goal of the discriminator is to output 1 for a real case and 0 for a generated case.

[0138] The input of the discriminator D is the use case parameter. The discriminator accepts use cases from real datasets or use cases generated by the generator as input. For example: The input features of the discriminator are: price ,quantity , transaction type code is . Suppose there are 2 neurons, corresponding to two classification types, and both neurons use the ReLU activation function.

[0139] The weight matrix of the hidden layer and the bias vector for: ; .

[0140] The weight matrix of the output layer and the bias vector for: ; .

[0141] Given an input feature vector: .

[0142] The calculation formula for the inactive output of the hidden layer is: .

[0143] Using the ReLU activation function Calculate the activation output of the hidden layer : .

[0144] The output of the hidden layer Passed to the output layer, the linear combination formula of the output layer is: .

[0145] Using the Sigmoid activation function, the output of the output layer is mapped to Range, indicating the probability of discrimination, the formula is: .

[0146] Substitution , calculate the output probability of the discriminator: .

[0147] That is, the output of the discriminator is 0.168, which means that the discriminator believes that the probability that the input case is a real sample is 16.8%, and the probability is closer to 0, indicating that the model believes that this case may be a fake sample generated by the generator.

[0148] Among the test cases generated by the Generative Adversarial Network (GAN), some of them may be redundant, that is, highly similar to existing cases, or of low value, that is, they cover fewer scenarios or have been tested multiple times. The purpose of the screening and optimization steps is to improve the test efficiency and give priority to retaining and executing high-value cases that cover key scenarios or uncovered areas. The following steps are included: (1) Remove redundant use cases Use cosine similarity or Euclidean distance to calculate the similarity between the generated use case and the existing use case, and filter out highly similar use cases. For example, if the generated use case "price = 92, quantity = 1000, transaction type = limit buy" is the same as the existing use case, then the use case is judged to be redundant.

[0149] The calculation formula for cosine similarity is: .

[0150] In the formula, , is the feature vector of the use case, including price, quantity, transaction type, etc. n is the dimension of the feature vector.

[0151] If the similarity exceeds a set threshold, which is set to 0.95 in the embodiment of the present invention, the use case is marked as redundant.

[0152] Determine whether the use cases cover the same scenarios. For example, if the transaction type, price range, and quantity range of multiple use cases are exactly the same, one or more of them can be considered redundant use cases.

[0153] (2) Use case scoring The scoring criteria for the use cases are: Scenario coverage: Prioritize covering untested scenarios.

[0154] Parameter boundary values: It is more important to test use cases that contain parameter boundary values.

[0155] Historical failure rate: Scenarios with high historical failure rates have higher priority.

[0156] Use Case Scores The calculation formula is: .

[0157] In the formula, coverage is the scenario coverage, which indicates the ratio of scenarios covered by the use case; boundary Whether the parameter is a boundary value, the boundary value weight is usually higher; failure The historical failure rate is used to increase the priority of use case scenarios that have failed; They are coverage、boundary , failure The weight of .

[0158] (3) Use case prioritization The generated use cases are sorted according to their scores, and use cases with higher scores are executed first.

[0159] Assume that the GAN generates the following use case:

[0160] Based on the score, G2 is executed first, followed by G3, and then G1.

[0161] Step S5: Repeat steps S2 to S4 until the set termination condition is reached, summarize the test results, and generate a test report.

[0162] Specifically, as the test cases are gradually optimized, the system will automatically generate a failure mode analysis report, including detailed information such as the modules involved and possible causes, so as to narrow the scope of problem location for testers and improve efficiency. By analyzing the current test results, the test results of the generated use cases are fed back to GAN, thereby improving the generation of future use cases. This process enables the model to gradually generate use cases with higher coverage and more representativeness. It includes the following steps: Step S51: Collect feedback data, including the execution results, coverage, and error types of the use cases.

[0163] Execution results: Record the execution results of each use case, success or failure.

[0164] Coverage: Collect the covered scenes and uncovered scenes.

[0165] Error type: record the error information of failed use cases, such as boundary value errors, logical errors, etc.

[0166] Step S52: Extract and analyze features of the feedback data, identify uncovered areas of the use cases, and analyze failed use cases.

[0167] Uncovered area: Find out the parameter combinations and scenarios that are not covered by the current use case and use them as the target area for generating use cases in the next step.

[0168] Failure case analysis: By analyzing failed cases, we can extract the features that lead to failure and help the generator generate more targeted test cases.

[0169] Step S53: Use the feedback data to update the parameters of the GAN model.

[0170] The GAN generator learns new uncovered scenarios through feedback, and the discriminator is retrained based on the success / failure labels of the feedback to help the generator generate more effective use cases.

[0171] The optimization algorithm used is reinforcement learning or adaptive generation algorithm.

[0172] The reinforcement learning algorithm Q-learning is used to reward use case generation behaviors with high coverage or success rates. Assume that the generator generates a use case of "price = 105, quantity = 1000, transaction type = market buy", and if the execution feedback of this use case is: covering new scenarios and successful execution. Then, according to the coverage and success rate of this use case, a positive reward is given: the generator increases the priority of generating market buy and use cases with a price near 105.

[0173] After multiple iterations, the reinforcement learning algorithm will gradually optimize the generator's generation strategy so that it prioritizes the generation of use cases with high coverage and high success rate.

[0174] Adaptive generation algorithms are used to adjust generation strategies based on failure rates, giving priority to boundary values ​​and high-risk scenarios. Adaptive generation algorithms use probability models or parameter adjustment methods to optimize the generator.

[0175] Assume that each parameter has a set of generation probabilities. If a certain parameter combination produces a valid use case, the probability of the combination is increased. Use the multi-armed bandit algorithm to update the generation rules in real time. The probability of choosing a random rule is The probability of selecting the optimal rule is: ; In the formula, To generate rules; The rule for optimal feedback effect; n is the number of generated rules; is the exploration coefficient.

[0176] After multiple feedback adjustments, the adaptive generator will automatically prioritize the generation of boundary scenario use cases with high coverage and high failure rates, thereby improving test coverage and error detection rates. Suppose the generator generates a use case: price = 92, quantity = 5000, transaction type = limit buy. If the use case covers the new scenario but triggers a failure, the cause of the failure may be a boundary value problem. Then perform an adaptive update on it: increase the probability of generating use cases near the price range of 92, and increase the probability of generating use cases near boundary values, such as when the quantity value is close to 5000.

[0177] By continuously improving and adjusting the test case set during the execution of the use case set, we ensure that the test cases are optimized layer by layer based on the feedback data and dynamically generate new scenarios, so as to finally achieve the goal of fully covering the test requirements with a small set of use cases. For example: if the result feedback after the use case execution shows that the use case is successfully executed, the use case iteration optimization module will automatically identify redundant use cases and exclude duplicate use cases with similar characteristics to reduce test duplication and improve efficiency. If the use case fails, the failure cause is analyzed, the failure type is correlated with other use cases, the priority of related use cases is automatically adjusted, and more detailed test cases related to the failed use cases are gradually generated. When a specific transaction process fails, the module can generate more variant test cases to verify whether different operation paths have the same problem until all error paths are covered. When all test cases are executed successfully or the set coverage threshold is reached, the system will automatically stop generating new use cases and end the test process.

[0178] like Figure 2 As shown, an embodiment of the present invention further provides an automated testing system for a securities trading system based on result feedback, including a use case generation module, a use case execution and feedback module, a use case iteration optimization module and an abnormality diagnosis module.

[0179] The use case generation module is used to read and parse the driver data file input by the tester, extract the features in the driver data file, generate several test case sets covering different feature combinations, and score and prioritize all test cases.

[0180] The use case execution and feedback module executes test cases in order of priority, collects various indicators and result data during the use case execution process, and generates feedback data based on the execution results.

[0181] The use case iteration optimization module identifies uncovered scenarios in the test case set based on feedback data, generates new test cases, scores and prioritizes the newly generated test cases, and feeds the newly generated test cases back to the use case execution and feedback module.

[0182] The anomaly diagnosis module is used to collect various indicators during the execution of test cases, set the baseline values ​​of the indicators, use the anomaly detection algorithm to mark abnormal use cases, classify the abnormal situations through the rule engine, and feed back the classified anomaly information to the use case iteration optimization module to improve the use case generation strategy.

[0183] When actually executing an automated testing method for a securities trading system based on result feedback provided by an embodiment of the present invention, the following steps are included: Step 1: The tester inputs the driver data, parses the driver data file to extract feature data, and then uses the extracted feature data as the input of the use case generation module to generate a set of limit price trading use cases in City A.

[0184] The driving data is generally input in the form of a file. The driving data file contains the scenario name of this use case set, such as limit price trading in City A, and the characteristic data of each use case in the use case set, such as positive or negative examples, market type, transaction category, verification level, expected results, etc.

[0185] Suppose 200 use cases are generated in the collection, for example, the use cases cover different account permissions, stock codes, prices, assets, etc.

[0186] Step 2: Use the AHP method to score the importance of each test case, and then prioritize each test case according to the final score.

[0187] The trained benchmark data has been stored in the database. Testers can modify these weight data in advance according to test requirements and actual conditions, but the weight data is static during the test execution.

[0188] Step 3: Execute the use cases in order of priority, collect the execution results, page refresh time, call interface return time, memory consumption and other indicators during the use case execution, and judge whether the use case execution results are abnormal based on these indicators. If the page jump of each step is normal, all performance indicators are normal, and the execution results are consistent with the expected results, then the use case execution is judged to be successful, otherwise the use case execution failure is fed back, and all collected test results are sent to the abnormal diagnosis module for abnormal diagnosis.

[0189] Step 4: If the previous use case feedback is executed successfully, the use case generation module will mark all redundant use cases that are highly similar to this use case as redundant use cases and filter them out, and then reorder the remaining use cases in the set and execute them on demand. If the previous use case feedback fails to execute, all indicator information and execution status results collected during the use case execution will be passed to the abnormal diagnosis module.

[0190] For example, if the positive example with stock code 000001 and price 10 is successfully executed, all use cases for verifying the transaction function with stock code of Shenzhen Main Board and price within the allowed entrustment range will be marked as redundant use cases.

[0191] Step 5: After receiving the information, the abnormal diagnosis module first classifies the abnormalities, such as the page does not jump normally, UI changes, etc., the execution result does not meet expectations, the front-end data is inconsistent with the transaction data stored in the back-end, the normal order process finally shows that the order fails, etc., a certain interface call returns a timeout, the page refresh timeout, etc. Then, according to the diagnosis rules, the abnormalities are carefully diagnosed, and the abnormalities that affect the subsequent execution of the entire test are repaired or advanced notifications such as emails and phone calls are made. For example, if the mobile phone network is disconnected, it can be repaired by reconnecting to the network or restarting the device under test. If the counter access fails, it can be repaired by telephone or email notifications. Then the diagnosis results and repair results are fed back to the use case generation module.

[0192] Step 6: After receiving the use case execution exception and diagnostic information, the use case generation module determines the impact scope of such exception based on the diagnostic content and regenerates a new use case for the use case set of the exception.

[0193] For example, if the price input box cannot be detected during the normal order process for the stock code 000001, it is judged as a UI change. The error use case set for this exception includes all use cases that need to operate the price input box to determine the impact range of the UI change. After obtaining the error type and impact range, if it is due to normal changes in the development process, the tester adjusts the UI element feature information in the use case generation model according to the test change range, otherwise the development needs to be notified to locate the source of the problem. If the abnormal diagnosis module feedback use case calls the available funds interface during execution and returns a timeout, the intelligent use case generation model generates other test cases that call this interface to form a use case set to determine the frequency of timeouts, and generates call cases for other interfaces related to funds to determine the impact range of the timeout interface.

[0194] Step 7: If the use case set has been filtered to the optimal set and executed, it is considered that the conditions for starting and terminating execution have been met, and the results of this round of testing are summarized and a report is generated.

[0195] An automated testing method for a securities trading system based on result feedback provided by an embodiment of the present invention can intelligently identify abnormal, uncovered or weak test areas, and generate corresponding test cases. Through the result feedback mechanism, the system can continuously correct test cases, eliminate redundant tests, optimize test paths, and ensure that the test case set is always in the optimal state, thereby improving the overall test efficiency. The system automatically detects anomalies that occur during the test process through feedback analysis, and automatically generates targeted test cases to narrow the scope of the problem, greatly shortening the time for testers to locate the problem and significantly improving the test efficiency. At the same time, the system can generate test cases for scenarios such as high-frequency trading and fast buying and selling operations based on result feedback, helping the securities trading system to cope with sudden situations such as rapid changes in the securities market and market fluctuations. Through multi-platform test feedback and differentiated comparative analysis, the system can generate test cases for different platforms to ensure that the user experience of the securities trading system on multiple platforms is consistent.

[0196] In summary, the intelligent test case generation mechanism based on result feedback makes the test process highly automated, reduces the need for manual writing and intervention by testers, and reduces labor costs. It can adjust the priority and test strategy of test cases based on feedback data, avoids the waste of test resources, and improves the utilization of test resources.

[0197] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. Only the preferred embodiments of the present invention are expressed. The description is more specific and detailed, but it cannot be understood as limiting the scope of the present invention. As long as there is no contradiction in the combination of these technical features, they should be considered as within the scope of this specification.

[0198] It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these modifications and improvements all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A method for automated testing of a securities trading system based on result feedback, characterized in that: The following steps are involved: Step S1: taking the account type, market type, transaction type and price range in the securities transaction request as transaction features, performing equivalence class division and boundary value analysis on all the transaction features, combining the equivalence classes and boundary values ​​to obtain a number of transaction feature combinations, and generating test cases covering all the transaction feature combinations to obtain a test case set; Step S2: performing abnormal identification on the execution results of the test case set, clustering the identified abnormal cases, and obtaining an abnormal scenario set; Step S3: According to the whole process of securities trading, the scenarios of price boundary values ​​not covered, market types not covered, transaction types not covered, and test paths not covered by the test case set are identified to obtain an uncovered scenario set; Step S4: Generate a new test case set based on the uncovered scenario set and the abnormal scenario set; Step S5: Repeat steps S2 to S4 until the set termination condition is reached, summarize the test results, and generate a test report.

2. The method for automated testing of a securities trading system based on result feedback according to claim 1, characterized in that: After the test case set is generated in step S1, all test cases in the test case set are scored and prioritized, including the following steps: Step S11: The risk level, business importance, fault impact and historical failure rate of the test case are used as evaluation indicators to calculate the evaluation index values ​​of all test cases; the risk level is the risk level of the scenario, the business importance is the importance of the scenario to the transaction business, the fault impact is the impact on the user experience or the transaction system when the test case fails, and the historical failure rate is the failure probability of the test case in the historical records; Step S12: adopting the hierarchical analysis method, taking the priority ranking of test cases as the target layer, taking the evaluation index as the criterion layer, taking the set of test cases to be evaluated as the solution layer, and constructing a hierarchical structure model; Step S13: construct a pairwise comparison matrix of evaluation indicators, calculate the relative weights of all evaluation indicators according to the pairwise comparison matrix, and perform consistency check on all relative weights; Step S14: Calculate the comprehensive score of each test case according to the relative weight of the evaluation index and the evaluation index value of the test case; Step S15: Prioritize all test cases in descending order of the comprehensive scores.

3. The method for automated testing of a securities trading system based on result feedback according to claim 1, characterized in that: In step S2, abnormal cases are identified based on the execution results of the test cases, which include functional indicators, performance indicators, error indicators, and behavioral indicators. The means of all indicators are calculated respectively, and the positive and negative three times standard deviation of the indicator mean are used as the baseline value of the indicator. Test cases whose indicator values ​​exceed the baseline value are regarded as abnormal cases.

4. The method for automated testing of a securities trading system based on result feedback according to claim 3, characterized in that: The functional indicators include the execution status of the test case, the reason for execution failure and the test coverage; the performance indicators include the system response time, resource utilization and throughput when executing the test case; the error indicators include the system exception type and crash frequency when executing the test case; the behavioral indicators include the number of repeated executions of the test case and the execution time.

5. The method for automated testing of a securities trading system based on result feedback according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: performing data cleaning and standardization on the execution results of all test cases; Step S42: converting uncovered scenes and abnormal scenes into feature vectors, randomly generating a number of noise vectors as a training set for a generator of a generative adversarial network (GAN), wherein the generator outputs a number of virtual test cases corresponding to the noise vectors; Step S43: using the plurality of virtual test cases and the feature vectors as a training set for a GAN discriminator, wherein the discriminator outputs a probability that the input test case is a real sample; Step S44: updating the parameters of the generator according to the discrimination result of the discriminator, and generating new test cases using the trained generator; Step S45: perform deduplication operation on the newly generated test cases, and score and prioritize the deduplicated test cases.

6. The method for automated testing of a securities trading system based on result feedback according to claim 5, characterized in that: The generator of the generative adversarial network GAN includes several hidden layers and an output layer, randomly generated noise vectors are input into several hidden layers for linear transformation, activation functions are used to calculate the activation outputs of the hidden layers, the activation outputs are transferred to the output layer, and test cases are generated; The discriminator of the generative adversarial network (GAN) consists of two hidden layers and an output layer. The test cases generated by the generator and the real test cases are input into the two hidden layers for linear transformation. The activation function is used to calculate the activation output of the hidden layer, and the activation output is passed to the output layer to obtain the probability that the input test case is a real test case.

7. The method for automated testing of a securities trading system based on result feedback according to claim 5, characterized in that: In step S44, a reinforcement learning algorithm or an adaptive generation algorithm is used to optimize the generation strategy of the GAN generator, wherein the reinforcement learning algorithm rewards test cases with high coverage or high success rate, so that the generator preferentially generates test cases with high coverage and high success rate; The adaptive generation algorithm adjusts the generation strategy according to the failure rate of the test cases, giving priority to generating boundary scenario cases with high coverage and high failure rate.

8. The method for automated testing of a securities trading system based on result feedback according to claim 5, characterized in that: In step S45, by calculating the similarity between the newly generated test cases and the existing test cases, the test cases in the newly generated test cases whose similarity is greater than the set similarity threshold are deleted, thereby achieving deduplication operation on the newly generated test cases.

9. The method for automated testing of a securities trading system based on result feedback according to claim 8, characterized in that: The expression for scoring the test cases after deduplication in step S45 is: ； In the formula, Score the test cases; coverage is the scene coverage; boundary is the boundary value; failure is the historical failure rate; The scene coverage 、 Boundary value, weight of historical failure rate.

10. The method for automated testing of a securities trading system based on result feedback according to claim 1, characterized in that: In step S1, an orthogonal experimental design method is used to combine equivalence classes and boundary values ​​to obtain several transaction feature combinations.

Images