Orthogonal test method, device and storage medium based on user account

Abstract

The application discloses a kind of orthogonal test methods, equipment and storage medium based on user account, comprising: obtaining real-time account state information and real-time service traffic data;From real-time account state information, target test account is allocated, and the mapping relationship of target test account and test case is established;Real-time service traffic data is analyzed for scene characteristics, and the parameter structure of orthogonal test table is generated parameterized test case combination based on characteristic dimension and value distribution dynamic adjustment;Parameterized test case combination and target test account are injected into test environment to execute traffic playback, and according to the difference feedback of playback response data and expected result, adjust the test parameter value weight and case execution priority in orthogonal test table.The application forms closed loop feedback by account state dynamic identification and orthogonal table adjustment driven by traffic characteristics, realizes the matching of test account and case and test parameter adaptive optimization, to improve test scene coverage and execution efficiency.

Description

Technical Field

[0001] This application relates to the field of software testing technology, and in particular to an orthogonal testing method, device and storage medium based on user accounts. Background Technology

[0002] As the scale and complexity of software systems continue to increase, software testing technology has become a core support means to ensure the quality of system delivery. In mainstream software testing engineering practices, the system under test and its dependent environment are usually deployed on a test server cluster. Network traffic acquisition devices (such as switch mirror ports or distributed traffic probes) are used to capture system interaction data, and test cases, traffic records, and execution results are saved through a distributed storage system.

[0003] In orthogonal testing, orthogonal arrays are typically generated based on the parameters of the system under test to determine test case combinations, and test accounts with fixed permissions are configured to execute them sequentially. However, the permissions and states of test accounts are statically fixed before the test starts, making it difficult to dynamically adapt to changes in system operation. The fixed account library cannot cover all user roles and permission combinations, and the state transition phase in the account lifecycle lacks systematic test coverage. Regarding traffic replay, production traffic is usually recorded independently and replayed in the test environment, but the replay process is separated from the orthogonal test case design phase. No dynamic correlation is established between traffic data and test parameter combinations, resulting in limited realism of complex business scenarios. The test case execution order is fixed and cannot be adaptively adjusted according to system state. These problems of rigid test account management and the disconnect between traffic replay and orthogonal testing restrict further improvements in test scenario coverage depth and execution efficiency.

[0004] Therefore, it is evident that in the current software testing technology system, orthogonal testing methods face prominent problems such as rigid test account management, relatively singular test dimensions, and limited scenario coverage. Traffic replay technology, on the other hand, is limited by its disconnect from the orthogonal testing process and the static pre-setting of test strategies. Dynamic adaptive identification and allocation of test account status, intelligent integration of real business traffic characteristics and orthogonal test parameter structures, and real-time closed-loop optimization of test strategies based on replay feedback information have become key technical bottlenecks that urgently need to be overcome to improve the testing efficiency of complex software systems.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main purpose of this application is to provide an orthogonal testing method, device and storage medium based on user accounts, which aims to solve the technical problems of low testing efficiency caused by rigid account management and the disconnect between traffic playback and orthogonal testing in the existing testing architecture.

[0007] To achieve the above objectives, this application proposes an orthogonal testing method based on user accounts, the method comprising: Obtain real-time account status information and real-time business traffic data in the production environment of the system under test; Based on the account permissions and account status required for the test scenario, at least one target test account is dynamically identified and allocated from the real-time account status information, and a mapping relationship between the target test account and the corresponding test case is established. The real-time business traffic data is analyzed for scenario features. Based on the analyzed feature dimensions and value distribution, the parameter structure of the initial orthogonal test table is dynamically adjusted to generate a combination of parameterized test cases that integrate real traffic constraints. The parameterized test case combination and the associated target test account are synchronously injected into the test environment to perform traffic replay. Based on the feedback of the difference between the replay response data and the expected results, the weight of the test parameter values ​​and the execution priority of the test cases in the orthogonal test table are adjusted to obtain the adaptively optimized orthogonal test parameter combination.

[0008] In one embodiment, the step of dynamically identifying and allocating at least one target test account from the real-time account status information based on the account permissions and account status required for the test scenario includes: Collect account creation events, permission change operations, login and logout behaviors, and resource access records in the system under test to construct a real-time account status time-series database; Based on the permission list and expected account status required by the test scenario, the real-time status time series database of the account is traversed to filter the corresponding candidate account set. Extract the historical operation behavior features of each user account in the candidate account set, calculate the similarity between the historical operation behavior features and the expected behavior features of the test scenario, and determine at least one target test account based on the similarity ranking results.

[0009] In one embodiment, the step of establishing the mapping relationship between the target test account and the corresponding test case includes: Extract the historical behavior feature vector of each user account in the candidate account set. The historical behavior feature vector includes operation frequency features, operation success features, and data access pattern features. Obtain the scenario expected behavior feature vector of the test case, and calculate the similarity between the historical behavior feature vector and the scenario expected behavior feature vector; The user accounts in the candidate account set are sorted based on the similarity, and a preset number of user accounts are selected from the sorting results as the target test accounts. The unique identifier of the target test account is then bound and stored with the identifier of the test case.

[0010] In one embodiment, before the step of dynamically adjusting the parameter structure of the initial orthogonal test table based on the parsed feature dimensions and value distribution, the method further includes: The real-time business traffic data is parsed for protocols and business logic is identified to extract operation sequence features, data access features and anomaly handling features. Based on the operation sequence characteristics, data access characteristics, and anomaly handling characteristics, the real-time business traffic data is clustered to obtain multiple business scenario clusters. In the business scenario cluster, identify the key test parameter dimensions and the set of parameter values ​​for each dimension, and generate an initial orthogonal test table based on the set of parameter values.

[0011] In one embodiment, the step of generating a combination of parameterized test cases that incorporate real traffic constraints includes: Obtain the business constraint relationships between each of the key test parameter dimensions, wherein the business constraint relationships define the mutual exclusion dependency rules between the values ​​of different parameter dimensions; The business constraint relationship is applied to filter the test case combinations in the initial orthogonal test table to remove invalid parameter combination records; The parameterized test case combination is obtained by assigning execution weights to the filtered test case combination based on historical test performance data.

[0012] In one embodiment, the steps of synchronously injecting the parameterized test case combination and the associated target test account into the test environment to perform traffic replay, and adjusting the test parameter values ​​and test case execution priorities in the orthogonal test table based on the feedback of the difference between the replay response data and the expected results to obtain an adaptively optimized orthogonal test parameter combination include: Obtain the response data after the traffic replay is performed in the test environment, compare the response data with the original feedback data in the real-time business traffic data, and identify the abnormal replay record in the comparison result; Extract the test case combinations and associated parameter dimension values ​​corresponding to the anomaly replay records to generate defect association rules; The test weights of the target parameter dimensions are adjusted according to the defect association rules, and test case combinations containing high-weight parameter values ​​are prioritized during subsequent test execution. The target parameter dimension values ​​refer to parameter dimension values ​​that are associated with the anomaly replay records.

[0013] In one embodiment, the orthogonal testing method based on user accounts further includes: During the traffic replay process in the test environment, preset abnormal events are injected into the system under test according to the parameter combination in the orthogonal test table. The abnormal events include network timeout events, service unavailability events, and data format error events. The process of injecting preset abnormal events includes controlling the time node, duration, and scope of impact of the abnormal events. Monitor the response behavior and processing flow of the system under test when the abnormal event occurs, and obtain abnormal handling feedback data; The anomaly handling feedback data is compared with a preset anomaly handling baseline, and the stability of the tested system under abnormal scenarios is determined based on the comparison results.

[0014] In one embodiment, the orthogonal testing method based on user accounts further includes: Monitor the module call chain of the system under test to obtain the call frequency and error rate of each functional module; Based on the call frequency and error rate, a risk score is calculated for each functional module, and the test resource allocation ratio of the relevant parameter combinations in the orthogonal test table is adjusted according to the risk score.

[0015] In addition, to achieve the above objectives, this application also proposes an orthogonal testing device based on user accounts, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the orthogonal testing method based on user accounts as described above.

[0016] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the orthogonal testing method based on user accounts as described above.

[0017] One or more technical solutions proposed in this application have at least the following technical effects: This application's technical solution addresses the core pain points of existing static configurations' inability to dynamically respond to account status changes and the disconnect between traffic replay and orthogonal testing. It achieves precise matching of test accounts and adaptive adjustment of test parameters through a deep integration of real-time account status perception, intelligent traffic feature analysis, and dynamic optimization of orthogonal testing. Specifically, the method first acquires real-time account status information of the system under test and real-time business traffic data in the production environment, constructing a data foundation for dynamic perception. Based on this, according to the test scenario's requirements for account permissions and status, target test accounts are dynamically identified and allocated from the real-time account status information, establishing a mapping relationship between accounts and test cases, enabling on-demand scheduling of account resources and adaptive coverage of permission boundaries. Simultaneously, scenario feature analysis is performed on the real-time business traffic data. Based on the analyzed feature dimensions and value distribution, the parameter structure of the initial orthogonal test table is dynamically adjusted to generate parameterized test case combinations that incorporate real traffic constraints, deeply coupling test cases with real business scenarios. Subsequently, the parameterized test case combination and the associated target test account are synchronously injected into the test environment to perform traffic replay. Based on the feedback of the difference between the replay response data and the expected results, the weight of the test parameters and the execution priority of the test cases in the orthogonal test table are adjusted, forming a complete closed-loop iterative mechanism from data collection, intelligent matching, dynamic generation to feedback optimization, so that the test strategy is always synchronized with the actual state of the system throughout the entire iteration process.

[0018] Through the above-mentioned technical means, this application significantly improves execution efficiency while ensuring test coverage depth, solves the technical problem that existing static configuration and separate execution cannot dynamically respond to system changes, and realizes the technical leap of software testing from static configuration to dynamic perception, from separate execution to deep integration, and from fixed strategy to adaptive closed loop. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the first embodiment of the orthogonal testing method based on user accounts in this application; Figure 2 This is a detailed process diagram based on step S20 in the first embodiment; Figure 3This is a schematic diagram of another detailed process based on step S20 in the first embodiment; Figure 4 This is a detailed schematic diagram of step S40 based on the first embodiment; Figure 5 This is a flowchart illustrating the second embodiment of the orthogonal testing method based on user accounts in this application; Figure 6 This is a flowchart illustrating the third embodiment of the orthogonal testing method based on user accounts in this application; Figure 7 This is a flowchart illustrating the fourth embodiment of the orthogonal testing method based on user accounts in this application; Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the orthogonal testing method based on user accounts in the embodiments of this application.

[0022] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0024] In related technologies, orthogonal testing based on user accounts mainly follows the technical path of static configuration and separate execution, but it has inherent defects and is difficult to meet the dual requirements of test scenario coverage depth and execution efficiency.

[0025] This type of method employs an offline planning strategy, determining test case combinations based on a pre-set orthogonal array before test execution and configuring test accounts with fixed permissions and states. Simultaneously, production traffic is recorded independently and replayed in a predetermined order within the test environment. This method is applicable when the system permission model is simple and the business scenario boundaries are clear, reducing test design complexity and facilitating test result comparison. However, this method is essentially an open-loop control, unable to dynamically adjust based on the real-time status of the system under test and replay feedback. When the system permission model changes or the business scenario becomes increasingly complex, statically configured test accounts struggle to adapt to the new permission structure, the fixed account library cannot cover all user roles and permission combinations, and the state transition boundary conditions during the account lifecycle are difficult to effectively trigger and verify. Furthermore, the traffic replay process is disconnected from the orthogonal test case design phase, lacking dynamic correlation between replay traffic and test parameter combinations. This limits the realistic reproduction of complex business scenarios, and the test case execution order becomes rigid, making it impossible to adaptively adjust the test strategy based on defect distribution and system state. Practical applications show that in iterative testing of microservice architectures or cloud-native systems, system states and business traffic patterns are highly time-varying. Static configuration and separate execution testing models struggle to predict potential risk areas, resulting in insufficient boundary condition coverage and decreased defect discovery efficiency. Furthermore, these methods lack real-time awareness of account status, intelligent analysis of traffic characteristics, and forward-looking optimization mechanisms for testing strategies. This makes it difficult to focus on high-risk modules under limited testing resources, forcing the adoption of equal resource allocation strategies, further sacrificing testing efficiency and quality.

[0026] A comprehensive analysis reveals that the core dilemma faced by the aforementioned technical approaches lies in the fact that while static configuration and separate execution methods are simple to implement and have clear processes, their offline and open-loop characteristics fundamentally contradict the inherent requirements of complex software systems for real-time and adaptive testing. They cannot achieve a balance between test coverage depth and execution efficiency in rapid iteration scenarios, and are difficult to adapt to the development requirements of intelligent software quality assurance.

[0027] Based on the aforementioned deficiencies in related technologies, this application proposes an orthogonal testing method based on user accounts. This method addresses the core pain points of existing static configurations' inability to dynamically respond to account status changes and the disconnect between traffic replay and orthogonal testing. It achieves precise matching of test accounts and adaptive adjustment of test parameters through a deep integration of real-time account status perception, intelligent traffic feature analysis, and dynamic optimization of orthogonal testing. Specifically, the method first acquires real-time account status information of the system under test and real-time business traffic data in the production environment to build a dynamically perceptible data foundation. Based on this, according to the requirements of the test scenario for account permissions and status, target test accounts are dynamically identified and allocated from the real-time account status information, establishing a mapping relationship between accounts and test cases, achieving on-demand scheduling of account resources and adaptive coverage of permission boundaries. Simultaneously, scenario feature analysis is performed on the real-time business traffic data. Based on the analyzed feature dimensions and value distribution, the parameter structure of the initial orthogonal test table is dynamically adjusted to generate a parameterized test case combination that integrates real traffic constraints, deeply coupling the test cases with the real business scenario. Subsequently, the parameterized test case combination and the associated target test account are synchronously injected into the test environment to perform traffic replay. Based on the feedback of the difference between the replay response data and the expected results, the weight of the test parameters and the execution priority of the test cases in the orthogonal test table are adjusted, forming a complete closed-loop iterative mechanism from data collection, intelligent matching, dynamic generation to feedback optimization, so that the test strategy is always synchronized with the actual state of the system throughout the entire iteration process.

[0028] Through the above-mentioned technical means, this application significantly improves execution efficiency while ensuring test coverage depth, solves the technical problem that existing static configuration and separate execution cannot dynamically respond to system changes, and realizes the technical leap of software testing from static configuration to dynamic perception, from separate execution to deep integration, and from fixed strategy to adaptive closed loop.

[0029] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0030] Based on this, embodiments of this application provide an orthogonal testing method based on user accounts, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the orthogonal testing method based on user accounts according to this application. In this embodiment, the orthogonal testing method based on user accounts includes steps S10 to S40: Step S10: Obtain real-time account status information and real-time business traffic data in the production environment of the system under test. During the test task initiation phase, the adaptive fusion test engine first establishes a data channel with the production and testing environments of the system under test. For acquiring account status information, the system deploys distributed probe clusters at key locations such as the network access layer, application service layer, and data access layer. The probes acquire and capture account-related operation events in a non-intrusive manner, covering account creation events, permission change operations, login / logout behaviors, and resource access records. The probes push the raw event data to the log collection server in real time. The log collection server then performs data cleaning, format standardization, timestamp alignment, and correlation analysis on the raw data. The cleaned account status data is written to a time-series database that supports high-speed writing and complex time window queries, capable of completely depicting the entire lifecycle status change trajectory of each account in the system under test from creation to cancellation.

[0031] To acquire real-time business traffic data, the system replicates the interactive traffic consisting of real user requests and system responses from the production environment using a traffic mirroring component deployed on an API gateway or load balancer. The traffic mirroring component sends the captured data packets to the traffic preprocessing engine, which parses the packets according to preset protocol specifications, extracting key elements such as request methods, resource locators, request header fields, and request body content. Simultaneously, the traffic preprocessing engine retains timestamp information and call chain identifiers from the traffic data to maintain the operational sequence and contextual dependencies of the original business scenario.

[0032] After the aforementioned account status information and business traffic data are collected, they are transmitted to the data aggregation layer of the adaptive fusion test engine via message queues. The data aggregation layer performs unified index management on both types of data, establishing correlation indexes based on time and account dimensions. This provides a real-time and complete data foundation for intelligent account matching and traffic feature analysis in subsequent steps. Through this acquisition process, the system establishes a dynamic perception capability of the operating status of the system under test and user behavior patterns, enabling the formulation of test strategies to be based on real and real-time system status information rather than static preset parameters.

[0033] Step S20: Based on the account permissions and account status required for the test scenario, dynamically identify and allocate at least one target test account from the real-time account status information, and establish a mapping relationship between the target test account and the corresponding test case. After acquiring the real-time account status information of the system under test, the account adaptive recognition engine first parses the account permission and status requirements in the test cases corresponding to the current test scenario. When a test case is generated or loaded, its metadata pre-marks the list of permissions necessary to execute the test case and the expected account status conditions. The permission list is represented in the form of a set of permission identifiers in the permission model of the system under test, and the expected account status conditions include constraints such as the account being in a normal active state, having a specific role binding, or not having triggered a rate limiting threshold.

[0034] The adaptive account recognition engine uses the aforementioned permission list and status conditions as matching input, traversing the real-time account status time-series database to retrieve candidate accounts. The retrieval process employs a multi-stage filtering strategy: the first stage performs rapid exclusion based on permission requirements, filtering accounts whose permission sets do not contain the required permissions; the second stage performs status matching verification, filtering accounts whose current status differs from the expected status by a preset threshold. After these two stages of filtering, a candidate account set is formed.

[0035] For each account in the candidate account set, the intelligent matching engine further extracts its historical operational behavior features to construct a multi-dimensional behavioral feature vector. The dimensions of the behavioral feature vector include operation frequency features such as average daily operation count and peak time distribution; operation success features such as request success rate and typical failure error code patterns; operation time features such as operation interval patterns and time period preferences; and data access features such as frequently used functional modules and data range preferences. Simultaneously, the intelligent matching engine extracts a scenario-expected behavioral feature vector from the test case scenario description. This vector characterizes the expected profile of the account's behavioral patterns in the test scenario.

[0036] The intelligent matching engine calculates the similarity between the behavioral feature vectors of candidate accounts and the expected behavioral feature vectors of the scenario. After the similarity calculation is completed, the candidate account set is sorted from high to low similarity, and a predetermined number of accounts at the top of the sorted list are selected as target test accounts. The system then establishes an association mapping record between the target test accounts and test cases, binding the unique identifier of the target test account with the identifier of the test case and storing it in the test session context.

[0037] Through the above dynamic identification and mapping process, the allocation of test accounts changes from static preset to on-demand matching based on real-time status and behavioral characteristics. When the permission model of the system under test changes or the account status is migrated, the matching mechanism can automatically adapt to the changes and reselect accounts that meet the conditions, so as to achieve accurate matching between account resources and test scenarios.

[0038] Step S30: Analyze the scene features of the real-time business traffic data, and dynamically adjust the parameter structure of the initial orthogonal test table based on the analyzed feature dimensions and value distribution to generate a parameterized test case combination that integrates real traffic constraints. After acquiring real-time business traffic data from the production environment, the traffic pattern analysis engine performs in-depth analysis of the traffic data. The analysis process first extracts structured fields from requests and responses according to communication protocol specifications, including operation type, resource path, input parameters and their values, and output result status. Based on protocol parsing, the traffic pattern analysis engine further performs business logic identification, aggregating discrete request-response pairs into user operation sequences with business meaning, thus reconstructing the complete user operation flow in a real business scenario.

[0039] The traffic pattern analysis engine extracts features from user operation sequences. These features include operation sequence characteristics (transition patterns and frequency distribution between adjacent operations), data access characteristics (types of data objects involved in the operation and the scope of access), and exception handling characteristics (types of exceptions and retry behavior patterns that occur during the operation). Based on these extracted features, the engine employs an improved clustering algorithm to cluster traffic data according to business scenarios, grouping traffic segments with feature similarity exceeding a preset threshold into the same business scenario cluster. After clustering, the system obtains several scenario clusters with business semantics, each cluster corresponding to a real business scenario.

[0040] For each business scenario cluster, the system identifies key test parameter dimensions that affect system behavior. The parameter dimension identification process includes both explicit parameters directly extracted from traffic data, such as input field values ​​in requests, and implicit parameters derived through analysis, such as user identity type, operation time period attributes, and pre-operation dependency states. For each parameter dimension, its value distribution within the corresponding scenario cluster is statistically analyzed, constructing a parameter value set.

[0041] The orthogonal array dynamic optimizer receives the aforementioned list of parameter dimensions and the set of values ​​for each dimension as input, and calls the orthogonal array generation algorithm to construct an initial orthogonal test table. During the construction process, the orthogonal array dynamic optimizer obtains the business constraint relationships between parameter dimensions. These constraints define the mutual exclusion dependency rules between different parameter values, such as the binding constraint between a specific operation type and a specific user role. The orthogonal array dynamic optimizer applies these business constraint relationships to filter the test case combinations in the initial orthogonal test table, removing invalid parameter combination records that violate the constraint rules. After filtering, the orthogonal array dynamic optimizer assigns execution weights to each test case combination based on historical test performance data, including information such as the defect detection rate and execution time of the parameter combination in past tests. After constraint filtering and weight allocation, the system generates parameterized test case combinations that incorporate real traffic constraints. This combination maintains the systematic coverage of parameter interaction relationships by the orthogonal array while introducing business constraints and distribution characteristics from real traffic, making the test cases highly consistent with the actual business scenarios in the production environment.

[0042] Step S40: Synchronously inject the parameterized test case combination and the associated target test account into the test environment to perform traffic replay, and adjust the test parameter value weights and test case execution priorities in the orthogonal test table according to the feedback of the difference between the replay response data and the expected results, so as to obtain the adaptively optimized orthogonal test parameter combination.

[0043] After completing the intelligent association of the target test account and the generation of parameterized test case combinations, the adaptive fusion test engine initiates the test execution phase. The test execution scheduler first reads each parameterized test case from the test case combination set and finds the target test account identifier bound to it based on the test case identifier. For each test case to be executed, the scheduler initializes the corresponding test session in the test environment, loads the authentication credentials and context state of the target test account into the session, and constructs the parameter values ​​in the parameterized test case into a specific request message according to the interface specification.

[0044] The intelligent traffic replay controller receives the constructed sequence of request messages and sends them one by one to the test environment interface of the system under test, according to the preset execution order or dynamically orchestrated execution priority in the test case combination. During the transmission process, the replay controller controls the transmission interval between requests based on the timestamp information in the raw traffic data, simulating the natural rhythm of real user operations. The system under test processes each request and returns response data. The replay controller collects the response data and compares it with the original feedback data corresponding to that request.

[0045] The comparison process is performed by the result analyzer. The result analyzer extracts key fields from the response data, including the response status code, business processing result identifier, returned data structure and content, and compares them item by item with the corresponding fields in the original feedback data. When there are differences between the response data and the original feedback data, the result analyzer marks the replay record as an abnormal replay record and extracts the test case combination identifier and associated parameter dimension value information corresponding to the abnormal replay record.

[0046] The defect pattern learning system collects all anomaly replay records, performs correlation analysis on the parameter values ​​in the anomaly records, and generates defect association rules. These rules describe the strength of the correlation between specific parameter value combinations and abnormal system behavior. Based on these rules, the policy optimizer adjusts the test weights for corresponding parameter values ​​in the orthogonal test table, increasing the weights of parameter value combinations involving anomaly records. Simultaneously, the policy optimizer recalculates the execution priority of each test case combination based on the adjusted weights, ensuring that test case combinations containing high-weight parameter values ​​are prioritized in subsequent test execution rounds.

[0047] The aforementioned feedback and adjustment process constitutes a closed-loop iterative mechanism. After one round of test execution and feedback adjustment, the system obtains an updated orthogonal test parameter combination, where the parameter weights and test case execution priorities reflect the focus on the system's risk areas. During continuous iteration, this closed-loop mechanism enables the test strategy to adaptively evolve with changes in the state of the system under test, gradually converging to an optimized state with high defect detection rate and high coverage efficiency, ultimately obtaining an adaptively optimized orthogonal test parameter combination.

[0048] Furthermore, you can also view Figure 2 , Figure 2 This is a detailed process diagram based on step S20 in the first embodiment. Figure 2 The step of dynamically identifying and allocating at least one target test account from the real-time account status information based on the account permissions and account status required for the test scenario includes steps S21-23: Step S21: Collect account creation events, permission change operations, login and logout behaviors, and resource access records in the system under test to construct a real-time account status time-series database; Step S22: Based on the permission list and expected account status required by the test scenario, traverse the real-time status time series database of the account and filter the corresponding candidate account set. Step S23: Extract the historical operation behavior features of each user account in the candidate account set, calculate the similarity between the historical operation behavior features and the expected behavior features of the test scenario, and determine at least one target test account based on the similarity ranking result.

[0049] During the account status information collection phase, the system comprehensively captures runtime events related to accounts through a distributed probe cluster pre-deployed in each access layer of the system under test. The probes acquire and record four types of core event data: the first type is account creation events, including the time of account unique identifier generation, initial role allocation, and activation status markers; the second type is permission change operations, including records of permission granting, permission revocation, and role rebinding; the third type is login and logout behavior, including account login time, login source address, session duration, and logout method; and the fourth type is resource access records, including details of the account's access to system functional modules, data interfaces, and business resources. The probes stream the above event data to the log collection server. The log collection server sequentially performs data cleaning to remove duplicate and invalid records, format standardization to unify the data structure of events from different sources, timestamp alignment to correct clock deviations in a distributed environment, and correlation analysis to establish event correlation links for the same account within a continuous time window. The processed event data is written to the real-time account status time-series database. This database uses the account identifier as the primary key and the event occurrence time as the time axis index to fully store the entire lifecycle status change sequence of each account from creation, activation, normal use, restriction, freezing to cancellation, providing real-time and complete data support for subsequent dynamic account matching.

[0050] During the candidate account screening phase, the adaptive account recognition engine reads the metadata of the test case to be executed and extracts two matching conditions: the permission list required by the test case and the expected account state. The permission list is an ordered set of permission identifiers describing the minimum permission range required to execute the test case; the expected account state defines the target state the account should be in before test execution, such as normal activation, specific role binding, or no rate limiting threshold triggered. Using the above permission list and expected account state as query conditions, the adaptive account recognition engine traverses the real-time account state time-series database for two-level screening. The first level is permission coverage screening, where the engine checks the current permission set of each account in the time-series database to determine if it completely includes all permission identifiers in the permission list; accounts that do not completely include all are directly excluded. The second level is state compliance screening, where the engine further checks whether the current state of accounts that passed the first level of screening matches the expected account state; accounts with state differences exceeding a preset tolerance threshold are excluded. The accounts retained after these two levels of screening constitute the candidate account set, where each account meets the basic requirements of the test case in both permission and state dimensions.

[0051] In the target account identification phase, the intelligent matching engine performs refined sorting of the candidate account set to select the final target test account. The intelligent matching engine first extracts the historical operational behavior features of each user account in the candidate account set, based on the sequence of past operation events recorded in the account's real-time status time-series database. These historical operational behavior features cover multiple analytical dimensions: operation frequency includes the account's average daily number of operations, peak operation time distribution, and average operation interval; operation success rate includes the success rate of account requests, error code distribution patterns of failed requests, and retry frequency; operation time includes the account's operation time preferences, duration of continuous activity, and operation interval patterns; and data access includes the types of commonly used functional modules, the range of accessed data objects, and the ratio of read to write operations. The intelligent matching engine organizes these multi-dimensional features into a fixed-length historical operational behavior feature vector. Simultaneously, the intelligent matching engine parses the expected behavior features from the scenario description field of the test case. These expected behavior features depict the typical expected profile of the account's behavior pattern in the business scenario simulated by the test case. The intelligent matching engine calculates the similarity between the candidate account's historical operational behavior feature vector and the expected behavior feature vector. The similarity calculation results reflect the degree to which the behavioral patterns of candidate accounts match the expected behavioral patterns in the test scenario. After the calculation is completed, the intelligent matching engine sorts the candidate account set from high to low according to the similarity value, and selects a preset number of accounts at the top of the ranking as target test accounts. Through the progressive processing of the above three stages of data collection and database construction, two-level screening, and similarity ranking, the system has achieved a technological leap from static preset test accounts to dynamic on-demand matching based on real-time status and behavioral characteristics.

[0052] You can also view Figure 3 , Figure 3 This is a schematic diagram of another detailed process based on step S20 in the first embodiment. Figure 3 The step of establishing the mapping relationship between the target test account and the corresponding test case includes S24~26: Step S24: Extract the historical behavior feature vector of each user account in the candidate account set. The historical behavior feature vector includes operation frequency features, operation success features, and data access pattern features. Step S25: Obtain the scenario expected behavior feature vector of the test case, and calculate the similarity between the historical behavior feature vector and the scenario expected behavior feature vector; Step S26: Sort the user accounts in the candidate account set according to the similarity, select a preset number of user accounts from the sorting results as the target test accounts, and bind and store the unique identifier of the target test account with the identifier of the test case.

[0053] After the target test accounts are initially determined, the system needs to establish a formal association mapping between accounts and test cases to ensure that the corresponding accounts can be accurately invoked during the test execution phase. This process begins with the extraction of historical behavior feature vectors for each user account in the candidate account set. The intelligent matching engine traverses the candidate account set and constructs a multi-dimensional historical behavior feature vector for each user account in the set. The construction of this historical behavior feature vector is based on the historical operation event records of the account stored in the account's real-time status time series database. The feature extraction algorithm performs statistical aggregation and pattern mining on the event records. The historical behavior feature vector contains three core feature dimension groups: the operation frequency feature group covers the average number of operations of the account within a unit time window, the operation interval time distribution curve, and the operation burst index, used to characterize the account's usage activity and operation rhythm; the operation success feature group covers the account's successful completion rate of requests, the error code clustering distribution of typical failure scenarios, and the abnormal recovery behavior pattern, used to characterize the stability and reliability of the account's operations; the data access pattern feature group covers the account's access preference vector for different data objects, the ratio coefficient of read and write operations, and the data volume distribution range, used to characterize the account's business operation tendencies and data usage habits. The three-dimensional features mentioned above together constitute a complete quantitative description of the account's historical behavior patterns.

[0054] After acquiring historical behavioral feature vectors, the intelligent matching engine further acquires the scenario-expected behavioral feature vectors corresponding to the test cases. During the design or generation phase of the test cases, the values ​​for various dimensions of the scenario-expected behavioral feature vectors are predefined in the test case description data. This vector describes the ideal behavioral profile of the executing account for the business scenario simulated by the test case from three dimensions: expected operation frequency, expected operation success, and expected data access pattern. For example, a test case simulating a high-frequency trading scenario might have a high value for the operation frequency feature and a high success rate for the operation success feature in its scenario-expected behavioral feature vector; while a test case simulating an anomaly recovery scenario might have a scenario-expected behavior vector where the operation success feature may include specific failure mode expectations.

[0055] The intelligent matching engine inputs the historical behavioral feature vectors of candidate accounts and the expected behavioral feature vectors of test cases into the similarity calculation module. The similarity calculation module employs a multi-dimensional vector similarity measurement algorithm, considering the varying importance of each feature dimension in the test scenario. It assigns scenario-appropriate weight coefficients to operation frequency features, operation success features, and data access pattern features before calculating the overall similarity. A higher similarity score indicates a closer alignment between the candidate account's historical behavioral patterns and the test scenario's expectations of the account's behavior. This means that using the account to execute test cases can more realistically reproduce the user behavior characteristics of the target business scenario.

[0056] After similarity calculation, the intelligent matching engine sorts all user accounts in the candidate account set in descending order based on the calculated similarity scores. Accounts ranking higher in the sorted results show a higher degree of behavioral pattern consistency with the test scenario. The intelligent matching engine then selects a corresponding number of user accounts from the sorted results, from highest to lowest, as the final target test accounts for this test case, according to a preset selection threshold.

[0057] After identifying the target test account, the system performs a binding and storage operation between the account and the test case. The intelligent matching engine generates an association mapping record containing two core identification fields: a unique identifier for the target test account and an identifier for the test case. The unique identifier for the target test account is a globally unique name for that account within the account system of the system under test, and the identifier for the test case is a globally unique number for that test case within the test management system. The intelligent matching engine writes the above association mapping record into the test session context storage area, which remains valid throughout the execution cycle of this test task. During the test execution phase, the scheduler can quickly locate the corresponding target test account identifier based on the test case identifier by querying the association mapping record in the test session context, thereby obtaining the account's authentication credentials and state context, and completing the initialization of the test session. Through the progressive processing of the above four stages—feature vector construction, similarity calculation, sorting and selection, and binding and storage—the system establishes a mapping relationship mechanism between test accounts and test cases based on behavioral pattern matching.

[0058] Furthermore, you can also view Figure 4 , Figure 4 This is a detailed process diagram based on step S40 in the first embodiment. Figure 4 The steps of synchronously injecting the parameterized test case combination and the associated target test account into the test environment to perform traffic replay, and adjusting the test parameter values ​​and test case execution priorities in the orthogonal test table based on the difference between the replay response data and the expected results to obtain the adaptively optimized orthogonal test parameter combination include S41~S43: Step S41: Obtain the response data after the traffic replay is performed in the test environment, compare the response data with the original feedback data in the real-time business traffic data, and determine the abnormal replay record in the comparison result; Step S42: Extract the test case combination and associated parameter dimension values ​​corresponding to the anomaly replay record, and generate defect association rules; Step S43: Adjust the test weight of the target parameter dimension value according to the defect association rule, and prioritize the scheduling of test case combinations containing high-weight parameter values ​​during subsequent test execution. The target parameter dimension value refers to the parameter dimension value that is associated with the anomaly replay record.

[0059] In this embodiment, the response data after the traffic replay is performed in the test environment is obtained, the response data is compared with the original feedback data in the real-time business traffic data, and the abnormal replay record is identified in the comparison result.

[0060] During traffic replay execution, the intelligent traffic replay controller sends each request message from the parameterized test case combination to the system under test. The system under test processes the request and generates corresponding response data. The result collector captures the response data corresponding to each request in real time. The response data includes key information such as the response status code, business processing result identifier, return data structure, and content body. The result collector associates the response data with the request identifier in the replay result temporary storage area according to the order in which the requests were sent.

[0061] After a round of traffic replay is completed, the result analyzer reads response data line by line from the replay result buffer and retrieves the original feedback data corresponding to the request from the traffic recording repository. The original feedback data is a record of the actual response generated by the target online service for the same request during the production environment traffic recording phase, and its data format is consistent with the response data. The result analyzer compares the response data of each request with the original feedback data field by field, covering the response status code field, the business processing result identifier field, and key business data fields in the response body.

[0062] The comparison process follows a preset set of comparison rules. For the response status code field, the result analyzer determines whether the two are equal; for the business processing result identifier field, the result analyzer determines whether the business semantics are consistent; for the key business data fields in the response body, the result analyzer determines whether the difference in field values ​​exceeds a preset tolerance threshold. When the response data and the original feedback data do not match in any comparison dimension, the result analyzer marks the replay record corresponding to that request as an abnormal replay record. The abnormal replay record simultaneously records the request identifier, test case combination identifier, the specific differences between the response data and the original feedback data, and the field location information where the difference occurred. The result analyzer summarizes all abnormal replay records to form an abnormal replay record set, which serves as input data for subsequent defect correlation analysis.

[0063] After receiving the set of exception replay records, the defect pattern learning system performs source analysis on each exception replay record. Based on the request identifier recorded in the exception replay record, the system traces back to the test case combination identifier to which the request belonged during the test case combination generation phase. Each test case combination identifier uniquely corresponds to a row of parameter value combinations in the orthogonal test table; this combination consists of several parameter dimensions and their specific values.

[0064] The defect pattern learning system extracts all parameter dimensions and their corresponding values ​​from the test case combination, forming a set of anomaly-related parameter values. For example, if the test case combination corresponding to an anomaly replay record includes a user role dimension of read-only user, an operation type dimension of data write, and a data range dimension of cross-departmental data, the defect pattern learning system uses the combination of these three dimensions as the associated features of the anomaly record.

[0065] After performing the aforementioned source extraction operation on all records in the anomaly replay record set, the defect pattern learning system employs an association rule mining algorithm to statistically analyze the set of anomaly association parameter values. The association rule mining algorithm calculates the co-occurrence frequency of each parameter dimension value in different anomaly records, as well as the association strength index between specific parameter value combinations and anomaly occurrence. When the frequency of a parameter value combination appearing in the anomaly replay record is significantly higher than its baseline frequency in all test case combinations, and the association strength index exceeds a preset confidence threshold, the defect pattern learning system outputs the correspondence between this parameter value combination and the anomaly type as a defect association rule. The defect association rule describes the probabilistic characteristics of behavioral deviations in the tested system under specific parameter value conditions, providing a quantitative decision-making basis for subsequent test strategy adjustments.

[0066] The policy optimizer receives a set of defect association rules generated by the defect pattern learning system. The policy optimizer first identifies the target parameter dimension values ​​involved in the defect association rules. These target parameter dimension values ​​refer to the parameter dimension values ​​that are associated with the abnormal playback records, that is, the specific parameter values ​​that are determined to have a significant statistical correlation with the abnormal behavior of the system in the defect association rules.

[0067] The strategy optimizer performs weight adjustments on test case combinations that contain the aforementioned target parameter dimension values ​​in the orthogonal test table. Specifically, the strategy optimizer iterates through each row of test case combinations in the orthogonal test table, determining whether the parameter value set of that combination contains the target parameter dimension value. For test case combinations that do contain the target parameter dimension value, the strategy optimizer proportionally increases the execution weight value of that combination based on the association strength index recorded in the defect association rules. The magnitude of the weight increase is positively correlated with the association strength index; the higher the association strength, the greater the weight increase.

[0068] After the weights are adjusted, the policy optimizer recalculates the execution priority for all test case combinations based on the updated execution weight values. The calculation of execution priority comprehensively considers the execution weight values ​​of the test case combinations, the coverage requirements of uncovered parameter combinations, and the availability of test resources. Test case combinations containing high-weight parameter values ​​receive a higher scheduling priority in the execution priority ranking.

[0069] In subsequent test execution rounds, the test execution scheduler schedules each test case combination for execution according to the updated execution priority. High-priority combinations are allocated test resources first and complete replay verification first. Through the progressive processing of the above three stages—anomaly identification through difference comparison, association rule mining, and weight-driven priority adjustment—the system forms a complete closed loop from replay feedback to test strategy optimization. The orthogonal test parameter combinations continuously converge towards high-risk areas during the iteration process, ultimately obtaining an adaptively optimized orthogonal test parameter combination.

[0070] Furthermore, you can also view Figure 5 , Figure 5 This is a flowchart illustrating the second embodiment of the orthogonal testing method based on user accounts in this application. In this embodiment, before the step of dynamically adjusting the parameter structure of the initial orthogonal testing table based on the parsed feature dimensions and value distribution, steps S50-70 are included: Step S50: Perform protocol parsing and business logic identification on the real-time business traffic data, and extract operation sequence features, data access features and anomaly handling features; Step S60: Based on the operation sequence characteristics, data access characteristics, and anomaly handling characteristics, the real-time business traffic data is clustered to obtain multiple business scenario clusters; Step S70: Identify key test parameter dimensions and the set of parameter values ​​for each dimension in the business scenario cluster, and generate an initial orthogonal test table based on the set of parameter values.

[0071] In this embodiment, after receiving real-time business traffic data from the data aggregation layer, the traffic pattern analysis engine first enters the protocol parsing stage. The protocol parser, based on the communication protocol identifier carried in the traffic data, calls the corresponding protocol parsing plugin to disassemble the original data packets layer by layer. The parsing process sequentially extracts the transport layer protocol header information, application layer protocol header information, and message body payload content. For traffic based on the Hypertext Transfer Protocol (HTTP), the protocol parser extracts the request method, resource locator, request header field set, and request body content; for response traffic, the protocol parser extracts the response status code, response header field set, and response body content. After protocol parsing is completed, a structured request-response pair sequence is obtained.

[0072] Based on protocol parsing, the traffic pattern analysis engine performs business logic identification. The business logic identifier reads structured request-response pair sequences and, according to the pattern characteristics of resource locators and the combination of request methods, aggregates discrete request-response pairs into user operation steps with business meaning. For example, consecutive product query requests, product detail requests, and order creation requests are identified as a complete shopping order process. The business logic identifier also extracts user session information from request headers, session identifiers, and user authentication credentials, chaining operation steps belonging to the same user session into a complete user operation sequence. Each user operation sequence corresponds to a complete behavioral trajectory of a real user in a business scenario.

[0073] The feature extractor performs multi-dimensional feature extraction on user operation sequences, resulting in three core feature categories. Operation sequence features describe the transition patterns between user operations, including the transition frequency matrix between adjacent operation types, the length distribution range of the operation sequence, and the pre- and post-departure dependencies of key operation nodes. Data access features describe the data object access patterns involved in the operation, including the type of data table or data interface accessed, the amount of data involved in a single operation, and the proportion of read and write operations. Anomaly handling features describe anomalies that occur during the operation and the corresponding responses, including the distribution of anomaly types and frequencies, the number of retries and retry intervals after an anomaly occurs, and the success rate of anomaly recovery. These three types of features together constitute a complete characterization of the implicit behavioral patterns in business traffic.

[0074] The traffic pattern analysis engine vectorizes the extracted three types of features to form a feature vector corresponding to each user's operation sequence. In the feature vector, the operation sequence feature is encoded as a fixed-length operation transition probability vector, the data access feature is encoded as a data object type distribution vector and data volume statistics, and the anomaly handling feature is encoded as an anomaly type distribution vector and retry behavior statistics.

[0075] The cluster analyzer receives feature vectors from all user operation sequences as input and uses a density-based clustering algorithm to spatially cluster the feature vectors. Based on the distribution density of the feature vectors in the high-dimensional feature space, the algorithm automatically identifies high-density regions as cluster centers and groups samples whose feature similarity meets a preset density connectivity condition into the same cluster. During the clustering process, the algorithm automatically excludes isolated noise sample points to avoid interference from occasional abnormal operation sequences on the clustering results.

[0076] After clustering analysis, each cluster contains a set of user operation sequences that are highly similar in terms of operation sequence patterns, data access behavior, and anomaly handling characteristics. The traffic pattern analysis engine assigns business semantic labels to each cluster. These labels are automatically generated by analyzing the functional characteristics of typical operation sequences within the cluster, such as high-frequency read-only query scenarios, low-frequency batch update scenarios, and scenarios with intensive anomaly retries. The multiple business scenario clusters obtained from clustering cover real business patterns in the production environment from different dimensions, providing a scenario-based basis for the subsequent design of the parameter structure of orthogonal test tables.

[0077] For each business scenario cluster, the parameter recognizer performs parameter traversal analysis on all user operation sequences contained within that cluster. The parameter recognizer extracts the input parameters of each request in the operation sequence as candidate test parameters. The input parameters include path parameters and query parameters in the resource locator, business fields in the request body, and control parameters in the request header.

[0078] The parameter identifier assesses the importance of candidate test parameters based on factors including the parameter's frequency of occurrence in different operation sequences within the cluster, the impact of parameter value changes on subsequent operation paths, and the parameter's association frequency in historical defect records. Candidate parameters whose assessment scores exceed a preset importance threshold are identified as key test parameter dimensions for that business scenario cluster. For each key test parameter dimension, the parameter identifier statistically analyzes its values ​​across all operation sequences within the cluster, removing duplicates to form a set of parameter values ​​for that dimension. This set of parameter values ​​reflects the actual value distribution range of the parameter in the production environment.

[0079] The orthogonal array generator receives all key test parameter dimensions and their corresponding sets of values ​​as input, and calls the orthogonal array construction algorithm to generate an initial orthogonal test table. The orthogonal array construction algorithm selects a matching orthogonal array template based on the number of parameter dimensions and the number of values ​​for each dimension, arranging and combining the parameter values ​​according to the orthogonal array structure to ensure that all combinations of values ​​between any two parameter dimensions appear at least once in the orthogonal array. The initial orthogonal test table is organized in tabular form, with each row representing a test case combination, each column corresponding to a key test parameter dimension, and the intersecting cells recording the specific value of that dimension in that combination.

[0080] In addition, the step of generating a parameterized test case combination that incorporates real traffic constraints includes: Obtain the business constraint relationships between each of the key test parameter dimensions, wherein the business constraint relationships define the mutual exclusion dependency rules between the values ​​of different parameter dimensions; The business constraint relationship is applied to filter the test case combinations in the initial orthogonal test table to remove invalid parameter combination records; The parameterized test case combination is obtained by assigning execution weights to the filtered test case combination based on historical test performance data.

[0081] After the initial orthogonal test table is generated, the system needs to perform constraint filtering and weight allocation processing on the orthogonal test table so that the generated test case combination not only conforms to the parameter dependency relationship in the real business scenario, but also focuses on the high-risk parameter value range.

[0082] First, the system acquires the business constraint relationships between key test parameter dimensions. These business constraint relationships define the mutual exclusion and dependency rules between different parameter dimension values. The constraint relationship acquisition module reads a predefined set of parameter constraint rules from the business rule configuration library of the system under test. These predefined parameter constraint rules originate from the business rule descriptions in the system design document or the parameter dependency descriptions in the interface definition document. Simultaneously, the constraint relationship acquisition module mines implicit parameter constraint patterns within the generated business scenario clusters. This mining method involves statistically analyzing the co-occurrence relationships between parameter values ​​in the operation sequences within the cluster. When a specific value of two parameter dimensions never appears simultaneously in the same operation sequence, this value combination is marked as a potential mutual exclusion constraint. The explicitly configured constraint rules are merged with the mined implicit constraint rules to form a complete set of business constraint relationships. Each constraint rule in the business constraint relationship set explicitly records the name of the constrained parameter dimension, the constrained value, and the constraint type. Constraint types include mutual exclusion constraints and dependency constraints. A mutual exclusion constraint indicates that two values ​​cannot appear simultaneously in the same test case combination, while a dependency constraint indicates that the occurrence of one value is a prerequisite for the occurrence of another value.

[0083] Secondly, the system applies business constraints to filter test case combinations in the initial orthogonal test table, removing invalid parameter combination records. The constraint filter iterates through the test case combinations in the initial orthogonal test table row by row, performing constraint compliance checks on the parameter value set in each row. The constraint filter matches the parameter value set of the current combination with each constraint rule in the business constraint relationship set. When a combination is found to contain value pairs that violate mutual exclusion constraints (i.e., two mutually exclusive values ​​appear simultaneously in the same combination), the constraint filter marks that row as invalid. Similarly, when a combination is found to contain values ​​that violate dependency constraints (i.e., a dependent value appears alone even when the dependent value is absent), the constraint filter also marks that row as invalid. After iteration, the constraint filter removes all records marked as invalid combinations from the orthogonal test table, retaining only the test case combinations that pass the constraint compliance check. Each row of parameter values ​​in the filtered test case combinations conforms to the constraints of the actual business rules of the system under test and can be effectively executed in the test environment.

[0084] Finally, the system assigns execution weights to the filtered test case combinations based on historical test performance data, resulting in parameterized test case combinations. The weight allocator reads execution records of past test tasks from the historical test performance database. This historical test performance data includes the number of defects discovered, the distribution of defect severity levels, and execution time statistics for each parameter dimension value combination in historical tests. The weight allocator matches each filtered test case combination with the historical test performance data, calculating the total frequency and average severity level of each parameter value in the combination being associated with defect records in historical tests. Based on a preset weight calculation function, the weight allocator maps defect association frequency and severity level to execution weight values. The higher the defect association frequency and severity level, the larger the assigned execution weight value. For entirely new parameter value combinations not found in the historical test performance data, the weight allocator assigns a default baseline weight value. The execution weight value reflects the expected detection value of the test case combination for potential system defects.

[0085] After constraint filtering and weight allocation, the initial orthogonal test table is transformed into a parameterized test case combination that incorporates real traffic constraints. This combination maintains the orthogonal table's systematic coverage of parameter interaction relationships while eliminating invalid combinations with valid constraints that do not conform to business rules. Furthermore, by assigning differentiated execution weights to each combination, test resources are automatically tilted towards high-risk areas.

[0086] You can also view Figure 6 , Figure 6 This is a flowchart illustrating the third embodiment of the orthogonal testing method based on user accounts according to this application. In this embodiment, the orthogonal testing method based on user accounts further includes steps S80-100: Step S80: During the traffic replay process in the test environment, a preset abnormal event is injected into the system under test according to the parameter combination in the orthogonal test table. The abnormal events include network timeout events, service unavailability events, and data format error events. The process of injecting the preset abnormal events includes controlling the time node, duration, and scope of impact of the abnormal events. Step S90: Monitor the response behavior and processing flow of the system under test when the abnormal event occurs, and obtain abnormal handling feedback data; Step S100: Compare the anomaly handling feedback data with a preset anomaly handling baseline, and determine the stability of the tested system under anomaly scenarios based on the comparison results.

[0087] In this embodiment, the anomaly injection controller and the intelligent traffic replay controller work collaboratively. During traffic replay execution, the anomaly injection controller reads the parameter values ​​associated with the currently executing test case combination from the orthogonal test table. The anomaly injection controller internally maintains an anomaly pattern library, which predefines various types of anomaly event templates, mainly including three categories: network timeout event templates, service unavailability event templates, and data format error event templates. The network timeout event template is used to simulate network connection timeout scenarios between the tested system and dependent services; the service unavailability event template is used to simulate scenarios where dependent services return unavailable responses; and the data format error event template is used to simulate scenarios where request parameters or response data formats do not conform to the agreed-upon format.

[0088] The exception injection controller selects an exception event type that matches the current parameter combination scenario from the exception pattern library based on the type characteristics of the parameter combination in the orthogonal test table. For example, when the parameter combination includes a parameter dimension for calling downstream services, the exception injection controller prioritizes injecting network timeout events or service unavailability events; when the parameter combination includes a parameter dimension with complex data structures, the exception injection controller prioritizes injecting data format error events.

[0089] The exception injection controller precisely controls the injection process, with control dimensions including the timing of the exception event, its duration, and its scope of impact. Timing control ensures the exception event is triggered at a specific time when the traffic replay request is sent, such as within a specified time window after the request is sent and before the response is returned. Duration control specifies the length of time the exception state persists; after this duration, the system returns to normal. Scope of impact control limits the range of requests affected by the exception event, specifying whether it affects only the current request, all requests within a specific user session, or all requests for a specific functional module.

[0090] Anomaly monitoring probes are deployed at key nodes of the system under test, including the request entry layer, business logic processing layer, and data persistence layer. When the anomaly injection controller triggers an anomaly event, the anomaly monitoring probes capture the system under test's response to the event in real time. The captured content includes whether the system under test successfully detected the anomaly, the type of anomaly handling branch triggered by the system under test, the error response content and error code returned by the system under test to the caller, and the anomaly stack information recorded in the system under test's internal logs.

[0091] The anomaly monitoring probe simultaneously tracks the processing flow of the system under test after an anomaly occurs, recording the number of retry operations performed by the system under test, the triggering of degradation processing logic, the activation status of the circuit breaker mechanism, and the final processing result returned to the client. This monitoring data is aggregated into anomaly handling feedback data, which is stored in a structured format, including the anomaly event identifier, the time of anomaly occurrence, the response time of the system under test, the status of key nodes in the processing flow, and the final processing result.

[0092] The stability analyzer receives anomaly handling feedback data and reads the preset anomaly handling baselines corresponding to the injected anomaly event types from the anomaly handling baseline library. The preset anomaly handling baselines define the expected behavior standards that the system under test should exhibit when encountering specific types of anomalies, including that the anomaly should be detected within a specified time threshold, that the specified anomaly handling branch should be triggered, that the agreed error response code should be returned to the caller, and that the preset retry or degradation strategy should be correctly executed.

[0093] The stability analyzer compares each actual metric in the anomaly handling feedback data with its corresponding metric in the preset anomaly handling baseline. The comparison items include whether the anomaly detection latency is within the threshold range, whether the triggered anomaly handling branch is consistent with the baseline, whether the returned error response code conforms to the convention, whether the number of retries meets the policy configuration, and whether degradation processing is correctly activated. When all metrics in the anomaly handling feedback data meet the requirements of the preset anomaly handling baseline, the stability analyzer determines that the system under test is stable under that anomaly scenario. When any metric deviates from the baseline requirements, the stability analyzer records the deviation and its degree, and determines the stability level based on the impact of the deviation on the overall system stability.

[0094] The stability analyzer writes the judgment results into the test report, which indicates the stability performance of the tested system under various abnormal scenarios and details of the indicators that did not meet the standards, providing a quantitative basis for the evaluation and improvement of the system's fault tolerance capability.

[0095] Furthermore, you can also view Figure 7 , Figure 7 This is a flowchart illustrating the fourth embodiment of the orthogonal testing method based on user accounts according to this application. In this embodiment, the orthogonal testing method based on user accounts further includes steps S110-120: Step S110: Monitor the module call chain of the system under test and obtain the call frequency and error rate of each functional module; Step S120: Calculate the risk score of each functional module based on the call frequency and error occurrence rate, and adjust the test resource allocation ratio of the relevant parameter combination in the orthogonal test table according to the risk score.

[0096] In this embodiment, the call chain monitor continuously tracks the inter-module call behavior of the system under test during test execution. The call chain monitor obtains detailed information for each module call by embedding monitoring code at the interceptor level of the service call framework or by parsing the link logs output by the distributed call chain tracing system. This detailed information includes the identifier of the calling module, the identifier of the called module, the time of the call, the call response time, and the call return status.

[0097] The call chain monitor periodically aggregates and statistically analyzes the collected call data. For each functional module, the monitor calculates two core metrics: call frequency and error rate. Call frequency refers to the total number of times the module is called by other modules within a unit time window, reflecting its traffic-carrying role in the system architecture. Error rate refers to the proportion of calls where the module returns an error status when responding to a call request, calculated by dividing the number of calls where the module returns an error status by the total number of calls to the module. The detailed statistics of error rate also include the distribution of error types, such as the proportion of different error categories like business logic errors, data validation errors, and dependency service errors. The call chain monitor stores the call frequency and error rate data for each functional module in a time-series format in the monitoring database.

[0098] The risk assessor retrieves statistical data on the call frequency and error rate of each functional module within the current testing cycle from the monitoring database. The risk assessor then calls a risk scoring model, using call frequency and error rate as input variables for weighted calculation. The risk scoring model assigns a positive weight to call frequency; modules with higher call frequencies occupy a more critical position in the system's business process, and their impact when a failure occurs is more widespread, thus resulting in a higher risk score. Similarly, the risk scoring model assigns a positive weight to error rate; modules with higher error rates indicate poorer stability in their current version and a higher probability of potential defects, thus also resulting in a higher risk score.

[0099] The risk scoring calculation model weights and sums the normalized values ​​of call frequency and error occurrence rate to obtain the initial risk score for each functional module. The risk assessor further introduces a module dependency factor to correct the initial risk score. This module dependency factor is obtained by analyzing the call chain topology. When a module is depended upon by multiple frequently called modules, a failure in that module will trigger a chain reaction, increasing the value of its dependency factor and thus improving its final risk score. The corrected risk score comprehensively reflects the module's criticality in the system, its current stability performance, and the risk of fault propagation.

[0100] The resource scheduler receives the risk score results from each functional module and adjusts the test resource allocation ratio for relevant parameter combinations in the orthogonal test table based on the risk score. The resource scheduler first identifies the association between each test case combination in the orthogonal test table and the functional modules. The association is determined based on whether the parameter values ​​in the test case combination involve calls to a specific functional module. For example, when the operation type parameter in a test case combination points to an interface provided by a functional module, the test case combination is associated with that functional module.

[0101] The resource scheduler adjusts the test resource allocation weights of test case combinations associated with each functional module proportionally based on the module's risk score. These weights determine the share of execution concurrency, execution priority, and available computing resource quotas for that test case combination. Functional modules with higher risk scores receive greater test resource weights for their associated test case combinations. The resource scheduler then distributes the adjusted test resource allocation ratios to the test execution scheduler, which schedules resources for each test case combination in subsequent test rounds based on the updated allocation ratios.

[0102] Through the aforementioned module call link monitoring and risk-scoring-based dynamic resource allocation mechanism, the system automatically focuses test resources on high-risk modules, maximizing defect discovery efficiency and system stability verification depth under the constraint of limited test resources.

[0103] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the orthogonal testing method based on user accounts in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0104] This application provides an orthogonal testing device based on user accounts. The orthogonal testing device based on user accounts includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the orthogonal testing method based on user accounts in the first embodiment described above.

[0105] The following is for reference. Figure 8The diagram illustrates a structural schematic of a user account-based orthogonal test device suitable for implementing embodiments of this application. The user account-based orthogonal test device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The orthogonal test device based on user accounts shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0106] like Figure 8 As shown, the user account-based orthogonal testing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the user account-based orthogonal testing device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows orthogonal test equipment based on user accounts to communicate wirelessly or wiredly with other devices to exchange data. Although various orthogonal test equipment based on user accounts are shown in the figures, it should be understood that it is not required to implement or possess all of those shown. More or fewer may be implemented alternatively.

[0107] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0108] The orthogonal testing device based on user accounts provided in this application, employing the orthogonal testing method based on user accounts in the above embodiments, can solve the technical problems of low testing efficiency caused by rigid account management and the disconnect between traffic replay and orthogonal testing in existing testing architectures. Compared with the prior art, the beneficial effects of the orthogonal testing device based on user accounts provided in this application are the same as those of the orthogonal testing method based on user accounts provided in the above embodiments, and other technical features of the orthogonal testing device based on user accounts are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0109] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0110] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0111] This application provides a storage medium, which is a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the orthogonal testing method based on user accounts in the above embodiments.

[0112] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be executed by instructions, used by devices, or used in conjunction with them. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0113] The aforementioned computer-readable storage medium may be included in a user account-based orthogonal test device; or it may exist independently and not assembled into a user account-based orthogonal test device.

[0114] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by a user account-based orthogonal testing device, enable the user account-based orthogonal testing device to implement the technical content of the user account-based orthogonal testing method embodiment shown above.

[0115] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0116] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using dedicated hardware-based implementations that perform the specified functions or operations, or can be implemented using a combination of dedicated hardware and computer instructions.

[0117] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0118] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described user account-based orthogonal testing method. This solves the technical problem of low testing efficiency caused by rigid account management and the disconnect between traffic replay and orthogonal testing in existing testing architectures. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the user account-based orthogonal testing method provided in the above embodiments, and will not be repeated here.

Claims

1. An orthogonal testing method based on user accounts, characterized in that, The orthogonal testing method based on user accounts includes the following steps: Obtain real-time account status information and real-time business traffic data in the production environment of the system under test; Based on the account permissions and account status required for the test scenario, at least one target test account is dynamically identified and allocated from the real-time account status information, and a mapping relationship between the target test account and the corresponding test case is established. The real-time business traffic data is analyzed for scenario features. Based on the analyzed feature dimensions and value distribution, the parameter structure of the initial orthogonal test table is dynamically adjusted to generate a combination of parameterized test cases that integrate real traffic constraints. The parameterized test case combination and the associated target test account are synchronously injected into the test environment to perform traffic replay. Based on the feedback of the difference between the replay response data and the expected results, the weight of the test parameter values ​​and the execution priority of the test cases in the orthogonal test table are adjusted to obtain the adaptively optimized orthogonal test parameter combination.

2. The orthogonal testing method based on user accounts as described in claim 1, characterized in that, The steps of dynamically identifying and allocating at least one target test account from the real-time account status information based on the account permissions and account status required for the test scenario include: Collect account creation events, permission change operations, login and logout behaviors, and resource access records in the system under test to construct a real-time account status time-series database; Based on the permission list and expected account status required by the test scenario, the real-time status time series database of the account is traversed to filter the corresponding candidate account set. Extract the historical operation behavior features of each user account in the candidate account set, calculate the similarity between the historical operation behavior features and the expected behavior features of the test scenario, and determine at least one target test account based on the similarity ranking results.

3. The orthogonal testing method based on user accounts as described in claim 2, characterized in that, The step of establishing the mapping relationship between the target test account and the corresponding test case includes: Extract the historical behavior feature vector of each user account in the candidate account set. The historical behavior feature vector includes operation frequency features, operation success features, and data access pattern features. Obtain the scenario expected behavior feature vector of the test case, and calculate the similarity between the historical behavior feature vector and the scenario expected behavior feature vector; The user accounts in the candidate account set are sorted based on the similarity, and a preset number of user accounts are selected from the sorting results as the target test accounts. The unique identifier of the target test account is then bound and stored with the identifier of the test case.

4. The orthogonal testing method based on user accounts as described in claim 2, characterized in that, Before the step of dynamically adjusting the parameter structure of the initial orthogonal test table based on the parsed feature dimensions and value distribution, the method further includes: The real-time business traffic data is parsed for protocols and business logic is identified to extract operation sequence features, data access features and anomaly handling features. Based on the operation sequence characteristics, data access characteristics, and anomaly handling characteristics, the real-time business traffic data is clustered to obtain multiple business scenario clusters. In the business scenario cluster, identify the key test parameter dimensions and the set of parameter values ​​for each dimension, and generate an initial orthogonal test table based on the set of parameter values.

5. The orthogonal testing method based on user accounts as described in claim 4, characterized in that, The step of generating a parameterized test case combination that incorporates real traffic constraints includes: Obtain the business constraint relationships between each of the key test parameter dimensions, wherein the business constraint relationships define the mutual exclusion dependency rules between the values ​​of different parameter dimensions; The business constraint relationship is applied to filter the test case combinations in the initial orthogonal test table to remove invalid parameter combination records; The parameterized test case combination is obtained by assigning execution weights to the filtered test case combination based on historical test performance data.

6. The orthogonal testing method based on user accounts as described in claim 1, characterized in that, The steps of synchronously injecting the parameterized test case combination and the associated target test account into the test environment to perform traffic replay, and adjusting the test parameter values ​​and test case execution priorities in the orthogonal test table based on the difference between the replay response data and the expected results to obtain the adaptively optimized orthogonal test parameter combination include: Obtain the response data after the traffic replay is performed in the test environment, compare the response data with the original feedback data in the real-time business traffic data, and identify the abnormal replay record in the comparison result; Extract the test case combinations and associated parameter dimension values ​​corresponding to the anomaly replay records to generate defect association rules; The test weights of the target parameter dimensions are adjusted according to the defect association rules, and test case combinations containing high-weight parameter values ​​are prioritized during subsequent test execution. The target parameter dimension values ​​refer to parameter dimension values ​​that are associated with the anomaly replay records.

7. The orthogonal testing method based on user accounts as described in claim 1, characterized in that, The orthogonal testing method based on user accounts also includes: During the traffic replay process in the test environment, preset abnormal events are injected into the system under test according to the parameter combination in the orthogonal test table. The abnormal events include network timeout events, service unavailability events, and data format error events. The process of injecting preset abnormal events includes controlling the time node, duration, and scope of impact of the abnormal events. Monitor the response behavior and processing flow of the system under test when the abnormal event occurs, and obtain abnormal handling feedback data; The anomaly handling feedback data is compared with a preset anomaly handling baseline, and the stability of the tested system under abnormal scenarios is determined based on the comparison results.

8. The orthogonal testing method based on user accounts as described in claim 1, characterized in that, The orthogonal testing method based on user accounts also includes: Monitor the module call chain of the system under test to obtain the call frequency and error rate of each functional module; Based on the call frequency and error rate, a risk score is calculated for each functional module, and the test resource allocation ratio of the relevant parameter combinations in the orthogonal test table is adjusted according to the risk score.

9. An orthogonal testing device based on user accounts, characterized in that, The orthogonal testing device based on user accounts stores a computer program, which, when executed by a processor, implements the orthogonal testing method based on any one of claims 1-8.

10. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the orthogonal testing method based on user accounts as described in any one of claims 1-8.

Images