Visual test process arrangement method and system
By collaborating with a low-code platform and AI, test cases are automatically generated, solving the problems of high technical barriers, low efficiency, and insufficient coverage of traditional software testing tools. This enables efficient automated testing in complex scenarios, adapts to multiple certification methods, and has self-optimization capabilities.
Patent Information
- Application Number
- CN202511168650.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-20
- Publication Date
- 2025-11-14
AI Technical Summary
Traditional software testing tools have high technical barriers, low efficiency, and insufficient coverage, making it difficult to automate complex scenarios such as CAPTCHA recognition and multi-factor authentication.
It adopts a low-code platform and AI collaboration, and automatically generates test cases through a visual test process orchestration method. It utilizes component libraries, AI parsing and code generation to support dynamic scenario handling.
It significantly improves test case design efficiency, lowers the technical threshold, enhances test coverage, adapts to complex scenarios, supports multiple authentication methods, and has self-optimizing learning capabilities.
Smart Images
Figure CN120950409A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of software testing technology, and in particular to a method and system for visual test process orchestration. Background Technology
[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.
[0003] In the field of software testing, traditional test case design relies on manually written scripts, which has the following drawbacks: 1. High technical threshold: Testers need to master programming languages and testing frameworks, making it difficult for business personnel to directly participate in test strategy formulation. 2. Low efficiency: Test cases for complex scenarios (such as CAPTCHA recognition and multi-factor authentication) require manual coding, which is time-consuming and prone to errors. 3. Insufficient coverage: Manual design cannot cover all boundary conditions and abnormal processes, leading to test blind spots.
[0004] While existing low-code testing tools simplify process configuration, they still require manual configuration of underlying scripts and cannot intelligently handle dynamic scenarios (such as image recognition logic for CAPTCHAs). Therefore, there is an urgent need for a test process orchestration tool with a higher degree of automation and adaptability to complex scenarios. Summary of the Invention
[0005] To address the aforementioned issues, this invention proposes a visual test process orchestration method and system. Through the collaboration of a low-code platform and AI, it shortens test case design time, automatically generates underlying scripts, and reduces the workload of manual coding and debugging, particularly improving the efficiency of test case generation for complex scenarios.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for orchestrating a visual test process, comprising: Design a component library, select the target component from the component library to the target area, define the execution order and logical relationship of the components, configure the parameters of the target component, and obtain the flowchart; The flowchart is parsed to obtain structured data including component attributes, parameters, and control flow. Generate generic code for the target testing framework based on structured data, and adjust the generic code according to the applicable domain and scenario to generate executable code; Select the corresponding test code template and fill the test code template with executable code. Adjust the generation of executable code based on the test results feedback of the filled test code template.
[0007] As an alternative implementation, the component library includes basic operation components, business logic components, and specific scenario components; the basic operation components support configuration of element location methods and operation parameters; the business logic components support defining the logical relationships of the test process through visual connections; the specific scenario components have built-in industry-standard test scenarios and can be adapted to specific business needs through parameter configuration.
[0008] As an alternative implementation, the process of parsing the flowchart includes: detecting components in the flowchart and constructing a topology graph structure, fusing component visual features and connection relationship features, using a relational graph convolutional network for classification to identify component types, parsing parameters and logical relationships, and finally generating structured data containing component attributes, parameters and control flow. Semantic enhancement and implicit test condition supplementation for structured data include: semantic role labeling: identifying predicate-argument structures, clarifying modification relationships, and deriving implicit conditions; domain knowledge mapping: combining knowledge graphs to map semantic tags to predefined attributes and recommend trigger conditions; test case design: expanding test coverage through boundary value analysis and equivalence class partitioning; and dynamically supplementing time-sensitive conditions based on historical feedback.
[0009] As an alternative implementation method, the process of generating general code for the target test framework based on structured data includes: converting the test process into a structured sequence, splitting the target code into a token sequence, and establishing an input-output mapping; generating code based on the input token sequence, and outputting general code after detoxification, formatting, and static verification.
[0010] As an alternative implementation, the model is fine-tuned through transfer learning to adapt general code to the domain. The transfer learning fine-tuning of the model includes: constructing a domain dataset; embedding a domain control token at the model input to prompt the model to generate test-specific code; freezing the underlying parameters of the model, adjusting the top attention layer, training with the domain dataset, and then fine-tuning the entire model; integrating a syntax parser during decoding to filter illegal code structures; and guiding the generation of code that conforms to the test framework conventions through pattern injection.
[0011] As an alternative implementation, the process of selecting and populating test code templates includes: storing templates categorized by test mode, with each template containing placeholders and conditional branches; parsing page elements and operation types in the test flow to extract flow features; calculating the similarity between flow features and template tags to select the optimal template; and dynamically populating the templates by replacing placeholders and performing conditional rendering based on the flow context.
[0012] Secondly, the present invention provides a visual test process orchestration system, comprising: The configuration module is configured as a design component library. After selecting the target component in the component library and moving it to the target area, the execution order and logical relationship of the components are defined, and the parameters of the target component are configured to obtain the flowchart. The parsing module is configured to parse the flowchart to obtain structured data including component properties, parameters, and control flow. The generation module is configured to generate generic code for the target test framework based on structured data, and adjust the generic code according to the applicable domain and scenario to generate executable code; The feedback module is configured to select the corresponding test code template, populate the executable code into the test code template, and adjust the generation of executable code based on the test results feedback of the populated test code template.
[0013] Thirdly, the present invention provides an electronic device including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.
[0014] Fourthly, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in the first aspect.
[0015] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.
[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: Significantly improved efficiency: By leveraging the collaboration between a low-code platform and AI, test case design time is shortened. AI automatically generates underlying scripts, reducing manual coding and debugging workload, especially improving the efficiency of test case generation for complex scenarios. Furthermore, it supports batch generation of regression test cases, greatly reducing repetitive manual work and accelerating the testing cycle.
[0017] Test coverage is significantly enhanced: AI models can automatically uncover potential boundary conditions and abnormal processes, thereby improving test coverage.
[0018] The technical threshold has been significantly reduced: business personnel can directly participate in the formulation of test strategies through a drag-and-drop interface, and can configure complex test processes without programming experience. This breaks the dependence of traditional test tools on technical personnel and promotes collaboration between business and test teams.
[0019] Intelligent adaptation to dynamic scenarios: For dynamic scenarios such as CAPTCHA recognition, AI calls OCR and image processing algorithms to generate simulated interaction code, supporting automated testing of various CAPTCHA types. For multi-factor authentication scenarios, dynamic token generation and verification logic is automatically inserted, covering multiple authentication methods such as SMS, email, and hardware tokens.
[0020] Self-optimizing learning capability: By collecting test execution results and user feedback, the AI model continuously optimizes the accuracy of script generation, ensuring the long-term effectiveness and adaptability of the testing tool.
[0021] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0023] Figure 1 The flowchart is for the visualization test process arrangement method provided in Embodiment 1 of the present invention. Detailed Implementation
[0024] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0025] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0026] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0027] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.
[0028] Example 1 This embodiment provides a method for orchestrating a visual test process, such as... Figure 1 As shown, it includes: Design a component library, select the target component from the component library to the target area, define the execution order and logical relationship of the components, configure the parameters of the target component, and obtain the flowchart; The flowchart is parsed to obtain structured data including component attributes, parameters, and control flow. Generate generic code for the target testing framework based on structured data, and adjust the generic code according to the applicable domain and scenario to generate executable code; Select the corresponding test code template and fill the test code template with executable code. Adjust the generation of executable code based on the test results feedback of the filled test code template.
[0029] The method of this embodiment will be described in detail below.
[0030] I. Low-code visual configuration.
[0031] 1. Component library design; provides a rich library of pre-built components, covering basic operation components, business logic components and specific scenario components.
[0032] (1) Basic operation components: such as click, input, wait, etc., supporting user configuration of element positioning methods (such as XPath, CSS selectors) and operation parameters (such as waiting time).
[0033] (2) Business logic components: such as conditional branches, loops, assertions, etc., allowing users to define the logical relationships of the test process through visual connections.
[0034] (3) Specific scenario components: such as verification code recognition, multi-factor authentication, payment gateway simulation, etc., with built-in industry-standard test scenarios. Users can adapt to specific business needs (such as verification code type and authentication method) through parameter configuration.
[0035] 2. Configure the interface interaction.
[0036] (1) Drag and drop arrangement: Users drag target components from the component library to the canvas and define the execution order and logical relationship of components by connecting them.
[0037] (2) Parameter configuration panel: After selecting a component, a parameter configuration panel will pop up on the right, which supports operations such as text input, drop-down selection, and writing conditional expressions.
[0038] (3) Real-time preview and verification: During the user configuration process, the system verifies the legality of the process in real time (such as whether the components are fully connected and whether the parameters are missing), and provides feedback on the verification results with highlighted or prompt information.
[0039] 3. Process version management; supports saving, rolling back, and branch management of test processes, allowing users to compare differences between different versions of the process and ensure the traceability of test strategies.
[0040] II. AI Script Generation Engine.
[0041] 1. Natural Language Processing (NLP) parsing layer.
[0042] (1) Flowchart to structured data: The flowchart configured by the user is parsed by graph neural network (GNN) to extract structured information such as component type, parameters, and connection relationship; First, the flowchart is preprocessed to detect components and build a topology graph structure. Then, the visual features of the components (such as type and text) and the features of the connection relationship (such as direction and condition) are fused together. Then, the Relationship Graph Convolutional Network (R-GCN) is used to classify the nodes to identify the component type. At the same time, the parameters and logical relationships are parsed, and finally, structured JSON data containing component attributes, parameters and control flow is generated.
[0043] (2) Semantic enhancement: Semantic enhancement of natural language descriptions and supplementation of implicit test conditions are achieved through pre-trained language models (such as BERT). BERT uses bidirectional context encoding and masked language models to accurately capture semantic associations in descriptions such as 4-digit numbers and derive implicit conditions (such as length and character range).
[0044] The supplementary strategies include: (2-1) Semantic role labeling: Identify predicate-argument structure, clarify modification relationship, and deduce implicit conditions (e.g., "verification code is 4 digits" is mapped to length=4, character set=0-9).
[0045] (2-2) Domain knowledge mapping: Combining knowledge graphs, semantic tags (such as numbers) are mapped to predefined attributes (such as character sets and expiration dates), triggering condition recommendations.
[0046] (2-3) Test case design: Expand test coverage through boundary value analysis (such as 3-digit / 5-digit numbers) and equivalence class partitioning (such as letter / symbol input).
[0047] (2-4) User feedback optimization: Analyze historical feedback (such as verification code timeout) and dynamically supplement time-sensitive conditions (such as validity period extension).
[0048] For example, the original description "the verification code is a 4-digit number" is supplemented to: test cases need to cover scenarios such as 3 / 5-digit numbers, letter / symbol input, and formats containing spaces to ensure system robustness. This method significantly improves test completeness and reduces the risk of missed tests.
[0049] In this embodiment, the collaborative mechanism between low-code and AI includes: (1) Standardized interface: The interaction interface between the low-code module and the AI engine is defined using the OpenAPI specification to ensure that the data format is consistent (e.g., flowcharts are transmitted in JSON format).
[0050] This also involves data interaction protocols: using the OpenAPI 3.0 specification to define the interface between the low-code module and the AI engine, ensuring that data such as flowcharts and component parameters are transmitted in a standardized JSON format.
[0051] Error handling mechanism: The interface supports custom error codes (such as INVALID_COMPONENT_TYPE, MISSING_PARAMETER) to ensure that exceptions between the low-code module and the AI engine are traceable.
[0052] (2) Knowledge graph construction: Based on the Neo4j graph database, a knowledge graph for the test domain is constructed to store the relationships between components, parameters, and test scenarios (such as "user login" associated with "verification code" and "password policy" nodes).
[0053] This includes: Entity and Relationship Extraction: Extract entities (such as user login, verification code) and relationships (such as user login → need → verification code) from the test case library and requirements document, and construct a graph; Knowledge graph storage and query: Stores knowledge graphs based on the Neo4j graph database, and supports the Cypher query language for fast retrieval of component relationships.
[0054] 2. Machine learning model layer.
[0055] (1) Model selection: The Sequence to Sequence (Seq2Seq) model is adopted, and the structured process data is used as input to generate the code of the target test framework (such as Selenium, Appium); The steps are as follows: (1-1) Data preprocessing: Transform the test process into a structured sequence (such as steps, operations, element location), and split the target code (such as Selenium) into a token sequence to establish input-output mapping.
[0056] (1-2) Model construction: Use bidirectional LSTM or Transformer as encoder to process input sequence, decoder to generate code token with attention mechanism, and embedding layer to map discrete operations into vector.
[0057] (1-3) Training optimization: The model is optimized using the cross-entropy loss function, teacher-forced convergence is accelerated, and label smoothing is used to prevent overfitting. Noise can be added to the input data to enhance robustness.
[0058] (1-4) Code generation and post-processing: The model generates code token by token based on the new input, and outputs general code after detoxification, formatting (such as automatic indentation) and static validation (such as checking bracket matching).
[0059] (1-5) Optimization strategy: Fine-tune the model at the API level of the test framework to adapt to domain differences (such as Selenium and Appium), introduce test case descriptions to enhance context awareness, and collect execution results to optimize the model.
[0060] (2) Domain adaptation: Based on the general code generation model, the model is fine-tuned through transfer learning to adapt it to the specific syntax of the test domain (such as element location method, assertion library call).
[0061] Specifically: (2-1) Constructing a domain dataset: Collect test framework code corpus (such as element location and assertion statements in Selenium), label the code function tags, and form a structured dataset.
[0062] (2-2) Design the model adaptation layer: Embed domain control tokens (such as...) at the model input end.<TEST_FRAMEWORK> The system prompts the model to generate test-specific code and expands the dictionary library to include test framework API vocabulary.
[0063] (2-3) Fine-tuning the model in stages: First, freeze the underlying parameters of the model and fine-tune only the top attention layer, and train using a small batch of test code data; then fine-tune the model in full, train with a larger scale of data at a lower learning rate, and strengthen the extraction of domain features.
[0064] (2-4) Enhance domain constraints: Integrate a syntax parser during decoding to filter illegal code structures; guide the model to generate code that conforms to the test framework conventions through pattern injection (such as automatically adding implicit waits).
[0065] (2-5) Evaluation and continuous optimization: Develop evaluation metrics (such as test case pass rate), continuously optimize the model through a combination of manual review and automated evaluation, and collect the execution results of newly generated test code for backpropagation updates.
[0066] (3) Dynamic scene processing: For scenarios such as CAPTCHA recognition, the model output includes code snippets containing OCR call interfaces (such as Tesseract, Baidu OCR API) and image processing logic (such as cropping, binarization).
[0067] The steps for generating CAPTCHA processing code using the model are as follows: (3-1) Scene analysis: Identify key information such as the type of verification code and image path in the input.
[0068] (3-2) Image processing logic generation: Based on the CAPTCHA type, the corresponding image preprocessing code is automatically inserted, such as perspective transformation and binarization, to optimize image quality.
[0069] (3-3) OCR service routing: Dynamically select Tesseract or Baidu OCR API according to the configuration and generate the corresponding calling code.
[0070] (3-4) Enhanced exception handling: Add retry mechanism and backup plan to deal with OCR recognition failure.
[0071] Finally, executable code is generated, enabling fully automated processing from image input to text output.
[0072] 3. Multimodal code generation.
[0073] (1) Template engine: Pre-set commonly used test code templates (such as page object model, data-driven test), and AI selects the appropriate template to fill in the generated code according to the process context.
[0074] The steps for AI to select and populate the test code template are as follows: (1-1) Template library design: Store templates according to test modes (such as POM, DDT). The templates contain placeholders (such as {{page_element}}) and conditional branches (such as {% if is_ajax %}).
[0075] (1-2) Context analysis: Analyze key information in the test process (such as page elements and operation types) and use NLP to extract structured features.
[0076] (1-3) Template matching: Calculate the similarity between process features and template labels, select the most suitable template, and prioritize matching templates that meet the necessary conditions.
[0077] (1-4) Dynamic filling: Replace placeholders and perform conditional rendering based on the process context (such as adding implicit wait code).
[0078] (1-5) Verification and optimization: Check code syntax through AST parsing and collect execution logs to optimize the template recommendation algorithm.
[0079] (2) Code optimization: Check the quality of the generated code through static code analysis tools (such as Pylint, ESLint) and automatically fix syntax errors and potential problems.
[0080] In this embodiment, the multimodal test script generation includes: (1) Cross-framework support: The test framework is decoupled through the adapter pattern. The AI model generates intermediate representation (IR) code, which is then converted into specific framework scripts by the framework adapter.
[0081] This includes: Intermediate Representation (IR) Design: Defines an intermediate code format independent of the testing framework, including fields such as operation type (e.g., CLICK, INPUT) and target element locator (e.g., XPath, CSS); Framework adapter implementation: Develop adapters for frameworks such as Selenium and Appium to convert IR code into specific framework scripts.
[0082] (2) Dynamic token processing: For multi-factor authentication scenarios, AI generates dynamic token generation logic that includes timestamps and encryption algorithms, and simulates interaction with the authentication server.
[0083] This includes: Timestamp algorithm: Embed dynamic token generation logic in the test script to calculate TOTP (Time-Based One-Time Password) based on the current timestamp. Simulate the authentication server: Use tools such as WireMock to simulate the authentication server interface and verify the validity of the dynamic token generated by the test script.
[0084] III. Test Execution and Feedback.
[0085] 1. Distributed execution engine; Task scheduling: A distributed test cluster is built based on Kubernetes, which supports the scaling of test nodes as needed to achieve large-scale concurrent testing.
[0086] Execution monitoring: Real-time display of test progress and resource utilization (such as CPU and memory), supporting test interruption and resumption.
[0087] 2. Intelligent feedback mechanism Execution result parsing: Automatically parse test reports (such as JUnit and Allure formats) and extract key metrics (such as pass rate and failed test cases).
[0088] Model optimization: Feed the test results back to the AI model, and use reinforcement learning to reward correctly generated code snippets and punish code patterns that lead to failure.
[0089] User feedback loop: Allows users to annotate generated test cases (such as insufficient coverage or logical errors), and the AI model continuously improves based on the annotated data.
[0090] In this embodiment, a self-optimizing learning mechanism is designed, including: (1) Data augmentation: Generate scarce test scenario data (such as abnormal verification codes and weak password combinations) through data synthesis techniques (such as GAN) to expand the training dataset.
[0091] This includes: Applications of Generative Adversarial Networks (GANs): Training GANs to generate scarce test scenario data (such as abnormal CAPTCHAs and weak password combinations) to expand the training dataset.
[0092] Example GAN structure: Generator: Input random noise, output synthesized CAPTCHA image. Discriminator: Distinguishes between real and synthesized CAPTCHAs, and feeds back to the generator to optimize the synthesis quality.
[0093] (2) Model interpretability: The AI model decision-making process is analyzed using SHAP values to show users the basis for code generation (e.g., calling the OCR interface because a CAPTCHA component was detected).
[0094] Specifically, SHAP (SHapley Additive exPlanations) values are used to analyze the AI model's decision-making process, quantifying the impact of each input feature (such as component type and parameter value) on the code generation results. User-annotated test case quality (such as insufficient coverage and logical errors) is used as an additional feature to retrain the AI model, improving the accuracy of code generation.
[0095] Example of implementation method.
[0096] Example 1: User login process test.
[0097] Configuration process: Drag and drop the "User Login" component to configure the username and password parameters.
[0098] Add a "CAPTCHA Recognition" component, select "Numbers + Letters" as the type, and associate it with the OCR service.
[0099] Add a "Multi-Factor Authentication" component, select "SMS" as the authentication method, and configure the mobile phone number parameters.
[0100] Execution and Feedback: Execute the generated script and record the verification code recognition success rate and SMS gateway response time. If the test fails, the user can indicate the reason for the failure (e.g., "Verification code recognition error"), and the AI model will optimize the OCR call logic based on the feedback.
[0101] The technical advantages of the method described in this embodiment are mainly reflected in the following aspects: 1. Significantly Improved Efficiency: By leveraging the collaboration between a low-code platform and AI, test case design time is shortened. AI automatically generates underlying scripts, reducing manual coding and debugging workload, especially improving the efficiency of test case generation for complex scenarios (such as CAPTCHA recognition and multi-factor authentication). Furthermore, it supports batch generation of regression test cases, significantly reducing repetitive manual work and accelerating the testing cycle.
[0102] 2. Enhanced Test Coverage: The AI model automatically identifies potential boundary conditions and abnormal processes (such as CAPTCHA timeouts and multi-factor authentication failures requiring retries), improving test coverage. Test reports provide coverage heatmaps, visually displaying blind spots and assisting testers in supplementing test cases accordingly.
[0103] 3. Significantly lower technical barriers: Business personnel can directly participate in test strategy formulation through a drag-and-drop interface, configuring complex test processes without programming experience (such as automatically generating test cases including CAPTCHA recognition and multi-factor authentication after configuring the user login process). This breaks the dependence of traditional testing tools on technical personnel and promotes collaboration between business and testing teams.
[0104] 4. Intelligent Adaptation to Dynamic Scenarios: For dynamic scenarios such as CAPTCHA recognition, AI utilizes OCR and image processing algorithms to generate simulated interactive code, supporting automated testing of various CAPTCHA types (such as numbers, letters, and arithmetic problems). For multi-factor authentication scenarios, AI automatically inserts dynamic token generation and verification logic, covering multiple authentication methods such as SMS, email, and hardware tokens.
[0105] 5. Self-optimizing learning capability: By collecting test execution results and user feedback, the AI model continuously optimizes the accuracy of script generation. For example, to address CAPTCHA recognition errors, the AI can adjust OCR parameters or switch recognition engines; to address insufficient test coverage, the AI can supplement boundary condition test cases. This self-optimizing mechanism ensures the long-term effectiveness and adaptability of the testing tool.
[0106] It should be noted that all data acquisition is conducted in accordance with laws and regulations and with user consent, and the data is used legally.
[0107] Example 2 This embodiment provides a visual test process orchestration system, including: The configuration module is configured as a design component library. After selecting the target component in the component library and moving it to the target area, the execution order and logical relationship of the components are defined, and the parameters of the target component are configured to obtain the flowchart. The parsing module is configured to parse the flowchart to obtain structured data including component properties, parameters, and control flow. The generation module is configured to generate generic code for the target test framework based on structured data, and adjust the generic code according to the applicable domain and scenario to generate executable code; The feedback module is configured to select the corresponding test code template, populate the executable code into the test code template, and adjust the generation of executable code based on the test results feedback of the populated test code template.
[0108] It should be noted that the above modules correspond to the steps described in Embodiment 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.
[0109] In further embodiments, the following is also provided: An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the method described in Embodiment 1. For brevity, further details are omitted here.
[0110] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.
[0111] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.
[0112] A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the method described in Embodiment 1.
[0113] The method in Example 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.
[0114] A computer program product includes a computer program that, when executed by a processor, implements the method described in Embodiment 1.
[0115] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, which execute in a device on a target real or virtual processor to perform the processes / methods described above. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided among program modules as needed. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside in both local and remote storage media.
[0116] The computer program code used to implement the methods of the present invention may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the computer or other programmable data processing device, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a stand-alone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.
[0117] In the context of this invention, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.
[0118] Those skilled in the art will recognize that the units and algorithm steps described in connection with the various examples of this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.
[0119] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.
Claims
1. A method for orchestrating a visual test process, characterized in that, include: Design a component library, select the target component from the component library to the target area, define the execution order and logical relationship of the components, configure the parameters of the target component, and obtain the flowchart; The flowchart is parsed to obtain structured data including component attributes, parameters, and control flow. Generate generic code for the target testing framework based on structured data, and adjust the generic code according to the applicable domain and scenario to generate executable code; Select the corresponding test code template and fill the test code template with executable code. Adjust the generation of executable code based on the test results feedback of the filled test code template.
2. The visual test process orchestration method as described in claim 1, characterized in that, The component library includes basic operation components, business logic components, and scenario-specific components; basic operation components support configuring element location methods and operation parameters; business logic components support defining the logical relationships of the test process through visual connections; The specific scenario component has built-in industry-standard test scenarios, which can be adapted to specific business needs through parameter configuration.
3. The visual test process orchestration method as described in claim 1, characterized in that, The process of parsing a flowchart includes: detecting components and constructing a topology graph structure, fusing component visual features and connection relationship features, using a relational graph convolutional network for classification to identify component types, parsing parameters and logical relationships, and finally generating structured data containing component attributes, parameters and control flow. Semantic enhancement and implicit test condition supplementation for structured data include: semantic role labeling: identifying predicate-argument structures, clarifying modification relationships, and deriving implicit conditions; domain knowledge mapping: combining knowledge graphs to map semantic tags to predefined attributes and recommend trigger conditions; test case design: expanding test coverage through boundary value analysis and equivalence class partitioning; and dynamically supplementing time-sensitive conditions based on historical feedback.
4. The visual test process orchestration method as described in claim 1, characterized in that, The process of generating generic code for the target test framework based on structured data includes: converting the test process into a structured sequence, splitting the target code into a token sequence, and establishing an input-output mapping; generating code based on the input token sequence, and outputting generic code after detoxification, formatting, and static validation.
5. The visual test process orchestration method as described in claim 1, characterized in that, By fine-tuning the model through transfer learning, domain-specific adjustments are made to the general code. Transfer learning fine-tuning of the model includes: building a domain dataset; embedding a domain control token at the model input to prompt the model to generate test-specific code; freezing the model's underlying parameters, adjusting the top-level attention layer, training with the domain dataset, and then fine-tuning it fully; integrating a syntax parser during decoding to filter illegal code structures; and guiding the generation of code that conforms to the testing framework conventions through pattern injection.
6. The visual test process orchestration method as described in claim 1, characterized in that, The process of selecting and populating test code templates includes: storing templates categorized by test mode, with each template containing placeholders and conditional branches; parsing page elements and operation types in the test flow to extract flow features; calculating the similarity between flow features and template tags to select the optimal template; and dynamically populating the templates by replacing placeholders and performing conditional rendering based on the flow context.
7. A visual test process orchestration system, characterized in that, include: The configuration module is configured as a design component library. After selecting the target component in the component library and moving it to the target area, the execution order and logical relationship of the components are defined, and the parameters of the target component are configured to obtain the flowchart. The parsing module is configured to parse the flowchart to obtain structured data including component properties, parameters, and control flow. The generation module is configured to generate generic code for the target test framework based on structured data, and adjust the generic code according to the applicable domain and scenario to generate executable code; The feedback module is configured to select the corresponding test code template, populate the executable code into the test code template, and adjust the generation of executable code based on the test results feedback of the populated test code template.
8. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-6.
10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the method described in any one of claims 1-6.