Creation method and device of automatic test framework and electronic equipment

Abstract

The invention relates to a creation method and device of an automatic test framework and electronic equipment. The method comprises the steps of obtaining business logic and test requirements of a recommendation engine service, determining a test type and a test environment according to the business logic, and generating a test case according to the test requirements; according to the test type, designing a framework architecture of the automatic test framework, and developing and implementing the framework architecture to obtain an initial automatic test framework; based on the test environment and the test case, the initial automatic test framework is tested multiple times, the initial automatic test framework is adjusted according to a test result after each test is completed, an automatic test framework is obtained, and the automatic test framework is used for conducting automatic test on the recommendation engine service. And generating a first test report of the recommended engine service. The technical problem that the service stability of the recommendation engine service cannot be guaranteed due to the fact that the manual test of the recommendation engine service is difficult to fully cover and low in efficiency is solved.

Description

Technical Field

[0001] The present application relates to the field of automated testing technology, and in particular to a method, device and electronic device for creating an automated testing framework. Background Art

[0002] When building a mature industrial-grade recommendation engine service, in addition to the personalized scoring and sorting strategies based on deep learning models, the re-ranking rules at the operation and user experience levels are also crucial. These rules usually involve multiple dimensions such as business needs, user preferences, content diversity, and real-time performance. As the system continues to evolve, these rules will continue to accumulate and form a complex rule system. Therefore, in the daily iteration process of the recommendation engine service, it is particularly important to ensure that the output of the recommendation list not only complies with the personalized strategy, but also meets these complex business rules. However, with the increase in the number and complexity of rules, manual manual testing is no longer feasible. Traditional manual testing methods are not only time-consuming and labor-intensive, but also difficult to achieve comprehensive coverage due to the diversity and variability of the rules, and are prone to missing potential problems. In addition, the recommendation engine service requires frequent rapid iteration and optimization. The efficiency and coverage of manual testing are difficult to meet the needs of daily business, which may cause system instability and affect user experience and operational results. Summary of the invention

[0003] The present application provides a method, device and electronic device for creating an automated testing framework to solve the technical problem that manual testing of recommendation engine services is difficult to fully cover and inefficient, and thus cannot guarantee the business stability of the recommendation engine services.

[0004] In the first aspect, the present application provides a method for creating an automated testing framework, including: obtaining the business logic and test requirements of a recommendation engine service, and determining the test type and test environment based on the above business logic, generating test cases based on the above test requirements, wherein there is at least one test case; according to the above test type, designing a framework architecture of the automated testing framework, and developing and implementing the above framework architecture to obtain an initial automated testing framework, wherein the above framework architecture includes a request simulator, a test manager, and a result validator, and when the above automated testing framework tests the above recommendation engine service, the above request simulator is used to generate a corresponding test request based on each of the above test cases, and each of the above test requests is The above-mentioned automated test framework is sent to the recommendation engine service, the above-mentioned test manager is used to execute each of the above-mentioned test cases, and record the execution process and execution results of each of the above-mentioned test cases, the above-mentioned result verifier is used to verify the recommendation results and corresponding expected results generated by the above-mentioned recommendation engine service according to each of the above-mentioned test requests, and generate the first test report of the above-mentioned recommendation engine service; based on the above-mentioned test environment and the above-mentioned test cases, the above-mentioned initial automated test framework is tested multiple times, and the above-mentioned initial automated test framework is adjusted according to the test results after each test is completed to obtain the above-mentioned automated test framework, wherein the above-mentioned automated test framework is used to perform automated testing on the above-mentioned recommendation engine service and generate the first test report of the above-mentioned recommendation engine service.

[0005] In the second aspect, the present application provides a device for creating an automated testing framework, including: a first acquisition module, used to acquire the business logic and test requirements of the recommendation engine service, and determine the test type and test environment based on the above business logic, and generate test cases based on the above test requirements, wherein the above test case is at least one; a first design module, used to design the framework architecture of the automated testing framework based on the above test type, and develop and implement the above framework architecture to obtain an initial automated testing framework, wherein the above framework architecture includes a request simulator, a test manager and a result validator, and when the above automated testing framework tests the above recommendation engine service, the above request simulator is used to generate a corresponding test request according to each of the above test cases, and each of the above A test request is sent to the recommendation engine service. The test manager is used to execute each of the test cases and record the execution process and results of each of the test cases. The result verifier is used to verify the recommendation results and the corresponding expected results generated by the recommendation engine service according to each of the test requests, and generate a first test report for the recommendation engine service. The first test module is used to perform multiple tests on the initial automated test framework based on the test environment and the test cases, and adjust the initial automated test framework according to the test results after each test is completed to obtain the automated test framework, wherein the automated test framework is used to perform automated testing on the recommendation engine service and generate a first test report for the recommendation engine service.

[0006] As an optional example, the above-mentioned device also includes: a second design module, which is used to design the framework architecture of the automated testing framework according to the above-mentioned test type, and at the same time, design the plug-in structure of the above-mentioned automated testing framework to be extensible, so that after obtaining the above-mentioned automated testing framework, the above-mentioned automated testing framework supports adding and / or deleting the above-mentioned test cases.

[0007] As an optional example, the above-mentioned first test module includes: a deployment unit, used to deploy the above-mentioned initial automated testing framework into the above-mentioned test environment, and integrate it into the continuous integration and continuous delivery process; a testing unit, used to perform multiple tests on the above-mentioned initial automated testing framework based on the above-mentioned test cases, and adjust the above-mentioned initial automated testing framework according to the test results after each test is completed, until the number of tests reaches the target threshold or the above-mentioned initial automated testing framework is determined to have passed the test according to the above-mentioned test results, thereby obtaining the above-mentioned automated testing framework, wherein the above-mentioned test results are a second test report of the above-mentioned initial automated testing framework generated after the above-mentioned initial automated testing framework executes the above-mentioned test cases.

[0008] As an optional example, the above-mentioned device also includes: a second acquisition module, which is used to obtain the business logic of the above-mentioned recommendation engine service and / or the execution information of the above-mentioned automated testing framework once every first time interval after obtaining the above-mentioned automated testing framework; a first update module, which is used to upgrade and update the framework architecture of the above-mentioned automated testing framework according to the re-acquired business logic of the above-mentioned recommendation engine service and / or the execution information of the above-mentioned automated testing framework.

[0009] As an optional example, the above-mentioned device also includes: a third acquisition module, used to obtain the test requirements of the above-mentioned recommendation engine service once every second time period after obtaining the above-mentioned automated testing framework; a second update module, used to update the above-mentioned test case according to the re-acquired test requirements of the above-mentioned recommendation engine service.

[0010] As an optional example, the above-mentioned device also includes: a generation module, which is used to generate a usage document of the above-mentioned automated testing framework after obtaining the above-mentioned automated testing framework, wherein the above-mentioned usage document includes the installation process, configuration process and usage process of the above-mentioned automated testing framework.

[0011] As an optional example, the above-mentioned device also includes: a second testing module, which is used to perform automated testing on the above-mentioned recommendation engine service using the above-mentioned automated testing framework after obtaining the above-mentioned automated testing framework, and generate a first test report for the above-mentioned recommendation engine service, including: through the request simulator of the above-mentioned automated testing framework, a corresponding test request is generated according to each of the above-mentioned test cases, and each of the above-mentioned test requests is sent to the above-mentioned recommendation engine service, so that the above-mentioned recommendation engine service executes each of the above-mentioned test requests and outputs the corresponding recommendation results; through the test manager of the above-mentioned automated testing framework, the execution process and execution results of each of the above-mentioned test cases are recorded; through the result verifier of the above-mentioned automated testing framework, each of the above-mentioned recommendation results is verified with the corresponding expected results, and the first test report for the above-mentioned recommendation engine service is generated.

[0012] In a third aspect, the present application provides a storage medium storing a computer program, wherein the computer program executes the method for creating the above-mentioned automated testing framework when executed by a processor.

[0013] In a fourth aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the method for creating an automated testing framework through the computer program.

[0014] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art:

[0015] The present application adopts the method of obtaining the business logic and test requirements of the recommendation engine service, and determines the test type and test environment according to the above business logic, and generates test cases according to the above test requirements, wherein there is at least one test case; according to the above test type, the framework architecture of the automated testing framework is designed, and the above framework architecture is developed and implemented to obtain an initial automated testing framework, wherein the above framework architecture includes a request simulator, a test manager and a result validator. When the above automated testing framework tests the above recommendation engine service, the above request simulator is used to generate a corresponding test request according to each of the above test cases, and each of the above test requests is sent to the above recommendation engine service. The above test manager is used to execute each of the above test cases and record each of the above test cases. The execution process and results of the test case, the above-mentioned result verifier is used to verify the recommendation result and the corresponding expected result generated by the above-mentioned recommendation engine service according to each of the above-mentioned test requests, and generate the first test report of the above-mentioned recommendation engine service; based on the above-mentioned test environment and the above-mentioned test case, the above-mentioned initial automated test framework is tested multiple times, and the above-mentioned initial automated test framework is adjusted according to the test results after each test is completed to obtain the above-mentioned automated test framework, wherein the above-mentioned automated test framework is used to perform automated testing on the above-mentioned recommendation engine service and generate the method of the first test report of the above-mentioned recommendation engine service, because in the above-mentioned method, by obtaining the business logic and test requirements of the recommendation engine service, the test type and test environment are determined, and the corresponding test cases are generated. Then the framework architecture of the automated test framework is designed and developed, including a request simulator, a test manager and a result verifier, to ensure that the automated test framework can perform automated execution tests on the recommendation engine service. Then, multiple rounds of testing are carried out based on the test environment and test cases, and the framework is adjusted and optimized according to the results after each round of testing until the framework is stable and efficient. The final automated testing framework is used to perform automated testing on the recommendation engine service, thereby improving the testing efficiency of the recommendation engine service, reducing the cost of manual testing, increasing test coverage, and improving the quality and stability of the recommendation engine service. This solves the technical problem that manual testing of the recommendation engine service is difficult to fully cover and inefficient, and thus cannot guarantee the business stability of the recommendation engine service. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0019] Figure 1 is a flowchart of an optional method for creating an automated testing framework according to an embodiment of the present application;

[0020] Figure 2 is an architecture diagram of an automated testing framework according to an optional method for creating an automated testing framework according to an embodiment of the present application;

[0021] Figure 3 is a test flow chart of a recommendation engine service according to an optional method for creating an automated testing framework in an embodiment of the present application;

[0022] Figure 4 is a structural schematic diagram of an optional device for creating an automated test framework according to an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0025] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0026] According to a first aspect of an embodiment of the present application, a method for creating an automated testing framework is provided. Optionally, as follows: Figure 1 As shown, the above method includes:

[0027] S102, obtaining the business logic and test requirements of the recommendation engine service, determining the test type and test environment according to the business logic, and generating a test case according to the test requirements, wherein there is at least one test case;

[0028] S104, designing a framework architecture of an automated testing framework according to the test type, and developing and implementing the framework architecture to obtain an initial automated testing framework, wherein the framework architecture includes a request simulator, a test manager, and a result verifier. When the automated testing framework tests the recommendation engine service, the request simulator is used to generate a corresponding test request according to each test case, and send each test request to the recommendation engine service. The test manager is used to execute each test case and record the execution process and execution result of each test case. The result verifier is used to verify the recommendation result and the corresponding expected result generated by the recommendation engine service according to each test request, and generate a first test report for the recommendation engine service.

[0029] S106, based on the test environment and test cases, the initial automated test framework is tested multiple times, and after each test is completed, the initial automated test framework is adjusted according to the test results to obtain an automated test framework, wherein the automated test framework is used to perform automated testing on the recommendation engine service and generate a first test report for the recommendation engine service.

[0030] Optionally, in order to improve the reliability of the recommendation engine service and optimize the test process, in this embodiment, an automated testing framework is designed and implemented. The framework is used to perform black box testing. By simulating user requests, it is automatically verified whether the recommendation results generated by the recommendation engine service comply with the established policy rules, thereby improving the testing efficiency of the recommendation engine service, improving the system stability of the recommendation engine service, and reducing labor costs. Specifically, when building an automated testing framework, it is first necessary to obtain the business logic of the recommendation engine service, understand the core functions, business rules and various factors that may affect the recommendation results of the recommendation system, and based on these business logics, the test type and test environment can be further determined, for example, whether load testing, performance testing, functional testing, etc. are required, and select a suitable test environment, such as a development environment, a production environment or a simulation environment. The generation of test cases is defined based on business logic and requirements. Test cases are specific test scenarios or processes designed to verify the correctness of the recommendation engine service. The design of each test case generally includes input data (such as user information, content information) and expected output (recommendation results). These use cases can cover different user scenarios and business logic to ensure the comprehensiveness of the recommendation system. Next, based on the test requirements and types, the architecture of the automated testing framework is designed, such as Figure 2 The architecture diagram of the automated testing framework shown in the figure usually includes three key components:

[0031] 1. Request simulator: It generates test requests based on each test case and sends these requests to the recommendation engine service. The request simulator simulates the interaction between the user and the recommendation engine service to ensure that the response of the recommendation engine service under different inputs can be verified.

[0032] 2. Test Manager: Responsible for executing each test case and recording the execution process and results of each test case. The test manager is the core of the entire automated testing process. It coordinates various modules and tracks test progress.

[0033] 3. Result validator: Verifies the recommendation results returned by the recommendation engine service according to preset rules or business logic. It compares the generated recommendation results with the expected results to ensure that the output of the recommendation engine service meets the predetermined business requirements.

[0034] After building the architecture of the automated testing framework, the framework will undergo continuous adjustments and optimizations through multiple rounds of testing based on the test environment and test cases. First, the test framework is deployed to the selected test environment and the designed test cases are executed. After each round of testing, the execution process and results are collected, and problems such as missing framework functions, performance issues or defects are analyzed and found, and the framework is adjusted based on these feedbacks. Adjustments may include improving functions, optimizing performance, fixing defects or adding new functions. This process gradually stabilizes the framework through feedback and iteration, and ensures that it can accurately and efficiently perform various tests to verify whether the recommendation engine service meets the expected results. Finally, after multiple tests and adjustments, the automated testing framework can provide continuous and reliable testing support for the recommendation engine service, ensuring the stability and correctness of the system in continuous evolution.

[0035] Optionally, in this embodiment, by obtaining the business logic and test requirements of the recommendation engine service, the test type and test environment are determined, and corresponding test cases are generated. Then, the framework architecture is designed and developed, including a request simulator, a test manager, and a result verifier, to ensure that the framework can automatically perform tests. Then, multiple rounds of testing are performed based on the test environment and test cases, and the framework is adjusted and optimized according to the results after each round of testing until the framework is stable and efficient. After each adjustment, the framework will gradually improve, improving the coverage and accuracy of the test. Through feedback and iterative testing, the test efficiency and accuracy are improved, and the cost and time of manual testing are reduced; it ensures that the recommendation engine service complies with complex business rules and expected outputs; it provides continuous integration and rapid iteration support, and improves the stability and maintainability of the automated testing framework. Ultimately, the automated testing framework can continuously ensure the quality and reliability of the recommendation engine service and meet daily operations and iteration needs.

[0036] As an optional example, while designing the framework architecture of the automated testing framework according to the test type, the above method also includes:

[0037] The plug-in structure of the automated testing framework is designed to be extensible, so that after the automated testing framework is obtained, the automated testing framework supports adding and / or deleting test cases.

[0038] Optionally, in this embodiment, the plug-in structure of the automated test framework is designed to be extensible, so that the design of the framework has flexibility and modular characteristics, and test cases can be easily added, modified or deleted as needed. The plug-in structure usually provides an independent module for each test case or test scenario, so that the test framework can be expanded and maintained in a modular manner without large-scale modifications to the entire framework. Through this design, users can quickly plug in and out new test cases or delete test scenarios that are no longer needed according to different business needs or test goals without affecting the overall structure and other functions of the framework. Specifically, the plug-in structure may include the following features: each test case or test component exists as an independent module, supporting independent deployment and replacement; new test cases can be dynamically loaded through configuration files or automated scripts without manually modifying the core code of the framework; the coupling between test cases and the core functions of the framework is low, and the dependency between modules is minimal, which is convenient for adding or deleting test cases; unified standards are usually defined between the framework interface and the plug-in, so that new test cases can be easily connected to the existing framework to avoid unnecessary complexity.

[0039] Optionally, in this embodiment, the plug-in design enables the automated testing framework to easily respond to different changes in requirements and quickly adapt to business and technical adjustments. New functions or new use cases can be inserted or deleted at any time without affecting the stability of the entire framework. As the business develops, new test scenarios or test requirements may emerge. Through the plug-in mechanism, test cases can be expanded as needed without the need to reconstruct the entire framework. When business requirements and system architecture change frequently, the plug-in structure can help the testing team respond quickly, quickly integrate new test cases or delete test cases that are no longer needed, thereby improving the adaptability and response speed of the automated testing framework.

[0040] As an optional example, based on the test environment and the test cases, the initial automated test framework is tested multiple times, and after each test is completed, the initial automated test framework is adjusted according to the test results, and the obtained automated test framework includes:

[0041] Deploy the initial automated testing framework into the test environment and integrate it into the continuous integration and continuous delivery process;

[0042] Based on the test cases, the initial automated test framework is tested multiple times, and the initial automated test framework is adjusted according to the test results after each test is completed, until the number of tests reaches a target threshold or the initial automated test framework is determined to have passed the test according to the test results, to obtain an automated test framework, wherein the test results are a second test report of the initial automated test framework generated after the initial automated test framework executes the test cases.

[0043] Optionally, in the present embodiment, in the process of developing and improving the automated test framework, multiple tests and continuous adjustments are one of the key steps. The purpose of this process is to optimize and stabilize the automated test framework through multiple tests, evaluation results and adjustment frameworks. Specifically, the initial automated test framework is first deployed in a test environment similar to the production environment. The test environment ensures that the framework runs under real workload and environmental conditions, can simulate actual application scenarios, and ensure the accuracy and reliability of the framework. The framework needs to be integrated with the existing continuous integration (CI) and continuous delivery (CD) processes so that after each code submission, the automated test framework will automatically execute predetermined test cases and generate reports to help the development team find and solve problems in a timely manner. Through this integration, testing can be highly automated to ensure that the code quality is continuously stable. According to the designed test cases, the automated test framework will perform tests in different scenarios, which usually include functional tests, performance tests, compatibility tests, regression tests, etc., to ensure that the framework can handle a variety of complex situations. After each test is completed, the test results will be fed back to the developers. If problems or deficiencies are found in the framework, the development team can adjust the framework based on the test report. For example, the logic of the framework may be adjusted, the problems found may be fixed, and the performance may be optimized. This adjustment is usually a cyclic process until the framework reaches the expected stability and performance. The test will continue until the number of tests reaches a predetermined threshold, or the test results show that the framework has been fully verified and all key issues have been resolved. This target threshold is usually determined by factors such as project requirements, test case coverage, and key function verification. When the framework passes all necessary test cases and can be executed stably, the framework is considered to have "passed the test". At this time, the final test report can be generated to confirm that the automated test framework can be applied in the actual environment. After each test is completed, a second test report will be generated to record the performance of the framework during the test, including all executed test cases, the results of each test case, the defects found, adjustments and optimizations, etc. This report is an important basis for evaluating whether the framework meets the requirements.

[0044] Optionally, in this embodiment, through continuous testing and adjustment, potential problems in the framework can be discovered and solved in a timely manner, ensuring that the framework is gradually improved and meets the requirements. The iterative testing process can help the development team improve the stability and reliability of the framework. The automated testing framework can quickly feedback and fix problems in the framework through continuous testing and adjustment. After being integrated into the continuous integration and continuous delivery process, the automation of testing greatly improves the efficiency of quality assurance and saves a lot of manual testing time.

[0045] As an optional example, after obtaining the automated testing framework, the method further includes:

[0046] At every first time interval, obtain the execution information of the business logic and / or automated testing framework of the recommendation engine service;

[0047] The framework architecture of the automated testing framework is upgraded and updated based on the re-acquired business logic of the recommendation engine service and / or the execution information of the automated testing framework.

[0048] Optionally, in this embodiment, after obtaining the automated test framework, continuous business logic monitoring and framework updates are important steps to ensure that the framework always meets business requirements and adapts to system changes. This process includes regularly obtaining the business logic and framework execution information of the recommendation engine service, and upgrading and updating the test framework based on this information. Specifically, the business logic of the recommendation engine service may change over time. Therefore, obtaining the business logic of the recommendation engine service once every first time interval can enable the automated test framework to keep abreast of business changes, including adjustments to the recommendation algorithm, newly added business rules, changes in recommendation dimensions, etc. Check these changes regularly to ensure that the automated test framework is synchronized with the business logic of the recommendation engine. The execution information includes the status of the framework operation, the executed test cases, the test results, etc. This information can help developers analyze the execution of the test framework, discover potential framework problems, performance bottlenecks, or blind spots covered by the test. By regularly obtaining this information, it is possible to promptly discover deficiencies in the test framework and make improvements. The architecture of the automated test framework needs to be adjusted according to the new business logic and execution information. For example, changes in business logic may require adjustment of the test strategy to cover new recommendation rules and algorithms. The execution information of the framework can also reveal performance problems, thereby optimizing the performance or improving the functionality of the framework. Based on the execution information, the performance, test efficiency, and stability of the framework may need to be optimized. Optimizing the framework, such as fixing issues, enhancing logging capabilities, and improving the accuracy of result validators, can make the automated testing framework more efficient and reliable.

[0049] As an optional example, after obtaining the automated testing framework, the method further includes:

[0050] At every second interval, obtain the test requirements of the recommendation engine service;

[0051] The test cases are updated according to the newly acquired test requirements of the recommendation engine service.

[0052] Optionally, in this embodiment, as the recommendation engine service continues to evolve, new functions, requirements, and rules may be introduced. Obtaining the latest test requirements for the recommendation engine service every second interval can help the test team understand which new functions or rules need to be verified, or which existing rules need to be adjusted. Test requirements usually include verification requirements for new functions of the recommendation engine, adjusted business logic, new performance requirements, etc. By regularly checking and obtaining the latest test requirements, it is possible to ensure that the test cases and test frameworks do not lag behind in business changes and respond to the evolution and update of the recommendation engine service in a timely manner. Once new test requirements are obtained, the test team will update the existing test cases, which may include adding new test cases to verify the new functions or requirements, or modifying existing test cases to adapt to the adjustment of business logic. For example, if the recommendation engine algorithm is optimized, it may be necessary to update the test cases for sorting and recommendation results. By updating the test cases, it can be ensured that all new requirements and business logic are effectively covered, avoiding omissions and defects, and ensuring that every change of the recommendation engine is fully and rigorously tested.

[0053] As an optional example, after obtaining the automated testing framework, the method further includes:

[0054] Generate a usage document for the automated testing framework, wherein the usage document includes the installation process, configuration process, and usage process of the automated testing framework.

[0055] Optionally, in this embodiment, after obtaining the automated test framework, a usage document is generated to ensure that the automated test framework can be used and maintained smoothly. The usage document provides developers, testers and operation and maintenance teams with a detailed guide to the framework to help them understand how to install, configure and use the framework. Specifically, the usage document includes the installation steps of the automated test framework, and the document describes in detail how to download, deploy and set up the environment, dependencies and tool chains required for the test framework, including operating system requirements, configuration requirements for third-party libraries and services, etc. The configuration document of the framework details how to personalize the framework according to specific project requirements, which can include how to set up test cases, select test environments, configure log output, configure report generation, etc. The document also provides steps on how to use the automated test framework, including how to perform automated testing through the framework, how to manage and execute test cases, how to view and analyze test reports, etc. For possible problems and common error scenarios, the usage document also includes a detailed troubleshooting guide to help users solve common problems encountered when installing, configuring and executing tests. Generating detailed usage documents can help developers, testers and other relevant personnel quickly get started with the automated test framework without additional training or repeated help. The document provides clear operating steps and guidance, which improves the ease of use of the framework.

[0056] As an optional example, after obtaining the automated testing framework, the method further includes:

[0057] Use the automated testing framework to perform automated testing on the recommendation engine service and generate the first test report of the recommendation engine service, including:

[0058] Generate a corresponding test request according to each test case through the request simulator of the automated testing framework, and send each test request to the recommendation engine service so that the recommendation engine service executes each test request and outputs the corresponding recommendation result;

[0059] Record the execution process and results of each test case through the test manager of the automated testing framework;

[0060] Through the result validator of the automated testing framework, each recommendation result is verified against the corresponding expected result, and the first test report of the recommendation engine service is generated.

[0061] Optionally, in this embodiment, the recommendation engine service is automatically tested using an automated testing framework to ensure that the system works stably as expected and to promptly detect and resolve potential problems. Figure 3As shown in the test flow chart, after the user sends a test instruction to the automated test framework, the request simulator in the automated test framework generates corresponding test requests according to each test case. These test requests represent different business scenarios and conditions, which are used to simulate the actual user's behavior or system call. Each request contains specific input data and parameters to ensure the comprehensiveness of the test. The generated request is then sent to the recommendation engine service, at which time the recommendation engine service performs the corresponding recommendation calculation according to the received request and generates the recommendation result. The test manager of the automated test framework is responsible for managing the execution process of each test case. It records the execution process (such as start time, end time, execution status) and execution result (such as success or failure) of each case. The test manager also handles errors, exceptions and retry mechanisms to ensure that they can be recorded and reported in a timely manner when faced with unexpected situations. After the test is completed, the result validator in the automated test framework compares the execution result (the recommendation result of the recommendation engine service) with the expected result set in advance. The expected result is pre-defined based on business requirements or historical data. The result validator verifies the accuracy, relevance and completeness of the recommendation result to ensure that the output of the recommendation engine meets the set rules and expectations. If the actual result does not match the expected result, the result validator will generate a corresponding failure report so that developers and testers can fix the problem in time. After the test is completed, the automated testing framework will integrate the information recorded by the test manager and the comparison results of the result validator to generate the first test report of the recommendation engine service and return it to the user. The report describes in detail the execution of each test case, the statistics of success and failure, and the specific error information and reasons. The test report also includes performance indicators, response time, stability test results, etc. to comprehensively evaluate the working status and performance of the recommendation engine service.

[0062] Optionally, in this embodiment, the automated testing framework can ensure that the recommendation engine service can run stably in various situations by simulating different business scenarios and test cases. It is more efficient and comprehensive than manual testing, and can cover more boundary conditions and special cases. The automated testing framework eliminates the need for human intervention in the testing process, thereby saving a lot of time, and the execution process and steps of each test are consistent, reducing deviations and errors introduced by human factors. The application of the automated framework greatly reduces the time and manpower required for manual testing, avoids manual operation errors, and can quickly respond to the rapid iteration and update of the recommendation engine service.

[0063] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0064] According to another aspect of the embodiment of the present application, a device for creating an automated testing framework is also provided. Figure 4 As shown, including:

[0065] The first acquisition module 402 is used to acquire the business logic and test requirements of the recommendation engine service, determine the test type and test environment according to the business logic, and generate a test case according to the test requirements, wherein there is at least one test case;

[0066] The first design module 404 is used to design a framework architecture of the automated test framework according to the test type, and to develop and implement the framework architecture to obtain an initial automated test framework, wherein the framework architecture includes a request simulator, a test manager, and a result verifier. When the automated test framework tests the recommendation engine service, the request simulator is used to generate a corresponding test request according to each test case, and send each test request to the recommendation engine service. The test manager is used to execute each test case and record the execution process and execution result of each test case. The result verifier is used to verify the recommendation result and the corresponding expected result generated by the recommendation engine service according to each test request, and generate a first test report for the recommendation engine service.

[0067] The first test module 406 is used to perform multiple tests on the initial automated test framework based on the test environment and test cases, and adjust the initial automated test framework according to the test results after each test is completed to obtain an automated test framework, wherein the automated test framework is used to perform automated testing on the recommendation engine service and generate a first test report for the recommendation engine service.

[0068] It should be noted that the first acquisition module 402 in this embodiment can be used to execute step S102 in the embodiment of the present application, the first design module 404 in this embodiment can be used to execute step S104 in the embodiment of the present application, and the first test module 406 in this embodiment can be used to execute step S106 in the embodiment of the present application.

[0069] As an optional example, the above device further includes:

[0070] The second design module is used to design the framework architecture of the automated testing framework according to the test type, and at the same time design the plug-in structure of the automated testing framework to be extensible, so that after the automated testing framework is obtained, the automated testing framework supports adding and / or deleting test cases.

[0071] As an optional example, the first test module includes:

[0072] Deployment unit, which is used to deploy the initial automated test framework into the test environment and integrate it into the continuous integration and continuous delivery process;

[0073] A testing unit is used to perform multiple tests on an initial automated testing framework based on test cases, and adjust the initial automated testing framework according to the test results after each test is completed, until the number of tests reaches a target threshold or the initial automated testing framework is determined to have passed the test according to the test results, thereby obtaining an automated testing framework, wherein the test results are a second test report of the initial automated testing framework generated after the initial automated testing framework executes the test cases.

[0074] As an optional example, the above device further includes:

[0075] A second acquisition module is used to acquire the business logic of the recommendation engine service and / or the execution information of the automated testing framework once at a first time interval after the automated testing framework is obtained;

[0076] The first updating module is used to upgrade the framework architecture of the automated testing framework according to the re-acquired business logic of the recommendation engine service and / or the execution information of the automated testing framework.

[0077] As an optional example, the above device further includes:

[0078] A third acquisition module is used to acquire the test requirements of the recommendation engine service once every second time period after obtaining the automated test framework;

[0079] The second updating module is used to update the test cases according to the re-acquired test requirements of the recommendation engine service.

[0080] As an optional example, the above device further includes:

[0081] The generation module is used to generate a usage document of the automated testing framework after obtaining the automated testing framework, wherein the usage document includes the installation process, configuration process and usage process of the automated testing framework.

[0082] As an optional example, the above device further includes:

[0083] The second testing module is used to perform automated testing on the recommendation engine service using the automated testing framework after obtaining the automated testing framework, and generate a first test report for the recommendation engine service, including:

[0084] Generate a corresponding test request according to each test case through the request simulator of the automated testing framework, and send each test request to the recommendation engine service so that the recommendation engine service executes each test request and outputs the corresponding recommendation result;

[0085] Record the execution process and results of each test case through the test manager of the automated testing framework;

[0086] Through the result validator of the automated testing framework, each recommendation result is verified against the corresponding expected result, and the first test report of the recommendation engine service is generated.

[0087] For other examples of this embodiment, please refer to the above examples and will not be repeated here.

[0088] Figure 5 is a schematic diagram of an optional electronic device according to an embodiment of the present application, such as Figure 5 As shown, it includes a processor 502, a communication interface 504, a memory 506 and a communication bus 508, wherein the processor 502, the communication interface 504 and the memory 506 communicate with each other through the communication bus 508, wherein,

[0089] Memory 506, used for storing computer programs;

[0090] The processor 502 is used to execute the computer program stored in the memory 506 to implement the following steps:

[0091] Obtain the business logic and test requirements of the recommendation engine service, determine the test type and test environment according to the business logic, and generate test cases according to the test requirements, wherein there is at least one test case;

[0092] According to the test type, the framework architecture of the automated testing framework is designed, and the framework architecture is developed and implemented to obtain an initial automated testing framework, wherein the framework architecture includes a request simulator, a test manager, and a result verifier. When the automated testing framework tests the recommendation engine service, the request simulator is used to generate a corresponding test request according to each test case, and send each test request to the recommendation engine service. The test manager is used to execute each test case and record the execution process and execution result of each test case. The result verifier is used to verify the recommendation result and the corresponding expected result generated by the recommendation engine service according to each test request, and generate a first test report for the recommendation engine service.

[0093] Based on the test environment and test cases, the initial automated testing framework is tested multiple times, and after each test is completed, the initial automated testing framework is adjusted according to the test results to obtain an automated testing framework, wherein the automated testing framework is used to perform automated testing on the recommendation engine service and generate a first test report for the recommendation engine service.

[0094] Optionally, in this embodiment, the communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industrial Standard Architecture) bus. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The communication interface is used for communication between the above electronic device and other devices.

[0095] The memory may include RAM, or may include non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0096] As an example, the above-mentioned memory 506 may include but is not limited to the first acquisition module 402, the first design module 404 and the first test module 406 in the creation device of the above-mentioned automated testing framework. In addition, it may also include but is not limited to other module units in the creation device of the above-mentioned automated testing framework, which will not be repeated in this example.

[0097] The above-mentioned processor can be a general-purpose processor, which can include but not be limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0098] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0099] It can be understood by those skilled in the art that Figure 5 The structure shown is for illustration only. The device for implementing the above-mentioned method for creating the automated testing framework may be a terminal device, which may be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Figure 5 The structure of the electronic device is not limited. For example, the electronic device may also include Figure 5 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 5 Different configurations shown.

[0100] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0101] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program executes the steps in the method for creating the automated testing framework when executed by a processor.

[0102] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0103] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0104] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0105] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0106] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0107] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0108] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0109] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for creating an automated testing framework, characterized in that: include: Obtaining the business logic and test requirements of the recommendation engine service, determining the test type and test environment according to the business logic, and generating a test case according to the test requirements, wherein the test case is at least one; According to the test type, a framework architecture of an automated testing framework is designed, and the framework architecture is developed and implemented to obtain an initial automated testing framework, wherein the framework architecture includes a request simulator, a test manager, and a result verifier. When the automated testing framework tests the recommendation engine service, the request simulator is used to generate a corresponding test request according to each of the test cases, and send each of the test requests to the recommendation engine service. The test manager is used to execute each of the test cases and record the execution process and results of each of the test cases. The result verifier is used to verify the recommendation results and corresponding expected results generated by the recommendation engine service according to each of the test requests, and generate a first test report for the recommendation engine service. Based on the test environment and the test cases, the initial automated testing framework is tested multiple times, and after each test is completed, the initial automated testing framework is adjusted according to the test results to obtain the automated testing framework, wherein the automated testing framework is used to perform automated testing on the recommendation engine service and generate a first test report for the recommendation engine service.

2. The method according to claim 1, characterized in that While designing the framework architecture of the automated testing framework according to the test type, the method further includes: The plug-in structure of the automated testing framework is designed to be extensible, so that after the automated testing framework is obtained, the automated testing framework supports adding and / or deleting the test cases.

3. The method according to claim 1, characterized in that The initial automated test framework is tested multiple times based on the test environment and the test case, and the initial automated test framework is adjusted according to the test result after each test is completed to obtain the automated test framework, including: Deploy the initial automated testing framework into the testing environment and integrate it into the continuous integration and continuous delivery process; Based on the test case, the initial automated test framework is tested multiple times, and the initial automated test framework is adjusted according to the test result after each test is completed, until the number of tests reaches a target threshold or the initial automated test framework is determined to have passed the test according to the test result, thereby obtaining the automated test framework, wherein the test result is a second test report of the initial automated test framework generated after the initial automated test framework executes the test case.

4. The method according to claim 1, characterized in that After obtaining the automated testing framework, the method further includes: Obtaining the business logic of the recommendation engine service and / or the execution information of the automated testing framework once every first time interval; The framework architecture of the automated testing framework is upgraded and updated according to the re-acquired business logic of the recommendation engine service and / or the execution information of the automated testing framework.

5. The method according to claim 1, characterized in that: After obtaining the automated testing framework, the method further includes: At every second time interval, obtaining a test requirement of the recommendation engine service; The test case is updated according to the re-acquired test requirements of the recommendation engine service.

6. The method according to claim 1, characterized in that After obtaining the automated testing framework, the method further includes: Generate a usage document for the automated testing framework, wherein the usage document includes an installation process, a configuration process, and a usage process of the automated testing framework.

7. The method according to claim 1, characterized in that After obtaining the automated testing framework, the method further includes: Using the automated testing framework to perform automated testing on the recommendation engine service and generate a first test report for the recommendation engine service, including: Generate a corresponding test request according to each test case through the request simulator of the automated testing framework, and send each test request to the recommendation engine service, so that the recommendation engine service executes each test request and outputs a corresponding recommendation result; Recording the execution process and execution result of each test case through the test manager of the automated testing framework; Each of the recommendation results is verified against the corresponding expected result through the result verifier of the automated testing framework, and a first test report of the recommendation engine service is generated.

8. A device for creating an automated testing framework, characterized in that: include: A first acquisition module is used to acquire the business logic and test requirements of the recommendation engine service, determine the test type and test environment according to the business logic, and generate a test case according to the test requirements, wherein the test case is at least one; A first design module is used to design a framework architecture of an automated testing framework according to the test type, and to develop and implement the framework architecture to obtain an initial automated testing framework, wherein the framework architecture includes a request simulator, a test manager, and a result verifier. When the automated testing framework tests the recommendation engine service, the request simulator is used to generate a corresponding test request according to each of the test cases, and send each of the test requests to the recommendation engine service. The test manager is used to execute each of the test cases and record the execution process and results of each of the test cases. The result verifier is used to verify the recommendation results and corresponding expected results generated by the recommendation engine service according to each of the test requests, and to generate a first test report for the recommendation engine service. The first testing module is used to perform multiple tests on the initial automated testing framework based on the testing environment and the test cases, and adjust the initial automated testing framework according to the test results after each test is completed to obtain the automated testing framework, wherein the automated testing framework is used to perform automated testing on the recommendation engine service and generate a first test report for the recommendation engine service.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images