Code testing method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a code testing method and device, electronic equipment and a storage medium. The method comprises: testing target source code according to test code corresponding to the target source code, obtaining a test result, if the test result indicates that the target source code fails the test, obtaining test data corresponding to the target source code and code change information of the target source code, wherein the code change information is used to indicate change information of the target source code relative to a historical version code of the target source code; thereby detecting a test failure cause according to the test data and the code change information, obtaining test failure cause information, and generating test repair instruction information according to the test failure cause information, so as to retest the target source code according to the test repair instruction information and the test code. The technical solution of the embodiments of the present application improves the accuracy of test failure cause detection and improves the test success rate.

Description

Technical Field

[0001] This application relates to the field of computers, and more specifically, to a code testing method and apparatus, electronic equipment, storage medium, and program product. Background Technology

[0002] To verify whether the code meets requirements, identify code defects, and ensure stable operation after deployment, the developed code needs to be tested. If a test fails, the cause of the failure needs to be investigated, and the test needs to be retested. In related technologies, this is usually done manually based on test logs to determine the cause of the failure, followed by manual repair based on the cause, and then retesting. However, manual analysis is time-consuming, and the subjectivity and limitations of manual inspection reduce the efficiency and accuracy of cause detection, resulting in lengthy code testing times. Summary of the Invention

[0003] Embodiments of this application provide a code testing method and apparatus, electronic device, storage medium, and program product, which can improve the efficiency and accuracy of detecting test failure causes and improve code testing efficiency.

[0004] According to one aspect of the embodiments of this application, a code testing method is provided, the method comprising:

[0005] Based on the test code corresponding to the target source code to be tested, the target source code is tested to obtain the test results; If the test result indicates that the target source code test failed, then the test data corresponding to the target source code and the code change information of the target source code are obtained; wherein, the code change information is used to indicate the changes of the target source code relative to the historical version of the target source code; Based on the test data and the code change information, test failure reasons are detected to obtain test failure reason information, and test repair instruction information is generated based on the test failure reason information. Based on the test repair instructions and the test code, the target source code is retested.

[0006] According to one aspect of the embodiments of this application, a code testing apparatus is provided, the apparatus comprising: The testing module is used to test the target source code based on the test code corresponding to the target source code to be tested, and obtain the test results. The acquisition module is used to acquire test data corresponding to the target source code and code change information of the target source code if the test result indicates that the target source code test failed; wherein, the code change information is used to indicate the change information of the target source code relative to the historical version of the target source code; The detection module is used to detect the reasons for test failure based on the test data and the code change information, obtain the reason information for test failure, and generate test repair instruction information based on the reason information for test failure. The testing module is also used to retest the target source code based on the test repair instruction information and the test code.

[0007] In an exemplary embodiment, based on the foregoing scheme, the detection module is specifically used for: Error information is obtained from the test data, and code segment location information related to the code segment contained in the target source code is extracted from the error information; Based on the code change information, detect whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code, obtain the change detection result, and perform test failure reason detection based on the change detection result to obtain the test failure reason information.

[0008] In an exemplary embodiment, based on the foregoing scheme, the detection module is specifically used for: If the error message indicates a logical error, then based on the code change information, it is detected whether the logical code corresponding to the code segment location information contained in the target source code has changed, and the change detection result is obtained; If the change detection result indicates that the logic code has been changed, test failure reason information indicating that there is a logical error in the target source code is generated; If the change detection result indicates that the logic code has not been changed, test failure reason information indicating that the test code is abnormal is generated.

[0009] In an exemplary embodiment, based on the foregoing scheme, the detection module is specifically used for: If the error message indicates a user interface error, then based on the code change information, detect whether the user interface corresponding to the code segment location information in the target source code has changed, and obtain the change detection result; If the change detection result indicates that the user interface has been changed, test failure reason information indicating that the user interface in the target source code has been changed is generated.

[0010] In an exemplary embodiment, based on the foregoing scheme, the detection module is specifically used for: If the error message indicates an error in the runtime environment, then based on the code change information, detect whether the environment configuration code indicated by the code segment location information in the target source code has changed, and obtain the change detection result; If the change detection result indicates that the environment configuration code has been changed, test failure reason information indicating that the environment configuration of the target source code is incorrect is generated.

[0011] In an exemplary embodiment, based on the foregoing scheme, the detection module is specifically configured to: if the test failure reason information indicates that the code segment corresponding to the code segment location information has changed, then determine the code segment to be modified from the test code and the target source code according to the error information and the test failure reason information, and generate modification instruction information corresponding to the code segment to be modified according to the code change data corresponding to the code segment location information contained in the code change information, thereby obtaining the test repair instruction information; The testing module is specifically used to: modify the code segment to be modified according to the modification instruction information to obtain the modified code segment, and retest the target source code according to the modified code segment.

[0012] In an exemplary embodiment, based on the foregoing scheme, the detection module is specifically used for: If the error message indicates a user interface error, and the test failure reason information indicates that the user interface corresponding to the code segment location information in the target source code has changed, then based on the code change data of the user interface contained in the code change information, the changed interface elements in the user interface and the change data of the changed interface elements are found, and the element selector corresponding to the changed interface elements in the test code is taken as the code segment to be modified. Based on the change data of the modified interface elements, the modification method of the element selector is generated, and the test repair instruction information is generated based on the modification method.

[0013] In an exemplary embodiment, based on the foregoing scheme, the detection module is specifically used for: Generate a cause detection prompt based on the test data and the code change information; The cause detection prompt information is input into the large language model so that the large language model responds to the cause detection prompt information, performs test failure cause detection based on the test data and the code change information, and obtains the test failure cause information output by the large language model; The test repair instruction information is generated based on the test failure reason information using the large language model.

[0014] In an exemplary embodiment, based on the foregoing scheme, the detection module is specifically used for: Obtain the preset cause detection rules, and generate the cause detection prompt information based on the cause detection rules, the test data, and the code change information; The cause detection prompt information is input into the large language model, so that the large language model can detect the test data and the code change information according to the cause detection rules contained in the cause detection prompt information, and obtain the test failure cause information output by the large language model.

[0015] In an exemplary embodiment, based on the foregoing scheme, the detection module is specifically used for: Generate target test failure features based on the test data of the target source code; The system searches the failure case database for similar test failure features that meet the similarity criteria to the target test failure feature, as well as the test failure reason information corresponding to the similar test failure feature; wherein, the failure case database contains multiple test failure features and test failure reason information corresponding to each test failure feature; The test failure reason information corresponding to the similar test failure features and the reason detection prompt information are input into the large language model, so that the large language model can perform test failure reason detection based on the test data of the target source code, the code change information and the test failure reason information corresponding to the similar test failure features, and obtain the test failure reason information of the target source code output by the large language model.

[0016] In an exemplary embodiment, based on the foregoing scheme, the detection module is specifically used for: Perform code semantic analysis on each piece of code change data contained in the code change information to obtain the change semantic information corresponding to each piece of code change data; Based on the change semantic information corresponding to each code change data, code change data to be filtered out is found from the multiple code change data contained in the code change information; wherein, the change semantic information of the code change data to be filtered out indicates that the change type of the code change data to be filtered out is a change type other than code logical structure change; The code change data to be filtered is deleted from the code change information to obtain the target code change information. Based on the test data and the target code change information, the test failure reason is detected to obtain the test failure reason information.

[0017] In an exemplary embodiment, based on the foregoing scheme, the detection module is specifically used for: Obtain the error information of the target source code from the test data, and find the stack line corresponding to the framework code of the target source code from the error stack contained in the error information; Delete the stack line corresponding to the framework code from the error stack to obtain the error stack after deletion, and find the stack line corresponding to the storage path of the target source code from the error stack after deletion to obtain the target error stack. Based on the target error stack and the code change information, the test failure reason is detected to obtain the test failure reason information.

[0018] According to one aspect of the embodiments of this application, an electronic device is provided, comprising: At least one processor; A storage device for storing at least one computer program, which, when executed by the at least one processor, causes the electronic device to implement the code testing method as described in any of the preceding claims.

[0019] According to one aspect of the present application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor of an electronic device, causes the electronic device to implement the code testing method as described in any of the preceding claims.

[0020] According to one aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the code testing method as described in any of the preceding claims.

[0021] In the technical solution provided in the embodiments of this application, the target source code is first tested according to the test code corresponding to the target source code to be tested, and the test result is obtained. If the test result indicates that the target source code test fails, the test data corresponding to the target source code and the code change information of the target source code are obtained. The code change information is used to indicate the change information of the target source code relative to the historical version of the target source code. Then, based on the test data and the code change information, the test failure reason detection is performed to obtain the test failure reason information. Based on the test failure reason information, test repair instruction information is generated. Then, based on the test repair instruction information and the test code, the target source code is retested. Compared to related technologies, this method offers several advantages. First, it can automatically detect test failures based on test data and code change information, improving detection efficiency and reducing costs. Furthermore, combining test data and code change information in test failure detection can help determine if a failure is related to code changes, thus improving the accuracy of the detection. Second, since the test failure cause information is determined based on test data and code change information, it can help identify whether modifications to the test code or the code under test are necessary. This allows for the generation of test repair instructions, which in turn enable retesting of the target source code, thereby improving testing efficiency and success rate.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0023] Figure 1 This is a schematic diagram illustrating an implementation environment as shown in an exemplary embodiment of this application; Figure 2 This is a flowchart illustrating a code testing method in an exemplary embodiment of this application; Figure 3 This is a flowchart illustrating a code testing method as shown in another exemplary embodiment of this application; Figure 4 This is a flowchart illustrating a code testing method as shown in another exemplary embodiment of this application; Figure 5 This is a schematic diagram illustrating cause detection prompt information in an exemplary embodiment of this application; Figure 6 This is a flowchart illustrating a code testing method as shown in another exemplary embodiment of this application; Figure 7 This is a schematic diagram illustrating a test failure cause detection process, as shown in an exemplary embodiment of this application. Figure 8 This is a schematic diagram illustrating the display interface of the failure detection result in an exemplary embodiment of this application; Figure 9 This is a schematic diagram of a code testing apparatus shown in an exemplary embodiment of this application; Figure 10 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0024] Exemplary embodiments will now be described in a more comprehensive manner with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be construed as limited to these examples; rather, these embodiments are provided so that this application will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art.

[0025] Furthermore, the features, structures, or characteristics described in this application can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to provide a full understanding of the embodiments of this application. However, those skilled in the art will recognize that when implementing the technical solutions of this application, not all the detailed features in the embodiments may be used, one or more specific details may be omitted, or other methods, elements, devices, steps, etc., may be employed.

[0026] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or at least two processors or memory) can be used to implement one or at least two modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0027] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or at least two hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0028] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0029] It should also be noted that "multiple" as mentioned in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0030] The technical solutions of the embodiments of this application are described in detail below: In related technologies, manual analysis of test logs is typically used to determine the causes of test failures, followed by manual repairs and retesting. However, manual analysis is time-consuming, and the subjectivity and limitations of manual detection reduce the efficiency and accuracy of cause detection, resulting in lengthy code testing times. Therefore, embodiments of this application provide a code testing method and apparatus, electronic device, storage medium, and program product that can improve the efficiency and accuracy of test failure cause detection.

[0031] Please see Figure 1 , Figure 1 This is a schematic diagram of an implementation environment involved in this application, which includes a terminal device 101 and a server 102. The terminal device 101 includes, but is not limited to, mobile phones, tablets, laptops, computers, voice interaction devices, home appliances, vehicle terminals, aircraft, etc.; the server 102 can be an independent physical server, a server cluster or distributed system consisting of at least two physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. This document does not limit the specific form of the terminal device or server.

[0032] In an exemplary embodiment, the code testing method provided in the embodiments of this application can be executed by a terminal device 101. For example, the terminal device 101 can first test the target source code according to the test code corresponding to the target source code to be tested, and obtain the test result; if the test result indicates that the target source code test failed, then it obtains the test data corresponding to the target source code and the code change information of the target source code, wherein the code change information is used to indicate the change information of the target source code relative to the historical version of the target source code; thereby, based on the test data and the code change information, it performs test failure reason detection, obtains test failure reason information, and generates test repair instruction information based on the test failure reason information, thereby retesting the target source code according to the test repair instruction information and the test code.

[0033] In an exemplary embodiment, the code testing method provided in the embodiments of this application can be executed by server 102. For example, server 102 can first test the target source code according to the test code corresponding to the target source code to be tested, and obtain the test result; if the test result indicates that the target source code test failed, then it obtains the test data corresponding to the target source code and the code change information of the target source code; thereby, based on the test data and the code change information, it performs test failure reason detection, obtains test failure reason information, and generates test repair instruction information based on the test failure reason information, and then retests the target source code according to the test repair instruction information and the test code.

[0034] In an exemplary embodiment, the code testing method provided in this application can be jointly executed by a terminal device 101 and a server 102. For example, the terminal device 101 can test the target source code according to the test code corresponding to the target source code to be tested, obtain the test result, and send it to the server 102. If the server 102 detects that the test result indicates that the target source code test has failed, it obtains the test data corresponding to the target source code and the code change information of the target source code; then, based on the test data and code change information, it performs test failure reason detection, obtains test failure reason information, and generates test repair instruction information based on the test failure reason information. Finally, based on the test repair instruction information and the test code, it re-tests the target source code.

[0035] It should be noted that, Figure 1 The number of terminal devices 101 and servers 102 shown is merely exemplary. Any number of terminal devices 101 and servers 102 can be set up according to actual needs.

[0036] The embodiments of this application involve user-related data such as test code and target source code. When the methods of this application are applied to specific products or technologies, user permission or consent is obtained, and the extraction, use and processing of related data comply with local security standards and local laws and regulations.

[0037] See Figure 2 , Figure 2 This is a flowchart illustrating a code testing method in an exemplary embodiment of this application. This method can be applied to... Figure 1 The implementation environment shown can be composed of Figure 1 The terminal device 101 in the implementation environment shown can execute this, or it can be executed by... Figure 1 The server 102 in the implementation environment shown can also be executed by Figure 1 The terminal device 101 and server 102 in the implementation environment shown are executed together.

[0038] like Figure 2 As shown, in an exemplary embodiment, the code testing method may include S201-S204, which are described in detail below: S201, Test the target source code according to the test code corresponding to the target source code to be tested, and obtain the test results.

[0039] It should be noted that target source code refers to any code that needs to be tested, that is, the code being tested. It can be code used to implement specific functions or business logic. For example, according to the runtime environment / purpose, the types of target source code include, but are not limited to, front-end code, back-end code, mobile code, embedded code, script code, etc. According to the inclusion / hierarchical relationship, the types of target source code include, but are not limited to, applications, application subsystems / modules, functional units, code blocks, etc. Among them, an application is a software whole that can run independently and realize complete business value, containing all the code and configurations that implement product functions. Applications are broken down into multiple subsystems / modules according to "business domains". Each module is responsible for an independent type of business function. Modules are further broken down into smaller "functional units". Each unit implements a specific and single function. Functional units can be broken down into smaller "executable code fragments", that is, code blocks.

[0040] Test code refers to code used to test the target source code, such as test script files. It can be used for functional testing and non-functional testing of the target source code. Functional testing is used to test whether the function of the target source code meets expectations. Non-functional testing includes, but is not limited to, boundary value / abnormal scenario testing (used to test the robustness of code under boundary conditions and abnormal inputs), performance testing (used to test whether the execution efficiency and resource consumption of code meet expectations), and security testing (used to verify whether the code has security vulnerabilities).

[0041] In order to test the target source code, the test code is called and run, and the target source code is run based on the test code, thereby achieving the test of the target source code and obtaining the test results.

[0042] Optionally, test code and target source code can be submitted to the automated testing system. The automated testing system then uses the test code and target source code to execute the corresponding test cases using automated testing tools, thus testing the target source code. Finally, a test report generation tool generates and outputs a corresponding test report based on the test results. Optionally, the automated testing system can be a continuous integration / continuous delivery (CI / CD) system, the automated testing tool can be at least one of JUnit, TestNG, Cucumber, etc., and the test report generation tool can be Allure.

[0043] S202, if the test result indicates that the target source code test failed, then obtain the test data corresponding to the target source code, as well as the code change information of the target source code; wherein, the code change information is used to indicate the change information of the target source code relative to the historical version of the target source code.

[0044] If errors occur during or after testing the target source code based on the test code, the test will fail. The corresponding test result will indicate that the test on the target source code failed. In order to detect the reason for the test failure, the test data corresponding to the target source code can be obtained.

[0045] The test result refers to the result obtained by testing the target source code based on the test code. It may include a standardized report output from at least one of the following: automated testing tools and test report generation tools. For example, reports from at least one of JUnit, TestNG, Cucumber, and Allure. The standardized report is a standard automated test result data format and can be a file / data written in Extensible Markup Language (XML) or JSON format. The report may include, but is not limited to, test scenarios, test steps, test status, and error messages. The test scenario is the smallest independent test unit of a test case, corresponding to a "complete business test scenario" used to verify a specific business behavior, such as verifying "correct username and password login" and "incorrect password login prompt error". The test step is the smallest execution unit of a test scenario, corresponding to a specific operation step in the test scenario. A test scenario usually contains at least one test step. For example, in the "correct username and password login" scenario, it includes test steps such as "accessing the login page" and "entering username and password". The test status is the execution result of the test scenario and test step, used to characterize whether the execution of the test scenario and test step was successful. The error information in the report is a detailed description of the error when the execution of the test scenario and test step fails, including key information such as the reason for failure, error stack, and comparison of expected and actual results. It is the core field for locating the root cause of test failure. For example, the standardized report output by Cucumber is the Cucumber Report, which includes test scenario, test step, test status and error information.

[0046] Test data refers to data related to the testing process of the target source code. It may include, but is not limited to, at least one of the following: test results, test context, and runtime data during the test process. This data can be obtained from the runtime logs of an automated testing system. For example, if the automated testing system is CI (Computer Intervention), test data can be obtained from CI logs. The test context refers to the environment information during test execution, including at least one of the following: test steps, error stack traces, console logs, and network request logs. Runtime data during testing includes at least one of the following: runtime data of the target source code, runtime data of the test code, and runtime data of the automated testing system. Runtime data of the target source code refers to the data captured during the testing of the target source code, including but not limited to at least one of the following: captured runtime console logs, Network HAR files, screenshots, and DOM Tree Dumps. A Network HAR file is a recording file of network requests generated during the runtime of the target source code, containing information about network requests during access. For example, it may include at least one of the following: API requests, images, Cascading Style Sheets (CSS), scripting languages ​​(JavaScript, JS), API return data, response headers, status, execution time, cache / session credentials (cookies), and parameters. Network HAR files can be used for problem reproduction and test case generation. Screenshots are obtained by taking a screenshot of the interface displayed after the target source code runs. A DOM Tree Dump refers to the HTML structure of the target source code captured at the moment of test failure. Runtime data of the test code is the runtime data of the test code captured during the testing of the target source code.

[0047] Optionally, test data corresponding to the target source code can be obtained through a Test Context Collector. For example, the Test Context Collector can parse standard XML / JSON reports output by JUnit / TestNG / Cucumber / Allure, and capture Console Logs, NetworkHAR packets, screenshots, and DOM Tree Dumps during the test process to obtain test data.

[0048] It should be understood that during code development or after code deployment, code is usually modified to obtain different versions of code. Among them, the historical version code is the historical version of the target source code. In other words, the version corresponding to the historical version code is earlier than the version corresponding to the target source code, and the target source code is obtained by modifying the historical version code.

[0049] Code change information describes changes to the target source code relative to historical versions. It may include at least one of the following: the line location of the changed code, the method of change, and data before and after the change. The line location refers to the position of the code line within its respective code, including at least one of the following: the file, function, or line number to which the code line belongs. Code change information can be in text format.

[0050] There are many reasons why tests on the target source code might fail. For example, the target source code may have changed compared to a previous version, but the test code hasn't been updated accordingly, causing a mismatch between the target source code and the test code, thus leading to test failure. For instance, the target source code might have changed its Document Object Model (DOM) structure compared to a previous version, such as changing the User Interface (UI) library, modifying class names, or deleting certain UI elements. However, if the test code hasn't been updated accordingly, the test code won't be able to find the corresponding UI elements, causing the test to fail. Therefore, to improve the accuracy of test failure detection, if the test results indicate that the target source code failed, it's also necessary to obtain information about code changes relative to the previous version. The UI library, or UI component library, contains various UI elements, CSS styles, and interaction logic. UI elements are visual elements on the UI, such as buttons, input boxes, text, icons, images, dropdowns, pop-ups, and navigation bars. The class name, or CSS class name, is a style identifier or tag; the corresponding CSS style can be found based on the class name.

[0051] Optionally, to improve the accuracy of test failure cause detection, code change information of historical versions of the target source code relative to adjacent versions can be obtained for test failure cause detection. For example, code change information of the target source code relative to the previous version can be obtained.

[0052] Optionally, to improve the accuracy of test failure detection, it is also possible to search for historical versions of code that were successfully tested based on the test code from multiple historical versions of code, and obtain code change information of the target source code relative to the searched historical versions of code.

[0053] Optionally, a code change analyzer can be used to analyze the target source code and historical versions of the code to obtain code change information. It should be understood that during code development, code management tools are typically used. Developers can trigger commits within these tools to save code changes as a version snapshot, which is the smallest version unit. Developers can also trigger merge requests to merge modified code into the official codebase. These saved different versions of code can be stored in a code repository. The code change analyzer can interface with the repository's Application Programming Interface (API) to obtain the metadata of the Merge Requests / Commits corresponding to the target source code. Based on this metadata, comparative analysis can be performed, and code change information can be generated according to the analysis results. The code management tool can be a distributed version control system (Git), and the corresponding code repository can be at least one of GitLab, GitHub, etc.

[0054] S203: Based on the test data and code change information, perform test failure reason detection, obtain test failure reason information, and generate test repair instruction information based on the test failure reason information.

[0055] After obtaining the test data and code change information corresponding to the target source code, the test data and code change information can be analyzed together to determine whether the test failure was caused by the code change, thereby obtaining the test failure reason information. The test failure reason information describes the reason for the test failure, which can include the error phenomenon and the reason for failure. The error phenomenon describes what error occurred, which can be determined based on the error message. If the test failure is caused by changes to the target source code, the test failure reason information can also include the change data of the code segment that caused the test failure. For example, if the "ElementNotVisible" error occurs after the test due to changes in the DOM of the target source code, the test failure reason information can be: This change modified the DOM structure of "MergeButton.vue" in the target source code, removing the class name ".toolbar-merge-btn", resulting in "ElementNotVisible".

[0056] Optionally, artificial intelligence models can also be used for test failure cause detection. In one optional example, test failure cause detection can be performed using a specially trained cause detection model, where training samples corresponding to the cause detection model can be obtained, and the model can be trained based on the training samples to obtain the cause detection model; in another optional example, a Large Language Model (LLM) can be used for test failure cause detection.

[0057] In an optional implementation, the process of detecting test failure reasons and obtaining test failure reason information based on test data and code change information may include: Obtain error information from the target source code in the test data, and find the stack line corresponding to the framework code of the target source code from the error stack contained in the error information; Remove the stack line corresponding to the framework code from the error stack to obtain the error stack after deletion. Then, find the stack line corresponding to the storage path of the target source code from the error stack after deletion to obtain the target error stack. Based on the target error stack and code change information, the cause of test failure is detected to obtain the reason for test failure.

[0058] It should be understood that the test data includes error information from the target source code, including the error stack trace. This error stack trace is automatically generated during the testing process of the target source code using test code when an exception or error is triggered; it is a snapshot of the call trajectory at the moment of the error. Error stack traces are typically large in volume and contain a lot of useless data. To improve the efficiency of test failure cause detection, and especially when using AI models for test failure cause detection, to avoid excessive error stack traces interfering with contextual judgment, the error stack traces can be filtered. During filtering, stack lines corresponding to the framework code of the target source code need to be removed, while stack lines corresponding to the storage path of the target source code are retained. This ensures that the obtained target error stack trace is closely related to the target source code. Detecting test failure causes based on the target error stack trace and code change information can reduce the amount of data analysis and improve detection efficiency and accuracy. Optionally, test failure cause detection can be performed based on the filtered error information and code change information.

[0059] The framework code of the target source code includes at least one of auxiliary code and calling code. Auxiliary code refers to code specifically written to support and run the "code under test" or provided by the test framework. Calling code refers to external code called by the target source code, such as third-party dependency packages, language base libraries, etc. For example, node_modules (the dependency directory of Node.js) and java.lang (the core package of Java).

[0060] In an optional implementation, the process of detecting test failure reasons and obtaining test failure reason information based on test data and code change information may include: Perform code semantic analysis on each piece of code change data contained in the code change information to obtain the change semantic information corresponding to each piece of code change data; Based on the change semantic information corresponding to each code change data, the code change data to be filtered out is found from the multiple code change data contained in the code change information; whereby the change semantic information of the code change data to be filtered out indicates that the change type of the code change data to be filtered out is a change type other than the code logical structure change. Remove the code change data to be filtered from the code change information to obtain the target code change information. Based on the test data and the target code change information, perform test failure reason detection to obtain test failure reason information.

[0061] The code change information contains multiple code change data entries. Each entry indicates which line of code in the target source code has been changed, the method of change (addition, deletion, modification, etc.), and the data before and after the change. It can only represent the textual changes in the code and cannot understand the meaning of the code change. Therefore, code semantic analysis can be performed on each entry of code change data in the code change information to obtain the change semantic information corresponding to each entry. This change semantic information is used to indicate the meaning of the code change, such as whether it involves changes in the code's logical structure or whether it is a change in format. The code logical structure refers to the way the syntactic elements that determine the program's behavior and execution flow are organized, including functions, loops, conditions, calculations, variables, etc. It affects the execution of the code and its functionality or operation. For each code change data entry, its semantic information is used to determine whether it represents a change in code logic structure. If not, it is identified as code change data to be filtered out; otherwise, it is identified as code change data to be retained. Then, the code change data to be filtered out is removed from the code change information, and the code change data corresponding to the code logic structure change is retained, thus obtaining the target code change information. Test failure reason detection is then performed based on test data and the target code change information. This not only reduces the amount of data processing but also improves the accuracy of test failure reason detection because the target code change information contains code change data corresponding to code logic structure changes, which have a higher probability of causing test failures. Specifically, code logic structure changes refer to changes that alter the code's logical structure, affecting its functionality and execution results, such as logic changes, UI changes, and configuration item changes. Other change types besides code logic structure changes refer to changes that do not alter the code's logical structure but only change the code's form, etc., and do not affect the code's functionality or execution structure, such as modifying blank lines, formatting, and modifying comments.

[0062] Optionally, semantic analysis can be code structure parsing, such as Abstract Syntax Tree (AST) analysis, thereby using the obtained code structure information as change semantic information, which can characterize whether the change involves the code's logical structure.

[0063] Optionally, the target source code typically contains multiple files. If too many files have been changed, to further reduce the amount of data analysis, after deleting the code change data to be filtered from the code change information, code change data corresponding to files related to the stack lines in the target error stack can be extracted from the deleted code change information to obtain the target code change information. The files related to the stack lines can include at least one of the following: files belonging to the same directory as the file to which the stack line belongs, files that have a reference relationship with the file to which the stack line belongs, etc. Optionally, the number of changed files can be counted based on the code change information. If the number exceeds a threshold, extraction can be performed based on the target error stack. Alternatively, extraction can be performed based on the target error stack if the target source code is long (e.g., longer than a threshold).

[0064] In an optional implementation, the process of detecting test failure reasons and obtaining test failure reason information based on test data and code change information may include: Obtain the association data between the test code and the target source code; the association data includes the mapping relationship between the test units in the test code and the code modules in the target source code; Based on the test data and mapping relationships, locate the tested code modules from the target source code; Based on the tested code modules, test data, and code change information, the reasons for test failures are detected, and the reasons for test failures are obtained.

[0065] It should be understood that test code typically contains multiple test units, such as test scenarios, test steps, test cases, and test files. The target source code typically contains multiple code modules, which can be functions, business logic modules, or functional modules. Business logic modules refer to code modules divided according to the business logic contained in the target source code. Different business logic modules can implement different business logic, such as payment, login, and user center functionalities. A business logic module can be a business logic file. To determine the code module being tested by a test unit, we can obtain the association data between the test code and the target source code. This association data contains the mapping relationship between test units and code modules. Based on this association data, we can determine which test unit is used to test which code module. Based on the test data and this mapping relationship, we can determine which test units were executed, thereby identifying the tested code modules and pinpointing the suspected code modules causing test failures. By analyzing the tested code modules, test data, and code change information, we can improve the accuracy of failure cause detection. For example, we can use code change information to check whether the error messages in the test data are caused by changes to the tested code modules.

[0066] Optionally, the associated data can be generated and stored before testing, i.e., obtained through static mapping. This association data between the test code and the target source code can be established using techniques such as file naming conventions and import analysis. The associated data can be in the form of topology diagrams, tables, or text. For example, if the target source code is developed using Vue (a front-end development framework), the file naming convention could be login.vue vs login.spec.ts, where login.vue refers to the name of the business component, and login.spec.ts is the name of the corresponding test file. When a test fails, code coverage tools (such as Istanbul / Jacoco) and this associated data can be used to reverse-engineer which code modules were covered during the execution of the test unit in the test code, thus obtaining the tested code modules. In other words, dynamic detection is performed when a test fails to identify the tested code modules.

[0067] After determining the cause of the test failure, test repair instructions, also known as repair suggestions, can be generated based on the cause. These instructions guide how to fix the issue to ensure successful testing of the target source code. For example, if the test failure is due to a DOM change causing the element selector to fail to locate the element, the test repair instructions might be: "The system generates a new element selector." If the failure is due to a style tweak (Visual Change), the instructions might be: "The system suggests updating the baseline screenshot." The element selector is a module in the test code used to locate the corresponding element from the code under test. It contains positioning rules and is typically based on element attributes. If an element attribute changes, the element selector cannot locate the element. The element selector can be a CSS selector (i.e., a selector that locates elements based on CSS attributes) or other types of selectors. If the element selector is a CSS selector, its positioning rules include the CSS class name bound to the element.

[0068] Optionally, if the test failure is caused by the test environment, test repair instructions can be generated to adjust the test environment. For example, if the test failure is caused by a CPU alarm during the test, the amount of CPU resources allocated to the test environment of the target source code can be increased, so that the target source code can be retested in the increased test environment. If the test failure is caused by accidental circumstances, such as random environmental fluctuations, page rendering delays, slow environment, etc., test repair instructions can be generated to indicate retesting, so that the target source code can be retested based on the test code.

[0069] Optionally, since the test failure reason is obtained by combining test data and code change information, it is possible to determine whether the test failure is caused by changes to the target source code, the target source code itself, or the test code itself. Therefore, at least one of the following can be generated based on the test failure reason: repair instructions for the test code, repair instructions for the target source code, and retesting. Specifically, if the test failure is caused by the test code (e.g., expired or dirty test code data, or changes to the target source code not being synchronized with the test code), repair instructions for the test code can be generated. If the test failure is caused by the target source code (e.g., bugs in the target source code, logical errors in the target source code, or incorrect environment configuration in the target source code), repair instructions for the target source code can be generated. If the test failure is caused by environmental issues such as environmental fluctuations or rendering delays, retesting or delay waiting time can be generated.

[0070] If the test failure is caused by the test code or the target source code, the code segment to be modified can be determined from both the test code and the target source code based on the test failure reason information. Test repair instructions can then be generated based on this code segment. The code segment to be modified refers to the code segment in the test code or target source code that needs to be fixed. For example, if the test failure is caused by a logical error in the target source code, the code segment to be modified is the logic code corresponding to the logical error; if the test failure is caused by a DOM change in the target source code that prevents the element locator from locating elements, the code segment to be modified is the code in the test code corresponding to that element locator. The test repair instructions can include the code segment to be modified itself and / or the location to be repaired. The location to be repaired indicates the position of the code segment to be modified in the test code or target source code. Examples include the file, function, line number (indicating which line of code), and code module to which the code segment belongs. If the code segment to be modified belongs to test code, the location to be repaired can also be the test unit or element selector to which the code segment belongs. Optionally, when outputting test repair instructions, the test code or target source code to which the code snippet to be modified belongs can also be displayed, and the code snippet to be modified can be marked in the displayed code. The marking method can be to distinguish the code snippet to be modified from other code snippets, such as highlighting the code snippet to be modified in red, adding a label, or adding comments.

[0071] Optionally, at least one of the following can be generated based on the test failure reason information: a modification method for the code snippet to be modified and a modified code snippet, and added to the test repair instruction information. The modification method indicates how to repair the code snippet to be modified, including but not limited to updating the code snippet to be modified and manual repair. For example, for test code, if the test failure is due to a change in the DOM of the target source code, causing the element selector in the test code to fail to locate the corresponding interface element, the modification method could be to update the element selector; if the test failure is due to the presence of expired or dirty data in the test code, the modification method could be to update the expired or dirty data; for the target source code, if the test failure is due to a logical error in the target source code, the modification method could be to adjust the corresponding logic function. The modified code snippet is relative to the code snippet to be modified; it is the corrected code snippet. For example, for test code, if the test fails because the DOM changes in the target source code cause the CSS selector in the test code to fail to locate the interface element, a new CSS selector can be generated based on the changed DOM, and the new CSS selector can be added to the test repair instruction information. For the target source code, if the test fails because the target source code has a logical error, the corresponding logic function can be adjusted based on the error message, and the adjusted logic function can be added to the test repair instruction information.

[0072] To improve accuracy, test repair instructions can be generated based on the test failure reason information, combined with at least one of the following: test code, target source code, and code change information.

[0073] S204. Based on the test repair instructions and test code, retest the target source code.

[0074] After receiving the test repair instructions, you can perform the test repair to retest the target source code.

[0075] Optionally, the repair methods include, but are not limited to, at least one of the following: if the test repair instruction information is used to instruct the modification of code snippets in the test code, then the test code is modified according to the test repair instruction information, and the target source code is retested based on the modified test code; if the test repair instruction information is used to instruct the modification of code snippets in the target source code, then the target source code is modified according to the test repair instruction information, and the modified target source code is tested based on the test code; if the test repair instruction information is used to instruct the modification of the test environment, for example, allocating more CPU or memory for testing, then the test environment is modified, and the target source code is tested based on the modified test environment and the test code; if the test repair instruction information is used to instruct retesting, then the target source code is tested based on the test code.

[0076] Optionally, if the test still fails after retesting the target source code, the test data from this test process can be obtained. Based on this test data and the code change information of the target source code, the cause of the test failure can be detected to obtain the test failure reason information. Then, based on this test failure reason information, corresponding test repair instruction information can be generated. The target source code can then be retested according to this test repair instruction information until the test succeeds. The method for generating the test failure reason information during this test process can be found in the description in S203 above.

[0077] Optionally, in S203, during the process of detecting test failure reasons based on test data and code change information to obtain test failure reason information, if this test process is not the first test of the target source code, test repair instruction information, test data, and test failure reason information from the target source code's historical test processes (test processes before this test process) can be obtained. Based on the target source code's code change information, the test data in this test process, and the test repair instruction information, test data, and test failure reason information from historical test processes, test failure reason detection is performed to obtain the test failure reason information for this test. To reduce the amount of data, test failure reason detection can also be performed based on the target source code's code change information, the test data in this test process, and the test data, test failure reason information, and test repair instruction information from the target source code's previous test process to obtain the test failure reason information for this test. Optionally, during the process of generating test repair instruction information based on the test failure reason information, the test failure reason information for this test process, the test failure reason information for historical test processes, and the test repair instruction information can also be combined to generate the test repair instruction information for this test process.

[0078] exist Figure 2 In the illustrated embodiment, on the one hand, test failure cause detection can be automatically performed based on test data and code change information, which can improve detection efficiency and reduce detection costs. Furthermore, by combining test data and code change information to detect test failure causes, it is possible to discover whether test failures are related to code changes, thereby improving the accuracy of cause detection. On the other hand, since the test failure cause information is determined based on test data and code change information, it is possible to determine whether the test code or the code under test needs to be modified, thereby generating test repair instruction information. Based on the test repair instruction information, the target source code is retested, thereby improving test efficiency and test success rate.

[0079] See Figure 3 , Figure 3Is Figure 2 The flowchart illustrates a code testing method proposed based on the given example. This method can be applied to... Figure 1 The implementation environment shown can be composed of Figure 1 The terminal device 101 in the implementation environment shown can execute this, or it can be executed by... Figure 1 The server 102 in the implementation environment shown can also be executed by Figure 1 The terminal device 101 and server 102 in the implementation environment shown are executed together.

[0080] like Figure 3 As shown, S203 may include S301-S303, which are described in detail below: S301 retrieves error information from the test data and extracts code segment location information related to the code segment contained in the target source code from the error information.

[0081] Optionally, the test data for the target source code may include at least one of the following: test report and runtime data of the target source code during the test.

[0082] If the test on the target source code fails, the test data usually contains error messages, including error messages triggered by the test code, error messages triggered by the target source code, and error messages triggered by the test system. The test report and runtime data contain error messages, which are important for locating the cause of the test failure. Therefore, error messages can be obtained from the test data.

[0083] Error messages typically contain information related to code snippets in the target source code, enabling the identification of the corresponding code snippet. The code snippet location information allows the corresponding code snippet to be found in the target source code and may include the snippet's path, file, function, module, and line number. If the error message pertains to a user interface element, it may also include the element's attributes, such as its CSS properties or ID.

[0084] In an optional example, if the error message is triggered by the target source code, it contains the error stack of the target source code. The error stack contains the call chain of the target source code, including the function names, file locations, line numbers, etc. The function names, file locations, line numbers, etc., can be extracted from the error stack as code snippet location information. If the error message is triggered by test code, it typically contains a description of the test unit that failed and the error phenomenon. The test unit description usually contains objects in the target source code that are being tested, such as the code module to be tested, or UI elements that need to be rendered or triggered during the test. Therefore, code snippet location information can be extracted from this. For example, in the "correct username and password login" scenario... The description can include: the target source code's login function needs to be tested. During the test, the login page needs to be opened, the account input area needs to be clicked to enter the account, the password input area needs to be clicked to enter the password, and the login button needs to be clicked to trigger the login. From this, the login function, the login page, the name / ID corresponding to the account input area, the name / ID corresponding to the password input area, and the name / ID corresponding to the login button can be extracted as code snippet location information. The error phenomenon usually also contains information related to code snippets in the target source code. For example, assuming the error phenomenon is that the element is not visible, it usually contains information related to the invisible element, such as the element's ID, CSS class name, etc. Therefore, code snippet location information can also be extracted from it.

[0085] S302, based on the code change information, detect whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code, obtain the change detection result, and perform test failure reason detection based on the change detection result to obtain test failure reason information.

[0086] Error messages typically include the cause of failure, error stack trace, and a comparison between expected and actual results. While these messages can provide a rough indication of the test failure, they cannot determine whether the failure is related to code changes or pinpoint the specific cause. Therefore, for root cause analysis, we can use code change information to detect whether the code segment corresponding to the code segment location information in the target source code has changed. In other words, we can check if the code change information contains the corresponding code change data, thus obtaining the change detection result. If the change detection result indicates that the code segment has changed, it means that the change to the code segment is highly likely the cause of the error message. If the change detection result indicates that the code segment has not changed, it means that the test failure is highly likely unrelated to the code segment. Therefore, the cause of the test failure can be determined based on the change detection result. In an optional implementation, the process of detecting whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code, based on the code change information, obtaining change detection results, and performing test failure reason detection based on the change detection results to obtain test failure reason information may include: selecting the target error type corresponding to the target source code from multiple error types based on the error information of the target source code, obtaining the reason detection sub-rule corresponding to the target error type from the reason detection rules, and based on the reason detection sub-rule corresponding to the target error type, detecting whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code, based on the code change information, obtaining change detection results, and performing test failure reason detection based on the change detection results to obtain test failure reason information.

[0087] This method sets up multiple error types and establishes cause detection rules. These rules contain sub-rules corresponding to each error type, used to detect the reasons for test failures of the corresponding error type. Based on the error information, the corresponding error type can be analyzed, and the corresponding cause detection sub-rule can be determined based on the error type. Root cause analysis is then performed according to this rule to obtain the test failure cause information. The error types and cause detection rules can be flexibly set according to actual needs. For example, error types can include runtime environment errors, logic errors, and UI errors. For descriptions of runtime environment errors, logic errors, and UI errors, as well as the root cause analysis process for each, please refer to the following description.

[0088] In an optional implementation, the process of detecting whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code, obtaining a change detection result, and performing test failure reason detection based on the change detection result to obtain test failure reason information may include: If the error message indicates an error in the runtime environment, then based on the code change information, check whether the environment configuration code indicated by the code segment location information in the target source code has changed, and obtain the change detection result; If the test results indicate that the environment configuration code has been changed, test failure information indicating an environment configuration error in the target source code will be generated.

[0089] This implementation describes a specific method for root cause analysis when the error type is a runtime environment error. A runtime environment error refers to an error related to the runtime environment of the target source code; it is an error specific to the runtime environment, such as errors related to network, computing resources (e.g., CPU), or storage resources (e.g., memory). For example, if the error message includes network error, 502 Bad Gateway error, or socket timeout, it indicates a network error, and the error type is a runtime environment error.

[0090] In this scenario, where the error message indicates an environment error, the code snippet in the target source code corresponding to the code snippet location information is the environment configuration code. Therefore, based on the code change information, it is possible to detect whether the environment configuration code corresponding to the target source code has changed compared to the environment configuration code in the historical version. For example, it is possible to detect whether the network or configuration code in the target source code has been modified relative to the historical version. If the change detection result indicates that the environment configuration code has changed, it means that the error message is related to the change in the environment configuration code, and the test failure is caused by an environment configuration error in the target source code. Therefore, generating test failure reason information indicating an environment configuration error in the target source code improves the accuracy of the test failure reason information and can accurately identify errors caused by configuration errors and errors caused by environment anomalies.

[0091] Optionally, if the change detection result indicates that the environment configuration code has not changed, it indicates that the test failure is not caused by the change in the environment configuration code. Therefore, the test failure is determined to be caused by a problem with the runtime environment during the test, i.e., a problem with the test environment, such as random environmental fluctuations or an unstable environment. Therefore, test failure reason information indicating that the test environment caused the test failure is generated.

[0092] Optionally, to improve the accuracy of test failure reason information, if the change detection result indicates that the environment configuration code has not changed, the timestamp of the error message corresponding to the runtime environment error in the test data can be obtained, and the environment detection data in the test data can be obtained. It can then be determined whether there are alarms in the environment detection data at or near the corresponding timestamp, such as CPU alarms, memory alarms, network alarms, etc. If alarms exist, the test failure reason is determined to be a test environment problem, and test failure reason information indicating that the test environment caused the test failure is generated. The environment detection data is data obtained by detecting the runtime environment of the target source code during the test. It can be obtained by detecting the test environment provided by the automated testing system through an environment detection system, including but not limited to Prometheus, Zabbix, etc.

[0093] In an optional implementation, the process of detecting whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code, obtaining a change detection result, and performing test failure reason detection based on the change detection result to obtain test failure reason information may include: If the error message indicates a logical error, then based on the code change information, check whether the logical code corresponding to the code segment location information contained in the target source code has changed, and obtain the change detection result; If the change detection result indicates that the logic code has been changed, test failure information indicating that there is a logical error in the target source code will be generated; If the logic code indicating the change in the test result has not changed, then test failure reason information indicating that the test code is abnormal will be generated.

[0094] This implementation describes a specific method for root cause analysis when the error type is a logical error. A logical error refers to an error in the execution logic or business logic of the target source code. For example, if the error message includes an assertion failure (AssertionError) or a discrepancy between the expected and actual results (e.g., Expected X but got Y, meaning the expected result is X, but the actual result is Y), then the error type is a logical error.

[0095] When the error message indicates a logical error, the error message is a logical error message. The code snippet location information can contain information extracted from the logical error message that is related to the logical code in the target source code, such as function names and variable names within functions. Correspondingly, the code snippet corresponding to the code snippet location information is logical code. Therefore, it is possible to detect whether the logical code corresponding to the code snippet location information in the target source code has changed based on the code change information. If the logical code has changed, it indicates that the logical code is abnormal, that is, the target source code has a bug. Therefore, test failure reason information indicating that the target source code has a logical error is generated. Optionally, the test failure reason information can include the indication information corresponding to the abnormal logical code and / or the code change data of the abnormal logical code. The code change data can be obtained from the code change information. The indication information corresponding to the abnormal logical code is used to inform the tester which logical code has a problem. This indication information can be the name of the logical code, its location in the target source code, etc. If the logical code has not changed, it indicates that the logical code itself is not problematic. The test failure reason is determined to be an abnormal test code, and test failure reason information indicating that the test code is abnormal is generated. In this implementation, test failures caused by changes to the logic code itself in the code under test, as well as test failures caused by errors in the test code, can be accurately detected.

[0096] Optionally, logical error messages typically contain incorrect function names and incorrectly calculated variable names. The calculation function corresponding to the variable name is used to calculate the value of the variable name. Therefore, the incorrect variable name or function name can be extracted from the logical error message, and the code change information can be checked to see if there is any code change information corresponding to the variable name or function name. If it is detected, it indicates that the logical code has been changed; if it is not detected, it indicates that the logical code has not been changed.

[0097] Optionally, after determining that the test failure is due to a test code exception, the cause of the exception can be further analyzed and added to the test failure reason information. For example, it can be determined whether the test data in the test code has expired, or whether the test code contains dirty data. Specifically, the expiration date of the test code can be used to determine if it has expired.

[0098] In an optional implementation, the process of detecting whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code, obtaining a change detection result, and performing test failure reason detection based on the change detection result to obtain test failure reason information may include: If the error message indicates a user interface error, then based on the code change information, check whether the user interface corresponding to the code change information in the target source code has changed, and obtain the change detection result; If the change detection results indicate that the user interface has been changed, test failure information indicating that the user interface in the target source code has been changed is generated.

[0099] This implementation describes the specific method for root cause analysis when the error type is a UI error. A UI error is an error report targeting the UI or interface elements. If the error message includes messages such as "Element Not Found (NoSuchElement)," "Invalid Element Reference (StaleElementReference)," or "Element Not Visible (ElementNotVisible)," then the error type is a UI error. It should be understood that during front-end development, developers define the UI in the code. This includes defining the UI elements, element attributes, the corresponding CSS style library, and the corresponding UI package. Element attributes include the element's structure, interaction logic, and CSS styles. Typically, CSS styles are defined by binding the class name of the CSS style to the element. During front-end development, these UI elements are frequently changed. If the test code does not adapt to these changes, UI errors are likely to be triggered, leading to test failures. For example, if element A has been removed from the target source code compared to the previous version, and the test code needs to click on element A, the test code will not be able to find element A during the test of the target source code, resulting in test failure. Another example is if the class name bound to element A in the target source code has been updated from CSS1 and CSS2 to CSS1 compared to the previous version, and the test code uses CSS selectors to locate the element, the test code will not be able to find element A, resulting in test failure. CSS styles include visual styles, layout styles, state styles, and responsive design. Visual styles include the color, size, shape, font, and effects of elements. Layout styles include the spacing, arrangement, and position of elements. State styles are used to indicate the appearance of elements in different states, such as a button turning gray when disabled or the pressing effect when clicked. Responsive design is used to indicate style changes of elements on different screen sizes.

[0100] When the error type is UI error, the code snippet location information typically includes the UI or the attributes of UI elements. The code snippet corresponds to the UI or the attributes of UI elements, and the UI has been changed based on the code change information. If the UI has changed, it indicates that the UI error was caused by a UI change that was not synchronized with the test code, generating test failure reason information indicating that the UI in the target source code has changed. Here, a UI change refers to a change in the content corresponding to the UI, such as changes to elements or element attributes in the UI. Adding, deleting, or changing the position of elements will cause changes to the UI's Document Object Model (DOM).

[0101] Optionally, if the test failure reason information is due to UI changes, test failure reason information can also be generated based on the code change data corresponding to the code snippet location information contained in the code change information, so that users can determine which UI change caused the test failure based on the test failure reason information.

[0102] Optionally, when the error type of the error message is a UI error, there will usually be abnormal UI elements. Abnormal UI elements refer to the UI elements corresponding to the error message, such as missing elements, invalid elements, invisible elements, etc. The code change information can be used to determine whether the abnormal UI elements corresponding to the error message have changed. For example, whether the corresponding components or attributes of the element have changed, or whether the element has been deleted, etc., and then a change detection result can be generated based on the judgment result. Specifically, the element identifier information of the abnormal UI elements can be extracted from the error message as code snippet location information. The element identifier information can be the element's name, ID, location rules, or other information that can identify the element. Based on the element identifier information, a search is performed in the code change information to determine whether there is code change data corresponding to the abnormal UI elements. The location rules are used to indicate how to locate the corresponding element. The element selector in the test code usually contains element identifier information such as location rules. The element selector is used to locate the corresponding element from the code under test based on the location rules. When the element selector cannot locate the corresponding element, it will generate an error message. When the error type of the error message is a UI error, the element identifier information can be obtained from the element selector corresponding to the error message. The element selector can be a CSS selector or other types of selectors. If the element selector is a CSS selector, its positioning rules include the CSS class name bound to the element. When the selector reports an error, the CSS class name can be retrieved from the positioning rules, and the code change information can be used to detect whether the CSS class name has changed, such as being deleted. For example, the CSS class name "submit-btn" can be extracted from the error-prone selector. If the string "submit-btn" is found in the deleted lines (data indicating that the target source code has been deleted relative to historical versions) of the code change information, it indicates that the corresponding code has changed. When the test failure is due to a change in an abnormal interface element, the test failure reason information can include the change information corresponding to the abnormal interface element, allowing the user to determine which change in which abnormal interface element caused the test failure.

[0103] Optionally, if the change detection result indicates that the UI has not changed, it means that the UI error is not caused by the UI change. The reason for the test failure may be a rendering anomaly, such as rendering delay, dynamic loading problem, slow environment, etc. Therefore, test failure reason information can be generated based on this reason.

[0104] In an optional implementation, the process of detecting whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code, based on code change information, and obtaining the change detection result, may include: From the multiple business modules contained in the target source code, obtain the error code module to which the error message belongs and the changed code module corresponding to the code change information, and check whether there is a code module in the changed code module that matches the error code module; If a code module matching the error-reporting code module exists in the changed code module, then based on the code change information, it is detected whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code, and the change detection result is obtained.

[0105] The target source code typically includes multiple business modules to implement different business functions, such as login and payment. If the business module reporting an error is unrelated to the changed business module, the error message is usually also unrelated to the code change. Therefore, we can first determine the business module to which the error message belongs (the error-reporting module) and the business module corresponding to the code change information (the changed business module) from the target source code. Then, we can determine whether the error-reporting module and the changed business module are related to obtain the module relevance detection result. Here, the error-reporting business module refers to the business module to which the error message belongs within the business modules included in the target source code; the changed business module refers to the business module in the target source code that has changed compared to the previous version, for example, the business module to which the line of code in the target source code that has changed compared to the previous version belongs. Matching an error-reporting business module with a changed business module can be achieved if they belong to the same or similar business. The similarity between the corresponding business module names can be calculated, and the matching degree is determined based on this similarity. If the similarity is greater than a threshold, a match is made. For example, if the error-reporting business module is on the payment page, while the changed business module is in the user center, they belong to different businesses, so there is no match and the relevance is low. Conversely, if both the error-reporting and changed business modules belong to the user login module, then there is a match and the relevance is high. If the changed business module corresponding to the code change information contains a business module that matches the error-reporting business module, it indicates that the error information may have been caused by a code change. Therefore, further investigation is needed to check whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version of the code.

[0106] Optionally, if the changed business module corresponding to the code change information does not contain a business module that matches the business module that reported the error, it indicates that the error information is highly unlikely to be caused by the code change. Therefore, the test failure reason can be detected based on the test data to obtain the test failure reason information.

[0107] Optionally, the process of retrieving the error-reporting code module and the modified code module corresponding to the code change information from multiple business modules contained in the target source code, and detecting whether there is a code module matching the error-reporting code module in the modified code module, may include: retrieving the description of the failed test unit corresponding to the error-reporting information; retrieving entity names from the failed test unit description; obtaining the target entity names; searching for entity names related to the target entity names from the code change information; and determining whether there is a code module matching the error-reporting code module in the modified code module based on the search results. Here, the failed test unit refers to the test unit to which the error-reporting information belongs, such as a test scenario, test case, or test step. The failed test unit description describes the function and purpose of the failed test unit. Test unit descriptions typically include the name of the business module to be tested, and code change information usually includes the name of the business module to which the changed code lines belong. Therefore, we can obtain entity names from the failed test unit descriptions and record these as failed entity names. These entity names are usually the names of the business modules that the test unit needs to test. Then, we search for entity names that are the same as or similar to these failed entity names in the code change information. If a match is found, it indicates that there is a business module in the changed business modules that matches the error-reporting business module; otherwise, it indicates that there is no business module in the changed business modules that matches the error-reporting business module. Optionally, we can obtain similar entity names to the failed entity names, such as their variants, and then search for entity names that are the same as or similar to the failed entity names, as well as entity names that are the same as or similar to the similar entity names, in the code change information. If either a match or a similar entity name is found, it indicates that there is a business module in the changed business modules that matches the error-reporting business module. For example, if the failed test step description for the error message "ElementNotFound" is "Given user clicks login", then extract the business entity names "Login Button" and "Submit" from the test step, search for "Login Button", "Submit" and its variants "login-btn" and "handleSubmit" in the code change information, and if found, it indicates that there is a business module in the changed business module that matches the business module that reported the error.

[0108] Optionally, if a code module matching the error-reporting code module is detected in the changed code module, the code change data corresponding to the matching changed code module can be filtered from the code change information. Then, based on the filtered code change data, it can be detected whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code. Specifically, the root cause analysis methods corresponding to runtime environment errors, logic errors, and user interface errors mentioned above can be used to detect whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code. For example, if the error message indicates that the element corresponding to the login module is not visible, the code change information is first used to determine whether the changed code module includes the login module. If it includes the login module, the change information corresponding to the login module is used to further detect whether the code corresponding to the invisible element has changed, and a change detection result is generated based on the detection result. If the login module is not included, a change detection result indicating that the user interface has not changed is directly generated.

[0109] S303, based on the test failure reason information, generate test repair instruction information.

[0110] The specific method for generating the test repair instruction information can be found in the description of the foregoing embodiments.

[0111] In an optional implementation, the process of generating test repair instruction information based on test failure reason information may include: if the code segment corresponding to the code segment location information indicated by the test failure reason information has changed, then based on the error information and the test failure reason information, determining the code segment to be modified from the test code and the target source code, and generating modification instruction information corresponding to the code segment to be modified based on the code change data corresponding to the code segment location information contained in the code change information, thus obtaining test repair instruction information. Correspondingly, the process of retesting the target source code based on the test repair instruction information and the test code may include: modifying the code segment to be modified according to the modification instruction information to obtain the modified code segment, and retesting the target source code based on the modified code segment.

[0112] The test failure reason information indicates that the code segment corresponding to the code segment location information has changed. In other words, the test failure reason information indicates that the error message is related to the change of this code segment. Since the change of the code segment caused the error message, it indicates that the test code and / or the target source code need to be modified. Therefore, the code segment to be modified can be determined from the test code and the target source code. Based on the code change data corresponding to the code segment location information contained in the code change information, modification instruction information corresponding to the code segment to be modified is generated, resulting in test repair instruction information. This modification instruction information may include at least one of the following: modification location, modification method, and modified code segment. The modification location indicates the position of the code segment to be modified in the test code or target source code. The modification method indicates how to modify the code snippet to be modified, including but not limited to updating the code snippet, manual modification, etc. For example, for test code, if the test failure is due to a change in the DOM of the target source code, causing the element selector in the test code to fail to locate the corresponding UI element, the modification method could be to update the element selector. If the test failure is due to expired or dirty data in the test code, the modification method could be to update the expired or dirty data. For the target source code, if the test failure is due to a logical error in the target source code, the modification method could be to adjust the corresponding logic function. The modified code snippet is relative to the code snippet to be modified; it is the modified, correct code snippet. The modification method and the modified code snippet can be generated based on the code change data of the code snippet. For example, if the test failure is due to a change in the CSS class name of the UI element in the target source code, causing the element selector in the test code to fail to locate the corresponding UI element based on the original CSS class name, then the CSS class name in the element selector can be updated according to the modified CSS class name corresponding to the UI element, so that the updated element selector can locate the UI element based on the updated CSS class name.

[0113] In an optional implementation, if the test failure reason information indicates that the code segment corresponding to the code segment location information has changed, then the process of determining the code segment to be modified from the test code and the target source code based on the error information and the test failure reason information, and generating modification instruction information corresponding to the code segment to be modified based on the code change data corresponding to the code segment location information contained in the code change information, and obtaining test repair instruction information, may include: if the error information indicates a user interface error, and the test failure reason information indicates that the user interface corresponding to the code segment location information in the target source code has changed, then based on the user interface code change data contained in the code change information, finding the changed interface elements in the user interface and the change data of the changed interface elements, and taking the element selector corresponding to the changed interface elements in the test code as the code segment to be modified; generating a modification method for the element selector based on the change data of the changed interface elements, and generating test repair instruction information based on the modification method.

[0114] If the error message indicates a user interface error, and the test failure reason information suggests a change in the user interface corresponding to the code snippet's location information in the target source code, meaning the error message is related to this user interface change, and user interface changes typically involve changes to the interface elements they contain, then the changed interface element and its modified data can be determined based on the code change information. This change will cause the element selector used to locate the element in the test code to fail. Therefore, the element selector is treated as the code snippet to be modified. Based on the changed data of the interface element, a modification method for the element selector is generated so that, after modification based on this method, the element selector can locate the interface element. For example, suppose that in the historical version of the code, the CSS properties of a certain interface element are CSS1 and CSS2, and the element selector in the test code locates the interface element based on CSS1 and CSS2. In the target source code, the CSS properties of the interface element are changed to CSS1 and CSS3. Based on this change, a modification method can be generated to update "CSS2" in the element selector to "CSS3," so that, after modification, the element selector can locate the interface element.

[0115] In another optional implementation, if the test failure reason information indicates that the corresponding logic code or environment configuration code in the target source code has changed, the logic code or environment configuration code in the target source code (i.e., the code segment corresponding to the code segment location information) can be used as the code segment to be modified, and modification instruction information corresponding to the code segment to be modified can be generated according to the code change data corresponding to the code segment location information contained in the code change information, such as rolling back the code segment to be modified.

[0116] It should be noted that, Figure 3For the specific implementation details of S201-S202 and S204 shown, please refer to Figure 2 S201-S202 and S204 shown will not be described again here.

[0117] exist Figure 3 In the illustrated embodiment, error information is obtained from the test data, and code segment location information related to the code segment contained in the target source code is extracted from the error information. Based on the code change information, it is detected whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code, and the change detection result is obtained. Based on the change detection result, the test failure reason detection is performed to obtain the test failure reason information, thereby connecting the test data and code change information, accurately detecting whether the test failure reason is related to the code change, and improving the detection accuracy.

[0118] See Figure 4 , Figure 4 Is Figure 2 The flowchart illustrates a code testing method proposed based on the given example. This method can be applied to... Figure 1 The implementation environment shown can be composed of Figure 1 The terminal device 101 in the implementation environment shown can execute this, or it can be executed by... Figure 1 The server 102 in the implementation environment shown can also be executed by Figure 1 The terminal device 101 and server 102 in the implementation environment shown are executed together.

[0119] like Figure 4 As shown, S203 can include S401-S403, which are described in detail below: S401 generates cause detection prompts based on test data and code change information.

[0120] In this embodiment, LLM is used for test failure cause detection. To clarify the detection purpose for LLM, cause detection prompts can be generated based on test data and code change information. These prompts instruct LLM to perform test failure detection.

[0121] The cause detection message may include test data and code change information. To avoid context overflow, the cause detection message may also include target code change information and error messages. For a description of target code change information and error messages, please refer to the aforementioned records.

[0122] Optionally, the cause detection prompt information may also include the detection purpose, role definition, and output requirements. The role definition is a pre-constraint on the model's identity, professional field, behavioral boundaries, and response style, allowing the model to respond in a more professional, stable, controllable, and demand-compliant manner. The specific definition can be flexibly set according to the test failure cause detection requirements. Optionally, in order to enable the model to more accurately parse the relationship between code changes and test failure causes, the role definition can be QA and / or architect, where QA stands for Quality Assurance, referring to test engineers / quality assurance engineers.

[0123] Output requirements can include output content requirements and output format requirements. Output content requirements specify the requirements for the output content. For example, the model might be required to output at least one of the following in addition to test failure reason information: fault classification, confidence score, and test repair instructions. The test failure reason, or root cause analysis, describes the specific cause of the test failure. Fault classification categorizes the types of test failures, such as DOM changes, code bugs, and environmental fluctuations. The confidence score indicates the reliability of the fault classification and root cause analysis results, typically a value between 0 and 1. Test repair instructions suggest how to fix the issues to avoid test failures. Output format requirements define the format of the output content, such as outputting in JSON format. LLM can be configured according to... Figure 3 The method shown in the diagram is used to generate test and repair instruction information.

[0124] S402, input the cause detection prompt information into the large language model so that the large language model responds to the cause detection prompt information, performs test failure cause detection based on test data and code change information, and obtains the test failure cause information output by the large language model.

[0125] Input the cause detection prompt information into the LLM. The LLM will perform test failure cause detection according to the cause detection prompt information and output the test failure cause information.

[0126] In an optional implementation, the process of generating cause detection prompts based on test data and code change information may include: obtaining preset cause detection rules, and generating cause detection prompts based on the cause detection rules, test data, and code change information.

[0127] Correspondingly, the process of inputting the cause detection prompt information into the large language model and obtaining the test failure cause information output by the large language model can include: inputting the cause detection prompt information into the large language model so that the large language model can detect the test data and code change information according to the cause detection rules contained in the cause detection prompt information and obtain the test failure cause information output by the large language model.

[0128] In this implementation, considering that LLM alone is prone to misleading test failure cause detection, this implementation pre-sets cause detection rules and adds them to the cause detection prompt information. This allows the large language model to detect test failure causes according to the cause detection rules, thereby outputting test failure cause information.

[0129] Optionally, the cause detection rule can include detection sub-rules corresponding to different error types. The cause detection rule can instruct the LLM to first perform error type analysis, and then perform root cause analysis based on the error type and the corresponding detection sub-rules to obtain the reason for the test failure. Among them, the error type classification rule can be flexibly set according to actual needs. In an optional example, the cause detection rule can include detection sub-rules corresponding to environmental errors, logic errors, and UI errors respectively.

[0130] In an optional implementation, the process of generating cause detection prompts based on test data and code change information may include: Generate target test failure features based on test data from the target source code; The database retrieves similar test failure features that meet the similarity criteria to the target test failure feature from the failure case database, along with the test failure reason information corresponding to the similar test failure feature. The failure case database contains multiple test failure features and the test failure reason information corresponding to each test failure feature. The test failure reason information corresponding to similar test failure features and the reason detection prompt information are input into the large language model, so that the large language model can perform test failure reason detection based on the test data, code change information and test failure reason information corresponding to similar test failure features of the target source code, and obtain the test failure reason information of the target source code output by the large language model.

[0131] In this implementation, considering that LLM is prone to misinterpretations when detecting test failure causes solely based on data from the current test of the target source code, a failure case database containing multiple test failure cases is set up to enhance the LLM's test failure cause detection capability. Each test failure case contains the test failure characteristics of the tested code and its corresponding test failure cause information. The test failure characteristics are generated based on the test data of the tested code and are used to characterize the test failure phenomenon. Optionally, the test failure characteristics can be generated based on error messages. For ease of comparison, the test failure characteristics can be vectors or vector embeddings; for example, the vector embedding of error messages can be extracted as test failure characteristics. Optionally, the test failure cases in the failure case database can be historical test cases, i.e., cases obtained based on historical data. Optionally, to further improve accuracy, the test failure cases in the failure case database can be cases whose test failure causes have been manually verified. For ease of storage, the test failure cause information in the database can also be in vector form.

[0132] After a test of the target source code fails, test failure features can be generated based on the test data of the target source code. These features can then be used as target test failure features. For example, the vector embeddings of error messages from the target source code can be extracted and used as target test failure features. Next, a search is performed in the failure case database based on these target test failure features to retrieve test failure features that meet the similarity criteria. These retrieved test failure features are designated as similar test failure features, and the test failure cases to which these similar features belong are cases similar to the target source code. Therefore, the test failure reason information corresponding to the similar test failure features is input into the LLM (Limited Management Module), allowing the LLM to analyze the test failure reasons of the target source code based on similar cases. This enhances the test failure reason detection capability and improves detection accuracy through experience. For example, the LLM can prioritize checking whether the test failure reasons of the target source code are the test failure reasons of similar cases.

[0133] The similarity condition can be a similarity score greater than a set similarity threshold, which can be flexibly set according to actual needs, such as 0.8. Alternatively, the similarity condition can be the K highest similarity cases, i.e., retrieving the K highest similarity cases from the failure case database. The specific value of K can be flexibly set according to actual needs. Optionally, reference prompts can be generated based on the test failure reason information corresponding to similar test failure features, and these prompts can be added to the reason detection prompt information, so that the test failure reason information corresponding to similar test failure features is input into the LLM along with the reason detection prompt information. Optionally, the reference prompts can instruct the LLM to first determine whether the test failure reason information of the target source code corresponds to the test failure reason information of the similar test failure features.

[0134] Optionally, in addition to inputting the test failure reason information corresponding to similar test failure characteristics into the LLM, the number of failures corresponding to each test failure reason can be counted based on the found similar test failure characteristics and their corresponding test failure reason information. This count, along with the LLM's input, allows the LLM to perform test failure reason detection based on the test failure reason information and corresponding failure count of similar cases, combined with the test data and code change information of the target source code. Optionally, reference prompts can be generated based on the test failure reason information and corresponding failure count of similar cases. These prompts can be added to the reason detection prompt information. For example, a reference prompt could be: "The past 5 similar 502 errors in the system were all caused by 'database connection pool full'. Please prioritize investigating this possibility."

[0135] Optionally, at least one of the following information can be input into the LLM: target source code, test code, description of failed test units, associated data, and tested code modules, so that the LLM can generate at least one of the following based on the input: test failure reason information, test repair instruction information, etc.

[0136] In an optional example, see Figure 5 As shown, Figure 5 This example shows an optional cause detection message, which includes role definition, error message, failure location, code change information, historical similar cases, cause detection rules, and output requirements. The cause detection rules contain detection sub-rules corresponding to different error types. It should be noted that... Figure 5 The error reporting types and rules provided are merely examples and can be flexibly adjusted according to actual needs.

[0137] S403 generates test repair instructions based on test failure reason information using a large language model.

[0138] In this embodiment, since LLM has text analysis and code analysis capabilities, test repair instruction information can be generated through LLM. For example, LLM can be used to determine the code segment to be modified from the test code and the target source code, and generate modification instruction information for the code segment to be modified.

[0139] It should be noted that, Figure 4 For the specific implementation details of S201-S202 and S204 shown, please refer to Figure 2 S201-S202 and S204 shown will not be described again here.

[0140] exist Figure 4 In the illustrated embodiment, cause detection prompts are generated based on test data and code change information; the cause detection prompts are input into the large language model to obtain test failure cause information and test repair instruction information output by the large language model. Thus, based on the powerful language analysis and code understanding capabilities of LLM, test data and code change information can be accurately analyzed, and test failure cause information and test repair instruction information can be output, thereby improving the accuracy of test failure cause information and test repair instruction information.

[0141] In one exemplary embodiment, to better understand the invention, a more specific example is described. See also: Figure 6 As shown, Figure 6 The flowchart for the code testing methods, including S601-S608, is described in detail below: S601 calls the test code corresponding to the target source code to test the target source code.

[0142] This involves using automated testing tools to run test code and launch the target source code, and then testing the target source code based on the test code.

[0143] The target source code refers to the current version of code that needs to be tested. It can be business code used to implement specific business logic, and the target source code can be specified by the testers. The test code is set up specifically for the business logic corresponding to the target source code and is used to test the business logic.

[0144] S602 collects runtime data of the target source code and runtime data of the test code during the testing process, and obtains the test report of the target source code.

[0145] During the testing of the target source code, runtime data can be collected, and after the target source code testing is completed, test reports output by automated testing tools can be obtained.

[0146] S603: If the test on the target source code fails, extract the error information and the description of the failed test steps from the runtime data and test report.

[0147] The error message describes the fault phenomenon, such as an element not being visible, while the failed test step description describes the test steps in the test code that failed.

[0148] To avoid LLM context overflow, stack focus extraction is performed on the error stacks of the target source code in the runtime data and test reports to obtain key stack lines. The target error stack is then constructed based on these key stack lines, and error information is generated based on the target error stack. Optionally, the stack focus extraction process includes: obtaining the error stacks corresponding to the target source code from the runtime data and test reports; filtering out stack lines corresponding to the framework code from the error stacks; and retaining only the stack lines for the target source code path to obtain key stack lines. Key stack lines may contain the corresponding path, line of code, method name, etc. For example, the output key stack line is: File: src / auth / login.ts, Line: 45, Method: submit(). Furthermore, error information from the test code can also be extracted from the runtime data and test reports.

[0149] S604 retrieves the target source code and target code change information corresponding to historical versions of the code.

[0150] The historical version code is the business code of the previous version, and the historical version code was successfully tested using the test code.

[0151] The process of obtaining target code change information may include: obtaining the target source code and historical version code, performing Git Diff to obtain code change information, performing AST analysis on the code change information, and then deleting the code change data corresponding to blank line modifications, formatting, and comment modifications in the code change information based on the analysis results, while retaining the code change data corresponding to logical changes, DOM attribute changes, and configuration item changes. If there are too many changed files, the code change data of files other than those in the same directory as the stack focus or those with reference relationships are extracted to obtain the target code change information.

[0152] S605, based on the error message, generate target test failure features of the target source code, and search the failure case database for similar test failure features that meet the similarity conditions with the target test failure features.

[0153] The target test failure feature is a vector obtained by encoding the error information of the target source code; the failure case database contains vectors of test failure features and corresponding test failure reasons. In the historical testing process, if the code under test fails and the test failure reason information is manually confirmed, the vector corresponding to the error information of the code under test and the vector corresponding to the test failure reason can be used as failure case conditions in the failure case database.

[0154] Similar test failure features are test failure features in the failure case database whose similarity to the target test failure feature is greater than a similarity threshold.

[0155] S606 generates cause detection prompts based on similar test failure characteristics, test failure reasons corresponding to similar test failure characteristics, error information of target source code, description of failed test steps, target code change information, and cause detection rules, and inputs the cause detection prompts into LLM.

[0156] Based on the test failure reasons corresponding to similar test failure characteristics, the frequency of each reason is counted. The frequency and reason are added to the prompt information as reference answers to guide LLM to prioritize checking whether the test failure reasons of the target source code are the same as the test failure reasons of similar cases. The prompt information also includes the error information of the target source code, the description of the failed test steps, the target code change information, and the reason detection rules.

[0157] Optionally, the prompt message may also include output requirements, role definitions, etc.

[0158] S607, through LLM, performs root cause analysis based on the cause detection prompts and outputs test failure reason information and test repair instructions.

[0159] The test failure reason information includes the fault classification, confidence score, test repair instructions, and test failure reason information.

[0160] The failure detection results can be output in JSON format.

[0161] LLM can perform root cause analysis based on the cause detection rules in the cause detection prompts, based on error messages, descriptions of failed test steps, target code change information, and the number of times different causes correspond to similar cases, to obtain information on the reasons for test failure.

[0162] Optionally, error information from the target source code can be categorized first to obtain error types. These error types are then input into the LLM (Local Management Module). The LLM, based on the input error types, searches for the corresponding sub-rules from the various error types included in the root cause detection rules. Based on the found sub-rules, target code change information, and error messages, root cause analysis is performed to obtain the reason for test failure. Optionally, during root cause analysis, it is necessary to check whether the error messages are related to code changes, thereby determining the reason for test failure based on this correlation.

[0163] In an optional example, the root cause analysis logic includes error type analysis and root cause analysis branches corresponding to different error types, detailed below: Error type analysis: The error message is examined. If the error message includes Network Error, 502 BadGateway, SocketTimeout, etc., it indicates an environment error, and the root cause analysis branch A corresponding to environment errors is entered. If the error message includes AssertionError (assertion failure), Expected X but got Y, etc., the error type is a logic error, and the root cause analysis branch B corresponding to logic errors is entered. If the error message includes NoSuchElement, StaleElementReference, ElementNotVisible, etc., the error type is a UI error, and the root cause analysis branch C corresponding to UI errors is entered.

[0164] Root cause analysis branch A: Obtain the timestamp from the runtime logs corresponding to the test process of the target source code generated by the automated testing system (e.g., CI), and determine whether the environment monitoring system (e.g., Prometheus / Zabbix) has CPU / memory / network alarms at this time. If so, determine that the test failure is due to environmental instability, and this conclusion is of high confidence.

[0165] Root cause analysis branch B: Extract the variable names from the error messages and, based on the target code change information, determine whether the calculation function corresponding to the variable name has been modified; if so, determine that the test failure is due to a potential product bug, which requires manual review; if not, continue to check whether the test data in the test code has expired (such as whether the hard-coded date has expired), and determine the reason for the test failure based on the detection result.

[0166] Root cause analysis branch C: Extract the corresponding strings from the element name and associated class name in the error element selector, such as ".submit-btn". Search for this string in the deleted lines (the content deleted from the target source code relative to the historical version) included in the target code change information. If a match is found, the test failure is determined to be due to UI changes not being synchronized with the test code. LLM can output the following test repair instruction: Generate a new element selector. If no match is found, the test failure is determined to be due to rendering delay / dynamic loading issues. LLM can output the following test repair instruction: It is recommended to increase the wait time.

[0167] In another optional example, see Figure 7 As shown, the root cause analysis process includes: first, parsing the error type based on the error message to enter different root cause analysis branches; then, based on the target code change information and the error message, LLM detects whether any modifications related to the error message have been made in the target test code, and generates test failure reason information based on the relevant detection results. Figure 7 The conclusions in the report) and the corresponding test and repair instructions ( Figure 7 (The suggestions in the document), among which, Figure 7 The error types and root cause analysis logic shown are merely illustrative and can be flexibly adjusted according to actual needs.

[0168] In another optional example, see Figure 8 As shown, the interface displaying the test failure reason information can include test repair instructions, as well as links to test reports, test environment, and target source code submission records.

[0169] Optionally, a spatio-temporal mapping (Spatio-Temporal Mapper) can be performed on the test code and the target source code. This mapping includes static mapping and dynamic detection. The static mapping process involves pre-establishing a static topology graph between the test files in the test code and the target source code (i.e., the code under test) based on file naming conventions (e.g., login.vue vs login.spec.ts) and reference path analysis. The dynamic detection process involves using code coverage tools (such as Istanbul / Jacoco) to reverse-engineer the static topology graph to determine which business code files in the target source code were covered during the execution of the test cases in the test code, thus identifying the tested business code files. This information is then input into the LLM (Local Management Library) so that the LLM can use this information to detect whether the error is related to changes in the tested business code files, thereby determining the cause of the test failure.

[0170] S607, based on the test repair instructions and test code, retest the target source code.

[0171] If the test repair instruction indicates modification of the test environment, the target source code will be automatically tested based on the test code in the adjusted test environment. If the test repair instruction indicates modification of the test code, the target source code will be automatically tested based on the modified test code after the modification is made. If the test repair instruction indicates modification of the target source code, the target source code will be automatically tested based on the modified test code after the modification is made, until the test is successful. If the test repair instruction indicates retesting, no modifications will be made, and the target source code will be tested again based on the test code. Optionally, if the retest fails, the process can return to S602 to loop until the test succeeds, or until the number of tests reaches a set threshold.

[0172] If the test fails to complete after reaching a set threshold number of attempts, a test failure report can be generated to notify the testers or developers that the test has failed and requires manual repair. The notification may include at least one of the following: error messages from each test session, reasons for the test failure, instructions for repair, and the test code to be repaired.

[0173] Optionally, if the test is successful, a test success report can be generated, which may include at least one of the following: error information for each test process, reason information for test failure, test repair instructions, and test code to be repaired. Optionally, if the test code or target source code is modified during the test, the report may also include a comparison before and after the modification, as well as the test code and target source code corresponding to the successful test process.

[0174] In an optional implementation, the code testing method can be executed by a test failure cause detection system, which may include a data awareness layer, a knowledge graph layer, a reasoning and decision-making layer, and a service interaction layer, as detailed below: The Data Perception Layer is used to collect test data from the target source code and obtain target code change information. It may include a Test Context Collector for collecting test data from the target source code and a Code Change Analyzer for obtaining target code change information. For example, the Test Context Collector is used to: parse standard XML / JSON reports such as JUnit / TestNG / Cucumber / Allure; and capture runtime Console Logs, Network HAR packages, screenshots, and DOM Tree Dumps. The Code Change Analyzer is used to: interface with GitLab / GitHub APIs to extract metadata from Merge Requests / Commits; and perform AST analysis on code change information to understand changes in code structure (e.g., "function name changes" have higher weight than "comment changes").

[0175] Knowledge Graph Layer: Used to align test code with target source code, solving the problem of "test code not knowing which business file it is testing". It can be used for the aforementioned spatiotemporal association mapping.

[0176] The inference and decision-making layer is used to denoise the code change information and test data of the target source code, including deduplication, truncation, and sensitive information anonymization, and to generate target code change information and error messages for the target source code. Based on the error messages of the target source code, it retrieves similar cases from the failure case database. Based on the cause detection prompt template, similar cases, error messages from the target source code, and target code change information, it generates cause detection prompts and calls LLM to perform root cause analysis based on the cause detection prompts and context. Optionally, it can also calculate the confidence level of the root cause analysis results based on the LogProbabilities (log probabilities) and rule matching degree detected by LLM.

[0177] Service Interaction Layer: Provides interfaces to external services, enabling them to connect to the test failure cause detection system for test failure analysis. Optionally, it includes a RESTful API gateway for invoking automated testing systems such as CI / CD; and an Integrated Development Environment (IDE) plugin backend service for invoking external services.

[0178] Optionally, the service interaction layer can also connect to automated testing tools to retest the target source code after testing and fixing based on test fix instructions.

[0179] It should be noted that the various embodiments in this application specification can be flexibly combined according to actual needs.

[0180] exist Figure 6 In the illustrated embodiment, LLM analysis of test data and code changes yields information on test failure causes. Compared to manual analysis, this automated output can be completed in seconds, significantly improving detection efficiency and reducing the Mean Time To Repair (MTTR). For UI changes, it provides repair suggestions such as "update element selector," enabling even novice testers to quickly fix test code without needing to delve into the tested front-end source code, lowering the barrier to automation maintenance and improving repair efficiency and accuracy. Furthermore, by combining code change information, it effectively filters out errors caused by environmental fluctuations (such as network timeouts even when the code remains unchanged), enhancing the authority, credibility, and irrefutability of the automated testing system's error reports. Combining test data and code change data analysis solves the data silo problem between test data and code changes, and the semantic understanding capabilities of LLM enable cross-modal root cause localization, improving the accuracy of test failure cause information. Moreover, it can automatically repair tests based on test modification instructions, retesting the target source code, which can improve testing efficiency and success rate.

[0181] See Figure 9 , Figure 9 This is a block diagram illustrating a code testing apparatus as shown in an exemplary embodiment of this application. Figure 9 As shown, the device includes: Test module 901 is used to test the target source code according to the test code corresponding to the target source code to be tested, and obtain the test results; The acquisition module 902 is used to acquire the test data corresponding to the target source code and the code change information of the target source code if the test result indicates that the target source code test failed; wherein, the code change information is used to indicate the change information of the target source code relative to the historical version of the target source code; The detection module 903 is used to detect the reasons for test failures based on test data and code change information, obtain test failure reason information, and generate test repair instruction information based on the test failure reason information. Test module 901 is also used to retest the target source code based on test fix instructions and test code.

[0182] In an exemplary embodiment, based on the foregoing scheme, the detection module 903 is specifically used for: Obtain error information from the test data, and extract code segment location information related to the code segments contained in the target source code from the error information; Based on the code change information, detect whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code, obtain the change detection result, and perform test failure reason detection based on the change detection result to obtain test failure reason information.

[0183] In an exemplary embodiment, based on the foregoing scheme, the detection module 903 is specifically used for: If the error message indicates a logical error, then based on the code change information, check whether the logical code corresponding to the code segment location information contained in the target source code has changed, and obtain the change detection result; If the change detection result indicates that the logic code has been changed, test failure information indicating that there is a logical error in the target source code will be generated; If the logic code indicating the change in the test result has not changed, then test failure reason information indicating that the test code is abnormal will be generated.

[0184] In an exemplary embodiment, based on the foregoing scheme, the detection module 903 is specifically used for: If the error message indicates a user interface error, then based on the code change information, check whether the user interface corresponding to the code segment location information in the target source code has changed, and obtain the change detection result; If the change detection results indicate that the user interface has been changed, test failure information indicating that the user interface in the target source code has been changed is generated.

[0185] In an exemplary embodiment, based on the foregoing scheme, the detection module 903 is specifically used for: If the error message indicates an error in the runtime environment, then based on the code change information, check whether the environment configuration code indicated by the code segment location information in the target source code has changed, and obtain the change detection result; If the test results indicate that the environment configuration code has been changed, test failure information indicating an environment configuration error in the target source code will be generated.

[0186] In an exemplary embodiment, based on the aforementioned scheme, the detection module 903 is specifically configured to: if the code segment corresponding to the code segment location information indicated by the test failure reason information has changed, then determine the code segment to be modified from the test code and the target source code according to the error information and the test failure reason information, and generate modification instruction information corresponding to the code segment to be modified according to the code change data corresponding to the code segment location information contained in the code change information, thereby obtaining test repair instruction information; The testing module 901 is specifically used to: modify the code segment to be modified according to the modification instruction information, obtain the modified code segment, and retest the target source code based on the modified code segment.

[0187] In an exemplary embodiment, based on the foregoing scheme, the detection module 903 is specifically used for: If the error message indicates a user interface error, and the test failure reason information indicates that the user interface corresponding to the code snippet location information in the target source code has changed, then based on the code change data of the user interface contained in the code change information, find the changed interface elements in the user interface and the change data of the changed interface elements, and take the element selector corresponding to the changed interface elements in the test code as the code snippet to be modified. Based on the changed data of the modified interface elements, generate the modification method of the element selector, and generate test repair instruction information based on the modification method.

[0188] In an exemplary embodiment, based on the foregoing scheme, the detection module 903 is specifically used for: Generate cause detection prompts based on test data and code change information; The cause detection prompt information is input into the large language model so that the large language model responds to the cause detection prompt information, performs test failure cause detection based on test data and code change information, and obtains the test failure cause information output by the large language model; Using a large language model, test repair instructions are generated based on the test failure reason information.

[0189] In an exemplary embodiment, based on the foregoing scheme, the detection module 903 is specifically used for: Obtain the preset cause detection rules, and generate cause detection prompts based on the cause detection rules, test data, and code change information; The cause detection prompts are input into the large language model, so that the large language model can detect the test data and code change information according to the cause detection rules contained in the cause detection prompts, and obtain the test failure cause information output by the large language model.

[0190] In an exemplary embodiment, based on the foregoing scheme, the detection module 903 is specifically used for: Generate target test failure features based on test data from the target source code; The database retrieves similar test failure features that meet the similarity criteria to the target test failure feature from the failure case database, along with the test failure reason information corresponding to the similar test failure feature. The failure case database contains multiple test failure features and the test failure reason information corresponding to each test failure feature. The test failure reason information corresponding to similar test failure features and the reason detection prompt information are input into the large language model, so that the large language model can perform test failure reason detection based on the test data, code change information and test failure reason information corresponding to similar test failure features of the target source code, and obtain the test failure reason information of the target source code output by the large language model.

[0191] In an exemplary embodiment, based on the foregoing scheme, the detection module 903 is specifically used for: Perform code semantic analysis on each piece of code change data contained in the code change information to obtain the change semantic information corresponding to each piece of code change data; Based on the change semantic information corresponding to each code change data, the code change data to be filtered out is found from the multiple code change data contained in the code change information; whereby the change semantic information of the code change data to be filtered out indicates that the change type of the code change data to be filtered out is a change type other than the code logical structure change. Remove the code change data to be filtered from the code change information to obtain the target code change information. Based on the test data and the target code change information, perform test failure reason detection to obtain test failure reason information.

[0192] In an exemplary embodiment, based on the foregoing scheme, the detection module 903 is specifically used for: Obtain error information from the target source code in the test data, and find the stack line corresponding to the framework code of the target source code from the error stack contained in the error information; Remove the stack line corresponding to the framework code from the error stack to obtain the error stack after deletion. Then, find the stack line corresponding to the storage path of the target source code from the error stack after deletion to obtain the target error stack. Based on the target error stack and code change information, the cause of test failure is detected to obtain the reason for test failure.

[0193] It should be noted that, Figure 9 The provided code testing device and the code testing method on the detection end side provided in the above embodiments belong to the same concept. The specific way in which each module and unit performs operations has been described in detail in the method embodiments, and will not be repeated here.

[0194] Embodiments of this application also provide an electronic device, including: at least one processor; and a storage device for storing at least one computer program, which, when executed by at least one processor, causes the electronic device to implement the code testing methods provided in the above embodiments.

[0195] Figure 10 A schematic diagram of a computer system suitable for implementing an electronic device according to embodiments of this application is shown. The electronic device may be... Figure 1 The terminal device or server shown.

[0196] It should be noted that, Figure 10 The computer system 1000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0197] like Figure 10 As shown, the computer system 1000 includes a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes based on a computer program stored in Read-Only Memory (ROM) 1002 or a computer program loaded from storage portion 1008 into Random Access Memory (RAM) 1003, such as executing the code testing method described in the above embodiments. Various computer programs and data required for system operation are also stored in RAM 1003. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. An input / output (I / O) interface 1005 is also connected to bus 1004.

[0198] In some embodiments, the following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1010 as needed so that computer programs read from it can be installed into the storage section 1008 as needed.

[0199] In particular, according to embodiments of this application, a computer program implementing the code testing method can be carried on a computer-readable medium, which can be downloaded and installed from a network via the communication section 1009, and / or installed from a removable medium 1011.

[0200] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or at least two wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a computer program that can be used by or in conjunction with an instruction execution system, apparatus, or device. Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer program contained in the computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0201] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or at least two executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and a computer program.

[0202] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0203] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor of an electronic device, causes the electronic device to implement the code testing method described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0204] Another aspect of this application provides a computer program product, which includes a computer program that, when executed by a processor, implements the code testing methods provided in the various embodiments described above. The computer program can be stored in a computer-readable storage medium. The computer program product can be a computer program as a product, such as an APP (Application), webpage, mini-program, etc.; or, the computer program product can also be a storage medium, device, terminal, virtual machine, etc., containing the computer program.

[0205] The above description is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. A code testing method, characterized in that, The method includes: Based on the test code corresponding to the target source code to be tested, the target source code is tested to obtain the test results; If the test result indicates that the target source code test failed, then the test data corresponding to the target source code and the code change information of the target source code are obtained; wherein, the code change information is used to indicate the changes of the target source code relative to the historical version of the target source code; Based on the test data and the code change information, test failure reasons are detected to obtain test failure reason information, and test repair instruction information is generated based on the test failure reason information. Based on the test repair instructions and the test code, the target source code is retested.

2. The method as described in claim 1, characterized in that, The step of detecting test failure reasons based on the test data and the code change information to obtain test failure reason information includes: Error information is obtained from the test data, and code segment location information related to the code segment contained in the target source code is extracted from the error information; Based on the code change information, detect whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code, obtain the change detection result, and perform test failure reason detection based on the change detection result to obtain the test failure reason information.

3. The method as described in claim 2, characterized in that, The step involves detecting, based on the code change information, whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code, obtaining a change detection result, and then performing test failure reason detection based on the change detection result to obtain the test failure reason information, including: If the error message indicates a logical error, then based on the code change information, it is detected whether the logical code corresponding to the code segment location information contained in the target source code has changed, and the change detection result is obtained; If the change detection result indicates that the logic code has been changed, test failure reason information indicating that there is a logical error in the target source code is generated; If the change detection result indicates that the logic code has not been changed, test failure reason information indicating that the test code is abnormal is generated.

4. The method as described in claim 2, characterized in that, The step involves detecting, based on the code change information, whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code, obtaining a change detection result, and then performing test failure reason detection based on the change detection result to obtain the test failure reason information, including: If the error message indicates a user interface error, then based on the code change information, detect whether the user interface corresponding to the code segment location information in the target source code has changed, and obtain the change detection result; If the change detection result indicates that the user interface has been changed, test failure reason information indicating that the user interface in the target source code has been changed is generated.

5. The method as described in claim 2, characterized in that, The step involves detecting, based on the code change information, whether the code segment corresponding to the code segment location information in the target source code has changed relative to the historical version code, obtaining a change detection result, and then performing test failure reason detection based on the change detection result to obtain the test failure reason information, including: If the error message indicates an error in the runtime environment, then based on the code change information, detect whether the environment configuration code indicated by the code segment location information in the target source code has changed, and obtain the change detection result; If the change detection result indicates that the environment configuration code has been changed, test failure reason information indicating that the environment configuration of the target source code is incorrect is generated.

6. The method as described in claim 2, characterized in that, The step of generating test repair instruction information based on the test failure reason information includes: If the test failure reason information indicates that the code segment corresponding to the code segment location information has changed, then based on the error information and the test failure reason information, the code segment to be modified is determined from the test code and the target source code, and based on the code change data corresponding to the code segment location information contained in the code change information, modification instruction information corresponding to the code segment to be modified is generated to obtain the test repair instruction information; The step of retesting the target source code based on the test repair instruction information and the test code includes: Based on the modification instructions, the code segment to be modified is modified to obtain the modified code segment, and the target source code is retested based on the modified code segment.

7. The method as described in claim 6, characterized in that, If the test failure reason information indicates that the code segment corresponding to the code segment location information has changed, then based on the error information and the test failure reason information, the code segment to be modified is determined from the test code and the target source code, and based on the code change data corresponding to the code segment location information contained in the code change information, modification instruction information corresponding to the code segment to be modified is generated to obtain the test repair instruction information, including: If the error message indicates a user interface error, and the test failure reason information indicates that the user interface corresponding to the code segment location information in the target source code has changed, then based on the code change data of the user interface contained in the code change information, the changed interface elements in the user interface and the change data of the changed interface elements are found, and the element selector corresponding to the changed interface elements in the test code is taken as the code segment to be modified. Based on the change data of the modified interface elements, the modification method of the element selector is generated, and the test repair instruction information is generated based on the modification method.

8. The method as described in claim 1, characterized in that, The step involves detecting test failure reasons based on the test data and code change information to obtain test failure reason information, and generating test repair instruction information based on the test failure reason information, including: Generate a cause detection prompt based on the test data and the code change information; The cause detection prompt information is input into the large language model so that the large language model responds to the cause detection prompt information, performs test failure cause detection based on the test data and the code change information, and obtains the test failure cause information output by the large language model; The test repair instruction information is generated based on the test failure reason information using the large language model.

9. The method as described in claim 8, characterized in that, The step of generating cause detection prompt information based on the test data and the code change information includes: Obtain the preset cause detection rules, and generate the cause detection prompt information based on the cause detection rules, the test data, and the code change information; The step of inputting the cause detection prompt information into the large language model, so that the large language model responds to the cause detection prompt information, performs test failure cause detection based on the test data and the code change information, and obtains the test failure cause information output by the large language model, including: The cause detection prompt information is input into the large language model, so that the large language model can detect the test data and the code change information according to the cause detection rules contained in the cause detection prompt information, and obtain the test failure cause information output by the large language model.

10. The method as described in claim 8, characterized in that, The step of inputting the cause detection prompt information into the large language model, so that the large language model responds to the cause detection prompt information, performs test failure cause detection based on the test data and the code change information, and obtains the test failure cause information output by the large language model, including: Generate target test failure features based on the test data of the target source code; The system searches the failure case database for similar test failure features that meet the similarity criteria to the target test failure feature, as well as the test failure reason information corresponding to the similar test failure feature; wherein, the failure case database contains multiple test failure features and test failure reason information corresponding to each test failure feature; The test failure reason information corresponding to the similar test failure features and the reason detection prompt information are input into the large language model, so that the large language model can perform test failure reason detection based on the test data of the target source code, the code change information and the test failure reason information corresponding to the similar test failure features, and obtain the test failure reason information of the target source code output by the large language model.

11. The method as described in claim 1, characterized in that, The step of detecting test failure reasons based on the test data and the code change information to obtain test failure reason information includes: Perform code semantic analysis on each piece of code change data contained in the code change information to obtain the change semantic information corresponding to each piece of code change data; Based on the change semantic information corresponding to each code change data, code change data to be filtered out is found from the multiple code change data contained in the code change information; wherein, the change semantic information of the code change data to be filtered out indicates that the change type of the code change data to be filtered out is a change type other than code logical structure change; The code change data to be filtered is deleted from the code change information to obtain the target code change information. Based on the test data and the target code change information, the test failure reason is detected to obtain the test failure reason information.

12. The method as described in claim 1, characterized in that, The step of detecting test failure reasons based on the test data and the code change information to obtain test failure reason information includes: Obtain the error information of the target source code from the test data, and find the stack line corresponding to the framework code of the target source code from the error stack contained in the error information; Delete the stack line corresponding to the framework code from the error stack to obtain the error stack after deletion, and find the stack line corresponding to the storage path of the target source code from the error stack after deletion to obtain the target error stack; Based on the target error stack and the code change information, the test failure reason is detected to obtain the test failure reason information.

13. A code testing device, characterized in that, The device includes: The testing module is used to test the target source code based on the test code corresponding to the target source code to be tested, and obtain the test results. The acquisition module is used to acquire test data corresponding to the target source code and code change information of the target source code if the test result indicates that the target source code test failed; wherein, the code change information is used to indicate the change information of the target source code relative to the historical version of the target source code; The detection module is used to detect the reasons for test failure based on the test data and the code change information, obtain the reason information for test failure, and generate test repair instruction information based on the reason information for test failure. The testing module is also used to retest the target source code based on the test repair instruction information and the test code.

14. An electronic device, characterized in that, include: At least one processor; A storage device for storing at least one computer program, which, when executed by the at least one processor, causes the electronic device to implement the code testing method according to any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the processor of the electronic device, causes the electronic device to implement the code testing method according to any one of claims 1-12.

16. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the code testing method according to any one of claims 1-12.

Images