Test information processing method, electronic device, and storage medium

Abstract

The application relates to software test automation technology and provides a test information processing method, an electronic device and a storage medium, the method comprising the following steps: determining information source addresses of a plurality of build information sources based on a build task address, wherein the plurality of build information sources comprise a plurality of build parameter pages, console logs, test report interfaces and test report pages; collecting multi-source build information from the plurality of build information sources based on the information source addresses of the plurality of build information sources; extracting a failed test instance from the multi-source build information, and extracting a fault signal segment from the multi-source build information and binding the fault signal segment to the failed test instance corresponding to the fault signal segment. The method, the device and the medium provided by the application overcome the problem of missed detection of a failed test item caused by dependence on a single information source, realize automatic processing of a continuous integration test failed item, reduce the workload of manual analysis and fault positioning, and improve the coverage and accuracy of test failure analysis.

Description

Technical Field

[0001] This invention relates to the field of software testing automation technology, and in particular to a test information processing method, electronic device, and storage medium. Background Technology

[0002] In a continuous integration (CI) platform, when automated tests for a build task fail, relevant personnel typically need to manually check the console logs to locate the failed test items and extract the error information; alternatively, a script based on keyword matching can identify failed tests from the console logs.

[0003] However, if the console logs are truncated, incomplete, or lack critical information, the aforementioned test information processing method, which relies entirely on console logs, will lead to missed test items. Furthermore, in concurrent testing scenarios, various fault signal fragments in the console logs are intertwined and mixed, making manual analysis and processing highly susceptible to errors in fault attribution. Summary of the Invention

[0004] This invention provides a test information processing method, electronic device, and storage medium to address the shortcomings of test information processing in continuous integration platforms in related technologies, which are prone to errors and omissions.

[0005] This invention provides a test information processing method, comprising: Based on the build task address, the information source addresses of multiple build information sources are determined. The multiple build information sources include multiple sources from the build parameter page, console logs, test report interface, and test report page. Based on the information source addresses of the multiple construction information sources, multi-source construction information is collected from the multiple construction information sources; Failed test instances are extracted from the multi-source construction information, and fault signal fragments are extracted from the multi-source construction information, and the fault signal fragments are bound to the failed test instances corresponding to the fault signal fragments.

[0006] According to a test information processing method provided by the present invention, the step of extracting failed test instances from the multi-source construction information includes: Identify the test node identifier and the node status of the test node identifier from the console logs in the multi-source build information, and establish a failed test instance based on the identified test node identifier whose node status is failed; If no test node identifier is identified as having failed, or if the console log is truncated or erroneous, test failure information is extracted from the test report in the multi-source build information, and the failed test instance is established or supplemented based on the test failure information.

[0007] According to a test information processing method provided by the present invention, the step of extracting fault signal segments from the multi-source constructed information includes: Based on at least one of the error values, test output values, runtime error keywords, and alarm keywords in the multi-source construction information, extract fault signal fragments from the multi-source construction information.

[0008] According to a test information processing method provided by the present invention, the step of binding the fault signal segment to a failed test instance corresponding to the fault signal segment includes: Based on the binding rules, the fault signal segment is bound to the failed test instance corresponding to the fault signal segment. The binding rules are used for at least one of the following: delaying the binding timing, preventing accidental binding due to proximity, abnormal backtracking mapping, numerical error instance isolation, and feature conflict determination.

[0009] A test information processing method provided by the present invention further includes: The failed test instances are aggregated based on the test file to which they belong and / or the logical test item to which they belong.

[0010] A test information processing method provided by the present invention further includes: Based on the fault signal fragments bound to the failed test instance, the fault category corresponding to the failed test instance is determined; Based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance, a build task defect report is generated or updated.

[0011] According to a test information processing method provided by the present invention, generating or updating a build task defect report based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance includes: Based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance, defect matching is performed with historical build task defect reports; In the event of defect matching failure, a build task defect report is generated based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance. If a defect match is successful, the successfully matched historical build task defect list is updated based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance. This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements any of the test information processing methods described above.

[0012] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the test information processing method as described above.

[0013] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the test information processing method as described above.

[0014] The test information processing method, electronic device, and storage medium provided by this invention collect information from multiple build information sources, such as build parameter pages, console logs, test report interfaces, and test report pages, to form multi-source build information. This overcomes the problem of missed failure test items caused by traditional solutions relying on a single information source. Furthermore, by extracting failed test instances and fault signal fragments from the multi-source build information and then binding these two together, the invention automates the processing of continuous integration test failures, significantly reducing the workload of manually analyzing build information and locating faults, and improving the coverage and accuracy of test failure analysis. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating the test information processing method provided by the present invention.

[0017] Figure 2 This is a schematic diagram of the test information processing device provided by the present invention.

[0018] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0020] All actions involving the acquisition of signal information or data in this invention are carried out in compliance with the relevant data protection laws and policies of the country where the device is located, and with the authorization granted by the owner of the device.

[0021] Continuous integration is a software development practice in which developers commit and merge code into a shared code repository. Each merge is verified through automated builds and automated tests to identify problems introduced into the code as early as possible.

[0022] In continuous integration platforms such as Jenkins, failure analysis of automated tests for build tasks is a crucial step in ensuring software quality. Currently, the methods for processing test information in continuous integration platforms, especially for analyzing test failures, typically involve testers or developers manually checking console logs in the build platform to locate failed test instances and extract relevant error information; alternatively, keyword-matching scripts are used to automatically extract fields such as FAILED, ERROR, and traceback from console logs to identify failed tests; or failure results are read from test reports, and then failure classification, issue organization, and defect reporting are done manually.

[0023] Among the various methods mentioned above, log parsing, fault location, and defect reporting are mostly independent processes that are separate from each other, and all rely on a single source of information and are output in a fixed format.

[0024] For test information processing methods that rely solely on console logs as the source of information, the actual operation of console logs may result in truncation, incomplete segmented output, or missing key content. Therefore, processing test information based on this method can easily lead to missed detection of failed test items.

[0025] Furthermore, in concurrent or parameterized testing scenarios, error messages, anomaly backtracking information, performance alarms, and numerical comparison snippets from different test nodes are often intertwined and mixed in the console logs. Manual analysis and processing of this test information using the methods described above is insufficient to handle such complex scenarios. This can easily lead to the association of fault signal fragments with incorrect test nodes, resulting in biased fault attribution and consequently affecting subsequent problem analysis and handling.

[0026] To address the aforementioned problems, this invention provides a test information processing method for analyzing and handling test failures on a continuous integration platform. Figure 1 This is a flowchart illustrating the test information processing method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes: Step 110: Based on the build task address, determine the information source addresses of multiple build information sources.

[0027] Specifically, the build task address refers to the access entry identifier of a specific build task in the continuous integration platform, such as the page address of a specific build task in the continuous integration platform. In this embodiment of the invention, by standardizing the build task address, the specific access addresses of various related information sources under the build task can be derived, that is, the information source addresses of multiple build information sources are obtained.

[0028] Here, "build information source" refers to a data source that stores information related to the build task execution process. In this embodiment of the invention, multiple build information sources include at least two of the following: build parameter page, console logs, test report interface, and test report page. Different build information sources each carry different categories of build information, and these different categories of build information complement each other, providing a data foundation for the subsequent comprehensive extraction of information related to build task test failures.

[0029] The information source address of the information source is the specific access address of the information source. The information source address can be used to obtain the corresponding construction information from the information source.

[0030] Furthermore, the build parameters page is the page in the continuous integration platform that records the configuration parameters of build tasks. The information source address of the build parameters page is the build parameters page address. The build parameters page can also store build parameter information such as build number, task name, execution node identifier, hardware model, and software version number.

[0031] Console logs are text logs output in real time by the continuous integration platform during the execution of build tasks. The source address of the console logs is the console log address. Console logs can record the running process of the test framework, the execution status of each test node, and error outputs and alarm information generated during the process.

[0032] The test report interface is a structured data interface provided by the testing framework within the continuous integration platform after test execution. The information source address of the test report interface is the test report interface address. The test report interface can output a test report containing structured information such as the name of each test item, execution result, error summary, and exception details. For example, the test report data can be in JSON (JavaScript Object Notation, JS key-value pair data) format.

[0033] The test report page is the page that displays test reports within the continuous integration platform. The information source address of the test report page is the same as the test report page address. The test report page can display test reports containing various structured information, such as page data displayed in HTML (Hypertext Markup Language) format.

[0034] In this embodiment of the invention, after obtaining the build task address, the build task address can be standardized to obtain the information source addresses of each derived build information source. Further, the standardization process can involve string parsing and path concatenation of the build task address.

[0035] Step 120: Based on the information source addresses of the multiple construction information sources, collect multi-source construction information from the multiple construction information sources.

[0036] Specifically, after determining the information source address of each construction information source, corresponding construction information can be collected from the corresponding construction information source based on each information source address, and then the information collected from each construction information source can be integrated into multi-source construction information.

[0037] It is understood that the resulting multi-source build information is the collection of all build information related to the build task. Furthermore, the multi-source build information in this embodiment of the invention originates from multiple build information sources. Compared to related technologies that only apply build information from a single source for analysis and processing, the multi-source build information in this embodiment of the invention contains more complete and comprehensive build information, effectively avoiding the problem of truncated or missing build information output from a single build information source.

[0038] For example, multi-source build information can include build parameter information collected from the build parameter page, console log text collected from the console logs, test report JSON data collected from the test report interface, and test report HTML page data collected from the test report page. These various types of build information complement each other in subsequent test information processing, thereby ensuring the comprehensiveness and completeness of test failure analysis for build tasks.

[0039] Step 130: Extract failed test instances from the multi-source construction information and extract fault signal fragments from the multi-source construction information, and bind the fault signal fragments to the failed test instances corresponding to the fault signal fragments.

[0040] Specifically, after obtaining the multi-source build information of the build task, failed test instances and fault signal fragments can be extracted from the multi-source build information respectively.

[0041] A failed test instance refers to the record information of a test node that fails during the test execution of a build task. A failed test instance can include the identifier of the test node and related information, such as the test file name, test class name, test function name, and parameterized instance identifier. One failed test instance corresponds to one failed test node.

[0042] Extracting failed test instances from multi-source build information involves parsing the build information to identify and extract relevant information about test nodes in a failed state, and then creating a one-to-one correspondence between these failed test nodes and the resulting failed test instances. Console log text, test report interface data, and test report page data can all serve as sources for extracting failed test instances. Furthermore, these sources can complement and corroborate each other, ensuring the completeness of the extracted failed test instances and preventing missed detections due to incomplete data from a single source.

[0043] Furthermore, a fault signal fragment refers to a text or data fragment identified from multi-source construction information that reflects a specific fault manifestation or fault characteristic. Fault signal fragments may include one or more of the following: precision mismatch fragments, zero output fragments, anomaly backtracking fragments, performance alarm fragments, runtime error fragments, and known limitation indication fragments.

[0044] Extracting fault signal fragments from multi-source construction information can specifically involve extracting fragments with fault characterization significance from multi-source construction information based on predefined identification strategies or rules, and using these fragments as fault signal fragments.

[0045] After extracting the failed test instances and fault signal segments, each fault signal segment can be bound to its logically related failed test instance, thus establishing a binding relationship between the failed test instance and the fault signal segment. By binding the failed test instance and the fault signal segment, the specific fault behavior corresponding to each failed test instance can be clearly identified, thereby enabling fault attribution for the failed test instance.

[0046] In the process of binding fault signal segments to failed test instances, factors such as the position of the fault signal segment in the multi-source construction information and the temporal logic correlation between the fault signal segment and each failed test instance can be referenced to determine which failed test instance the fault signal segment should be bound to, thereby realizing the binding of the fault signal segment to the failed test instance.

[0047] In the test information processing method provided in this embodiment of the invention, information from multiple build information sources, such as the build parameter page, console logs, test report interface, and test report page, is collected to form multi-source build information. This overcomes the problem of missed detection of failed test items caused by the reliance on a single information source in traditional solutions. Furthermore, by extracting failed test instances and fault signal fragments from the multi-source build information and then binding these two together, the automated processing of continuous integration test failure items is achieved. This significantly reduces the workload of manually analyzing build information and locating faults, and improves the coverage and accuracy of test failure analysis.

[0048] Based on the above embodiments, step 130, extracting failed test instances from the multi-source construction information, includes: Identify the test node identifier and the node status of the test node identifier from the console log in the multi-source build information. Based on the identified test node identifier whose node status is failed, establish a failed test instance. If no test node identifier is identified as having failed, or if the console log is truncated or erroneous, test failure information is extracted from the test report in the multi-source build information, and the failed test instance is established or supplemented based on the test failure information.

[0049] Specifically, the multi-source build information may include console logs and test reports. Console logs are the text logs output in real time by the continuous integration platform during test execution, while test reports are the test result records output by the test framework within the continuous integration platform after test execution. Test reports can come from a test report interface, a test report page, or a combination of data collected by both the test report interface and the test report page. This embodiment of the invention does not impose specific limitations on these aspects.

[0050] During the extraction of failed test instances, the first step is to identify and classify the test nodes in the console logs. This yields the test node identifiers and statuses of each test node contained within the logs. Here, the test node identifier is the information that uniquely identifies a test node during test execution. It can include one or more combinations of test file names, test class names, test function names, and parameterized instance identifiers. The node status is the state of the test node during test execution, and can include three categories: FAILED, PASSED, and SKIPPED. A FAILED status indicates that the test node failed in this test execution; a PASSED status indicates that the test node executed successfully; and a SKIPPED status indicates that the test node was skipped in this test execution and was not actually run.

[0051] Here, the identification of test nodes and the identification of node status are performed on the console logs. This can be achieved through various text analysis methods such as regular expression matching and log format template parsing. This embodiment of the invention does not impose specific limitations on this.

[0052] Based on this, test node identifiers with a failed status can be filtered from the identified test node identifiers, thereby establishing failed test instances. Here, each test node identifier with a failed status corresponds to a failed test instance, which can record the test node identifier of that test node, such as test file name, test class name, test function name, and parameterized instance identifier.

[0053] In addition, for cases where test node identifiers with a failed status are not identified from the console logs, and for cases where console logs are truncated or erroneous, test reports from multi-source build information can be introduced as supplementary information to the console logs, thereby ensuring the completeness and comprehensiveness of the acquisition of failed test instances.

[0054] Furthermore, when a test node with a failed status is not identified, or when the console log is truncated or erroneous, the test report in the multi-source build information can be further parsed to extract test failure information. Failed test instances can be constructed based on the extracted test failure information, or existing failed test instances can be supplemented based on the extracted test failure information, thereby restoring the failed test instances that were missing due to incomplete console logs.

[0055] Test failure information refers to specific information extracted from the test report that reflects test execution failures. This may include, for example, the name of the file that failed, the name of the class to which the test belongs, the name of the function that failed, and the parameterized instance identifier of the failed test. The test failure information semantically corresponds to the test node identifier parsed from the console logs and can be used to create or supplement failed test instances.

[0056] After receiving test failure information, creating a failed test instance based on the test failure information can be done if a failed test instance cannot be created based on the console logs. The way to supplement a failed test instance based on the test failure information can be to add missing information to an existing failed test instance, or to create other failed test instances based on an existing failed test instance. This embodiment of the invention does not limit this.

[0057] In this embodiment of the invention, test failure information is extracted from the test report to supplement the test log when the console log is unavailable or incomplete. This effectively solves the problem of missing failed test instances in scenarios such as truncation, anomalies, or incomplete output of the console log in related technologies, and improves the completeness and reliability of the extraction of failed test instances.

[0058] Based on any of the above embodiments, step 130, extracting fault signal fragments from the multi-source construction information, includes: Based on at least one of the error values, test output values, runtime error keywords, and alarm keywords in the multi-source construction information, extract fault signal fragments from the multi-source construction information.

[0059] Specifically, to accurately pinpoint the cause of test failure, this embodiment of the invention extracts fault signal segments from the multi-source build information based on at least one dimension: error value, test output value, runtime error keywords, and alarm keywords. Specifically, this can be done by extracting fault signal segments from console logs and / or test reports within the multi-source build information.

[0060] In this context, error values ​​typically refer to the numerical difference between the actual calculated output data and the expected reference data during automated testing. When error values ​​are identified in multi-source construction information, the segment containing these error values ​​can be extracted as a precision mismatch segment. For example, if multi-source construction information contains expected reference data, actual output data, and error data, this portion of data can be extracted as a type of fault signal segment, specifically a precision mismatch segment.

[0061] Test output values ​​refer to the calculation results or status values ​​directly output by the test program after execution. Test output values ​​may exhibit unexpected or unusual forms, such as all zeros. When an abnormal form of test output value is identified in multi-source build information, a segment containing the test output value can be extracted as an anomalous output segment. For example, if all actual output values ​​in multi-source build information are zero, while the expected reference value is non-zero, this data can be extracted as a fault signal segment, specifically a zero-output segment.

[0062] Runtime error keywords are specific identifier strings output when a test program crashes, terminates abnormally, or throws an exception. Examples include traceback, ErrorCode, or [ERROR]. When runtime error keywords are identified in multi-source build information, the fragment containing these keywords can be extracted as a runtime error fragment. Specifically, in the case of extracted traceback, the runtime error fragment can be an exception backtracking fragment.

[0063] Alarm keywords refer to suggestive strings displayed when a program, though not completely crashing, triggers certain soft limits, performance thresholds, or known defects. Examples include "Warning" and "threshold missed." When alarm keywords are identified in multi-source build information, segments containing these keywords can be extracted as fault signal segments. For example, when a "Warning" is identified, a type of fault signal segment can be extracted, specifically a performance alarm segment; when a "threshold missed" is identified, a type of fault signal segment can be extracted, specifically a performance rollback segment.

[0064] In this embodiment of the invention, fault signal fragments are extracted based on dimensions such as error value, test output value, runtime error keywords and alarm keywords, which can effectively improve the completeness and precision of test failure information feature extraction, thereby distinguishing different types of faults such as precision mismatch, zero output, runtime error, performance alarm, and performance rollback.

[0065] Based on any of the above embodiments, step 130, which involves binding the fault signal segment to the failed test instance corresponding to the fault signal segment, includes: Based on the binding rules, the fault signal segment is bound to the failed test instance corresponding to the fault signal segment. The binding rules are used for at least one of the following: delaying the binding timing, preventing accidental binding due to proximity, abnormal backtracking mapping, numerical error instance isolation, and feature conflict determination.

[0066] Specifically, in concurrent execution testing or parameterized testing scenarios, error messages, anomaly backtracking information, performance alarms, and numerical comparison segments from multiple test nodes are often intertwined and mixed in output. To further improve the reliability and accuracy of the binding between fault signal segments and failed test instances, in this embodiment of the invention, the binding process between fault signal segments and failed test instances is constrained based on pre-defined binding rules.

[0067] Here, the binding rules can be used for at least one of the following: delaying binding timing, preventing accidental binding due to proximity, abnormal backtracking mapping, isolating numerical error instances, and determining feature conflicts.

[0068] Among them, the binding rule for delaying the binding timing means that during the binding process, the binding operation is not performed immediately after the fault signal segment is identified. Instead, it waits until it is confirmed that the current test node has been clearly identified as a failure state before binding the fault signal segment that occurs during the execution of the test node to the corresponding failed test instance.

[0069] The binding rule for preventing accidental binding based on proximity refers to the practice of not directly binding isolated fault signal fragments that appear in multi-source build information and do not carry explicit test node identifiers to the most recently failed test instance during the binding process. In some embodiments, isolated fault signal fragments may be FAILED markers that appear in isolation and do not carry test node identifiers.

[0070] The binding rule used for exception backtracking mapping refers to binding the exception backtracking fragment in the multi-source build information to the actual failed test instance that triggered the exception backtracking, rather than simply binding it to any failed test instance that is adjacent to the exception backtracking information in the multi-source build information.

[0071] The binding rule for numerical error instance isolation means that the precision mismatch fragment is bound only to the corresponding failed test instance in the current context, and the precision mismatch fragment is not allowed to spread across the test instance boundary to other failed test instances.

[0072] The binding rules used for feature conflict determination refer to the process where, when the same failed test instance is simultaneously associated with multiple categories of fault signal segments during the binding process, and there are conflicts or overlaps between the fault categories indicated by these fault signal segments, the fault signal segments can be selected or sorted according to a preset priority rule. For example, when a failed test instance is associated with both a precision mismatch segment and an anomaly backtracking segment, since the anomaly backtracking segment better reflects runtime errors, it can be prioritized to bind the anomaly backtracking segment to the failed test instance.

[0073] In this embodiment of the invention, the binding process between fault signal segments and failed test instances is constrained by binding rules that include at least one of delayed binding timing, prevention of accidental binding due to proximity, abnormal backtracking mapping, numerical error instance isolation, and feature conflict determination, thereby effectively avoiding the problem of accidental binding.

[0074] Based on any of the above embodiments, the test information processing method further includes: The failed test instances are aggregated based on the test file to which they belong and / or the logical test item to which they belong.

[0075] Specifically, a test file refers to a file that stores automated test code. Typically, a test file contains one or more test instances targeting a specific module or function. A logical test item, on the other hand, refers to the test unit within a test file, such as a test class or test function. For example, in a parameterized testing scenario, the same logical test item may be assigned multiple different parameter configurations and executed repeatedly; each execution can be considered a test instance.

[0076] Therefore, when multiple failed test instances are obtained, failed test instances belonging to the same test file or the same logical test item can be aggregated to obtain a set or group of failed test instances based on the test file or logical test item.

[0077] It is understandable that the resulting set or group of failed test instances includes multiple failed test instances belonging to the same test file or the same logical test item.

[0078] Furthermore, the resulting set or group of failed test instances can retain the configuration list of each instance, the number of failures, the identifier of the first failed test node in the set or group, and a representative error summary extracted from that set or group. This approach reduces the number of failure records while fully preserving the configuration details needed to locate the fault.

[0079] In this embodiment of the invention, the aggregation of failed test instances based on test files or logical test items can effectively address the large number of similar or originating failed test instances generated in parameterized testing and concurrent execution scenarios. While preserving configuration details, it significantly reduces duplicate failure records, which helps to reduce the situation of repeated analysis and repeated creation of defect tickets.

[0080] Based on any of the above embodiments, the test information processing method further includes: Based on the fault signal fragments bound to the failed test instance, the fault category corresponding to the failed test instance is determined; Based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance, a build task defect report is generated or updated.

[0081] Specifically, after binding a fault signal segment to a corresponding failed test instance, the failed test instance can be classified based on the fault signal segment it is bound to, thereby obtaining the fault category of the failed test instance. Here, the identified fault category may include one or more of the following: precision mismatch, zero output, performance regression, runtime error, and known limitations.

[0082] In addition to determining the fault category corresponding to the failed test instance, the fault level of the failed test instance can be further determined based on one or more of the error range, error segment category, and anomaly severity in the fault signal segment bound to the failed test instance, such as warning level or critical level.

[0083] After obtaining the failure category of the failed test instance, the build task defect report can be generated or updated by combining the failure category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance.

[0084] The build parameters in the multi-source build information are the build information collected from the build parameters page, which may include metadata such as build number, build task address, hardware model, software version, and execution node. Build task defect reports, on the other hand, are structured information created in the defect management system to record test failure issues; they can also be simply referred to as defect reports.

[0085] Furthermore, by combining the failure category of the failed test instance, the corresponding build parameters, and the test node identifier, the defect report for the build task can be automatically generated by calling the defect management system's interface. Alternatively, the status of existing related build task defect reports in the defect management system can be updated and information can be added. For example, the failure category, build parameters, and test node identifier can be automatically populated into the template fields of the build task defect report. Here, the template fields of the build task defect report may include build number, build address, hardware model, software version, test node identifier, failure category, number of failed configurations, representative error summary, priority, tags, and assignment information.

[0086] Furthermore, a single build task defect list can be created for all failed test instances under a single build task. In this defect list, the build task corresponds to a parent defect list. Under the parent defect list, multiple sub-task defect lists are created for each test file, and the aforementioned information of the failed test instances belonging to that test file is filled into the defect list of the corresponding sub-task of that test file.

[0087] Alternatively, a separate defect report could be created for each test file or failed test instance within a single build task. For example, a separate build task defect report could be created for each test file, and the aforementioned information for the failed test instances belonging to that test file could be populated into the corresponding build task defect report for that test file. As another example, a separate build task defect report could be created for each failed test instance or test node identifier, and the aforementioned information for each failed test instance could be populated into the corresponding build task defect report for that failed test instance.

[0088] In this embodiment of the invention, based on the fault signal fragments bound to failed test instances, fault classification is performed on the failed test instances, facilitating problem flow and responsibility assignment. Furthermore, by combining fault categories, build parameters from multi-source build information, and test node identifiers, the defect slip for the build task is automatically populated. This achieves full automation from information parsing and fault classification to defect reporting, reducing the workload and time spent on manually compiling logs, analyzing errors, and entering defect information, and shortening the closed-loop time from fault discovery to problem tracking.

[0089] Based on any of the above embodiments, generating or updating a build task defect report based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance includes: Based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance, defect matching is performed with historical build task defect reports; In the event of defect matching failure, a build task defect report is generated based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance. If a defect match is successful, the historical build task defect list that was successfully matched is updated based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance.

[0090] Specifically, before generating or updating a build task defect form, defect matching can be performed by combining the fault category corresponding to the failed test instance to be filled in, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance with the historical build task defect forms.

[0091] Specifically, based on at least one of the fault category, build parameters, and test node identifier, the failed test instance to be filled in is matched with the defects in the historical build task defect forms that have already been filled in. This allows us to determine whether the defect reflected in the failed test instance to be filled in has already been reflected in the historical build task defect forms. In other words, it determines whether the defect reflected in the historical build task defect forms and the defect reflected in the failed test instance to be filled in are the same defects.

[0092] For example, a matching key can be generated using the fault category corresponding to the failed test instance, the hardware model and software version in the build parameters, the test file name and test function name in the test node identifier, etc., thereby achieving defect matching between failed test instances and historical build task defect lists.

[0093] If a match is successful, meaning the failed test instance to be filled in matches a set or group of failed test instances already filled in the historical build task defect form, then there is no need to create a new build task defect form for the failed test instance to be filled in. Instead, the failed test instance to be filled in is added to the historical build task defect form that has been successfully matched, thereby updating the historical build task defect form and avoiding duplicate reporting of defects.

[0094] If the matching fails, meaning the failed test instance to be filled in does not match the set or group of failed test instances already filled in the historical build task defect form, then it is necessary to generate or update the build task defect form based on the fault category corresponding to the failed test instance to be filled in, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance.

[0095] In this embodiment of the invention, defect matching is performed before generating or updating the defect ticket for the construction task, which can avoid the situation of duplicate ticket creation.

[0096] In addition, based on at least one of the fault category, build parameters, and test node identifier, defects can be deduplicated among the failed test instances to be filled, thereby avoiding duplicate creation of the same defect.

[0097] Based on any of the above embodiments, this invention provides a test information processing method, which includes the following steps: First, obtain the build task address of the continuous integration platform. Then, standardize the build task address to obtain the source addresses of multiple build information sources. These source addresses include the build parameter page address, the console log address, the test report interface address, and the test report page address.

[0098] Secondly, based on the information source addresses of the aforementioned multiple build information sources, multi-source build information is collected. This multi-source build information includes build parameter information collected from the build parameter page, console log text collected from the console logs, test report JSON data collected from the test report interface, and test report HTML page data collected from the test report page.

[0099] After obtaining multi-source build information, test execution results can be parsed from the console logs within the multi-source build information to identify test node identifiers and their node states. Furthermore, for test node identifiers with no failed node states in the console logs, or in cases where the console logs are truncated or erroneous, test failure information such as the failed test name, failed class name, error summary, and exception details can be extracted from the test report in the multi-source build information. This obtained test failure information is then used to supplement the parsed console log results, thereby constructing a complete and comprehensive failure test instance.

[0100] In addition, fault signal fragments can be extracted from console logs and / or test reports in multi-source build information based on pre-defined rules. Here, fault signal fragments can include one or more of the following: precision mismatch fragments, zero output fragments, anomaly backtracking fragments, performance alarm fragments, runtime error fragments, and known limitation prompt fragments.

[0101] After obtaining the failed test instance and the fault signal segment respectively, the binding process between the fault signal segment and the failed test instance can be constrained based on the pre-set binding rules, thereby realizing the binding between the failed test instance and the fault signal segment.

[0102] After binding is completed, failed test instances belonging to the same test file or the same logical test item can be merged and aggregated, and information such as the configuration list, number of failures, identifier of the first failed node, and representative error summary of the failed test instances can be retained.

[0103] Furthermore, based on the fault signal segments associated with the failed test instance, the failed test instance can be classified into fault categories and fault levels. Here, the fault level can also be understood as the severity of the fault. Fault categories can include one or more of the following: precision mismatch, zero output, performance regression, runtime errors, and known limitations.

[0104] Finally, deduplication can be performed based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance. After deduplication is completed, a new build task defect form can be created in the defect management system for failed test instances that do not match the set or group of failed test instances already filled in the historical build task defect form; for failed test instances that match the set or group of failed test instances already filled in the historical build task defect form, the matched historical build task defect form can be updated in the defect management system, which can also be understood as associating the failed test instance with that historical build task defect form.

[0105] In this embodiment of the invention, by fusing build information from multiple build information sources to obtain multi-source build information, failed test instances can still be recovered even if console logs are truncated or missing, reducing the false negative rate and improving the completeness of failure identification. Furthermore, binding fault signal fragments to failed test instances based on preset rules avoids misbinding issues caused by concurrent log output and isolated failure markers, improving the accuracy of error attribution. Additionally, by merging and aggregating failed test instances, duplicate recordings and duplicate order creation can be reduced while preserving configuration details. Fault classification of failed test instances effectively distinguishes various fault categories, facilitating problem flow and responsibility assignment. Moreover, through defect deduplication and defect order creation and updating based on defect matching, the workload of manually organizing logs and entering defect information can be effectively reduced, shortening the closed-loop time from fault discovery to problem tracking. Furthermore, the above test information processing method is not only applicable to Jenkins and pytest scenarios but can also be extended to other continuous integration platforms, testing frameworks, and defect management systems.

[0106] The test information processing apparatus provided by the present invention is described below. The test information processing apparatus described below and the test information processing method described above can be referred to in correspondence.

[0107] Figure 2 This is a schematic diagram of the test information processing device provided by the present invention, as shown below. Figure 2 As shown, the device includes: The acquisition unit 210 is used to determine the information source address of multiple build information sources based on the build task address. The multiple build information sources include multiple ones from the build parameter page, console log, test report interface and test report page. The acquisition unit 220 is used to acquire multi-source construction information from the multiple construction information sources based on the information source addresses of the multiple construction information sources; The parsing unit 230 is used to extract failed test instances from the multi-source construction information, extract fault signal segments from the multi-source construction information, and bind the fault signal segments to the failed test instances corresponding to the fault signal segments.

[0108] In the test information processing device provided in this embodiment of the invention, multi-source build information is formed by collecting information from multiple build information sources such as build parameter page, console log, test report interface, and test report page. This overcomes the problem of missed detection of failed test items caused by the reliance on a single information source in traditional solutions. Furthermore, by extracting failed test instances and fault signal fragments from the multi-source build information and binding them together, the automated processing of continuous integration test failure items is achieved, which greatly reduces the workload of manually analyzing build information and locating faults, and improves the coverage and accuracy of test failure analysis.

[0109] Based on any of the above embodiments, the parsing unit is specifically used for: Identify the test node identifier and the node status of the test node identifier from the console log in the multi-source build information. Based on the identified test node identifier whose node status is failed, establish a failed test instance. If no test node identifier is identified as having failed, or if the console log is truncated or erroneous, test failure information is extracted from the test report in the multi-source build information, and the failed test instance is established or supplemented based on the test failure information.

[0110] Based on any of the above embodiments, the parsing unit is specifically used for: Based on at least one of the error values, test output values, runtime error keywords, and alarm keywords in the multi-source construction information, extract fault signal fragments from the multi-source construction information.

[0111] Based on any of the above embodiments, the parsing unit is specifically used for: Based on the binding rules, the fault signal segment is bound to the failed test instance corresponding to the fault signal segment. The binding rules are used for at least one of the following: delaying the binding timing, preventing accidental binding due to proximity, abnormal backtracking mapping, numerical error instance isolation, and feature conflict determination.

[0112] Based on any of the above embodiments, the parsing unit is further configured to: The failed test instances are aggregated based on the test file to which they belong and / or the logical test item to which they belong.

[0113] Based on any of the above embodiments, the parsing unit is further configured to: Based on the fault signal fragments bound to the failed test instance, the fault category corresponding to the failed test instance is determined; Based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance, a build task defect report is generated or updated.

[0114] Based on any of the above embodiments, the parsing unit is specifically used for: Based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance, defect matching is performed with historical build task defect reports; In the event of defect matching failure, a build task defect report is generated based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance. If a defect match is successful, the historical build task defect list that was successfully matched is updated based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance.

[0115] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a test information processing method, which includes: Based on the build task address, the information source addresses of multiple build information sources are determined. The multiple build information sources include multiple sources from the build parameter page, console logs, test report interface, and test report page. Based on the information source addresses of the multiple construction information sources, multi-source construction information is collected from the multiple construction information sources; Failed test instances are extracted from the multi-source construction information, and fault signal fragments are extracted from the multi-source construction information, and the fault signal fragments are bound to the failed test instances corresponding to the fault signal fragments.

[0116] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0117] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer being able to execute the test information processing method provided by the above methods, the method comprising: Based on the build task address, the information source addresses of multiple build information sources are determined. The multiple build information sources include multiple sources from the build parameter page, console logs, test report interface, and test report page. Based on the information source addresses of the multiple construction information sources, multi-source construction information is collected from the multiple construction information sources; Failed test instances are extracted from the multi-source construction information, and fault signal fragments are extracted from the multi-source construction information, and the fault signal fragments are bound to the failed test instances corresponding to the fault signal fragments.

[0118] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the test information processing methods provided by the methods described above, the method comprising: Based on the build task address, the information source addresses of multiple build information sources are determined. The multiple build information sources include multiple sources from the build parameter page, console logs, test report interface, and test report page. Based on the information source addresses of the multiple construction information sources, multi-source construction information is collected from the multiple construction information sources; Failed test instances are extracted from the multi-source construction information, and fault signal fragments are extracted from the multi-source construction information, and the fault signal fragments are bound to the failed test instances corresponding to the fault signal fragments.

[0119] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0120] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A test information processing method, characterized in that, include: Based on the build task address, the information source addresses of multiple build information sources are determined. The multiple build information sources include multiple sources from the build parameter page, console logs, test report interface, and test report page. Based on the information source addresses of the multiple construction information sources, multi-source construction information is collected from the multiple construction information sources; Failed test instances are extracted from the multi-source construction information, and fault signal fragments are extracted from the multi-source construction information, and the fault signal fragments are bound to the failed test instances corresponding to the fault signal fragments.

2. The test information processing method according to claim 1, characterized in that, The step of extracting failed test instances from the multi-source construction information includes: Identify the test node identifier and the node status of the test node identifier from the console log in the multi-source build information. Based on the identified test node identifier whose node status is failed, establish a failed test instance. If no test node identifier is identified as having failed, or if the console log is truncated or erroneous, test failure information is extracted from the test report in the multi-source build information, and the failed test instance is established or supplemented based on the test failure information.

3. The test information processing method according to claim 1, characterized in that, Extracting fault signal fragments from the multi-source construction information includes: Based on at least one of the error values, test output values, runtime error keywords, and alarm keywords in the multi-source construction information, extract fault signal fragments from the multi-source construction information.

4. The test information processing method according to claim 1, characterized in that, The step of binding the fault signal segment to the failed test instance corresponding to the fault signal segment includes: Based on the binding rules, the fault signal segment is bound to the failed test instance corresponding to the fault signal segment. The binding rules are used for at least one of the following: delaying the binding timing, preventing accidental binding due to proximity, abnormal backtracking mapping, numerical error instance isolation, and feature conflict determination.

5. The test information processing method according to any one of claims 1 to 4, characterized in that, Also includes: The failed test instances are aggregated based on the test file to which they belong and / or the logical test item to which they belong.

6. The test information processing method according to any one of claims 1 to 4, characterized in that, Also includes: Based on the fault signal fragments bound to the failed test instance, the fault category corresponding to the failed test instance is determined; Based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance, a build task defect report is generated or updated.

7. The test information processing method according to claim 6, characterized in that, The process of generating or updating a build task defect report based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance includes: Based on the fault category corresponding to the failed test instance, the construction parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance, defect matching is performed with historical build task defect reports; In the event of defect matching failure, a build task defect report is generated based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance. If a defect match is successful, the historical build task defect list that was successfully matched is updated based on the fault category corresponding to the failed test instance, the build parameters in the multi-source build information to which the failed test instance belongs, and the test node identifier of the failed test instance.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the test information processing method as described in any one of claims 1 to 7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the test information processing method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the test information processing method as described in any one of claims 1 to 7.

Images