Continuous integration method and system, electronic equipment, storage medium and program product

Abstract

The invention provides a continuous integration method and system, electronic equipment, a storage medium and a program product, and the method comprises the steps: triggering a first test corresponding to code change information under the condition that the code change information is detected; if the test result of the first test is passing, acquiring a change feature of the code change information, and predicting a prior probability that the code change information is abnormal according to the change feature; taking a test result of the first test as an observation evidence, and determining a posterior probability that the code change information is abnormal according to the observation evidence and the prior probability; triggering a second test corresponding to the code change information under the condition that the posterior probability is greater than a preset triggering threshold value; the test scale of the second test is larger than the test scale of the first test. According to the mode, the association between the first test and the second test is established, so that the second test is subjected to linkage triggering based on the test condition of the first test, and the code detection efficiency and accuracy are improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a continuous integration method, system, electronic device, storage medium, and program product. Background Technology

[0002] The core idea of ​​continuous integration is that developers frequently integrate code changes into the main branch. Each integration is verified through automated builds and tests, aiming to quickly identify integration errors and ensure software quality. In a typical continuous integration process, when the codebase detects changes (such as new commits or merge requests), a series of predefined tests are automatically triggered.

[0003] Currently, tests are generally divided into two main types: the first type (such as unit tests and basic integration tests) runs quickly and consumes few resources, aiming to quickly provide feedback on basic issues. The second type (such as more comprehensive integration tests and end-to-end tests) has a wider coverage and larger data volume, but takes longer to execute and consumes more computational resources. In related technologies, these two types of tests are usually triggered according to mutually independent logic. Summary of the Invention

[0004] This disclosure provides a continuous integration method, system, electronic device, storage medium, and program product.

[0005] In a first aspect, this disclosure provides a continuous integration method, the method comprising:

[0006] Upon detecting code change information, a first test corresponding to the code change information is triggered;

[0007] If the test result of the first test is passed, the change characteristics of the code change information are obtained, and the prior probability that the code change information is abnormal is predicted based on the change characteristics;

[0008] The test result of the first test being passed is used as observational evidence. Based on the observational evidence and the prior probability, the posterior probability that the code change information is abnormal is determined.

[0009] If the posterior probability is greater than a preset trigger threshold, a second test corresponding to the code change information is triggered; the test scale of the second test is greater than that of the first test.

[0010] Secondly, this disclosure provides a continuous integration system, including:

[0011] The detection module is adapted to trigger a first test corresponding to the code change information when code change information is detected;

[0012] The prediction module is adapted to obtain the change features of the code change information if the test result of the first test is passed, and predict the prior probability that the code change information is abnormal based on the change features;

[0013] The determination module is adapted to use the test result of the first test as observation evidence, and determine the posterior probability that the code change information is abnormal based on the observation evidence and the prior probability.

[0014] The triggering module is adapted to trigger a second test corresponding to the code change information when the posterior probability is greater than a preset triggering threshold; the test scale of the second test is greater than the test scale of the first test.

[0015] Thirdly, this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the methods described above.

[0016] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described method.

[0017] Fifthly, this disclosure provides a computer program product comprising computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the method described above.

[0018] The embodiments provided in this disclosure, when the first test result is a pass, further acquire the change characteristics of the code change information, and predict the prior probability that the code change information is abnormal based on the change characteristics. Furthermore, the pass result of the first test is used as observational evidence, and based on the observational evidence and the prior probability, the posterior probability that the code change information is abnormal is determined. If the posterior probability is greater than a preset trigger threshold, a second test with a larger test scale is triggered. This method uses the pass result of the first test as observational evidence, determines the posterior probability based on the observational evidence and the prior probability, and then triggers the second test based on the posterior probability. Therefore, this method establishes a correlation between the first and second tests, enabling the second test to be triggered in conjunction with the test results of the first test, thus making the triggering timing of the second test more reasonable and improving the efficiency and accuracy of code detection.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0021] Figure 1 This is a flowchart of a continuous integration method in one possible implementation of this application.

[0022] Figure 2 This is a flowchart illustrating a possible implementation of the continuous integration method in this application.

[0023] Figure 3 This is a schematic diagram of the feature indicators of code change characteristics in one possible implementation of this application.

[0024] Figure 4 This is a schematic diagram of the structure of a continuous integration system in one possible implementation of this application.

[0025] Figure 5 This is a block diagram of an electronic device in one possible implementation of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0027] Unless otherwise specified, the various embodiments and features of this disclosure may be combined with each other. As used herein, the term "and / or" includes any and all combinations of one or more of the associated enumerated entries.

[0028] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0029] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0030] Figure 1 This is a flowchart illustrating one possible implementation of a continuous integration method according to this application. (Refer to...) Figure 1 The method includes:

[0031] Step S110: If code change information is detected, trigger the first test corresponding to the code change information.

[0032] Code change information refers to change events and related metadata that occur in the codebase during software development. This includes, but is not limited to, various events such as code commits, merge requests, and branch pushes, as well as associated metadata such as the content of the changes, committer information, timestamps, and a list of modified files. Detection can be achieved through the continuous integration system's mechanism for detecting code change events. For example, code change events can be captured by polling the version control system or through event notifications issued by the version control system.

[0033] Accordingly, the first test can be a set of rapid verification tests that run preferentially for code changes within the continuous integration process. It is characterized by fast execution speed and low resource consumption, aiming to verify the basic correctness and core functionality of the code. For example, the first test may include at least one of the following: compilation tests, a subset of unit tests, or smoke tests. The first test can be triggered in various ways, such as calling a test script, creating a containerized test environment, or adding a task to the test queue.

[0034] Step S120: If the test result of the first test is passed, obtain the change characteristics of the code change information, and predict the prior probability that the code change information is abnormal based on the change characteristics.

[0035] In this context, a "pass" result for the first test means that all test cases in the first test executed successfully without any blocking issues. Examples include passing the compilation test (no compilation errors), passing the smoke test (core functionality is normal), and passing the static code analysis (no code violations).

[0036] Change characteristics of code change information can be extracted from the code change content, code context, and related metadata. Specifically, these can be feature metrics used to characterize change attributes. For example, these include, but are not limited to, quantitative metrics based on dimensions such as code change scale, change complexity, change type, and developer's historical performance.

[0037] The prior probability of anomalies in code change information refers to the probability predicted based on the change characteristics of the code change information before obtaining the observational evidence of the first test result. Here, "anomalies in code change information" means that the code change will cause an anomaly or introduce a defect. Specifically, the prior probability can be represented by prior information in Bayes' theorem, indicating an initial risk estimate determined based on historical experience.

[0038] In obtaining change features of code change information, this can be achieved based on metadata in the version control system, such as extracting features by parsing commit information and difference comparisons. Examples include extracting the number of newly added / deleted lines of code, the number of modified files, and the depth of modified directories. Alternatively, it can be achieved through static code analysis, such as using static code analysis tools to perform in-depth analysis of the changed code and extract features like complexity, coupling, and code style compliance. Furthermore, it can be achieved based on development context information, such as combining development process data like the number of code review comments, review approval time, and associated task priorities to obtain change features. Correspondingly, predicting prior probabilities can be achieved through various methods such as logistic regression models, gradient boosting decision trees, and deep learning models; this application does not limit the specific prediction method.

[0039] Step S130: Take the test result of the first test as the observation evidence, and determine the posterior probability that the code change information is abnormal based on the observation evidence and the prior probability.

[0040] Observational evidence refers to objective facts obtained during the testing process that can be used to update probability judgments. In this embodiment, it specifically refers to the test result "the first test passed," which is used as conditional evidence in the Bayesian update. Posterior probability refers to the probability that the code change is abnormal after obtaining observational evidence (the first test passed). Specifically, it can be the posterior probability in Bayes' theorem, representing the comprehensive judgment result obtained by combining prior probability and observational evidence.

[0041] In one alternative implementation, the first test result can be simplified to two states: pass and fail, serving as discrete observational evidence. This approach enables rapid probability updates, meeting the response speed requirements of continuous integration.

[0042] In another alternative implementation, the first test result can be further subdivided into multiple result dimensions. Specifically, the first test result can be subdivided into multiple technical dimensions, and the strength of observed evidence can be determined based on the results of each dimension to calculate the posterior probability. Accordingly, when determining the observed evidence based on the test result of the first test, the subdivision results of the first test in multiple predetermined technical dimensions are obtained; based on the multiple subdivision results, the comprehensive strength of observed evidence is determined. Among them, the multiple technical dimensions include at least one of the following: (1) Test case complexity dimension: assess the complexity of the test case itself (such as path coverage, state machine coverage). If a high-complexity test passes, a higher confidence level is given. (2) Test environment stability dimension: assess the environmental state during test execution (such as network latency, resource utilization). A stable environment makes the result more credible. (3) Test data authenticity dimension: assess whether the data used in the test is close to the production environment data. The more authentic the data, the higher the confidence level of the result. (4) Historical consistency dimension: compare the current test result (such as execution time) with the historical average result. If the difference is too large, the confidence level is reduced.

[0043] For example, based on the specific pass / fail status of the first test, the first test result can be refined into three result dimensions: a first result dimension corresponding to test coverage, a second result dimension corresponding to execution time, and a third result dimension corresponding to pass rate. Furthermore, different confidence weights can be assigned to different result dimensions. Accordingly, based on the actual situation of the above-mentioned subdivided result dimensions, the strength of observed evidence can be adjusted more precisely, thereby optimizing the calculation accuracy of the posterior probability. Specifically, the first result dimension (test coverage) is used to characterize the degree of code coverage by the first test, such as statement coverage, branch coverage, etc. The weight of this dimension can be set according to project quality requirements; the higher the coverage, the higher the confidence weight is usually. The second result dimension (execution time) is used to characterize the complete execution time of the first test, used to evaluate test efficiency. If the execution time is within the expected range, it may be given a higher weight, while abnormal timeouts may have a lower weight. The third result dimension (pass rate) is used to characterize the pass rate of the first test cases, such as all passes, partial passes, etc. The higher the pass rate, the higher the confidence weight is usually. This subdivision method allows the system to dynamically adjust the reliability of observed evidence based on test quality (such as insufficient coverage or execution anomalies). For example, when test coverage is low, even if the test passes, the overall confidence level may be reduced, leading to a more conservative assessment of the posterior probability. Therefore, a multi-dimensional assessment approach can reduce misjudgments and improve the accuracy of risk prediction.

[0044] In another alternative implementation, multi-source evidence can be combined for comprehensive judgment. For example, the first test result can be combined with other quality indicators (such as code review results and static analysis results) to serve as comprehensive observational evidence. Accordingly, the posterior probability can be accurately determined by combining comprehensive observational evidence. This approach avoids the limitations of a single method, thereby improving the reliability of probability estimation.

[0045] The posterior probability can be achieved through various methods such as Bayesian formula, incremental Bayesian update, and Bayesian network modeling. This application does not limit the specific details.

[0046] Step S140: If the posterior probability is greater than the preset trigger threshold, trigger the second test corresponding to the code change information; the test size of the second test is greater than the test size of the first test.

[0047] The preset trigger threshold can be a pre-defined probability threshold used to determine whether in-depth testing is necessary. This threshold can be dynamically calculated or statically set based on factors such as testing cost and risk tolerance. The second test refers to a more comprehensive and time-consuming testing phase in the continuous integration process than the first test, typically including a wider range of test cases, more complex test scenarios, and a larger test scale. Therefore, test scale refers to metrics such as the scale of input data used in the testing process, the number of test cases, and the test coverage. A larger test scale usually means more thorough testing, but it also consumes more execution time and resources.

[0048] The preset trigger threshold can be a fixed probability threshold (e.g., 0.1) set based on historical project data or experience. A second test is triggered when the posterior probability exceeds this threshold. Alternatively, the preset trigger threshold can be a dynamic threshold set based on multiple factors such as the current system resource utilization, the current time (e.g., working or non-working hours), and the project stage (e.g., development phase or pre-release phase), allowing for flexible adaptation to different scenarios. For example, the threshold can be increased (reducing testing) when resources are scarce and decreased (increasing testing) when resources are abundant, thus achieving resource self-adaptation. Alternatively, the preset trigger threshold can also be a threshold set in conjunction with a cost model. For example, based on a decision model, considering factors such as the cost of the second test, the loss from defect leakage, and the detection rate of the second test, an optimal threshold can be calculated to maximize cost-effectiveness.

[0049] Additionally, when the posterior probability exceeds a threshold, a second test can be initiated immediately to meet the needs of development scenarios requiring rapid feedback, thereby quickly identifying deep-seated defects. Alternatively, when the posterior probability exceeds a threshold, a second test can also be triggered via a queue. For example, the test task corresponding to the second test can be added to a task queue and executed in priority order. The priority of the test task corresponding to the second test can be configured based on factors such as the magnitude of the posterior probability and the importance of the change, thus ensuring that high-priority tasks are executed first through a priority queue scheduling method.

[0050] The second test, which is larger in scale than the first test, can include at least one of the following: (1) Number of test cases: The number of test cases in the second test can be several times to tens of times that of the first test, in order to cover more functional points and abnormal scenarios; (2) Scale of test input data: The second test uses a data volume closer to the production environment. For example, the first test can use hundreds of records, while the second test can use tens of thousands to millions of records; (3) Test coverage: The second test includes more types of tests, such as performance tests, security tests, compatibility tests, etc., while the first test only includes basic functional tests; (4) Test execution time: The execution time of the second test is usually a multiple of that of the first test (such as more than 5 times), or even several hours or days.

[0051] Therefore, in this application, if the first test result is passed, the change characteristics of the code change information are further obtained, and the prior probability of anomalies in the code change information is predicted based on the change characteristics. Furthermore, the passing result of the first test is used as observational evidence. Based on the observational evidence and the prior probability, the posterior probability of anomalies in the code change information is determined. Thus, if the posterior probability is greater than a preset trigger threshold, a second test with a larger test scale is triggered. This method uses the passing result of the first test as observational evidence, and determines the posterior probability based on the observational evidence and the prior probability, thereby triggering the second test based on the posterior probability. Therefore, this method establishes a correlation between the first and second tests, enabling the second test to be triggered in conjunction with the test results of the first test, thus making the triggering timing of the second test more reasonable and improving the efficiency and accuracy of code detection.

[0052] Furthermore, those skilled in the art can make various modifications and variations to the embodiments disclosed herein:

[0053] In one alternative implementation, to improve the accuracy of prior probability prediction, code change information can be transformed into a structured multidimensional feature vector. This allows for probability prediction using a pre-trained machine learning model, facilitating accurate identification of high-risk changes. Specifically, the change features may include a multidimensional feature vector. Correspondingly, the prior probability of anomalies in the code change information can be predicted as follows: a multidimensional feature vector of the code change information is obtained based on multiple preset feature dimensions; the prior probability of anomalies in the code change information is predicted based on a pre-trained prior probability prediction model and the multidimensional feature vector of the code change information.

[0054] In this context, feature dimensions refer to abstract categories used to describe the attributes of code changes. Each dimension represents a specific type of change characteristic (such as code size, complexity, developer experience, etc.). Feature dimensions are the foundation for feature vectors, ensuring the systematicity and comparability of feature extraction through predefined dimensions. A multidimensional feature vector is a mathematical vector composed of specific values ​​from multiple feature dimensions, used to comprehensively characterize the attributes of code changes. For example, a three-dimensional feature vector can be used to characterize information such as the number of lines of code added, the number of modified files, and the developer defect rate, where each dimension corresponds to a feature type. A prior probability prediction model refers to a machine learning model trained based on historical code change data and corresponding defect records, used to map the input multidimensional feature vector to a defect probability value (i.e., the posterior probability that code change information is abnormal). Therefore, prior probability refers to the probability of defect introduction predicted solely based on the inherent characteristics of the code change before obtaining observational evidence; this probability value is between 0 and 1. Feature dimensions can include the following basic dimensions: code change size, change complexity, and developer historical performance. For example, the code change scale dimension includes sub-features such as the number of lines of code added / deleted and the number of files modified; the change complexity dimension includes sub-features such as changes in loop complexity and nesting depth. Optionally, feature dimensions can also include the following extended dimensions: architecture impact dimension, test coverage dimension, etc. For example, the architecture impact dimension includes sub-features such as changes in module coupling and the scope of interface modifications; the test coverage dimension includes sub-features such as the proportion of affected test cases and the amount of test code modified.

[0055] The prior probability prediction model can be trained as follows: First, obtain historical code change information and corresponding anomaly labels. Second, construct a multi-dimensional feature vector based on the historical code change information. For example, extract multiple feature values ​​corresponding to multiple feature dimensions from the historical code change information, and construct a multi-dimensional feature vector based on these feature values. Third, train the prior probability prediction model using the multi-dimensional feature vector and anomaly labels. The multiple feature dimensions can include at least one of the following: code change scale, code change complexity, code change type, user history information, and dependency impact information. Code change scale measures the scope of a code modification, and may include, for example, the number of lines of code modified, the number of affected files, and the time spent. Code change complexity assesses the structural and logical complexity of the code modification, and may include, for example, loop complexity, nesting depth, the number of dependent modules, and the level of abstraction of the design patterns or algorithms involved. Code change type categorizes the purpose and nature of code modifications, and may include: bug fixing, new feature development, code refactoring (optimizing structure while maintaining functionality), performance optimization, and documentation updates. User history information can be a collection of historical behavioral data related to a specific developer. For example, it can include: historical code commit frequency and habits, frequently involved modules, density of introduced defects, coding style preferences, and the distribution of types of issues fixed. Dependency impact information describes the scope and extent to which other system components or modules may be involved in a code modification. For example, it can include: direct dependencies (modules explicitly called), transitive dependencies (dependencies of dependencies), etc. Historical code change information refers to past code change information recorded in the version control system, including commit metadata (committer, time, description, etc.) and change content (modified files, specific code differences, etc.). Typically, historical code change information can be determined based on historical code commit data. By comparing the changes in the code after the commit based on historical commit data, the code change information can be determined. Anomaly labels are binary or multi-valued labels used to identify whether a historical code commit introduced a defect. Eigenvalues ​​are specific numerical values ​​that quantify code commit attributes on a specific feature dimension. Through prior probability prediction models, the anomalies of the current code change can be accurately predicted.

[0056] Furthermore, in traditional continuous integration systems, test results typically affect subsequent processes in a binary (pass or fail) manner. However, this approach overlooks the possibility of missed detections in the first test. Therefore, to further improve the accuracy of the posterior probability, one alternative implementation can use the test anomaly detection rate as a key parameter and apply Bayes' theorem to combine observed evidence with prior probabilities, thereby achieving dynamic updates to the posterior probability. This approach can more accurately assess the true risk level after a test passes, thus reducing the probability of defect leakage.

[0057] Accordingly, the posterior probability of anomalies in the code change information is determined as follows: The anomaly detection rate of the first test is obtained; whereby the anomaly detection rate of the first test characterizes the probability that the anomaly information in the program code corresponding to the code change information can be detected by the first test; based on the anomaly detection rate of the first test and the prior probability, the posterior probability of anomalies in the code change information is determined. The anomaly detection rate refers to the probability that the first test can correctly identify and report defects given that the code does indeed have defects. Specifically, it can be characterized by conditional probability to reflect the first test's ability to detect defects. The program code corresponding to the code change information includes: program source code entities associated with the code change information. For example, a higher detection rate indicates a more reliable first test. Observational evidence specifically refers to the observable event that the first test passes. The posterior probability refers to the probability of a defect in the code obtained after re-evaluating the observable evidence (test passed). The anomaly detection rate can be determined based on historical statistics: obtaining all confirmed defects within a certain period and statistically analyzing the proportion successfully detected by the first test.

[0058] Optionally, when determining the posterior probability of an anomaly in the code change information based on the anomaly detection rate and prior probability of the first test, it can be achieved in the following way: based on Bayes' theorem, the prior probability and anomaly detection rate are pre-calculated, and the posterior probability of anomaly in the code change information is determined based on the calculation results; wherein, the posterior probability is positively correlated with the prior probability and negatively correlated with the anomaly detection rate.

[0059] For example, according to Bayes' theorem, the posterior probability of an anomaly in code change information can be determined using the following formula:

[0060] q=

[0061] Here, q is used to characterize the posterior probability that there is an anomaly in the code change information, that is, the probability that the code has a defect under the condition that the first test passes. Used to characterize the anomaly detection rate of the first test, and to characterize the ability of the first test to detect defects. Used to characterize the prior probability that code change information is abnormal.

[0062] In the above formula, the anomaly detection rate of the first test is objectively calculated by dynamically statistically analyzing the actual abnormal data within a preset time period, thereby accurately reflecting the current effectiveness of the first test and providing reliable parameters for Bayesian probability updates.

[0063] In specific implementation, the anomaly detection rate of the first test can be obtained in the following ways: First, obtain the total number of anomalies appearing in the program code corresponding to the code change information within a preset time period; second, obtain the first number of anomalies detected by the first test within the preset time period; determine the anomaly detection rate of the first test based on the ratio between the first number and the total number; where the preset time period is a period of time dynamically determined based on the current time. Accordingly, the anomaly detection rate of the first test is dynamically updated as the preset time period changes. Through anomaly statistics and ratio calculation within the dynamic time window, the anomaly detection rate can be dynamically updated according to the actual situation of the online code. This method ensures the reliability and timeliness of the probability update parameters through the extraction and calculation of key features (dynamic time period, total number of anomalies, number of detections). Among them, anomaly information mainly refers to defects, errors, or non-standard aspects existing in the code, which are usually recorded and classified through an issue tracking system. For example, anomaly information includes various types such as functional defects, performance problems, and security vulnerabilities. The preset time period is a dynamic time window used for statistical anomaly data, and its time range and length can be automatically adjusted according to the current time. A dynamically determined time period refers to a time window that is automatically calculated based on the current time point, and its start point and duration can be automatically adjusted as the system runs.

[0064] The preset time period can be dynamically determined based on a sliding time window: starting from the current time, a fixed duration is slid forward as the statistical period. This method ensures a fixed window size and stable data volume, guaranteeing the stability of the statistical results and avoiding excessive data fluctuations. Alternatively, the preset time period can also be dynamically determined based on an adaptive time window, allowing for dynamic adjustment of the window size according to code commit frequency. For example, the window is shortened when code commits are frequent and extended when code commits are infrequent, ensuring a moderate statistical sample size. This method adapts to the project development pace, making it particularly suitable for projects with rapidly changing development rhythms, preventing outdated or insufficient data. Alternatively, the window length of the preset time period can be dynamically adjusted based on the project stage. For example, a shorter window is used in the early stages of development for quick adaptation; a longer window is used during the stable development period to reduce fluctuations; and the latest window is used before release to ensure timeliness.

[0065] The preset trigger threshold mentioned above refers to the critical posterior probability value required to initiate the second test. When the posterior probability exceeds this threshold, it is determined that the second test needs to be initiated. To improve the accuracy of the preset trigger threshold, it can be comprehensively determined based on the test cost information of the second test, the anomaly detection rate of the second test, and the loss information of anomalies in the code change information. The anomaly detection rate of the second test characterizes the probability that anomalies in the program code corresponding to the code change information can be detected by the second test. The test cost information of the second test can be determined based on the resource consumption of the second test per unit time and the test duration. Specifically, the test cost information of the second test refers to the cost of all resources consumed in executing the second test, including computing resources, storage resources, time costs, and manpower costs, which can be determined by multiplying the resource consumption of the second test per unit time by the test duration. The anomaly detection rate of the second test refers to the probability that the second test can find code defects, reflecting the effectiveness of the test suite, and is obtained by statistically analyzing historical test results. The loss information regarding abnormal code change information refers to the estimated economic losses caused if a defect were to leak into the production environment, including direct costs (repair costs) and indirect costs (reputational damage, user impact, etc.). Therefore, the threshold determination method described above is used to ensure that the expected benefits of testing outweigh the testing costs.

[0066] Furthermore, in the above method, the anomaly detection rate of the first test, the anomaly detection rate of the second test, test cost information, and loss information can be recalculated periodically. This allows for dynamic adjustment of the preset trigger threshold based on the recalculation results, achieving adaptive updating of the threshold. For example, the test cost information of the second test, the anomaly detection rate of the second test, and / or the loss information for anomalies in code change information can be dynamically adjusted based on the time period to which the current time belongs. Accordingly, if at least one of the test cost information of the second test, the anomaly detection rate of the second test, or the loss information for anomalies in code change information is detected to be adjusted, the preset trigger threshold is recalculated and updated based on the adjusted test cost information of the second test, the adjusted anomaly detection rate of the second test, and the adjusted loss information for anomalies in code change information, thereby achieving adaptive adjustment of the preset trigger threshold. The time period to which the current time belongs can be determined through various methods such as the sliding time window and the adaptive time window mentioned above. Alternatively, the time period to which the current time belongs can also be dynamically determined by the project stage to flexibly adapt to the characteristics of different stages.

[0067] Optionally, the first test mentioned above may include at least one of the following: compilation testing, unit testing, static code analysis, smoke testing, and code change impact analysis. Compilation testing verifies whether source code can be successfully converted into executable code, including basic compilation steps such as syntax checking, dependency resolution, and link verification. Unit testing is isolated testing of the smallest testable unit of software (function, method, class) to verify the correctness of unit-level logic. Static code analysis detects code quality, security vulnerabilities, and compliance issues through source code analysis without executing the code. Smoke testing verifies the core functional links of the system, ensuring basic functions are normal and preventing serious blocking problems. Code change impact analysis identifies the scope and modules that may be affected by code changes through dependency analysis, call chain tracing, and other methods.

[0068] The second type of testing may include at least one of the following: full-feature testing, end-to-end testing, performance testing, security testing, and compatibility testing. Full-feature testing involves a complete verification of all implemented functionalities of the software, covering test scenarios such as normal processes, abnormal processes, and boundary conditions. End-to-end testing simulates real user operation scenarios to verify the complete business process of the system from the user interface to the backend service. Performance testing evaluates the system's performance under various load conditions, including metrics such as response time, throughput, and resource utilization. Security testing verifies the system's security attributes, including vulnerability scanning, penetration testing, and security compliance checks. Compatibility testing verifies the system's ability to operate normally in different environments (browsers, operating systems, devices).

[0069] To facilitate understanding, the following example illustrates the specific implementation details of the continuous integration method provided in this application.

[0070] This example provides a method and system for continuous integration of dedicated computing chip software based on the expected value decision method. Specifically, it relates to the field of computer technology, and more specifically, to technologies such as chips, graphics processing units (GPUs), graphics cards, continuous integration, continuous deployment, smoke testing, and daily build and test. This example relates to a method, system, and computer-readable storage medium for continuous integration of chip software.

[0071] In the current era of large-scale AI models, the development of artificial intelligence is advancing at an astonishing pace. Various large-scale pre-trained models, such as chatGPT and DeepSeek, have made significant progress in various fields. These models typically require substantial computing resources and data for training and inference, driving the demand for high-performance dedicated computing chips, especially graphics processing units (GPUs). Against this backdrop, the development of domestically produced GPUs is particularly important and urgent. With the rapid development of domestic GPU technology in recent years, enterprises often face challenges in their daily domestic GPU software development work, such as insufficient resources, low efficiency, inadequate testing, or test redundancy in Continuous Integration (CI). This example aims to address these issues. Given the limited resources of enterprise test cluster machines and GPUs, coupled with the very frequent code commits within GPU companies, the CI pipeline needs to run numerous CI tasks, involving multiple code repositories, multiple code repository branches, various test GPU models, and various test types (GPU driver, AI accelerated computing operator library, neural network model training, neural network model inference functional testing, performance testing, stress testing, anomaly testing, etc.). Each CI task consumes machine and GPU resources.

[0072] In the daily development of domestically produced GPU software, enterprise R&D often encounters problems such as insufficient resources, low efficiency, inadequate testing, or redundant testing. The reasons include at least the following:

[0073] (1) If all code submissions are only smoked in the CI pipeline, the testing is insufficient and there is a high probability that defects (bugs) cannot be effectively intercepted, which poses a great quality risk.

[0074] (2) If all code submissions are fully tested in the CI pipeline, there will be insufficient hardware resources, which may lead to queuing and low efficiency. For some very simple code submissions, the risk is extremely low, and full testing may result in test redundancy.

[0075] (3) The quality of software code repositories is not fixed or predictable at different time periods. For example, the quality of code submissions does not increase or decrease over time (the quality is poor at the beginning of the year and better at the end of the year). Therefore, it is impossible to simply judge the quality changes, and thus it is impossible to adopt a one-size-fits-all approach to smoke testing or full testing.

[0076] This example leverages smoke testing and full testing to optimize CI strategies. The core objective is to strike a balance between testing costs and the losses from missed bugs. By using expected value decision-making based on the risk level of the decision type, it dynamically triggers full tests, fully utilizing probabilistic information to achieve intelligent allocation of testing resources. This ensures development efficiency while mitigating risk, thereby minimizing expected losses. Furthermore, this example uses machine learning models to dynamically predict the risk of code commits, making the entire CI decision-making system more intelligent, accurate, and efficient. It truly prioritizes testing resources for the highest-risk changes, avoiding both insufficient and redundant testing.

[0077] The following section will explain the basic concepts and implementation basis of this example:

[0078] Smoke testing: Smoke tests typically last 0.5 to 2 hours and are used for rapid basic testing to verify that basic functions are working correctly, including compilation, unit testing, and basic functionality verification. Smoke tests are low-cost and provide quick feedback, but have low test coverage and must be run with every code commit.

[0079] Full-scale testing: Full-scale testing typically lasts 2-12 hours and is used for comprehensive testing, covering a wide range of functions, including integration testing, end-to-end testing, performance testing, multi-GPU testing, and anomaly testing. It has high test coverage but is also costly and is run on a daily schedule (e.g., at midnight).

[0080] Decision-making mechanism: Based on the expected value criterion, after the smoke test passes, the posterior probability (the posterior probability that the code has a bug) is calculated, and it is decided whether to immediately trigger a full test. This avoids unnecessary full test execution, saves resources, and enables timely execution of full tests in high-risk situations, reducing code quality risks and minimizing the loss from missed bugs.

[0081] In addition, regarding tool integration, this example can use the following tools:

[0082] Version control systems, such as GitLab;

[0083] Bug tracking systems, such as JIRA;

[0084] CI / CD tools, such as Jenkins;

[0085] Automation scripts, such as those using Python or Groovy code;

[0086] The test cluster can be deployed based on Kubernetes and Docker.

[0087] In addition, this example does not specify a particular development code repository and test code repository, and can be extended to one or more repositories.

[0088] Figure 2 The flowchart of the continuous integration method in this example is shown below. Figure 2 As shown, the method includes the following steps:

[0089] Step S201: If a code commit operation is detected, a compilation operation is triggered based on the code change information corresponding to the code commit operation.

[0090] After code is committed, a CI (Compilation and Integration) build is triggered. Specifically, after code is committed to the version control system (GitLab), a CI build is triggered using a CI tool (such as Jenkins). This involves creating a build pod (a temporary computing unit that executes the CI task) and pulling the latest committed code from the development code repository to that build pod. Furthermore, project dependencies need to be installed, the development code needs to be compiled into a build artifact (the object to be tested), and the build artifact (the binary file or software package generated from the compiled source code that can be directly tested or deployed) needs to be uploaded to a specified storage path.

[0091] Step S202: Based on the compilation results, trigger the smoke test corresponding to the code change information.

[0092] In this example, smoke testing (also referred to as Smoke CI below) is a specific implementation of the first test. Specifically, the CI pipeline first executes smoke testing: a test pod (i.e., a temporary test container) is created, the compilation artifacts from step S201 are downloaded and installed, and relevant code information is pulled from the test code repository to the test pod. Furthermore, project dependencies also need to be installed to enable the smoke test to run via the test code.

[0093] If the smoke test fails, the developers can be notified immediately to fix it and resubmit the code to re-trigger the CI and smoke test. If the smoke test passes, the process continues.

[0094] Step S203: If the smoke test passes, predict the prior probability that the code change information is abnormal based on the change characteristics of the code change information.

[0095] The prior probability p can be calculated in the following way:

[0096] First, in the CI pipeline, feature extraction is performed using automated scripts: parsing the git diff (comparison of code commit differences), commit information, author information, etc., of this code commit, and calculating the feature values ​​of the code change information corresponding to this code commit in real time. , , ..., Examples of features include `lines_added` and `author_bug_rate`, which are used to construct feature vectors using feature values. Here, the feature values ​​are specific numerical values ​​used to characterize the change features of code change information.

[0097] Then, the pre-trained prior probability prediction model is invoked to predict the prior probability: the feature vector [ , ,..., The input is fed into a pre-trained prior probability prediction model. Specifically, the prior probability prediction model can be a logistic regression model. This model returns a predicted probability value p, which is the prior probability used in this Bayesian calculation.

[0098] Step S204: Take the smoke test as observational evidence, and determine the posterior probability that the code change information is abnormal based on the observational evidence and the prior probability.

[0099] Specifically, in a CI pipeline, the posterior probability q can be obtained by an automated script in the following way: based on the premise that the smoke test has passed, Bayes' theorem is used to calculate the posterior probability that there is a bug in this code submission.

[0100] Step S205: Determine the preset trigger threshold.

[0101] The preset trigger threshold T can be dynamically determined to serve as the threshold T for running the full test (hereinafter also referred to as DailyCI). The full test is a specific implementation of the second test. Specifically, in the CI pipeline, the threshold T for running the full test is automatically calculated by an automated script.

[0102] Step S206: If the posterior probability is greater than the preset trigger threshold, trigger the second test corresponding to the code change information.

[0103] If q > T, a full test can be triggered immediately without waiting for the scheduled time to arrive.

[0104] If q <= T, the full test will not be triggered immediately. The full test can be started after the scheduled time has elapsed. For example, the scheduled time could be once a day, such as at midnight, to ensure quality.

[0105] If q > T, the CI pipeline can immediately execute full tests: create a test pod, download and install the compilation artifacts (i.e., the objects to be tested) from step S201, and pull the relevant code information from the test code repository to the test pod. Additionally, project dependencies need to be installed to run full tests using the test code.

[0106] Step S207: Dynamically monitor the code execution status and dynamically adjust relevant parameters based on the monitoring results.

[0107] Specifically, parameters can be continuously optimized through dynamic monitoring to make the subsequent triggering process more accurate.

[0108] This involves recording decision-making results and actual bug reports to ensure all parameters are updated regularly, thereby guaranteeing the timeliness and accuracy of decisions. For example, continuously optimizing the prior probability prediction model, such as retraining the model weekly based on the latest historical data. Another example is... , , , Parameters such as Loss may dynamically change with variations in code quality, test suites, and costs, requiring continuous adjustment and optimization, such as recalculating these parameters weekly based on the latest historical data. In this example, independent of CI triggered by code commits, a scheduled full test can be triggered, such as running a full test daily, for example at midnight, to ensure that even if smoke tests pass and the probability of subsequent testing is low, potential bugs are still caught, providing an additional safety net. If the full test finds a bug, the developers are notified to fix it.

[0109] The definitions and calculation methods of the above parameters are as follows:

[0110] Given a bug in the code, the probability that a smoke test can detect it. For example, the percentage of bugs caught by smoke tests can be calculated from historical data. Therefore, This refers to the anomaly detection rate of the first test mentioned above.

[0111] =

[0112] In the above formula, the numerator represents the number of defects (bugs) detected by smoke testing, and the denominator represents the total number of bugs introduced after code submission. In bug tracking systems (such as JIRA), it's necessary to record how bugs were discovered, such as through smoke testing, full testing, or the production environment. For example, if there are a total of 100 bugs, and 20 of them are caught by smoke testing, then... =0.2.

[0113] Given a bug in the code, what is the probability that a full test will detect it? The percentage of bugs caught by a full test is calculated from historical data. Therefore, it can be seen that... This refers to the anomaly detection rate of the second test mentioned above.

[0114] =

[0115] In the formula above, the numerator represents the number of bugs detected by the full test, and the denominator represents the total number of bugs introduced after code submission. For example, if there are a total of 100 bugs, and 60 of them are caught by the full test, then... =0.6.

[0116] The resource cost of running a smoke test, including time, resources (GPU, CPU, memory, storage, etc.), and manpower, is usually relatively low. Specifically, measure the average runtime of the smoke test (e.g., in hours) and multiply it by the hourly resource cost:

[0117] =Time taken in hours of Smoke CI×Hourly cost of Smoke CI resources

[0118] Time taken in hours of Smoke CI: refers to the number of hours spent executing the smoke testing continuous integration process.

[0119] Hourly cost of Smoke CI resources: This refers to the hourly cost of the resources consumed in running these tests.

[0120] CI resource costs can be determined based on the costs of the cloud service provider or internal infrastructure. For example, if a smoke test runs for 1.5 hours and the smoke test resource cost is 10 yuan per hour, then... =15 yuan.

[0121] The resource cost of running a full test, including time, resources (GPU, CPU, memory, storage), and manpower, is typically high, exceeding [amount missing]. .

[0122] = Time taken in hours of Daily CI × Hourly cost of Daily CIresources

[0123] Time taken in hours of Daily CI: refers to the total number of hours actually spent executing the daily continuous integration process.

[0124] Hourly cost of Daily CI resources: refers to the hourly cost of resources consumed in running daily continuous integration tasks.

[0125] For example, if a full test runs for 10 hours, and the resource cost for a full test is 30 yuan per hour, then... =300 yuan.

[0126] It's important to note that smoke testing and full testing have different time and resource consumption:

[0127] Smoke testing requires fewer resources, such as 1 GPU, 8 CPUs, 20 Gi of memory resources (e.g., RAM), and 100 Gi of temporary storage resources (e.g., ephemeral storage). Full testing requires more resources, such as 8 GPUs, 32 CPUs, 80 Gi of memory resources (e.g., RAM), and 200 Gi of temporary storage resources (e.g., ephemeral storage).

[0128] Loss: The average loss caused by a bug leaking into the production environment, including fixing costs, user impact, and business losses, which can be estimated based on the importance of the project.

[0129] Loss=

[0130] In the formula above, the numerator represents the total loss caused by all bugs in the project, and the denominator represents the number of bugs in the project. For example, if a total of 10 bugs leak into the production environment, causing a total loss of 100,000 for a project, then Loss = 10,000.

[0131] p: The prior probability of a bug existing after code submission can be dynamically predicted using a machine learning model based on code submission features. Code submission features refer to the change characteristics of code change information. These features can include multiple dimensions such as: code change scale, code change complexity, code change type, risk level of the affected modules (low, medium, high), developer history information, and the number of code reviewers.

[0132] The process involves training a machine learning model (such as logistic regression) to obtain a prior probability prediction model. This model takes the code submission features mentioned above as input and outputs a prior probability p. For example, each code submission can be considered a data point, and a series of features can be extracted from the metadata and code content corresponding to this submission as code submission features. Then, the trained prior probability prediction model outputs a probability value between 0 and 1 based on these code submission features. This probability value is the dynamic prior probability p of this code submission, representing the prior probability that this code submission will introduce a defect.

[0133] The prior probability prediction model can be trained in the following ways:

[0134] First, prepare and label the dataset to build a labeled historical dataset for model training. When collecting data, historical data can be extracted from version control systems and issue tracking systems. The time period for extracting historical data can be flexibly configured, such as the past 6 months. Additionally, code commits that introduced bugs can be filtered: by finding the associated commit hash in the bug reports of the issue tracking system, labels can be created as follows: for each historical commit, a label is added. For example, bug_commit=1 indicates that the commit was later confirmed to have introduced a bug, while bug_commit=0 indicates that the commit did not introduce any known bugs.

[0135] Then, a series of indicators that may indicate risks are calculated for each code commit. By calculating the feature values ​​corresponding to this code commit in real time, the corresponding feature vector can be obtained. Figure 3 A diagram illustrating the characteristic metrics of code change features is provided. For example... Figure 3 As shown, feature indicators can be characterized from three aspects: feature category, feature name, and feature description.

[0136] Next, model training is performed. A logistic regression model can be used as the prior probability prediction model. The advantages of logistic regression are its simplicity, fast computation speed, and the fact that its output is naturally a probability value between 0 and 1. Furthermore, the model is highly interpretable, as the model weight coefficients can explain the degree of influence of each feature on the prediction result.

[0137] During training, the prepared dataset can be divided into a training set and a test set (e.g., 80% as the training set and 20% as the test set). Furthermore, to facilitate quantification, numerical features (such as the number of lines of code) can be standardized to have a mean of 0 and a variance of 1.

[0138] The standardized calculation formula is as follows: =

[0139] in, These are the original feature values ​​(e.g., lines_added: 10).

[0140] μ is the average value of this feature on the training dataset.

[0141] σ is the standard deviation of this feature on the training dataset.

[0142] It is the standardized value.

[0143] We fit a logistic regression model using the training set data. The trained logistic regression model is used to output the dynamic probability p, which is calculated using the following formula:

[0144] p=P(bug|x)=σ(z)=

[0145] Where σ(z) is the Sigmoid function (activation function), which is used to map the linear combination z to the interval (0, 1).

[0146] z: Linear combination, which is a linear weighted sum of all features:

[0147] z = + × + × + ... + ×

[0148] , , ..., The feature value corresponding to this code submission (such as lines_added, author_bug_rate, etc.).

[0149] , , ..., The weight coefficients learned by the model for each feature. Positive weights indicate that the feature increases the risk of bugs, while negative weights indicate that the feature may reduce the risk of bugs.

[0150] : Intercept (model bias).

[0151] After training the aforementioned prior probability prediction model, it can be integrated into the CI pipeline to dynamically predict the prior probability p.

[0152] For example, in a specific instance, to simplify the description, assume that the prior probability prediction model uses only the following two features:

[0153] =lines_added;

[0154] =author_bug_rate.

[0155] The coefficients obtained after model training are:

[0156] (Model bias);

[0157] =0.8 (weight of lines_added);

[0158] =1.5 (weight of author_bug_rate).

[0159] Scene 1:

[0160] In this scenario, assuming a senior developer submits a minor fix, the relevant data would be as follows:

[0161] lines_added=10 (0.2 after standardization);

[0162] author_bug_rate=0.02 (normalized to -1.0);

[0163] Substituting the parameter values ​​mentioned above into the formula yields the following:

[0164] p= ≈0.0215

[0165] Therefore, the model predicts that there is only a 2.15% probability that this code submission in this scenario will introduce a bug, and based on the prior probability, the risk is very low.

[0166] Scene 2:

[0167] In this scenario, assuming a new developer submits code for a large feature, the relevant data would be as follows:

[0168] lines_added = 500 (2.5 after standardization);

[0169] author_bug_rate = 0.25 (1.8 after standardization);

[0170] ;

[0171] p = ≈ 0.9003

[0172] Therefore, the model predicts that this code submission in this scenario has a 90.03% probability of introducing a bug, which is extremely risky.

[0173] Next, we calculate the posterior probability q.

[0174] Q: What is the posterior probability that a code submission still contains bugs after passing a smoke test?

[0175] Bayes' theorem general formula is P(A|B) =

[0176] Event A: The code has a bug;

[0177] Event B: Smoke test passed.

[0178] P(A) = the prior probability that the code has a bug = p

[0179] P(B|A) = The probability that the smoke test will pass even if the code has a bug = 1 - ;

[0180] P(B) = The total probability of passing the smoke test, which can include the following two cases:

[0181] (1) The code has a bug and the smoke test passes: the probability is p×(1- );

[0182] (2) The code has no bugs and the smoke test passes: the probability is (1-p)×1.

[0183] Therefore, the total probability P(B) = (p × (1 - ))+((1-p)×1)

[0184] Derivation using Bayes' theorem:

[0185] q=P(A|B)=P(Bug| Smoke CI pass) = = =

[0186]

[0187] Therefore, the final formula is as follows:

[0188] q = P(Bug | Smoke CI pass) =

[0189] Accordingly, in scenario 1 above, when a senior developer submits a small fix,

[0190] p ≈ 0.0215, = 0.2,

[0191] Then q = ≈ 0.0173

[0192] In scenario 2 above, when a new developer submits a large feature,

[0193] p≈0.9003

[0194] Then q= ≈0.8784

[0195] Assuming the preset trigger threshold is T, this threshold is determined based on the expected value criterion and is used to run full-scale tests. The specific threshold can be flexibly adjusted according to the project stage. For example, T can be lowered before a version release to increase the testing frequency.

[0196] T=

[0197] The resource cost consumed by running a full test;

[0198] Loss: The average loss caused by a bug leaking into the production environment, including fixing costs, user impact, and business losses, which can be estimated based on the importance of the project.

[0199] The probability that a bug in the given code can be detected by a full test.

[0200] For example, if =300, =0.6, Loss=10000

[0201] Then T= =0.05

[0202] In scenario 1, a senior developer submits a small fix, q≈0.0173, which is less than T, so a full test is not triggered.

[0203] In scenario 2, a new developer submits a large feature, q≈0.8784, which is greater than T, thus triggering a full test.

[0204] In summary, this example has at least the following characteristics:

[0205] (1) During the training of the prior probability prediction model, the model is trained based on the features of all code submissions over a certain period of time. The feature data used to train the model is scientific and effective, and can fully reflect the quality of the code submissions.

[0206] (2) By making full use of various data, a CI process based on the expected value decision method was designed scientifically and comprehensively. Among them, various data include: the probability that smoke test / full test can detect (i.e., anomaly detection rate); the resource cost consumed by running smoke test / full test, including time, resources, manpower, etc.; and the average loss caused by the bug leaking into the production environment.

[0207] (3) When calculating the prior probability p, the prior probability used for this Bayes theorem is calculated based on the various features of this code submission, which can more scientifically quantify the prior probability that the code submission has bugs.

[0208] (4) When calculating the posterior probability q, Bayes' theorem is used to calculate the posterior probability that there is a bug in this code submission, which can more scientifically quantify the posterior probability that there is a bug in the code submission.

[0209] (5) When calculating the threshold T for running full tests, the threshold for whether code submissions should run full tests can be dynamically quantified, and the threshold can be flexibly adjusted according to various data and actual needs.

[0210] Therefore, this example has at least the following beneficial effects:

[0211] (1) If all code submissions are only smoked in the CI pipeline, the testing is insufficient and there is a high probability that bugs cannot be effectively intercepted, which poses a great quality risk. This problem can be solved by the method in this example.

[0212] (2) If all code submissions are fully tested in the CI pipeline, hardware resources will be insufficient, queuing may occur, and efficiency will be low. For some very simple code submissions, the risk is extremely low. At this time, full testing may result in test redundancy. The method in this example can solve this problem.

[0213] (3) The quality of software code repositories is not fixed or predictable at different time periods. For example, the quality of code submissions does not increase or decrease over time (the quality is poor at the beginning of the year and better at the end of the year). It is impossible to simply judge the quality changes, so it is impossible to perform smoke testing or full testing in a one-size-fits-all manner. The method in this example can solve this problem.

[0214] Figure 4This is a schematic diagram of the structure of a continuous integration system provided in an embodiment of this disclosure. (Refer to...) Figure 4 The system includes:

[0215] Detection module 41 is adapted to trigger a first test corresponding to the code change information when code change information is detected;

[0216] The prediction module 42 is adapted to obtain the change features of the code change information if the test result of the first test is passed, and predict the prior probability that the code change information is abnormal based on the change features;

[0217] The determination module 43 is adapted to take the test result of the first test as passing as observation evidence, and determine the posterior probability that the code change information is abnormal based on the observation evidence and the prior probability.

[0218] Trigger module 44 is adapted to trigger a second test corresponding to the code change information when the posterior probability is greater than a preset trigger threshold; the test scale of the second test is greater than the test scale of the first test.

[0219] In one alternative implementation, the prediction module is specifically adapted to:

[0220] Based on multiple preset feature dimensions, obtain a multidimensional feature vector of the code change information;

[0221] Based on a pre-trained prior probability prediction model and the multi-dimensional feature vector of the code change information, the prior probability that the code change information is abnormal is predicted.

[0222] In one alternative implementation, the prior probability prediction model is trained in the following manner:

[0223] Obtain historical code change information and the corresponding exception tags for the historical code change information;

[0224] Extract multiple feature values ​​corresponding to the multiple feature dimensions from the historical code change information, and construct a multi-dimensional feature vector corresponding to the historical code change information based on the multiple feature values;

[0225] The prior probability prediction model is trained based on the multidimensional feature vector corresponding to the historical code change information and the anomaly label.

[0226] Furthermore, the multiple feature dimensions include at least one of the following: code change scale, code change complexity, code change type, user history information, and dependency impact information.

[0227] In one alternative implementation, the determining module is specifically adapted to:

[0228] Obtain the anomaly detection rate of the first test; wherein, the anomaly detection rate of the first test is used to characterize the probability that anomaly information in the code can be detected by the first test;

[0229] Based on the anomaly detection rate of the first test and the prior probability, the posterior probability that the code change information is abnormal is determined.

[0230] In one alternative implementation, the determining module is specifically adapted to:

[0231] According to Bayes' theorem, the posterior probability of the code change information being abnormal is determined by the following formula:

[0232] q=

[0233] Where q represents the posterior probability that the code change information is abnormal. Used to characterize the anomaly detection rate of the first test, This is used to characterize the prior probability that the code change information is abnormal.

[0234] In one optional implementation, obtaining the anomaly detection rate of the first test includes:

[0235] Get the total number of exception messages that appear in the code within a preset time period;

[0236] Obtain the first number of abnormal information detected by the first test within the preset time period;

[0237] The anomaly detection rate of the first test is determined based on the ratio between the first quantity and the total quantity.

[0238] The preset time period is a period of time dynamically determined based on the current time; and the anomaly detection rate of the first test is dynamically updated as the preset time period changes.

[0239] In one optional implementation, the preset trigger threshold is determined based on the test cost information of the second test, the anomaly detection rate of the second test, and the loss information of the anomaly in the code change information;

[0240] The anomaly detection rate of the second test is used to characterize the probability that anomalies in the code can be detected by the second test; the test cost information of the second test is determined based on the resource consumption of the second test per unit time and the test duration of the second test.

[0241] In one optional implementation, the test cost information of the second test, the anomaly detection rate of the second test, and / or the loss information of anomalies in the code change information are dynamically adjusted according to the time period to which the current time belongs;

[0242] Furthermore, if adjustments are made to the test cost information of the second test, the anomaly detection rate of the second test, and / or the loss information of the code change information that are abnormal, the preset trigger threshold is recalculated based on the adjusted test cost information of the second test, the anomaly detection rate of the second test, and / or the loss information of the code change information that are abnormal.

[0243] In one alternative implementation, the first test includes at least one of the following: compilation test, unit test, static code analysis, smoke test, and code change impact analysis;

[0244] The second test includes at least one of the following: full-featured testing, end-to-end testing, performance testing, security testing, and compatibility testing.

[0245] The specific implementation methods and related details of each of the above modules can be found in the descriptions of the corresponding parts in the method embodiments, and will not be repeated here.

[0246] Figure 5 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0247] Reference Figure 5 This disclosure provides an electronic device, which includes: at least one processor 701; at least one memory 702; and one or more I / O interfaces 703 connected between the processor 701 and the memory 702; wherein the memory 702 stores one or more computer programs that can be executed by the at least one processor 701, and the one or more computer programs are executed by the at least one processor 701 to enable the at least one processor 701 to perform the above-described method.

[0248] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the above-described method. The computer-readable storage medium may be volatile or non-volatile.

[0249] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0250] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0251] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0252] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0253] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0254] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0255] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0256] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0257] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0258] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0259] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A continuous integration method, characterized in that, The method includes: Upon detecting code change information, a first test corresponding to the code change information is triggered; If the test result of the first test is passed, the change characteristics of the code change information are obtained, and the prior probability that the code change information is abnormal is predicted based on the change characteristics; The test result of the first test being passed is used as observational evidence. Based on the observational evidence and the prior probability, the posterior probability that the code change information is abnormal is determined. If the posterior probability is greater than a preset trigger threshold, a second test corresponding to the code change information is triggered; the test scale of the second test is greater than that of the first test.

2. The method according to claim 1, characterized in that, The change features include multi-dimensional feature vectors; The change features for obtaining the code change information include: Based on multiple preset feature dimensions, obtain a multidimensional feature vector of the code change information; The prior probability of predicting the anomaly of the code change information based on the change characteristics includes: Based on a pre-trained prior probability prediction model and the multi-dimensional feature vector of the code change information, the prior probability that the code change information is abnormal is predicted.

3. The method according to claim 2, characterized in that, The prior probability prediction model is trained in the following way: Obtain historical code change information and the corresponding exception tags for the historical code change information; Based on the historical code change information, construct a multi-dimensional feature vector corresponding to the historical code change information; The prior probability prediction model is trained based on the multidimensional feature vector corresponding to the historical code change information and the anomaly label. Furthermore, the multiple feature dimensions include at least one of the following: code change scale, code change complexity, code change type, user history information, and dependency impact information.

4. The method according to any one of claims 1-3, characterized in that, The step of determining the posterior probability that the code change information is abnormal based on the observed evidence and the prior probability includes: Obtain the anomaly detection rate of the first test; wherein, the anomaly detection rate of the first test is used to characterize the probability that the anomaly information in the program code corresponding to the code change information can be detected by the first test; Based on the anomaly detection rate of the first test and the prior probability, the posterior probability that the code change information is abnormal is determined.

5. The method according to claim 4, characterized in that, The step of determining the posterior probability that the code change information is abnormal based on the anomaly detection rate of the first test and the prior probability includes: Based on Bayes' theorem, the prior probability and the anomaly detection rate are pre-calculated, and the posterior probability that the code change information is abnormal is determined according to the calculation results; wherein, the posterior probability is positively correlated with the prior probability and negatively correlated with the anomaly detection rate.

6. The method according to claim 4, characterized in that, The process of obtaining the anomaly detection rate of the first test includes: Obtain the total number of exception messages that appear in the program code within a preset time period; Obtain the first number of abnormal information detected by the first test within the preset time period; The anomaly detection rate of the first test is determined based on the ratio between the first quantity and the total quantity. The preset time period is a period of time dynamically determined based on the current time; and the anomaly detection rate of the first test is dynamically updated as the preset time period changes.

7. The method according to any one of claims 1-3, characterized in that, The preset trigger threshold is determined based on at least one of the following: the test cost information of the second test, the anomaly detection rate of the second test, and the loss information of anomalies in the code change information; The anomaly detection rate of the second test is used to characterize the probability that the anomaly information in the program code corresponding to the code change information can be detected by the second test; the test cost information of the second test is determined based on the resource consumption of the second test per unit time and the test duration of the second test.

8. The method according to claim 7, characterized in that, The method further includes: If at least one of the test cost information of the second test, the anomaly detection rate of the second test, and the loss information is adjusted, the preset trigger threshold is recalculated based on the adjusted test cost information of the second test, the adjusted anomaly detection rate of the second test, and the loss information indicating anomalies in the adjusted code change information.

9. The method according to any one of claims 1-3, characterized in that, The first test includes at least one of the following: compilation test, unit test, static code analysis, smoke test, and code change impact analysis; The second test includes at least one of the following: full-featured testing, end-to-end testing, performance testing, security testing, and compatibility testing.

10. A continuous integration system, characterized in that, include: The detection module is adapted to trigger a first test corresponding to the code change information when code change information is detected; The prediction module is adapted to obtain the change features of the code change information if the test result of the first test is passed, and predict the prior probability that the code change information is abnormal based on the change features; The determination module is adapted to use the test result of the first test as observation evidence, and determine the posterior probability that the code change information is abnormal based on the observation evidence and the prior probability. The triggering module is adapted to trigger a second test corresponding to the code change information when the posterior probability is greater than a preset triggering threshold; the test scale of the second test is greater than the test scale of the first test.

11. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, implements the method as described in any one of claims 1-9.

13. A computer program product comprising computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, characterized in that, When the computer-readable code is run in an electronic device, the processor in the electronic device performs the method of any one of claims 1-9.

Images