Method and program product for determining test cases

Abstract

The application discloses a test case determination method and a program product. The method comprises the following steps: obtaining difference change content corresponding to the tested code, determining a semantic feature vector of the tested code according to the difference change content, selecting a plurality of second test cases according to the semantic similarity between the semantic feature vector and a query vector, and forming a semantic candidate set; obtaining a co-occurrence record, determining an evolution correlation strength according to the co-occurrence record, selecting a plurality of third test cases according to the evolution correlation strength, and forming an evolution candidate set; generating a candidate context set based on the semantic candidate set and the evolution candidate set, inputting the candidate context set into a large language model, and obtaining a performance influence score; and sorting a plurality of fourth test cases according to the semantic similarity, the evolution correlation strength and the performance influence score, so as to obtain a test case priority sequence of the tested code. The technical scheme realizes the accuracy of test case priority sequence determination.

Description

Technical Field

[0001] This invention relates to the field of software performance testing technology, and in particular to a method and program product for determining test cases. Background Technology

[0002] In continuous integration processes, each code commit triggers an automated test pipeline. However, performance testing requires multiple rounds of warm-up and repeated sampling to establish statistical reliability, and a single complete test suite can take several hours, creating a significant conflict with the need for minute-level feedback in continuous integration. Addressing this conflict through test case prioritization is currently the mainstream strategy.

[0003] Currently, determining the priority sequence of test cases involves using a bidirectional code encoder model to map code changes and test cases to the same semantic space, recalling relevant changes through vector retrieval, inferring performance impact using a large language model, and finally outputting the ranking results by a machine learning model.

[0004] However, this method relies solely on the semantic similarity of the code text and establishes the association between code changes and test cases based solely on semantic similarity. When code modifications involve adding new logic, refactoring, or changes with low semantic similarity, the recall of test cases is incomplete, and the determination of test case priorities is not accurate enough. Summary of the Invention

[0005] This invention provides a method and program product for determining test cases to solve the problems of incomplete test case recall and inaccurate determination of test case priority.

[0006] According to one aspect of the present invention, a method for determining test cases is provided, comprising: Obtain the difference change content corresponding to the code under test, determine the semantic feature vector of the code under test based on the difference change content, and select multiple second test cases from multiple first test cases based on the semantic similarity between the semantic feature vector and the query vector of multiple first test cases in the test suite to form a semantic candidate set. Obtain common occurrence records of modified code files and corresponding first test cases with performance changes in historical versions of the code under test. Determine the evolutionary association strength between each modified code file and each test case based on the common occurrence records. Select multiple third test cases from multiple first test cases based on the evolutionary association strength to form an evolutionary candidate set. A candidate context set is generated based on the semantic candidate set and the evolutionary candidate set. The candidate context set is then input into a large language model to obtain a performance impact score. The candidate context set includes multiple fourth test cases. Based on the semantic similarity, evolutionary association strength, and performance impact score of the multiple fourth test cases, the multiple fourth test cases are sorted to obtain the test case priority sequence of the code under test.

[0007] According to another aspect of the present invention, a test case determination apparatus is provided, the apparatus comprising: The semantic candidate set acquisition module is used to acquire the difference change content corresponding to the code under test, determine the semantic feature vector of the code under test based on the difference change content, and select multiple second test cases from multiple first test cases based on the semantic similarity between the semantic feature vector and the query vector of multiple first test cases in the test suite to form a semantic candidate set. An evolution candidate set acquisition module is used to acquire co-occurrence records in the historical versions of the code under test where the code under test was modified and the corresponding first test cases experienced performance changes. Based on the co-occurrence records, the evolutionary association strength between each modified code file and each test case is determined. Based on the evolutionary association strength, multiple third test cases are selected from multiple first test cases to form an evolution candidate set. The performance impact score acquisition module is used to generate a candidate context set based on the semantic candidate set and the evolution candidate set, input the candidate context set into the large language model, and obtain the performance impact score. The candidate context set includes multiple fourth test cases. The test case priority sequence determination module is used to sort the multiple fourth test cases according to the semantic similarity, evolutionary association strength and performance impact score of the multiple fourth test cases to obtain the test case priority sequence of the code under test.

[0008] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the test case determination method according to any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the method for determining test cases according to any embodiment of the present invention.

[0010] According to another aspect of the present invention, embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements a method for determining test cases as described in any of the embodiments of this disclosure.

[0011] The technical solution of this invention obtains the difference changes corresponding to the code under test, determines the semantic feature vector of the code under test based on the difference changes, and selects multiple second test cases from the multiple first test cases based on the semantic similarity between the semantic feature vector and the query vectors of multiple first test cases in the test suite, forming a semantic candidate set. By converting the difference changes into a semantic feature vector and calculating the similarity with the query vectors of the first test cases, semantically relevant test cases can be quickly filtered out from the test suite. The invention also obtains co-occurrence records of modifications to the code under test in historical versions where the corresponding first test cases show performance changes. Based on these co-occurrence records, the evolutionary association strength between each modified code file and each test case is determined. Based on the evolutionary association strength, multiple third test cases are selected from the multiple first test cases, forming an evolutionary candidate set. By obtaining co-occurrence records and quantifying the evolutionary association strength, an evolutionary candidate set independent of semantic similarity is formed. This allows for the effective recall of relevant test cases from historical co-evolutionary patterns even in scenarios where semantic retrieval fails, such as new code additions, refactoring, or runtime behavior adjustments, thus enhancing the comprehensiveness and robustness of test case recall. A candidate context set is generated based on the semantic candidate set and the evolutionary candidate set. This candidate context set is then input into a large language model to obtain a performance impact score. The candidate context set includes multiple fourth test cases. By fusing the semantic candidate set and the evolutionary candidate set, the candidate contexts input into the large language model simultaneously possess dual evidence of semantic similarity and historical evolutionary association. The performance impact score, output by the large language model, provides a performance impact score for subsequent multi-dimensional feature ranking. Based on the semantic similarity, evolutionary association strength, and performance impact score of the multiple fourth test cases, they are ranked to obtain a test case priority sequence for the code under test. By comprehensively ranking multiple features such as semantic similarity, evolutionary association strength, and the large language model's impact score, it is ensured that the test cases most likely to expose performance issues are executed first. In summary, the technical solution of this invention selects multiple second test cases from multiple first test cases based on semantic similarity to form a semantic candidate set, and selects multiple third test cases from multiple first test cases based on evolutionary association strength to form an evolutionary candidate set. A candidate context set is generated based on the semantic candidate set and the evolutionary candidate set. The candidate context set is input into a large language model to obtain a performance impact score. Based on the semantic similarity, evolutionary association strength, and performance impact score of multiple fourth test cases, the multiple fourth test cases are sorted to obtain the test case priority sequence of the code under test. This achieves the accuracy of the test case priority sequence determination and improves the efficiency of testing.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of a method for determining test cases according to the present invention; Figure 2 This is a flowchart of a method for determining test cases according to the present invention; Figure 3 This is a general framework diagram of the test case determination method provided by the present invention; Figure 4 This is a schematic diagram of the structure of a test case determination device provided by the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device implemented in an embodiment of the present invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0016] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0017] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0018] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0019] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0020] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0021] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0022] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0023] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0024] Figure 1 This invention provides a flowchart of a method for determining test cases. This embodiment is applicable to determining the priority sequence of test cases in a continuous integration environment. This method can be executed by a test case determination device, which can be implemented in hardware and / or software, optionally through an electronic device such as a mobile terminal, PC, or server. Figure 1 As shown, the method may specifically include: S110. Obtain the difference change content corresponding to the code under test, determine the semantic feature vector of the code under test based on the difference change content, and select multiple second test cases from multiple first test cases to form a semantic candidate set based on the semantic similarity between the semantic feature vector and the query vector of multiple first test cases in the test suite.

[0025] In this embodiment of the invention, the code under test can be specifically understood as code submitted by developers that needs to be tested. The code under test is organized in code files as the basic unit. The code under test is the original data source for extracting semantic feature vectors. The difference changes can be specifically understood as the content representing code-level changes that have occurred in the code under test. These difference changes can be extracted through the Git application programming interface. The difference changes are the original data basis for subsequent semantic feature vector extraction. The difference changes corresponding to the code under test include, but are not limited to, the addition, deletion, or modification of lines of code. The semantic feature vector can be specifically understood as a vector representing the semantic features of the code file. The semantic feature vector can be generated through a bidirectional code encoder model; for example, it can be generated through the CodeBERT model (CodeBERT: A Pre-Trained Model for Programming and Natural Languages). The test suite can be specifically understood as a collection of all test cases configured for the code under test. The test suite is used to provide a set of test cases to be ordered for the code under test. The test suite contains multiple first test cases. A first test case can be specifically understood as a test case included in the test suite. The first test case is the original test case without semantic similarity filtering. The query vector can be understood as a high-dimensional real-number vector representing the semantic features of the first test case. The query vector can be generated using a bidirectional code encoder model. For example, semantic feature vectors can be generated using the CodeBERT model. Semantic similarity can be understood as the semantic similarity between the query vector and the semantic feature vector of a test case. Higher semantic similarity indicates that the test case is more likely to experience performance changes due to differences in the tested code, and therefore should be given higher execution priority in the test case priority sequence. The second test case can be understood as selecting test cases with higher semantic similarity scores to the query vector from multiple first test cases. The second test cases constitute the semantic candidate set. The second test cases are selected from the first test cases that have a high semantic correlation with code changes. The semantic candidate set can be understood as a set based on the second test cases. The semantic candidate set is the result obtained based on semantic similarity. The semantic candidate set is one of the inputs for the subsequent dual-path retrieval result fusion stage.

[0026] Specifically, the difference changes corresponding to the code under test are obtained through the Git application programming interface. These difference changes are then input into a bidirectional code encoder model to obtain the semantic feature vector of the code under test. The query vector of the first test case is obtained based on the first test case and the bidirectional code encoder model. The semantic similarity between this semantic feature vector and the query vectors of multiple first test cases in the test suite is calculated using the following formula: in, Indicates the first test case The query vector; Indicates the code being tested The semantic feature vector. Semantic similarity value. The value range is [-1, 1]. The closer the semantic similarity value is to 1, the more semantically similar the semantic feature vector is to the query vector of the first test case. The semantic similarity between the semantic feature vector and the query vectors of multiple first test cases in the test suite is obtained. The semantic similarity is sorted in descending order, and the test cases whose semantic similarity ranks before the first preset position are selected as second test cases. All second test cases are then set to obtain a semantic candidate set. For example, the test cases ranked in the top 2K by semantic similarity are selected as second test cases, and all second test cases are set. Here, K is a preset positive integer, and the value of K can be changed according to application requirements; this invention does not limit this. By converting the difference changes into semantic feature vectors and calculating their semantic similarity with the query vectors of each first test case in the test suite, the semantic association between code changes and test cases, which were originally impossible to directly compare numerically, is transformed into a calculable similarity score. Based on semantic similarity ranking, a set of semantically related candidate cases is quickly filtered out from the full set of test cases, providing a foundation of input data that balances efficiency and recall, and effectively narrowing the scope of analysis in subsequent stages.

[0027] Optionally, determining the semantic feature vector of the code under test based on the difference changes includes: using a sliding window strategy with overlapping parts to divide the difference changes into multiple content segments, inputting each content segment into a bidirectional code encoder model to obtain the content vector corresponding to each content segment, and performing average pooling on the multiple content vectors corresponding to the code under test to obtain the semantic feature vector of the code under test.

[0028] In this embodiment of the invention, a content fragment can be specifically understood as a fragment obtained by segmenting the differing content according to the overlapping sliding window. The content fragment is the basic input unit for semantic encoding by the bidirectional code encoder model. Adjacent content fragments have overlapping portions. These overlapping portions are used to prevent key semantic information from being truncated and lost at the segmentation boundaries. The bidirectional code encoder model can be specifically understood as a code encoding model capable of capturing contextual semantic information from both the left and right directions of the input sequence. The bidirectional code encoder model is specifically designed to understand the semantic structure of programming language code and can map the input content fragment into a dense numerical vector in a high-dimensional space. For example, the bidirectional code encoder model can be the CodeBERT model. The content vector can be specifically understood as the vector obtained by inputting the content fragment into the bidirectional code encoder. The content vector has a fixed length. The content vector is a high-dimensional real-number vector used to represent the semantic information of the content fragment.

[0029] Specifically, when determining the semantic feature vector of the tested code based on the differences in content, the differences in content can be segmented using an overlapping sliding window strategy to obtain multiple content fragments. Each segmented code fragment is then input into a bidirectional code encoder model to obtain the content vector for each fragment. Average pooling is then applied to the multiple content vectors corresponding to the tested code to obtain the semantic feature vector of the tested code. By using average pooling to obtain the semantic feature vector of the tested code, complete semantic information of the changes is preserved while significantly reducing the amount of data and computational overhead for subsequent similarity calculations, providing a standardized numerical basis for rapid semantic retrieval in a continuous integration environment.

[0030] Optionally, the method further includes: for each of the first test cases in the test suite, obtaining the method signature, method body, and function call signature list of the first test case, concatenating the method signature, method body, and function call signature list, and inputting them into the bidirectional code encoder model to obtain the query vector of the first test case.

[0031] In this embodiment of the invention, the method signature can be specifically understood as a signature used to identify the identity information of a test method. Since each test case corresponds to an independent test method, the method signature also serves as a unique identifier for that test case. The method body can be specifically understood as the collection of all executable source code statements contained within the test case. The method body is the core representation of the test case's execution behavior, providing semantic information at the test case's execution logic level for the generation of query vectors. The function call signature list can be specifically understood as a list of signatures of all external functions called within the method body. The function call signature list reflects the call dependency relationship between the test case and the code being tested, providing semantic information at the test case dependency level for the generation of query vectors.

[0032] Specifically, for each first test case in the test suite, the corresponding method signature, method body, and function call signature list are extracted. The method signature, method body, and function call signature list for each first test case are concatenated, and the concatenated content is input into the bidirectional code encoder model to obtain the query vector for the first test case. By extracting and encoding these three types of information—method signature, method body, and function call signature list—into a query vector, the query vector simultaneously integrates semantic information from three dimensions: test case identity, execution behavior, and external dependencies. This allows for a more comprehensive and accurate representation of the complete semantic features of the test case.

[0033] S120. Obtain the co-occurrence records of the tested code being modified and the corresponding first test cases experiencing performance changes in historical versions of the tested code. Determine the evolutionary association strength between each modified code file and each test case based on the co-occurrence records. Select multiple third test cases from multiple first test cases based on the evolutionary association strength to form an evolutionary candidate set.

[0034] In this embodiment of the invention, co-occurrence records can be specifically understood as records of related events where test code is modified and test cases experience performance changes simultaneously. Co-occurrence records are used for subsequent quantitative calculations of evolutionary correlation strength. Modified code files can be specifically understood as code files that have been modified during the historical version evolution of the tested code. Taking each modified code file as a dimension, co-occurrence records between it and each test case in historical versions are statistically analyzed, thereby quantifying the evolutionary correlation strength between the two. Evolutionary correlation strength can be specifically understood as an indicator used to quantify the degree of collaborative change between the modified code file and the test case during historical version evolution. A higher evolutionary correlation strength indicates a stronger collaborative change relationship between the modified code file and the test case in historical evolution. The third test case can be specifically understood as a test case selected from multiple first test cases based on the evolutionary correlation strength. The third test case constitutes an evolutionary candidate set. The selection criterion for the third test case is the evolutionary correlation strength. The second and third test cases respectively filter the first test cases from the semantic dimension and the historical evolution dimension, jointly constituting the parallel output of dual-path retrieval, providing complementary candidate information for the subsequent fusion stage. The evolutionary candidate set can be understood as a set composed of third test cases. It is derived based on the strength of evolutionary associations. The evolutionary candidate set is then fused with the semantic candidate set to generate a candidate context set that provides both semantic association and historical evolutionary evidence.

[0035] Specifically, the Git application programming interface (API) is used to obtain co-occurrence records of modifications to the tested code in historical versions, where the corresponding first test cases showed performance changes. The evolutionary correlation strength between each modified code file and each test case is calculated based on these co-occurrence records. Multiple third test cases are selected from the multiple first test cases based on the evolutionary correlation strength, and the evolutionary correlation strength is sorted in descending order. Test cases with evolutionary correlation strength preceding a second preset position are selected as third test cases, and all third test cases are grouped into a set to obtain an evolutionary candidate set. For example, the top M test cases in terms of evolutionary correlation strength are selected as third test cases, and all third test cases are grouped into a set. Here, M is a preset positive integer, and the value of M can be changed according to application requirements; this invention does not limit this. Since semantic similarity filtering cannot detect implicit runtime dependencies, obtaining the evolutionary candidate set can effectively recall relevant test cases from a historical evolutionary dimension, significantly enhancing the accuracy and stability of test case determination.

[0036] The co-occurrence record can be obtained by traversing the historical versions of the code under test through the Git application programming interface, extracting a list of modified code files in each historical version. Performance test execution records corresponding to the same historical version are then retrieved, containing performance metrics data for each test case in that version. If a code file is modified and the performance metric of a test case changes by more than a preset threshold, both occurring in the same historical version, a co-occurrence event is recorded. After traversing all historical versions, all co-occurrence events are aggregated to obtain the co-occurrence record.

[0037] Optionally, determining the evolutionary association strength between each modified code file and each test case based on the co-occurrence record includes: constructing a weighted bipartite graph between the code under test and the first test case based on the co-occurrence record; determining the evolutionary association strength between the code file and the test case based on the weighted bipartite graph; the weighted bipartite graph includes test nodes, nodes under test, and association edges between the test nodes and the nodes under test; the test nodes are used to indicate the first test case; the nodes under test are used to indicate the code under test; the association edges are used to indicate that the test nodes and the nodes under test co-occur in the same version; and each association edge is associated with a corresponding version number.

[0038] In this embodiment of the invention, the weighted bipartite graph can be specifically understood as a graph used to characterize the historical co-evolutionary relationship between the code under test and the test cases. The weighted bipartite graph includes test nodes, nodes under test, and the associated edges between the test nodes and the nodes under test. The weighted bipartite graph provides a structured data foundation for calculating the strength of evolutionary associations. A test node can be specifically understood as a node in the weighted bipartite graph used to indicate a first test case. Each test node uniquely corresponds to a first test case in a test suite. Test nodes and nodes under test are connected by associated edges. A node under test can be specifically understood as a node in the weighted bipartite graph used to indicate the code under test. Each node under test uniquely corresponds to one piece of code under test. An associated edge can be specifically understood as an edge connecting a test node and a node under test. Associated edges are used to indicate that the test node and the node under test co-occur in the same version. Each associated edge connects one and only one test node and one node under test. Each associated edge is associated with a corresponding version number, used to record the specific historical version in which the co-occurrence event occurred. Associated edges are the core component of the weighted bipartite graph.

[0039] Specifically, a weighted bipartite graph is constructed between the code under test and the first test case based on the co-occurrence records. The weighted bipartite graph can be represented as follows: Here, VB is the set of test nodes, VF is the set of nodes under test, E is the set of associated edges, and H is the set of historical version records. The evolutionary association strength between code files and test cases is determined based on a weighted bipartite graph. By constructing a weighted bipartite graph, co-occurring records are transformed into structured graph data, providing a computable graph structure foundation for subsequently quantifying the evolutionary association strength between code files and test cases.

[0040] Optionally, determining the evolutionary association strength between the code file and the test case based on the weighted bipartite graph includes: counting the number of associated edges between each test node and each tested node in the weighted bipartite graph, dividing the number of associated edges by the sum of the number of associated edges and the smoothing parameter to obtain a co-occurrence score; finding the shortest path between each test node and each tested node in the weighted bipartite graph, and using the reciprocal of the number of edges of the shortest path as the topological distance score; for test nodes and tested nodes with associated edges, calculating the absolute value of the difference between the version number corresponding to each associated edge and the current version number, averaging all absolute values, adding one, and taking the reciprocal to obtain a time decay score; and weighting and summing the co-occurrence score, the topological distance score, and the time decay score according to their respective adjustable weight coefficients to obtain the evolutionary association strength.

[0041] In this embodiment of the invention, the number of associated edges can be specifically understood as the total number of associated edges established between any test node and any tested node in the weighted bipartite graph. Since the same test node and the same tested node may co-occur multiple times in multiple historical versions, and each co-occurrence corresponds to the establishment of an associated edge, the number of associated edges is the actual number of historical co-occurrences between the test node and the tested node. The more associated edges there are, the higher the frequency of co-occurrence between the test node and the tested node in history, and the stronger the historical association between them. The smoothing parameter can be specifically understood as a parameter introduced when calculating the co-occurrence score. The smoothing parameter is used to prevent the denominator from being zero. The smoothing parameter avoids division by zero when the number of associated edges is zero by adding a positive number to the denominator. At the same time, when the number of associated edges is small, the co-occurrence score is smoothed to prevent low-frequency co-occurrence events from receiving excessively high scores, which would lead to statistical bias. For example, the smoothing parameter can be 1. The co-occurrence score can be specifically understood as a score used to quantify the frequency of co-occurrence between the test node and each tested node in historical versions. The co-occurrence score is calculated based on the number of associated edges between the test node and the tested node in the weighted bipartite graph. A higher number of associated edges indicates a greater frequency of co-occurrence in history. The co-occurrence score is one of the three dimensions for calculating evolutionary association strength, directly measuring the degree of association between code files and test cases from a historical frequency perspective. The topological distance score can be understood as a score calculated based on the shortest path between the test node and the tested node in the weighted bipartite graph. The topological distance score quantifies the topological proximity of the test node and the tested node in the weighted bipartite graph. It is calculated by taking the reciprocal of the number of edges in the shortest path between nodes. A shorter shortest path indicates a closer topological distance between the test node and the tested node, and a stronger historical association between them. The time decay score can be understood as a score calculated based on the proximity between the occurrence time of a co-occurrence record and the current time. The time decay score is obtained by calculating the average interval between the version number corresponding to all co-occurrence records between the test node and the tested node and the current version number, and then taking the reciprocal of this average interval. The closer the co-occurrence record is to the current version, the higher the time decay score; conversely, the further away the co-occurrence record is from the current version, the lower the time decay score, thus reducing the interference of premature historical data on the current evolutionary association strength. The adjustable weight coefficients can be understood as coefficients used to adjust the weight of each of the three dimensions—co-occurrence score, topological distance score, and time decay score—in the overall score. The adjustable weight coefficients include a first weight coefficient corresponding to the co-occurrence score, a second weight coefficient corresponding to the topological distance score, and a third weight coefficient corresponding to the time decay score. The sum of these three weight coefficients is 1.By adjusting the magnitude of each weight coefficient, the influence of the three scoring dimensions on the strength of evolutionary association can be flexibly controlled.

[0042] Specifically, the number of associated edges between each test node and each node in the weighted bipartite graph is counted. This number of associated edges is then divided by the sum of the number of associated edges and the smoothing parameter to obtain the co-occurrence score. The calculation formula is as follows: in, This represents the number of associated edges; The smoothing parameter is used. The co-occurrence score reflects the historical association strength between each test node and each tested node. The shortest path between each test node and each tested node is found in the weighted bipartite graph, and the reciprocal of the number of edges in the shortest path is used as the topological distance score. The formula for calculating the topological distance score is as follows: in, This represents the number of edges in the shortest path. For test nodes and tested nodes with associated edges, calculate the absolute value of the difference between the version number corresponding to each associated edge and the current version number. Take the average of all absolute values, add one, and then take the reciprocal to obtain the time decay score. The formula for calculating the time decay score is as follows: in, Indicates the current version number. This represents the version number corresponding to the associated edge. The evolutionary association strength is obtained by weighting and summing the co-occurrence score, topological distance score, and time decay score according to their respective adjustable weight coefficients. By weighting and fusing these scores from three dimensions—historical co-occurrence frequency, graph topological proximity, and co-occurrence time—into a single comprehensive score, the evaluation of evolutionary association strength takes into account both the frequency evidence of direct co-occurrence and the propagation path of indirect associations. Furthermore, the time decay mechanism focuses on recent effective historical experience and suppresses early noise interference, enabling a comprehensive and accurate determination of test cases from a historical evolutionary perspective.

[0043] Optionally, the co-occurrence record contains multiple co-occurrence events, and each co-occurrence event record includes the version number where the co-occurrence occurred, the identifier of the modified code file in that version, and the identifier of the test case in that version where performance changes occurred.

[0044] In this embodiment of the invention, a co-occurrence event can be specifically understood as an event in which the code under test is modified in a certain version and the first test case experiences a performance change. A co-occurrence event is the basic unit constituting a co-occurrence record. Each co-occurrence event corresponds to an associated edge in a weighted bipartite graph. Each co-occurrence event includes at least the version number where the co-occurrence occurred, the identifier of the modified code file in that version, and the identifier of the test case in that version that experienced a performance change. The version number where the co-occurrence occurred can be specifically understood as the unique identifier information of the version corresponding to the co-occurrence event. The version number is used to record the specific point in time in the code evolution history where the co-occurrence event occurred. When the same test node and the same tested node co-occur multiple times in different historical versions, the version number is used to distinguish each co-occurrence event. The identifier of the modified code file in that version can be specifically understood as the unique identity information of the modified code under test corresponding to the co-occurrence event. The code file identifier is used to accurately locate the code file under test. The identifier of the test case in that version that experienced a performance change can be specifically understood as the unique identity information of the test case corresponding to the co-occurrence event that experienced a significant performance change. The test case identifier is used to accurately locate the test case.

[0045] Optionally, constructing a weighted bipartite graph between the code under test and the first test case based on the co-occurrence records includes: for each co-occurrence event, if there is no node of the first test case corresponding to the co-occurrence event in the weighted bipartite graph, creating a test node corresponding to the first test case; if there is no node of the code under test corresponding to the co-occurrence event in the weighted bipartite graph, creating a test node corresponding to the code under test; and establishing an association edge between the test node and the test node, and recording the version number in the co-occurrence event as the version number of the association edge; wherein, when the same test node and the same test node co-occur multiple times in different historical versions, multiple association edges are established, and each association edge records its corresponding version number.

[0046] Specifically, each co-occurrence event in the co-occurrence record is traversed. For the currently traversed co-occurrence event, the version number of the co-occurrence, the identifier of the modified code file in that version, and the identifier of the test case with performance changes in that version are extracted. Using the identifier of the test case with performance changes in that version as an index, the weighted bipartite graph is checked to see if a corresponding test node exists. If no corresponding test node exists, a new test node is created; if a corresponding test node already exists, it is directly located. Using the identifier of the modified code file in that version as an index, the weighted bipartite graph is checked to see if a corresponding node under test exists. If no corresponding node under test exists, a new node under test is created; if a corresponding node under test already exists, it is directly located. After determining the test node and the node under test, an association edge is established between the test node and the node under test, and the version number in the current co-occurrence event is recorded as the version number of the association edge. When the same test node and the same node under test co-occur multiple times in different historical versions, since each co-occurrence corresponds to a different co-occurrence event with a different version number, a new independent association edge is added between the two identical nodes when processing each co-occurrence event, and the corresponding version number is recorded for each. Multiple association edges can exist between the same pair of nodes, each associated with a different version number, providing an accurate graph structure data foundation for subsequent statistics on the number of association edges and calculation of time decay scores.

[0047] S130. Generate a candidate context set based on the semantic candidate set and the evolution candidate set, input the candidate context set into the large language model to obtain a performance impact score, and the candidate context set includes multiple fourth test cases.

[0048] In this embodiment of the invention, the candidate context set can be specifically understood as the set obtained by fusing the semantic candidate set and the evolutionary candidate set. The candidate context set is used as input to a large language model to obtain a performance impact score. The candidate context set includes multiple fourth test cases. The large language model can be specifically understood as a generative artificial intelligence model pre-trained on a large-scale text corpus. The large language model is used to receive the performance impact score output by the candidate context set. The performance impact score can be specifically understood as the score output by the large language model after the candidate context set is input. The performance impact score ranges between 0 and 1. The numerical value of the performance impact score is used to represent the degree of risk of the current code change causing a performance change in the test case. The closer the performance impact score is to 1, the more the large language model judges that the test case is affected by the code change; the closer the performance impact score is to 0, the less likely the large language model judges that the test case is affected by the code change. The fourth test case can be specifically understood as the test case in the candidate context set used as input to the large language model for performance impact analysis. The fourth test case is the test case that finally enters the candidate context set after being filtered by semantic similarity and evolutionary association strength.

[0049] Specifically, a candidate context set is generated based on the semantic candidate set and the evolutionary candidate set. This candidate context set is then input into the large language model to obtain a performance impact score. By combining the semantic and evolutionary candidate sets to obtain the candidate context set, the large language model outputs a performance impact score based on both semantic association and historical evolution evidence. Compared to directly inputting all test cases or generating a candidate set based solely on a single retrieval path, this significantly reduces contextual noise interference, enhances the sufficiency of the reasoning basis and the credibility of the score, and provides a high-quality performance impact evaluation dimension for subsequent multi-dimensional feature ranking.

[0050] Optionally, generating a candidate context set based on the semantic candidate set and the evolutionary candidate set includes: calculating the intersection of the semantic candidate set and the evolutionary candidate set; when the number of intersections is less than a preset number, selecting supplementary results from the remaining results of the semantic candidate set according to semantic similarity from high to low; if still insufficient, selecting supplementary results from the remaining results of the evolutionary candidate set according to evolutionary association strength from high to low, so that the number of candidate test cases reaches the preset number, thereby generating the candidate context set.

[0051] In this embodiment of the invention, the preset quantity can be specifically understood as the target number of candidate test cases that need to be included in the candidate context set. The preset quantity is a pre-set fixed positive integer. The preset quantity can be adjusted according to the application scenario, and this invention does not limit it.

[0052] Specifically, the intersection of the semantic candidate set and the evolutionary candidate set is calculated. When the number of intersections is less than a preset number, the remaining results in the semantic candidate set are selected to supplement the existing ones according to semantic similarity from high to low, until the preset number is reached. If the number of semantic candidate sets is still less than the preset number after supplementation, the remaining results in the evolutionary candidate set are selected to supplement the existing ones according to evolutionary association strength from high to low, so that the number of candidate test cases reaches the preset number, generating a candidate context set. By generating a candidate context set, the two dimensions of semantic association and historical evolution are integrated to the greatest extent.

[0053] Optionally, inputting the candidate context set into the large language model to obtain a performance impact score includes: inputting the code implementation of each of the fourth test cases in the candidate context set, the difference changes in the modified code file, and the chained reasoning hint template into the large language model, so that the large language model outputs a performance impact score between zero and one according to the chained reasoning method.

[0054] In this embodiment of the invention, the chain-reasoning prompt template can be specifically understood as a prompt template used to guide a large language model in a step-by-step reasoning manner. By decomposing the complex performance impact analysis task into multiple ordered reasoning steps, the chain-reasoning prompt template enables the large language model to progressively construct a complete causal chain and output a performance impact score at each step of the reasoning.

[0055] Specifically, the code implementation of each fourth test case in the candidate context set, the differences in the modified code files, and the chain-reasoning hint template are all input into the large language model. The large language model analyzes the potential impact of the differences in the modified code files on the performance of the fourth test cases based on chain-reasoning, obtaining a performance impact score between zero and one. Specifically, the large language model outputs a performance impact score between zero and one using chain-reasoning. Building upon the semantics of the code text and historical evolution evidence, the deep logical reasoning capability of the large language model is introduced, elevating the performance impact assessment from shallow feature matching to a credible judgment level with causal basis, providing high-quality semantic analysis dimensions for the subsequent ranking of the fourth test cases.

[0056] S140. Based on the semantic similarity, evolutionary association strength, and performance impact score of the multiple fourth test cases, sort the multiple fourth test cases to obtain the test case priority sequence of the code under test.

[0057] Specifically, based on the semantic similarity, evolutionary association strength, and performance impact scores of multiple fourth test cases, the test cases are ranked to obtain a test case priority sequence for the code under test. By integrating semantic similarity, evolutionary association strength, and large language model scores for comprehensive ranking, high-risk test cases are given higher priority, thus achieving accuracy in determining the test case priority sequence.

[0058] The technical solution of this invention obtains the difference changes corresponding to the code under test, determines the semantic feature vector of the code under test based on the difference changes, and selects multiple second test cases from the multiple first test cases based on the semantic similarity between the semantic feature vector and the query vectors of multiple first test cases in the test suite, forming a semantic candidate set. By converting the difference changes into a semantic feature vector and calculating the similarity with the query vectors of the first test cases, semantically relevant test cases can be quickly filtered out from the test suite. The invention also obtains co-occurrence records of modifications to the code under test in historical versions where the corresponding first test cases show performance changes. Based on these co-occurrence records, the evolutionary association strength between each modified code file and each test case is determined. Based on the evolutionary association strength, multiple third test cases are selected from the multiple first test cases, forming an evolutionary candidate set. By obtaining co-occurrence records and quantifying the evolutionary association strength, an evolutionary candidate set independent of semantic similarity is formed. This allows for the effective recall of relevant test cases from historical co-evolutionary patterns even in scenarios where semantic retrieval fails, such as new code additions, refactoring, or runtime behavior adjustments, thus enhancing the comprehensiveness and robustness of test case recall. A candidate context set is generated based on the semantic candidate set and the evolutionary candidate set. This candidate context set is then input into a large language model to obtain a performance impact score. The candidate context set includes multiple fourth test cases. By fusing the semantic candidate set and the evolutionary candidate set, the candidate contexts input into the large language model simultaneously possess dual evidence of semantic similarity and historical evolutionary association. The performance impact score, output by the large language model, provides a performance impact score for subsequent multi-dimensional feature ranking. Based on the semantic similarity, evolutionary association strength, and performance impact score of the multiple fourth test cases, they are ranked to obtain a test case priority sequence for the code under test. By comprehensively ranking multiple features such as semantic similarity, evolutionary association strength, and the large language model's impact score, it is ensured that the test cases most likely to expose performance issues are executed first. In summary, the technical solution of this invention selects multiple second test cases from multiple first test cases based on semantic similarity to form a semantic candidate set, and selects multiple third test cases from multiple first test cases based on evolutionary association strength to form an evolutionary candidate set. A candidate context set is generated based on the semantic candidate set and the evolutionary candidate set. The candidate context set is input into a large language model to obtain a performance impact score. Based on the semantic similarity, evolutionary association strength, and performance impact score of multiple fourth test cases, the multiple fourth test cases are sorted to obtain the test case priority sequence of the code under test. This achieves the accuracy of the test case priority sequence determination and improves the efficiency of testing.

[0059] Figure 2This is a flowchart illustrating a method for determining test cases provided by the present invention. This embodiment provides a specific process for determining the priority sequence of test cases. For detailed implementation methods, please refer to the description of this embodiment. Technical features that are the same as or similar to those in the foregoing embodiments will not be repeated here. Figure 2 As shown, the method may specifically include: S210. Obtain the difference change content corresponding to the code under test, determine the semantic feature vector of the code under test based on the difference change content, and select multiple second test cases from multiple first test cases to form a semantic candidate set based on the semantic similarity between the semantic feature vector and the query vector of multiple first test cases in the test suite.

[0060] S220. Obtain common occurrence records in the historical versions of the code under test where the code under test was modified and the corresponding first test case experienced performance changes. Determine the evolutionary association strength between each modified code file and each test case based on the common occurrence records. Select multiple third test cases from multiple first test cases based on the evolutionary association strength to form an evolutionary candidate set.

[0061] S230. Generate a candidate context set based on the semantic candidate set and the evolution candidate set, input the candidate context set into the large language model, and obtain a performance impact score. The candidate context set includes multiple fourth test cases.

[0062] S240. Obtain the number of function calls during the execution of each of the fourth test cases as a coverage feature, and obtain the performance fluctuation range of each of the fourth test cases in historical versions as a historical performance stability feature.

[0063] In this embodiment of the invention, the coverage feature can be specifically understood as a feature used to measure the breadth of coverage of the tested code by the test case during execution. The coverage feature is characterized by the total number of different functions called during the execution of the fourth test case. The more functions called, the wider the scope of the tested code covered by the test case. The historical performance stability feature can be specifically understood as a feature used to represent the performance fluctuation of the test case in historical versions. It is characterized by the performance fluctuation of the test case in historical versions. The performance fluctuation amplitude can be specifically understood as a measure of the dispersion of the performance indicator data of the test case in multiple historical versions. The performance fluctuation amplitude is calculated based on the average absolute fluctuation amplitude of the performance indicator data such as response time or throughput of the test case in historical versions.

[0064] Specifically, the number of function calls during the execution of each fourth test case is obtained as a coverage feature, and the performance fluctuation range of each fourth test case in historical versions is obtained as a historical performance stability feature. After executing each fourth test case, a list of all called functions is collected, duplicates are removed, and the number of function calls is counted to obtain the coverage feature. The historical performance stability feature is obtained by extracting performance metric data of the fourth test case in historical versions, comparing the performance metric values ​​in the current version with those in historical versions, and calculating the average of the absolute values ​​of the performance deviations between versions as the performance fluctuation range of that test case. By obtaining both coverage and historical performance stability features, the priority of test cases can be comprehensively evaluated from these two dimensions, resulting in an accurate test case priority sequence.

[0065] S250. After normalizing the semantic similarity, evolutionary association strength, performance impact score, coverage feature, and historical performance stability feature, input them into the random forest model. The random forest model outputs the probability score of performance change for each of the fourth test cases. The test cases are then arranged in descending order of probability score to generate the test case priority sequence.

[0066] In this embodiment of the invention, the random forest model can be specifically understood as a machine learning model based on decision tree ensemble. The random forest model outputs a probability score for each fourth test case to indicate a performance change. This probability score serves as the direct basis for generating a priority sequence of test cases. The probability score ranges from 0 to 1. A higher probability score indicates a greater risk of performance change for the fourth test case under the current changes in the tested code, resulting in a higher priority in the test execution sequence and the earlier it should be executed. Conversely, a lower probability score indicates a lower risk of performance change for the test case, resulting in a lower priority in the test execution sequence.

[0067] Specifically, semantic similarity, evolutionary association strength, performance impact score, coverage feature, and historical performance stability feature are normalized. These normalized features are then input into a random forest model. The random forest model outputs the probability score of performance changes for each fourth test case. The fourth test cases are then sorted in descending order of probability scores to generate a test case priority sequence. By normalizing these five dimensions (semantic similarity, evolutionary association strength, large language model impact score, coverage feature, and historical performance stability feature) and inputting them into the random forest model, the ranking decision incorporates complementary signals from code semantics, historical evolution, deep reasoning, execution complexity, and historical sensitivity. This allows the test cases most likely to expose performance issues to be executed first within a limited testing time, improving the accuracy of test case selection and further enhancing testing efficiency and the effectiveness of test resource utilization.

[0068] This invention, in its embodiments, obtains the difference changes corresponding to the code under test, determines the semantic feature vector of the code under test based on the difference changes, and selects multiple second test cases from the multiple first test cases based on the semantic similarity between the semantic feature vector and the query vectors of multiple first test cases in the test suite, forming a semantic candidate set. It also obtains co-occurrence records in historical versions of the code under test where the code under test was modified and the corresponding first test cases experienced performance changes, determines the evolutionary association strength between each modified code file and each test case based on the co-occurrence records, and selects multiple third test cases from the multiple first test cases based on the evolutionary association strength, forming an evolutionary candidate set. Finally, it generates a candidate context set based on the semantic candidate set and the evolutionary candidate set, inputs the candidate context set into a large language model to obtain a performance impact score, and the candidate context set includes multiple fourth test cases. The method obtains the number of function calls during the execution of each fourth test case as a coverage feature, and the performance fluctuation range of each fourth test case in historical versions as a historical performance stability feature. This allows for a comprehensive evaluation of the priority of test cases from both coverage and historical performance stability dimensions, resulting in an accurate test case priority sequence. The semantic similarity, evolutionary association strength, performance impact score, coverage feature, and historical performance stability feature are normalized and input into a random forest model. The random forest model outputs a probability score for performance changes in each fourth test case, which is then sorted in descending order of probability scores to generate the test case priority sequence. This ensures that test resources are allocated preferentially to the highest-risk test cases, improving testing efficiency. In summary, the technical solution of this embodiment of the invention obtains the coverage and historical performance stability features of each fourth test case. After normalizing the semantic similarity, evolutionary association strength, performance impact score, coverage feature, and historical performance stability feature, the method inputs the random forest model to output a probability score for performance changes in each fourth test case, which is then sorted in descending order of probability scores to generate a test case priority sequence, thus improving the accuracy of test case priority ranking.

[0069] Based on the above embodiments, an optional example is provided, which can be used for scenarios involving test case determination. The overall framework diagram of the test case determination method is as follows: Figure 3 As shown. Figure 3As shown, in a CI (Continuous Integration) / CD (Continuous Delivery) continuous integration environment, the differences in the code under test and the first test case are taken as input. A candidate context set is generated based on semantic similarity retrieval and evolutionary association strength retrieval. The candidate context set is input into a large language model to obtain a performance impact score. The number of function calls during the execution of each fourth test case in the candidate context set is obtained as a coverage feature, and the performance fluctuation range of each fourth test case in historical versions is obtained as a historical performance stability feature. Then, multi-dimensional feature fusion and ranking are performed. Specifically, the semantic similarity, evolutionary association strength, performance impact score, coverage feature, and historical performance stability feature are normalized and input into a random forest model to obtain a test case priority sequence. The external environment and tools provide the FAISS (Facebook AI Similarity Search) database, graph database, and large language model interface. The overall architecture achieves comprehensiveness and accuracy in test case priority ranking. The FAISS database is used to store semantic feature vectors and perform similarity retrieval based on the query vectors of test cases. The graph database is used to store weighted bipartite graph structures.

[0070] Figure 4 This is a schematic diagram of a test case determination device provided by the present invention. This embodiment is applicable to determining the priority sequence of test cases in a continuous integration environment. The device can be implemented using software and / or hardware, and can be integrated into any device that provides test case determination functionality, such as… Figure 4 As shown, the device for determining test cases specifically includes: a semantic candidate set acquisition module 310, an evolution candidate set acquisition module 320, a performance impact score acquisition module 330, and a test case priority sequence determination module 340.

[0071] The semantic candidate set acquisition module 310 is used to acquire the difference change content corresponding to the code under test, determine the semantic feature vector of the code under test based on the difference change content, and select multiple second test cases from multiple first test cases to form a semantic candidate set based on the semantic similarity between the semantic feature vector and the query vector of multiple first test cases in the test suite. The evolution candidate set acquisition module 320 is used to acquire co-occurrence records in the historical versions of the code under test where the code under test was modified and the corresponding first test case experienced performance changes, determine the evolutionary association strength between each modified code file and each test case based on the co-occurrence records, and select multiple third test cases from multiple first test cases based on the evolutionary association strength to form an evolution candidate set. The performance impact score acquisition module 330 is used to generate a candidate context set based on the semantic candidate set and the evolution candidate set, input the candidate context set into the large language model, and obtain the performance impact score. The candidate context set includes multiple fourth test cases. The test case priority sequence determination module 340 is used to sort the multiple fourth test cases according to the semantic similarity, evolutionary association strength and performance impact score of the multiple fourth test cases to obtain the test case priority sequence of the code under test.

[0072] The technical solution of this invention obtains the difference changes corresponding to the code under test, determines the semantic feature vector of the code under test based on the difference changes, and selects multiple second test cases from the multiple first test cases based on the semantic similarity between the semantic feature vector and the query vectors of multiple first test cases in the test suite, forming a semantic candidate set. By converting the difference changes into a semantic feature vector and calculating the similarity with the query vectors of the first test cases, semantically relevant test cases can be quickly filtered out from the test suite. The invention also obtains co-occurrence records of modifications to the code under test in historical versions where the corresponding first test cases show performance changes. Based on these co-occurrence records, the evolutionary association strength between each modified code file and each test case is determined. Based on the evolutionary association strength, multiple third test cases are selected from the multiple first test cases, forming an evolutionary candidate set. By obtaining co-occurrence records and quantifying the evolutionary association strength, an evolutionary candidate set independent of semantic similarity is formed. This allows for the effective recall of relevant test cases from historical co-evolutionary patterns even in scenarios where semantic retrieval fails, such as new code additions, refactoring, or runtime behavior adjustments, thus enhancing the comprehensiveness and robustness of test case recall. A candidate context set is generated based on the semantic candidate set and the evolutionary candidate set. This candidate context set is then input into a large language model to obtain a performance impact score. The candidate context set includes multiple fourth test cases. By fusing the semantic candidate set and the evolutionary candidate set, the candidate contexts input into the large language model simultaneously possess dual evidence of semantic similarity and historical evolutionary association. The performance impact score, output by the large language model, provides a performance impact score for subsequent multi-dimensional feature ranking. Based on the semantic similarity, evolutionary association strength, and performance impact score of the multiple fourth test cases, they are ranked to obtain a test case priority sequence for the code under test. By comprehensively ranking multiple features such as semantic similarity, evolutionary association strength, and the large language model's impact score, it is ensured that the test cases most likely to expose performance issues are executed first. In summary, the technical solution of this invention selects multiple second test cases from multiple first test cases based on semantic similarity to form a semantic candidate set, and selects multiple third test cases from multiple first test cases based on evolutionary association strength to form an evolutionary candidate set. A candidate context set is generated based on the semantic candidate set and the evolutionary candidate set. The candidate context set is input into a large language model to obtain a performance impact score. Based on the semantic similarity, evolutionary association strength, and performance impact score of multiple fourth test cases, the multiple fourth test cases are sorted to obtain the test case priority sequence of the code under test. This achieves the accuracy of the test case priority sequence determination and improves the efficiency of testing.

[0073] Based on the above embodiments, optionally, the semantic candidate set acquisition module 310 is used to: adopt the strategy of overlapping part sliding window to divide the difference change content into multiple content segments, input each content segment into the code bidirectional encoder model, obtain the content vector corresponding to each content segment, and perform average pooling on the multiple content vectors corresponding to the code under test to obtain the semantic feature vector of the code under test.

[0074] Based on the above embodiments, optionally, the evolution candidate set acquisition module 320 is used to: construct a weighted bipartite graph between the code under test and the first test case according to the co-occurrence records, and determine the evolutionary association strength between the code file and the test case according to the weighted bipartite graph. The weighted bipartite graph includes test nodes, nodes under test, and association edges between the test nodes and the nodes under test. The test nodes are used to indicate the first test case, the nodes under test are used to indicate the code under test, and the association edges are used to indicate that the test nodes and the nodes under test co-occur in the same version. Each association edge is associated with a corresponding version number.

[0075] Based on the above embodiments, optionally, the evolutionary candidate set acquisition module 320 is further configured to: count the number of associated edges between each test node and each tested node in the weighted bipartite graph, divide the number of associated edges by the sum of the number of associated edges and the smoothing parameter to obtain a co-occurrence score; find the shortest path between each test node and each tested node in the weighted bipartite graph, and use the reciprocal of the number of edges of the shortest path as the topological distance score; for each test node and the tested node with associated edges, calculate the absolute value of the difference between the version number corresponding to each associated edge and the current version number, add one to the average of all absolute values ​​and take the reciprocal to obtain a time decay score; and sum the co-occurrence score, the topological distance score and the time decay score according to their respective adjustable weight coefficients to obtain the evolutionary association strength.

[0076] Based on the above embodiments, optionally, the co-occurrence record includes multiple co-occurrence events, and each co-occurrence event record includes the version number where the co-occurrence occurred, the identifier of the modified code file in that version, and the identifier of the test case in that version where performance changes occurred.

[0077] Based on the above embodiments, optionally, the evolution candidate set acquisition module 320 is further configured to: for each co-occurrence event, if there is no node of the first test case corresponding to the co-occurrence event in the weighted bipartite graph, create a test node corresponding to the first test case; if there is no node of the code under test corresponding to the co-occurrence event in the weighted bipartite graph, create a test node corresponding to the code under test; and establish an association edge between the test node and the test node, and record the version number in the co-occurrence event as the version number of the association edge; wherein, when the same test node and the same test node co-occur multiple times in different historical versions, multiple association edges are established, and each association edge records its corresponding version number.

[0078] Based on the above embodiments, optionally, the performance impact score acquisition module 330 is used to: calculate the intersection of the semantic candidate set and the evolutionary candidate set; when the number of intersections is less than a preset number, select supplementary results from the remaining results of the semantic candidate set according to semantic similarity from high to low; if it is still insufficient, select supplementary results from the remaining results of the evolutionary candidate set according to evolutionary association strength from high to low, so that the number of candidate test cases reaches the preset number, and generate the candidate context set.

[0079] Based on the above embodiments, optionally, the performance impact score acquisition module 330 is used to: input the code implementation of each of the fourth test cases in the candidate context set, the difference change content of the modified code file, and the chain reasoning prompt template into the large language model, so that the large language model outputs a performance impact score between zero and one according to the chain reasoning method.

[0080] Based on the above embodiments, optionally, the test case priority sequence determination module 340 is used to: obtain the number of function calls during the execution of each fourth test case as a coverage feature, and obtain the performance fluctuation range of each fourth test case in historical versions as a historical performance stability feature; input the normalized semantic similarity, evolutionary association strength, performance impact score, coverage feature and historical performance stability feature into a random forest model, and have the random forest model output the probability score of performance change for each fourth test case, and arrange them in descending order of probability score to generate the test case priority sequence.

[0081] Optionally, based on the above embodiments, the device further includes a query vector acquisition module, configured to: for each of the first test cases in the test suite, acquire the method signature, method body, and function call signature list of the first test case, concatenate the method signature, method body, and function call signature list, and input them into the bidirectional code encoder model to obtain the query vector of the first test case.

[0082] The above-described products can perform the methods provided in any embodiment of the present invention, and have the corresponding functional modules and beneficial effects for performing the methods.

[0083] Figure 5 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0084] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0085] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0086] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the method for determining test cases.

[0087] In some embodiments, the test case determination method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the test case determination method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the test case determination method by any other suitable means (e.g., by means of firmware).

[0088] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0089] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0090] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0091] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0092] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0093] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0094] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 19, or installed from storage unit 18, or installed from ROM 12. When the computer program is executed by processor 11, it performs the functions defined in the methods of the embodiments of the present invention.

[0095] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0096] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for determining test cases, characterized in that, include: Obtain the difference change content corresponding to the code under test, determine the semantic feature vector of the code under test based on the difference change content, and select multiple second test cases from multiple first test cases based on the semantic similarity between the semantic feature vector and the query vector of multiple first test cases in the test suite to form a semantic candidate set. Obtain common occurrence records of modified code files and corresponding first test cases with performance changes in historical versions of the code under test. Determine the evolutionary association strength between each modified code file and each test case based on the common occurrence records. Select multiple third test cases from multiple first test cases based on the evolutionary association strength to form an evolutionary candidate set. A candidate context set is generated based on the semantic candidate set and the evolutionary candidate set. The candidate context set is then input into a large language model to obtain a performance impact score. The candidate context set includes multiple fourth test cases. Based on the semantic similarity, evolutionary association strength, and performance impact score of the multiple fourth test cases, the multiple fourth test cases are sorted to obtain the test case priority sequence of the code under test.

2. The method for determining test cases according to claim 1, characterized in that, Determining the semantic feature vector of the tested code based on the differences in content includes: Using an overlapping sliding window strategy, the differentially changed content is divided into multiple content segments. Each content segment is input into a bidirectional encoder model to obtain a content vector corresponding to each content segment. Average pooling is then performed on the multiple content vectors corresponding to the code under test to obtain the semantic feature vector of the code under test.

3. The method for determining test cases according to claim 1, characterized in that, The evolutionary correlation strength between each modified code file and each test case is determined based on the co-occurrence records, including: A weighted bipartite graph is constructed based on the co-occurrence records between the code under test and the first test case. The evolutionary association strength between the code file and the test case is determined based on the weighted bipartite graph. The weighted bipartite graph includes test nodes, nodes under test, and association edges between the test nodes and the nodes under test. The test nodes are used to indicate the first test case, the nodes under test are used to indicate the code under test, and the association edges are used to indicate that the test nodes and the nodes under test co-occur in the same version. Each association edge is associated with a corresponding version number.

4. The method for determining test cases according to claim 3, characterized in that, The step of determining the evolutionary correlation strength between code files and test cases based on the weighted bipartite graph includes: The number of associated edges between each test node and each tested node in the weighted bipartite graph is counted, and the number of associated edges is divided by the sum of the number of associated edges and the smoothing parameter to obtain the co-occurrence score. Find the shortest path between each test node and each node under test in the weighted bipartite graph, and use the reciprocal of the number of edges of the shortest path as the topological distance score. For the test node and the tested node with associated edges, calculate the absolute value of the difference between the version number corresponding to each associated edge and the current version number, add one to the average of all absolute values ​​and take the reciprocal to obtain the time decay score; The evolutionary association strength is obtained by weighting and summing the co-occurrence score, topological distance score, and time decay score according to their respective adjustable weight coefficients.

5. The method for determining test cases according to claim 3, characterized in that, The co-occurrence record contains multiple co-occurrence events. Each co-occurrence event record includes the version number where the co-occurrence occurred, the identifier of the modified code file in that version, and the identifier of the test case where the performance change occurred in that version. The step of constructing a weighted bipartite graph between the code under test and the first test case based on the co-occurrence records includes: For each co-occurrence event, if there is no node in the weighted bipartite graph corresponding to the first test case for the co-occurrence event, a test node corresponding to the first test case is created. If there is no node in the weighted bipartite graph corresponding to the code under test for the co-occurrence event, a test node corresponding to the code under test is created. Furthermore, an association edge is established between the test node and the test node, and the version number in the co-occurrence event is recorded as the version number of the association edge. When the same test node and the same test node co-occur multiple times in different historical versions, multiple association edges are established, and each association edge records its corresponding version number.

6. The method for determining test cases according to claim 1, characterized in that, The step of generating a candidate context set based on the semantic candidate set and the evolutionary candidate set includes: Calculate the intersection of the semantic candidate set and the evolution candidate set; When the number of intersections is less than the preset number, supplementary results are selected from the remaining results of the semantic candidate set according to semantic similarity from high to low. If the number of candidate test cases is still insufficient, the remaining results of the evolutionary candidate set are selected from high to low according to the evolutionary association strength to supplement the candidate test cases, so that the number of candidate test cases reaches the preset number, and the candidate context set is generated.

7. The method for determining test cases according to claim 1, characterized in that, The step of inputting the candidate context set into the large language model to obtain a performance impact score includes: The code implementation of each of the fourth test cases in the candidate context set, the difference changes in the modified code file, and the chain reasoning prompt template are input into the large language model so that the large language model outputs a performance impact score between zero and one according to the chain reasoning method.

8. The method for determining test cases according to claim 1, characterized in that, The step of sorting the multiple fourth test cases according to their semantic similarity, evolutionary association strength, and performance impact score to obtain a test case priority sequence for the code under test includes: The number of function calls during the execution of each fourth test case is obtained as a coverage feature, and the performance fluctuation of each fourth test case in historical versions is obtained as a historical performance stability feature. The semantic similarity, evolutionary association strength, performance impact score, coverage feature, and historical performance stability feature are normalized and then input into a random forest model. The random forest model outputs the probability score of performance change for each of the fourth test cases. The test cases are then sorted in descending order of probability score to generate the test case priority sequence.

9. The method for determining test cases according to claim 1, characterized in that, Also includes: For each of the first test cases in the test suite, obtain the method signature, method body, and function call signature list of the first test case. Concatenate the method signature, method body, and function call signature list and input them into the code bidirectional encoder model to obtain the query vector of the first test case.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for determining test cases as described in any one of claims 1-9.

Images