CUDA (Compute Unified Device Architecture) compiler-oriented test failure case duplicate removal method

Abstract

The invention provides a CUDA (Compute Unified Device Architecture) compiler-oriented test failure case duplicate removal method, and belongs to the technical field of CUDA compiler testing. In order to solve the problem that a large number of redundant test failure cases are caused by fuzzy testing of the CUDA compiler, code features of the test failure cases are extracted through a deep learning model, feature vectors are generated, runtime information of the test failure cases is collected based on an executable file of the CUDA compiler, and runtime feature vectors of the test failure cases are generated; the method comprises the steps of calculating a distance matrix of code dimensions of all test failure cases and a distance matrix of runtime dimensions, normalizing the two matrixes and generating a multi-dimension fused comprehensive distance matrix, and finally obtaining a priority sequence of test failure through a farthest point priority algorithm so as to realize duplicate removal of the test failure cases. According to the method, the test failure cases triggering the same CUDA compiler defects can be accurately identified, the repeated work of a development team is effectively reduced, and the debugging efficiency of the test failure cases is improved.

Description

Technical Field

[0001] This invention belongs to the field of CUDA compiler testing technology, specifically a method for deduplicating failed test cases for CUDA compilers. Background Technology

[0002] CUDA (Compute Unified Device Architecture) compilers, such as NVIDIA NVCC, are core tools for compiling CUDA code into executable instructions for GPUs (Graphics Processing Units). They are also the core infrastructure of the CPU-GPU heterogeneous computing ecosystem, and their correctness directly affects the program stability and computational efficiency in key areas such as high-performance computing, deep learning training, and scientific simulation.

[0003] Fuzzing is currently the mainstream technique for testing CUDA compilers, generating massive numbers of test cases and inputting them into the compiler to expose compiler defects. However, this introduces new problems: failed test cases generated by fuzzing may repeatedly trigger the same CUDA compiler defects. Furthermore, the distribution of these redundant failed test cases across unique bugs is extremely uneven, making manual analysis of these test cases inefficient and severely hindering the CUDA compiler defect fixing process. Therefore, there is an urgent need for effective techniques to deduplicatize failed CUDA compiler test cases to solve this problem.

[0004] Previously, many researchers have proposed bug deduplication methods for C / C++ compilers such as LLVM and GCC, but research on bug deduplication for CUDA compilers remains limited. Although GPU parallel programming architectures such as CUDA share some syntactic similarities with C / C++ programming, the thread and memory management requirements of the multi-core GPU hardware structure significantly increase the difficulty of GPU parallel programming. Therefore, bug deduplication techniques from C / C++ compilers cannot be simply transferred to research on bug deduplication for CUDA compilers.

[0005] Compilers are highly complex software systems involving multi-stage processing, intricate optimization logic, and support for various hardware and language features. Defects in compilers have diverse causes and characteristics. Against this backdrop, feature extraction has become a crucial tool for understanding test failures and the compiler defects they trigger. Summary of the Invention

[0006] Currently, there is no readily available technology to deduplicat test cases caused by fuzz testing of the CUDA compiler, which greatly hinders the further optimization and development of the CUDA compiler. To address this issue, this invention proposes a method for deduplicating test cases in the CUDA compiler. Based on multi-dimensional feature extraction and feature similarity calculation, it deduplicates test cases in the CUDA compiler, aiming to fundamentally solve the aforementioned problem of redundant test cases.

[0007] This invention provides a method for deduplicating test failure cases for CUDA compilers, comprising the following steps:

[0008] Step 1: Preprocess the code of the test failure cases. Preprocessing includes code trimming, comment removal, and blank line removal, etc., to transform the original test failure case code into a minimal code program that can trigger the same compiler defect.

[0009] Step 2: Use a deep learning model to embed code into the preprocessed test failure cases and extract code features;

[0010] Step 3: Use cosine similarity to measure the similarity of code features of different failed test cases, and generate a code dimension distance matrix. The number of rows and columns of the matrix are the total number of failed test cases, and the elements in the matrix correspond to the distance between two failed test cases.

[0011] Step 4: Use a software dynamic binary instrumentation tool to collect runtime coverage information of failed test cases in the CUDA compiler;

[0012] Step 5: Parse and process the runtime coverage information of the failed test cases obtained in Step 4 to generate runtime feature vectors for the failed test cases;

[0013] The runtime coverage information obtained in step four is stored in binary format. The runtime coverage information processing in this step first parses it, then filters out the basic block coverage information from the executable file of the compiler under test, counts the number of times the basic block is executed during the compilation process of the current failed test case, and finally stores the above information in CSV file format. After processing the runtime coverage information of all failed test cases in the compiler under test, the following SBFL formula is used to calculate the suspicion degree of the basic blocks covered by the runtime of each failed test case, and the basic blocks are calculated. The suspiciousness score is :

[0014] ;

[0015] in For basic blocks The number of times it is covered by the currently failed test case; For basic blocks The number of times a test case is covered by all test cases that pass. For basic blocks The number of times the test case is covered by all failed test cases. The basic block identifier and suspicion score are stored as a CSV file.

[0016] Finally, based on the following rules, a runtime feature vector is generated for each failed test case, with the dimension being the total number of basic blocks covered by the compiler under test at runtime for all failed test cases. The index number of the feature vector The corresponding basic block is , For the basic blocks in the current failed test case Suspicion score; elements of the runtime feature vector for:

[0017] ;

[0018] Step 6: Calculate the distance between the runtime feature vectors of different failed test cases and generate a runtime dimension distance matrix. The number of rows and columns of this matrix is ​​the total number of failed test cases, and the elements in the matrix correspond to the distance between two failed test cases in the runtime dimension.

[0019] Step 7: Normalize the code dimension distance matrix and the runtime dimension distance matrix and generate a multi-dimensional fused comprehensive distance matrix;

[0020] Step 8: Randomly apply the farthest point priority algorithm to the obtained comprehensive distance matrix to obtain the priority sequence of test failure cases.

[0021] The advantages and positive effects of this invention are as follows:

[0022] 1) This invention proposes a method for deduplicating test failure cases for CUDA compilers. It proposes a test failure code feature extraction technique based on a deep learning model, which is used to efficiently extract the syntactic and semantic features of test failure case code and use them for test failure case similarity measurement. Compared with manual feature engineering, it is simple and convenient to use, easy to understand, and can capture more test failure case code features at a lower cost.

[0023] 2) This invention presents a method for deduplicating test failure cases for CUDA compilers. It proposes a runtime feature extraction technique for test failures based on the CUDA compiler executable file, enabling the collection of runtime information for test failure cases without requiring access to the compiler's source code. Theoretically, this technique can be transferred to any compiler to extract runtime information from its test failure cases, demonstrating strong generalization and scalability.

[0024] 3) This invention provides a method for deduplicating test failure cases for CUDA compilers. It measures the similarity of test failure cases based on the code and runtime features of the test failure cases. This multi-dimensional feature extraction can more comprehensively describe test failure cases. Compared with single-dimensional feature extraction, it is more sensitive to the identification of duplicate test failure cases and significantly improves the debugging efficiency of test failure cases.

[0025] 4) This invention provides a method for deduplicating test failure cases for CUDA compilers. Experimental results effectively demonstrate its effectiveness and robustness. On multiple datasets, this method can improve the debugging efficiency of compiler defects by more than 60%, greatly reducing the repetitive work of the CUDA compiler development team.

[0026] 5) This invention provides a method for deduplicating test failure cases for CUDA compilers. The code feature extraction and runtime feature extraction of test failure cases are loosely coupled with the CUDA compiler and CUDA test failure cases in design. It has strong generalization ability and can be applied to the deduplication problem of test failure cases for compilers of different programming languages. It provides a feasible solution to the problem of redundant test failure cases caused by fuzz testing of various compilers. Attached Figure Description

[0027] Figure 1 This is a flowchart of a method for deduplicating test failure cases for CUDA compilers according to the present invention;

[0028] Figure 2 This is a flowchart of the present invention for extracting test failure case features based on dynamic binary instrumentation;

[0029] Figure 3 This is a basic block overwrite log example generated by DynamoRIO. Detailed Implementation

[0030] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0031] To address the issue of redundant test failure cases arising from fuzz testing of the CUDA compiler, this invention proposes a method for deduplicating test failure cases in the CUDA compiler. The core of this method involves using existing deep learning models and binary dynamic instrumentation tools to extract and encode multi-dimensional features of test failure cases. A distance matrix is ​​then calculated based on the multi-dimensional feature vectors of the test failure cases. Finally, a priority sequence of test failure cases is obtained using a farthest-point-first algorithm, thereby indirectly deduplicating test failure cases in the CUDA compiler. This invention can accurately identify test failure cases that trigger the same CUDA compiler defects, thereby reducing repetitive work for the CUDA compiler development team, improving the debugging efficiency of test failure cases, and providing insights for the improvement and development of the CUDA compiler.

[0032] This invention extracts features from test failure cases from multiple dimensions to analyze and identify compiler defects triggered by these failures. Compared to single-dimensional features, multi-dimensional features can more comprehensively describe test failures and more accurately reveal their underlying causes. Therefore, systematically extracting and analyzing these features, rather than single features, is crucial for identifying recurring bugs. These features may originate from the test failure cases themselves or from the CUDA compiler under test. For the features of the test cases themselves, traditional code feature extraction techniques, such as Abstract Syntax Tree (AST) recognition and keyword statistics, require manually designing specific rules to extract features, often only capturing the syntactic structure information of the code while losing its semantic information. In contrast, embedding test failures into the code using deep learning models is a better choice, as it can capture both syntactic and semantic information simultaneously without requiring additional manual feature engineering. For the runtime features of test failure cases during test execution—that is, the coverage information of test failure cases during CUDA compiler compilation—common static instrumentation techniques are not applicable. This is because the CUDA compiler is not open source and its source code is unavailable. Therefore, using runtime feature extraction techniques based on software dynamic binary instrumentation is a better choice.

[0033] The method for deduplicating failed test cases for CUDA compilers according to embodiments of the present invention, such as... Figure 1 As shown, ① to ⑧ correspond to the following steps, and the specific steps and explanations are as follows:

[0034] Step 1: Preprocess the code of the test failure cases. Preprocessing includes code trimming, comment removal, and blank line removal, aiming to transform the original test failure case code into minimal test failure code that can trigger the same compiler defect.

[0035] On the one hand, to improve test coverage, the test cases generated by the fuzzer are mostly complex, with code lengths exceeding a thousand lines. However, only a small, crucial code snippet can actually trigger a compiler defect, leading to test failure. On the other hand, the input length of deep learning models is limited to 512 bytes to several kilobytes, and excessively long code in failed test cases can result in inaccurate code feature extraction. Therefore, this step first uses a code trimmer to trim the code of each failed test case generated by the fuzzer, simplifying it into the minimum program that can trigger a CUDA compiler defect. Then, Python regular expressions are used to eliminate comments and blank lines in the code, removing as much of the code as possible that is irrelevant to triggering the compiler defect.

[0036] This step is optional. If the original test failure is already very brief and concise, the preprocessing step can be skipped and the subsequent steps can be performed directly.

[0037] Step 2: Use a deep learning model to embed code into the preprocessed test failure cases and extract code features.

[0038] This step uses the Jina-embedding deep learning model, which is specifically designed for code embedding, to extract code-dimensional features from the preprocessed test failures. Jina-embedding is a small model based on the Transformer architecture for natural language and code embedding. It supports English and 30 widely used programming languages, and also supports a maximum sequence length of 8192. In most cases, it can completely embed 80-100 lines of CUDA C / C++ code.

[0039] Step 3: Use cosine similarity to measure the similarity of the features of the failed test code and generate a code-dimensional distance matrix.

[0040] Code embedding typically converts code snippets into vectors, whose directions convey semantic information about the code. Cosine similarity measures the cosine of the angle between two vectors, focusing more on direction than length. In code scenarios, even if two code snippets differ in length, if they are semantically similar, their corresponding vector directions will be similar, and cosine similarity accurately captures this semantic similarity. For example, two functions that perform the same functionality but have different line counts may have similar embedding vector directions; cosine similarity can effectively determine their semantic similarity.

[0041] In addition, cosine similarity calculation is relatively simple, and its calculation still has good stability and efficiency even in high-dimensional spaces, making it very suitable for computational scenarios involving code embedding.

[0042] Let the code features of two failed test cases be represented as vectors. and The vectors all contain 1 eigenvalue, For positive integers, this step uses cosine similarity to measure the similarity of the test failure code features, and the formula is as follows:

[0043]

[0044] in, They are vectors The Middle Each feature value. It represents the angle between two vectors. It is a vector and dot product, and They are vectors and The norm of the vector, i.e., the length of the vector, and by Calculate the distance between test cases to obtain the code dimension distance matrix M between them, with the total number of failed test cases as the dimension.

[0045] Step 4: Use a software dynamic binary instrumentation tool to collect runtime coverage information of failed test cases in the CUDA compiler.

[0046] Use binary instrumentation tools to collect runtime information from failed test cases. Binary instrumentation tools can capture basic block coverage information of all processes during compilation and generate binary log files on a process-by-process basis. This step only needs to collect runtime information of the CUDA compiler executable program from the failed test cases.

[0047] This step uses dynamic binary instrumentation to collect runtime coverage information for failed test cases during the compilation process of the CUDA compiler. The reasons are as follows: 1. CUDA compilers are not open-source, meaning it's difficult to collect line-by-line runtime coverage information using common static instrumentation methods; 2. Dynamic binary instrumentation collects coverage information on a block-by-block basis. A block is a sequentially executed sequence of instructions without any branches. Each failed test case has only two clear states for a given block: execution or non-execution. This makes it ideal for statistical analysis of coverage information instead of lines of code.

[0048] This step ultimately uses DynamoRIO to collect runtime information about test failures. DynamoRIO is an open-source dynamic binary instrumentation tool that supports multiple operating systems and hardware architectures and has excellent cross-platform compatibility.

[0049] Step 5: Parse and process the runtime coverage information of the failed test cases to generate runtime feature vectors for the failed test cases.

[0050] In step four, the coverage information collected by DynamoRIO is stored in binary format. This step first parses it, converting the binary basic block table into human-readable text. Each basic block belongs to a specific module, i.e., an independent code unit, such as an executable file or a dynamic link library. Next, the basic blocks covered by all failed test executions are filtered according to module, recording the basic block information belonging to the target CUDA compiler executable file under test and generating a list. This list belongs to the current set of failed test cases. Then, the number of times each basic block in the basic block list is executed during the compilation process of the current failed test case is counted, i.e., the number of times the basic block is covered by the current failed test case.

[0051] After preprocessing, the suspicion level of all basic blocks covered by the runtime of failed test cases is calculated using the following SBFL formula, and the basic block identifiers and scores are stored in a CSV file, where For basic blocks The number of times it is covered by the currently failed test case; For basic blocks The number of times a test case is covered by all test cases that pass. For basic blocks The number of times it is covered by all failed test cases. It is a basic block The degree of suspicion.

[0052]

[0053] Finally, a runtime feature vector with the total number of basic blocks as the dimension is generated for each failed test case based on the following rules, where The index number of the feature vector , For the present The corresponding basic blocks, The suspiciousness number of the current basic block indicates the degree of suspicion or importance of the current basic block triggering a compiler bug. The elements of the runtime feature vector are... for:

[0054]

[0055] As can be seen from the SBFL formula, calculating the suspiciousness of a basic block requires runtime information from both the failed test case set and the successful test case set. Considering the basic block coverage of both sets allows for a more impartial assessment of basic block suspiciousness. The methods for collecting and statistically analyzing runtime information from the successful test case set are exactly the same as those for the failed test case set.

[0056] After processing the basic block coverage information of the test failure and test success test case sets, the suspicion score of the basic blocks in all test failure sets can be calculated according to formulas (2) and (3), and a runtime feature vector can be generated for each test failure test case. Figure 2 The above process has been described.

[0057] Step 6: Use Euclidean distance to measure the similarity of runtime features during test failures and generate a runtime dimensional distance matrix.

[0058] Euclidean distance is a metric based on geometric straight-line distance, suitable for low-dimensional, continuous data and scenarios with spatial geometric characteristics, such as calculating the distance between two geographical locations or spatial points in two-dimensional or three-dimensional coordinates. Euclidean distance is also a commonly used distance metric in many clustering algorithms (such as K-Means). During clustering, a data point is assigned to a cluster if its Euclidean distance to the cluster center is smaller. Therefore, this step uses Euclidean distance to calculate the similarity of runtime feature vectors.

[0059] This step uses the Euclidean distance formula to calculate the distance between the runtime feature vectors of failed test cases. The specific formula is as follows:

[0060]

[0061] in dimensional vector and These are the runtime feature vectors of two failed test cases. The total number of basic blocks covered by the compiler under test when compiling all failed test cases.

[0062] Step 7: Normalize the code dimension distance matrix and the runtime dimension distance matrix and generate a multi-dimensional combined distance matrix;

[0063]

[0064] This step uses Min-Max normalization for normalization before fusing the two distance matrices, and the formula is shown above. These are sample values ​​from the original data. It is the minimum value in the original data. It is the maximum value in the original data. These are the normalized data values. Min-Max, a commonly used data normalization method, focuses on linearly transforming data to a specific interval. For each distance matrix, by calculating each element in the matrix according to the Min-Max normalization formula, the data in the original value range is mapped to the interval [0,1], thereby eliminating the influence of different units on the distance metric and making the data comparable at the same scale.

[0065] After normalizing the two matrices, the corresponding elements of the normalized code-dimensional distance matrix and the runtime distance matrix are added one by one to obtain a multi-dimensional fused feature matrix. This addition process is not a simple numerical superposition, but rather a fusion of the results of the test failure similarity measurement in their respective dimensions. In this way, a comprehensive distance matrix that integrates code-dimensional and runtime-dimensional information is successfully obtained, which comprehensively reflects the similarity of test failure cases across different dimensions.

[0066] Step 8: Apply the farthest point first algorithm multiple times to the combined test failure distance matrix to obtain the priority sequence of test failure cases.

[0067] This step applies the farthest point first algorithm, which randomly selects initial points, multiple times to the comprehensive distance matrix of the failed test cases generated earlier. The farthest point first algorithm is an effective strategy for filtering data points, and introducing random initial points allows the algorithm to sort the comprehensive distance matrix of failed test cases under different starting conditions, thereby avoiding the limitations of initial point selection that could affect the comprehensiveness and objectivity of the results.

[0068] After each application of the algorithm, a priority sequence of failed test cases is obtained. This step then statistically analyzes the priority sequences obtained from multiple calculations, focusing on those failed test cases that consistently rank high in the priority sequence. These consistently high-ranking test failures are highly likely to represent more representative and critical test failures. Based on this statistically derived subset of consistently high-ranking test failures, a Top-N sequence is generated as the final output of this method.

[0069] Example:

[0070] This embodiment focuses on applying a method for deduplicating test failure cases for the CUDA compiler. It processes the actual collected CUDA compiler test failure cases, demonstrating the entire process from dataset input to deduplication result output, and verifying the effectiveness and feasibility of the method.

[0071] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described in complete and clear form below:

[0072] Test environment setup:

[0073] Hardware environment: A server with a 12th Gen Intel(R) Core(TM) i7-12700 CPU, 32GB of memory, and an NVIDIA GeForce RTX 3090Ti GPU was selected for the experiment.

[0074] Software environment: Ubuntu 18.04.6 LTS system, with NVIDIA CUDA compiler NVCC-12.4, CUDA compiler M-1.0.0, Python-3.8 and DynamoRIO-10.0.0 installed.

[0075] Input dataset:

[0076] In this embodiment, CUDASmith was used as a fuzzer to perform fuzz testing on the M compiler. A total of 1,144 test cases were collected that could trigger test failures of compiler defects, including 11 compiler defects. The mapping between these test failure test cases and compiler defects is extremely uneven.

[0077] To ensure the successful generation of runtime feature vectors for test failures in step five, this embodiment also generates 1000 test cases that can pass the M compiler test using CUDASmith.

[0078] Step 1: Preprocess failed test cases:

[0079] This step uses the CUDA Reducer from the CUDASmith project to trim the code of 1144 original failed test cases. At the same time, it uses the regular expression library in Python to remove comments and blank lines from the failed test case code, reducing the amount of code for all failed tests to less than 100 lines.

[0080] Step 2: Use a deep learning model to embed code into the preprocessed test failure cases and extract code features:

[0081] This step uses jina-embeddings-v2-base-code to extract features from the code dimensions of the failed tests. Calling the model's encode() function will give the embedded code of the failed tests, which has a dimension of 768. All embedded code of the failed tests will be stored in the corresponding npy file.

[0082] Step 3: Use cosine similarity to measure the similarity of the features of the failed test code, and generate a code-dimensional distance matrix:

[0083] This step reads the npy files generated in step two sequentially, calculates the similarity of each pair of failed test cases using cosine similarity, and then... The corresponding distances are obtained, and finally a 1144*1144 dimensional symmetric distance matrix of the test failure code dimension is generated.

[0084] Step 4: Use a software dynamic binary instrumentation tool to collect runtime coverage information in the CUDA compiler for test failures:

[0085] This step uses DynamoRIO-10.0.0 (https: / / dynamorio.org / ) as a dynamic binary instrumentation tool to collect coverage information from the M compiler for compilation processes that fail during testing. DynamoRIO generates binary coverage information files by default to save space, and will generate different log files based on the different processes involved in the compilation process, named according to their program name and process ID.

[0086] At the same time, runtime coverage information was collected from 1,000 passed tests using the same method.

[0087] Step 5: Parse and process the runtime coverage information of failed tests to generate runtime feature vectors for failed tests:

[0088] This step first parses the binary log file generated by DynamoRIO to obtain a human-readable basic block table, as shown below. Figure 3 The basic block overlay log is shown, where the basic block is uniquely identified by its module and module offset address.

[0089] Figure 3 The format of a drcov file is shown, which mainly includes the following three parts:

[0090] 1) Lines 1-2: DRCOV version and style. DRCOV VERSION indicates the version number of the tool, and DRCOV FLAVOR indicates the specific version style of the tool, such as drcov-64.

[0091] 2) Lines 3-10 display the Module Table, which lists information about all modules in the program. Each module is represented by a single row of data. Each row includes the following fields:

[0092] a) Module ID (id): The unique identifier of the module.

[0093] b) Contains module ID (containing_id): If the module belongs to another module, it represents the ID of its parent module.

[0094] c) Start address: The starting address of the module in memory.

[0095] d) End address: The end address of the module in memory.

[0096] e) Entry address: The entry address of the module.

[0097] f) Offset: The offset when the module is loaded, usually 0.

[0098] g) Preferred base address: The preferred base address for loading the module.

[0099] h) Path: The file path or name of the module, such as gcc, libdrcov.so.

[0100] 3) Lines 11-17 display the Basic Block Table (BB Table), which is in binary format in the actual drcov file. Figure 3 To simplify, Figure 3 Line 11 shows that the log recorded 4340 executions of basic blocks. The basic block table lists information about the basic blocks within each module, with each basic block represented by a single row. Each row includes the following fields:

[0101] a) Module ID: The module ID to which this basic block belongs.

[0102] b) Start address: The starting address of the basic block in memory.

[0103] c) Size: The size of the basic block, in bytes.

[0104] Secondly, this step records all the basic blocks covered by the test failure test case set and the test success test case set, and generates a list of basic blocks for the test failure test case set and a list of basic blocks for the test success test case set. It also records the number of times each of them is covered in the corresponding test case set, and generates a separate list of basic blocks covered by each test failure test case.

[0105] Then, based on the basic blocks in the list of basic blocks in the test failure test case set and the number of times these basic blocks are covered in the test failure test case set and the test success test case set, the suspicious scores of these basic blocks are calculated by formula (2).

[0106] Finally, based on the list of basic blocks covered by each test case, the list of basic blocks of the test failure test case set, and the suspicious scores of each basic block, the runtime feature vector of each test failure test case is generated by formula (3). These runtime vectors are stored in the corresponding npy file, where each element of the feature vector represents one of the basic blocks covered by the current 1144 test failures.

[0107] Step 6: Use Euclidean distance to measure the similarity of runtime features of failed test cases, and generate a runtime dimensional distance matrix:

[0108] This step reads the npy files generated in step five in sequence, and uses Euclidean distance to calculate the distance between the features of the test failure runtime pairwise, finally generating a 1144*1144 dimensional test failure runtime dimension symmetric distance matrix.

[0109] Step 7: Normalize the code-dimensional distance matrix and the runtime-dimensional distance matrix, and generate a multi-dimensional combined distance matrix:

[0110] Formula (5) is used to normalize the feature vectors of the two dimensions respectively, and all element values ​​of the distance matrix are scaled to the [0,1] scale. The two matrices are then simply added together to obtain a comprehensive distance matrix that integrates the features of the two dimensions.

[0111] Step 8: Apply the combined test failure distance matrix multiple times using the farthest point first algorithm to obtain the test failure priority sequence.

[0112] Applying the farthest point priority algorithm with random initial point selection to the distance matrix yields the complete test failure priority sequence.

[0113] Apply the farthest point priority algorithm 20 times to the distance matrix. Statistically, the tests that are in the top 10% of the priority queue with a 90% probability fail in 20 tests, and output them as the Top-N sequence.

[0114] Example Result Analysis:

[0115] For an input dataset containing 1144 test failures, after implementing this method, the number of test failures that actually need in-depth analysis is reduced to 190 according to the complete priority sequence output by this method, improving the efficiency of finding CUDA compiler defects by more than 80%. Debugging according to the Top-N sequence output by this method can find 7 compiler defects in the first 8 test failure cases, greatly reducing the repetitive work of CUDA compiler developers.

[0116] This implementation verifies the feasibility and effectiveness of the invention in practical applications. It can efficiently extract key information from a large number of test failures, generate a guiding priority sequence, significantly reduce the workload of test failure case analysis, and greatly improve the debugging efficiency of CUDA compiler test failure cases.

[0117] Contents not described in detail in this specification are prior art known to those skilled in the art. It is hereby indicated that the above description is intended to help those skilled in the art understand this invention, but does not limit the scope of protection of this invention. Any equivalent substitutions, modifications, improvements, and / or simplifications of the above descriptions that do not depart from the essential content of this invention fall within the scope of protection of this invention.

Claims

1. A method for deduplicating failed test cases for CUDA compilers, characterized in that, Includes the following steps: Step 1: Preprocess the code of the failed test cases to transform the test case code into a minimal code program that can trigger the same compiler defect; Step 2: Use a deep learning model to embed code into the preprocessed test failure cases and extract code features; Step 3: Use cosine similarity to measure the similarity of the code features of the failed test cases and generate a code-dimensional distance matrix. The number of rows and columns of this matrix are the total number of failed test cases, and the elements in the matrix correspond to the distance between two failed test cases in the code dimension. Step 4: Use a software dynamic binary instrumentation tool to collect runtime coverage information of failed test cases in the CUDA compiler; Step 5: Parse and process the runtime coverage information of the failed test cases obtained in Step 4 to generate runtime feature vectors of test failures; The runtime coverage information of each failed test case is parsed, the basic blocks from the executable file of the compiler under test are filtered out, the number of times the basic blocks are executed in the compilation process of the current test case is counted, and the number of times the basic blocks are covered by the current test case is obtained. After obtaining the basic blocks covered by the compiler under test and the coverage count for all failed test cases, calculate the suspicion level for each failed test case's basic blocks covered by the compiler under test. Suspicion score for: ； in, For basic blocks The number of times it is covered by the currently failed test case; For basic blocks The number of times a test case is covered by all test cases that pass. For basic blocks The number of times it is covered by all failed test cases; For each failed test case, generate the total number of basic blocks covered by the compiler under test when all failed test cases are run. Let be the runtime feature vector of dimension , where The index number of the feature vector The corresponding basic block is , For the basic blocks in the current failed test case Suspicion score; elements of the runtime feature vector for: ； Step 6: Calculate the distance between the runtime feature vectors of different failed test cases and generate a runtime dimension distance matrix. The number of rows and columns of this matrix is ​​the total number of failed test cases, and the elements in the matrix correspond to the distance between two failed test cases in the runtime dimension. Step 7: Normalize the code dimension distance matrix and the runtime dimension distance matrix and generate a multi-dimensional fused comprehensive distance matrix; Step 8: Randomly apply the farthest point priority algorithm to the obtained comprehensive distance matrix to obtain the priority sequence of test failure cases.

2. The method for deduplicating test cases that fail to meet the requirements of the CUDA compiler as described in claim 1, characterized in that, in step one, if the length of the test case code that fails to meet the requirements of the input length of the deep learning model, a code trimmer is used to trim the code of the test case that fails to meet the requirements of the deep learning model, and the code is trimmed into the minimum code program that can trigger the same compiler defect. Then, Python regular expressions are used to eliminate comments and blank lines in the code.

3. The method for deduplicating failed test cases for CUDA compiler as described in claim 1, characterized in that, in step two, the deep learning model Jina-embedding is used for code feature extraction.

4. The method for deduplicating failed test cases for CUDA compilers as described in claim 1, characterized in that, in step three, the code features of two failed test cases are represented as vectors. and Calculate vector and The cosine similarity cos, with 1-cos as the vector. and The distance between the two corresponding failed test cases in the code dimension.

5. A method for deduplicating failed test cases for CUDA compilers as described in claim 1, characterized in that, in step six, the runtime feature vectors of the two failed test cases are respectively... and The distance between the runtime feature vectors of the two failed test cases is then... The calculation is as follows: 。 6. The method for deduplicating failed test cases for CUDA compiler as described in claim 1, characterized in that, in step seven, the code dimension distance matrix and the runtime dimension distance matrix are normalized using Min-Max respectively, and then the normalized code dimension distance matrix and the runtime distance matrix are added together to obtain the comprehensive distance matrix of feature fusion.

7. The method for deduplicating test failure cases for CUDA compiler as described in claim 1, characterized in that, in step eight, the farthest point first algorithm is applied to the synthesis distance matrix each time, an initial point is randomly selected to obtain a priority sequence of test failure cases, and after applying the farthest point first algorithm multiple times, all priority sequences are statistically analyzed to filter out the priority sequence of Top-N test failure cases.

Images