Automatic Generation Method of Software Configuration Item Test Cases Based on Large Language Model

Abstract

The present invention discloses an automatic generation method for software configuration item test cases based on large language models, which relates to the technical field of software testing. In order to solve the problems in the prior art that the flexibility and portability of the system are limited, it is difficult to adapt to the changing software requirements and designs, and manual intervention is still required to generate and parse test sequences; the automatic generation method for software configuration item test cases based on large language models not only improves the efficiency and accuracy of test case generation through intelligent text segmentation, prompt word construction and test case generation, but also ensures the pertinence and coverage of test cases, significantly reducing the workload of manually writing test cases; constructing a knowledge graph of software configuration items enhances the model's understanding of the relevance and similarity between texts and improves the quality of test case generation; through the real-time feedback and optimization mechanism, the automation level and test efficiency of software testing are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and particularly to a method for automatically generating test cases for software configuration items based on large language models. Background Art

[0002] With the continuous increase in the scale and complexity of software, the traditional manual test case writing method has been difficult to meet the requirements of efficient and comprehensive testing. For example, Chinese Patent No. CN104866423B discloses a test method and system for software configuration items. The system includes a network interface module, a CPU processing module, an FPGA module, a signal conversion module, an interface conversion module, and a storage module; the network interface module is used to receive test sequences sent by a test control terminal; the CPU processing module is used to receive test sequences, parse and schedule the test sequences, generate corresponding test instructions, and configure test interface parameters corresponding to the test instructions; the FPGA module generates configuration interface timing relationships and test data according to the test interface parameters; the signal conversion module is used to convert the interface timing relationships and test data into test excitation information matching the test equipment; the interface conversion module is used to feedback response information made by the device under test according to the test excitation information to the FPGA module; the storage module is used to store the response information.

[0003] Although the above patent improves the automation level of testing to a certain extent, it depends on modules such as the FPGA module and the signal conversion module, resulting in limited flexibility and portability of the system, being difficult to adapt to changing software requirements and designs, and still requiring manual intervention to generate and parse test sequences. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for automatically generating test cases for software configuration items based on large language models. Through intelligent text segmentation, prompt construction, and test case generation, it not only improves the efficiency and accuracy of test case generation, but also ensures the pertinence and coverage of test cases, significantly reducing the workload of manually writing test cases, so as to solve the problems raised in the above background art.

[0005] To achieve the above purpose, the present invention provides the following technical solutions:

[0006] A method for automatically generating test cases for software configuration items based on large language models includes the following steps:

[0007] Step 1: Set up the environment: Select a corresponding large language model according to the characteristics of the software configuration item and the test requirements, install and configure a framework for building applications based on large language models, and at the same time, install and configure associated components;

[0008] Step 2: Construct Prompt: Based on the framework and associated components, establish a software project document library containing requirement documents and design documents related to software configuration items. Split the requirement documents and design documents obtained from the software project document library into multiple small text fragments to generate a small text list;

[0009] Meanwhile, store and memorize the context information of the small text fragments, parse the content of the small text list, and construct a prompt;

[0010] Step 3: Test Case Generation: Pass the constructed prompt to a large language model to generate test cases for software configuration items, and in real time feedback the generated test cases to testers through the framework;

[0011] Step 4: Test Case Evaluation and Optimization: Save the generated test cases in a predetermined format. Meanwhile, establish a test case library to classify, annotate, and optimize the test cases.

[0012] Further, the environment setup is specifically as follows:

[0013] Analyze software configuration items: Analyze software configuration items to determine key information about software function features, performance indicators, and interface design;

[0014] Select a large prediction model: Based on the key information of software function features, performance indicators, and interface design, determine the required natural language processing capabilities, and combine with the requirements for generating test cases to select the corresponding large language model;

[0015] Install and configure the framework: Create a Python virtual environment, install the application development library that supports the large language model in this environment, and configure the large language model parameters;

[0016] Configure associated components: Install and configure associated components, including a text splitter, a file format parser, a requirement document analysis tool, and a design document visualization tool, and after configuration, test each associated component.

[0017] Further, the environment setup also includes training the selected large language model, and the specific steps are as follows:

[0018] Data processing: Collect the text data of software configuration items, and identify the text data to screen out the target text data that meets the test requirements;

[0019] Feature extraction: Extract text features from the screened target text data, and generate a text feature dataset for training based on the extraction results;

[0020] Model training: Train a large language model based on a text feature dataset to optimize the large language model's ability to understand and generate software configuration item-related texts;

[0021] Model output: Verify the trained large language model, determine whether the performance of the trained large language model meets the predetermined standard according to the verification result, and output the large language model that meets the predetermined standard.

[0022] Furthermore, the data processing also includes constructing a knowledge graph of software configuration items, specifically:

[0023] Define the specific instances of software configuration items as key nodes in the software configuration item knowledge graph, match the specific instances with existing software libraries, and input the detailed data of software configuration items into the software libraries based on the matching results;

[0024] Draw a software configuration item knowledge graph based on the detailed data of the software library, and perform entity alignment between the software configuration item knowledge graphs;

[0025] Train the large language model based on the information in the software configuration item knowledge graph.

[0026] Furthermore, constructing the prompt words in step two also includes: Classify the requirement documents and design documents collected from the software project document library according to document types and relevance, and preprocess the classified requirement documents and design documents.

[0027] Furthermore, in step two, the requirement documents and design documents obtained from the software project document library are segmented into multiple small text fragments, specifically including:

[0028] Load the requirement documents and design documents into a file format parser to parse the document content and determine the document structures of the requirement documents and design documents;

[0029] Identify the key information in the requirement documents and design documents based on the document structures, and set the maximum text fragment length;

[0030] Use the natural paragraphs in the requirement documents and design documents as the preliminary segmentation units, obtain the content lengths of each paragraph, and if it exceeds the set maximum text fragment length, segment the natural paragraph into sentences that meet the maximum text fragment length and maintain the integrity of the sentences;

[0031] Set a context window at each segmentation point, including relevant sentences and words before and after, where the segmentation point avoids being in the middle of the key information;

[0032] Generate multiple small text segments based on the segmentation result, assign a unique number to each small text segment, and arrange each small text segment according to the unique number to generate a small text list.

[0033] Furthermore, set the maximum text segment length, including:

[0034] Extract the key information from the requirement document and the design document;

[0035] Extract the keyword phrases contained in each key information from each key information;

[0036] Extract the number of characters corresponding to each keyword phrase and the frequency of each keyword phrase appearing in the requirement document and the design document;

[0037] Retrieve the keyword phrases whose frequencies of appearance in the requirement document and the design document are lower than the preset first frequency threshold as the first keyword data;

[0038] Obtain the first setting coefficient by using the number of characters of the keyword phrases contained in the first keyword data and their corresponding frequencies of appearance in the requirement document and the design document;

[0039] Among them, the first setting coefficient is obtained through the following formula:

[0040]

[0041] Among them, J 01 represents the first setting coefficient; n represents the total number of keyword phrases contained in the first keyword data; L 01i represents the number of characters of the i-th keyword phrase in the first keyword data; L 01b represents the standard deviation of the number of characters of the n keyword phrases in the first keyword data; R 01 represents the preset first reference coefficient, and the value range of the first reference coefficient is 1.37 - 1.48; f 01i represents the frequency of appearance of the i-th keyword phrase in the first keyword data in the requirement document and the design document; f 01c represents the first frequency threshold; f 01b represents the standard deviation of the frequencies of appearance of the n keyword phrases in the first keyword data in the requirement document and the design document;

[0042] Retrieve the keyword phrases whose frequencies of appearance in the requirement document and the design document exceed the preset second frequency threshold as the second keyword data;

[0043] Obtain the second setting coefficient by using the number of characters of the keyword phrases contained in the second keyword data and their corresponding frequencies of appearance in the requirement document and the design document;

[0044] Among them, the second setting coefficient is obtained through the following formula:

[0045]

[0046] Among them, J 02 represents the second setting coefficient; m represents the total number of key words included in the second key word data; L 02i represents the number of characters of the i-th key word in the second key word data; L 02b represents the standard deviation of the number of characters of the m key words in the second key word data; R 01 represents a preset first reference coefficient, and the value range of the first reference coefficient is 1.37 - 1.48; R 02 represents a preset second reference coefficient, and the value range of the second reference coefficient is 1.24 - 1.53; f 02i represents the frequency of occurrence of the i-th key word in the second key word data in the requirement document and the design document; f 02c represents the second frequency threshold; f 02b represents the standard deviation of the frequencies of occurrence of the m key words in the second key word data in the requirement document and the design document;

[0047] Set the maximum text segment length by using the first setting coefficient and the second setting coefficient.

[0048] Furthermore, setting the maximum text segment length by using the first setting coefficient and the second setting coefficient includes:

[0049] Extract the first setting coefficient and the second setting coefficient;

[0050] Compare the first setting coefficient with a preset first coefficient threshold;

[0051] Compare the second setting coefficient with a preset second coefficient threshold;

[0052] When the first setting coefficient exceeds the preset first coefficient threshold and the second setting coefficient exceeds the preset second coefficient threshold, then set the maximum text segment length by using the first setting coefficient and the second setting coefficient;

[0053] Among them, the maximum text segment length is obtained through the following formula:

[0054]

[0055] Among them, H max represents the maximum text segment length, and H max represents rounding up; J 02 represents the second setting coefficient; J01 represents the first setting coefficient; J 02c represents the second coefficient threshold; J 01c represents the first coefficient threshold; R 01 represents a preset first reference coefficient, and the value range of the first reference coefficient is 1.37 - 1.48; R 02 represents a preset second reference coefficient, and the value range of the second reference coefficient is 1.24 - 1.53; H c represents a preset initial text segment length;

[0056] When the first setting coefficient does not exceed the preset first coefficient threshold or the second setting coefficient does not exceed the preset second coefficient threshold, and when the first setting coefficient does not exceed the preset first coefficient threshold and the second setting coefficient does not exceed the preset second coefficient threshold, then use the keyword phrases that appear in the requirement document and the design document with a frequency not lower than the preset first frequency threshold but lower than the preset second frequency threshold to set the maximum text segment length.

[0057] Further, when the first setting coefficient does not exceed the preset first coefficient threshold or the second setting coefficient does not exceed the preset second coefficient threshold, and when the first setting coefficient does not exceed the preset first coefficient threshold and the second setting coefficient does not exceed the preset second coefficient threshold, then use the keyword phrases that appear in the requirement document and the design document with a frequency not lower than the preset first frequency threshold but lower than the preset second frequency threshold to set the maximum text segment length, including:

[0058] When the first setting coefficient does not exceed the preset first coefficient threshold or the second setting coefficient does not exceed the preset second coefficient threshold, and when the first setting coefficient does not exceed the preset first coefficient threshold and the second setting coefficient does not exceed the preset second coefficient threshold, then retrieve the keyword phrases that appear in the requirement document and the design document with a frequency not lower than the preset first frequency threshold but lower than the preset second frequency threshold as the third keyword data;

[0059] Use the number of characters of the keyword phrases included in the third keyword data and their corresponding frequencies in the requirement document and the design document to obtain the third setting coefficient;

[0060] Among them, the third setting coefficient is obtained through the following formula:

[0061]

[0062] Among them, J 03 represents the third setting coefficient; k represents the total number of keyword phrases included in the third keyword data; L 03iIndicates the number of characters of the i-th keyword in the third keyword data; L 03b Indicates the standard deviation of the number of characters of k keywords in the third keyword data; R 03 Indicates a preset third reference coefficient, and the value range of the third reference coefficient is 1.19 - 1.42; f 03i Indicates the frequency of the i-th keyword in the third keyword data appearing in the requirements document and the design document; f 01c Indicates the first frequency threshold; f 02c Indicates the second frequency threshold; f 03b Indicates the standard deviation of the frequencies of k keywords in the third keyword data appearing in the requirements document and the design document;

[0063] Set the maximum text segment length by using the third setting coefficient, where the maximum text segment length is obtained through the following formula:

[0064]

[0065] Where, H max Indicates the maximum text segment length, and, H max Indicates rounding up; H c Indicates a preset initial text segment length; J 03 Indicates the third setting coefficient; R 03 Indicates a preset third reference coefficient, and the value range of the third reference coefficient is 1.19 - 1.42; J 02 Indicates the second setting coefficient; J 01 Indicates the first setting coefficient.

[0066] Further, parsing the content of the small text list in step two to construct prompt words specifically includes:

[0067] Traverse the small text list, read each small text segment, and analyze the key information in each small text segment;

[0068] Match the key information with a preset prompt word template, and input the key information into the corresponding prompt word template based on the matching result to generate a complete prompt word;

[0069] At the same time, obtain the test case feedback data generated by the large language model, and optimize the prompt words based on the test case feedback data.

[0070] Further, the test cases generated in step three specifically include:

[0071] Test case ID: Assign a unique identifier to each test case; Test purpose: The purpose and objective of the test case; Prerequisites: Conditions that must be met before executing the test case; Test steps: Steps to execute the test; Input data: Data input during the test; Expected result: The expected result after executing the test steps; Actual result: The actual result after executing the test steps; Test conclusion: Whether the test result is successful.

[0072] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0073] Through intelligent text segmentation, prompt construction, and test case generation, it not only improves the efficiency and accuracy of test case generation, but also ensures the pertinence and coverage of test cases, significantly reducing the workload of manually writing test cases; By constructing a knowledge graph of software configuration items, it enhances the model's understanding of the relevance and similarity between texts, thereby improving the quality of test case generation; At the same time, through a real-time feedback and optimization mechanism, continuously adjust and improve test cases, making the generated test cases more in line with actual needs, effectively improving the automation level and test efficiency of software testing. Description of the Drawings

[0074] Figure 1 It is a step diagram of the method for automatically generating test cases for software configuration items based on the large language model of the present invention. Detailed Embodiments

[0075] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0076] In order to solve the technical problems in the prior art that rely on FPGA modules, signal conversion modules, etc., resulting in limited flexibility and portability of the system, making it difficult to adapt to changing software requirements and designs, and still requiring manual intervention to generate and parse test sequences, please refer to Figure 1 , this embodiment provides the following technical solutions,

[0077] A method for automatically generating test cases for software configuration items based on a large language model, comprising the following steps:

[0078] Step 1: Set up the environment: Select the corresponding large language model according to the characteristics of the software configuration items and test requirements. For example, select the GPT series models with powerful natural language processing and generation capabilities, and install and configure the frameworks for building applications based on large language models, such as Langchain, Hugging Face’s Transformers, etc.; at the same time, install and configure the associated components, including text splitters, file format parsers, etc.;

[0079] Step 2: Build the prompt: Establish a software project document library containing the requirement documents and design documents related to the software configuration items. Collect the corresponding requirement documents and design documents from the software project document library, and classify them according to document type and relevance. Preprocess the classified requirement documents and design documents, such as removing irrelevant format information, unifying terms and naming conventions, to ensure higher quality of the information received by the model;

[0080] Based on the framework and associated components (such as csv, json, html, pdf, etc.), split the requirement documents (PRD) and design documents (TDD) into multiple small text fragments to generate a list of small texts. Among them, PRD and TDD contain key information such as the functional requirements, performance requirements, and interface design of the software configuration items;

[0081] At the same time, use the Memory module (such as ConversationBufferMemory and ConversationSummaryBufferMemory) to store and remember the context information of the small text fragments, reduce the number of interactions with the large language model, parse the content of the small text list, and build the prompt to ensure that the prompt can guide the large language model to generate test cases that meet the requirements;

[0082] Step 3: Test case generation: Pass the built prompt to the large language model to generate test cases for the software configuration items. The test cases include: various functional scenarios, boundary conditions, and exception handling of the software configuration items. The generated test cases are fed back to the testers in real time through the framework for further review and execution, specifically including:

[0083] Test case ID: Assign a unique identifier to each test case; Test purpose: The purpose and goal of the test case; Prerequisite conditions: Conditions that must be met before executing the test case; Test steps: Steps to execute the test; Input data: Data input during the test; Expected result: The expected result after executing the test steps; Actual result: The actual result after executing the test steps; Test conclusion: Whether the test result is successful;

[0084] Step 4: Test Case Evaluation and Optimization: Save the generated test cases in a predefined format, such as Markdown, Excel, etc., for easy subsequent use and management. At the same time, establish a test case library to classify, label, and optimize the test cases to improve the efficiency and accuracy of subsequent use.

[0085] In this embodiment, to prevent the large model from experiencing response delays or exceeding the token limit due to excessive content passed in at once, the text splitter of the framework is used to split the file into multiple small text lists. Based on the effectiveness and coverage of the test cases generated by the evaluation of test engineers, feedback is provided to the large language model according to the evaluation results for further optimization to improve the generation of test cases.

[0086] In this embodiment, through intelligent text splitting, prompt construction, and test case generation, not only the efficiency and accuracy of test case generation are improved, but also the pertinence and coverage of test cases are ensured, significantly reducing the workload of manually writing test cases; by constructing a knowledge graph of software configuration items, the model's understanding of the relevance and similarity between texts is enhanced, thus improving the quality of test case generation; at the same time, through a real-time feedback and optimization mechanism, the test cases are continuously adjusted and improved, making the generated test cases more in line with actual requirements, effectively improving the automation level and test efficiency of software testing.

[0087] In this embodiment, the described environment setup is specifically as follows:

[0088] Analyze software configuration items: Analyze the software configuration items to determine the key information of software function characteristics, performance indicators, and interface design;

[0089] Select a large prediction model: Based on the key information of software function characteristics, performance indicators, and interface design, determine the required natural language processing capabilities, and combine with the requirements for generating test cases to select the corresponding large language model, such as GPT-3 or GPT-4, to ensure that the model has sufficient natural language processing and generation capabilities;

[0090] Install and configure the framework: Create a Python virtual environment, install the application development library that supports the large language model in this environment, and configure the large language model parameters to provide support for subsequent test case generation;

[0091] Configure associated components: Install and configure associated components, including a text splitter, a file format parser, a requirements document analysis tool, and a design document visualization tool, to facilitate the processing of requirements documents and design documents. After the configuration is completed, test each associated component to ensure that they can correctly process actual requirements documents and design documents.

[0092] In this embodiment, the software configuration items are analyzed, including clarifying all the function points of the software, including user interfaces, business logics, and data processing, etc., identifying which functions are core functions according to the functional characteristics and performing priority sorting, understanding the interaction methods between different functions and the interaction methods for responding to user inputs; determining performance indicators such as the response time, concurrent user number, and data processing ability of the software, and understanding the hardware resource limitations, such as CPU, memory, storage, etc.; listing all external and internal interfaces, including APIs, database connections, service calls, etc., analyzing the data formats used by the interfaces, such as JSON, XML, etc., and checking the integrity and accuracy of the interface documents.

[0093] In this embodiment, the environment setup further includes training the selected large language model, and the specific steps include:

[0094] Data processing: Collect the text data of the software configuration items, including requirement documents, design documents, design specifications, historical test cases, user feedback, etc., and identify the text data, screening out the target text data that meets the test requirements, for example:

[0095] For functional testing, screen out the text that describes function points, user stories, and business processes;

[0096] For performance testing, identify the text that contains performance indicators and hardware resource limitations;

[0097] For interface testing, find out the document parts that describe API interfaces and database connections;

[0098] Feature extraction: Extract text features from the screened target text data, and generate a text feature dataset for training based on the extraction results;

[0099] Model training: Train the large language model based on the text feature dataset to optimize the large language model's understanding and generation capabilities for software configuration item-related texts;

[0100] Model output: Verify the trained large language model, judge whether the performance of the trained large language model meets the predetermined standards according to the verification results, and output the large language model that meets the predetermined standards.

[0101] In this embodiment, the data processing further includes constructing a knowledge graph of software configuration items, specifically:

[0102] Define the specific instances of software configuration items as key nodes in the software configuration item knowledge graph, such as specific function modules, business logic units, etc., and match the specific instances with the existing software libraries. Based on the matching results, input the detailed data of software configuration items into the software libraries, including function descriptions, interface definitions, performance parameters, etc.;

[0103] Draw a knowledge graph of software configuration items based on the detailed data in the software library, where the nodes represent function points, business processes, etc., and the edges represent the association relationships between them; and perform entity alignment between the knowledge graphs of software configuration items to ensure that the same or similar function points can be corresponding in different graphs;

[0104] Train a large language model based on the information in the knowledge graph of software configuration items so that it can understand the relevance and similarity between texts related to software configuration items.

[0105] In this embodiment, by combining the training of the large language model and the construction of the knowledge graph of software configuration items, the efficiency and quality of test case generation are improved. Developers and testers can obtain test cases faster, reducing the workload of manually writing test cases. At the same time, the generated test cases are more targeted and comprehensive, and can better meet the needs of software testing.

[0106] In this embodiment, in step two, the requirement documents and design documents obtained from the software project document library are segmented into multiple small text fragments, specifically including:

[0107] Load the requirement documents and design documents into a file format parser to parse the document content, determine the document structures of the requirement documents and design documents, such as chapters, paragraphs, lists, etc., to determine appropriate segmentation points;

[0108] Identify the key information in the requirement documents and design documents based on the document structures, such as function descriptions, performance indicators, interface definitions, etc., and set the maximum text fragment length;

[0109] Take the natural paragraphs in the requirement documents and design documents as the preliminary segmentation units, obtain the content lengths of each paragraph. If it exceeds the set maximum text fragment length, then segment the natural paragraph into sentences that meet the maximum text fragment length and maintain the integrity of the sentences;

[0110] Set a context window at each segmentation point, including the relevant sentences and words before and after, to provide sufficient context information; among them, the segmentation points avoid being in the middle of the key information, such as not in the middle of the function point description or performance indicator, to ensure that each small text fragment contains complete key information, such as a single function point or performance indicator;

[0111] Generate multiple small text fragments based on the segmentation results, and assign a unique number to each small text fragment. Arrange each small text fragment according to the unique number to generate a small text list for subsequent tracking and context reconstruction;

[0112] Specifically, setting the maximum text fragment length includes:

[0113] Extract the key information from the requirement document and the design document;

[0114] Extract the keyword phrases contained in each key information from each key information;

[0115] Extract the number of characters corresponding to each keyword phrase and the frequency of occurrence of each keyword phrase in the requirement document and the design document;

[0116] Retrieve the keyword phrases with a frequency of occurrence lower than the preset first frequency threshold in the requirement document and the design document as the first keyword data;

[0117] Obtain the first setting coefficient by using the number of characters of the keyword phrases contained in the first keyword data and their corresponding frequencies of occurrence in the requirement document and the design document;

[0118] Among them, the first setting coefficient is obtained through the following formula:

[0119]

[0120] Among them, J 01 represents the first setting coefficient; n represents the total number of keyword phrases contained in the first keyword data; L 01i represents the number of characters of the i-th keyword phrase in the first keyword data; L 01b represents the standard deviation of the number of characters of the n keyword phrases in the first keyword data; R 01 represents the preset first reference coefficient, and the value range of the first reference coefficient is 1.37 - 1.48; f 01i represents the frequency of occurrence of the i-th keyword phrase in the first keyword data in the requirement document and the design document; f 01c represents the first frequency threshold; f 01b represents the standard deviation of the frequencies of occurrence of the n keyword phrases in the first keyword data in the requirement document and the design document;

[0121] Retrieve the keyword phrases with a frequency of occurrence exceeding the preset second frequency threshold in the requirement document and the design document as the second keyword data;

[0122] Obtain the second setting coefficient by using the number of characters of the keyword phrases contained in the second keyword data and their corresponding frequencies of occurrence in the requirement document and the design document;

[0123] Among them, the second setting coefficient is obtained through the following formula:

[0124]

[0125] Among them, J 02represents the second setting coefficient; m represents the total number of keyword phrases contained in the second keyword data; L 02i represents the number of characters of the i-th keyword phrase in the second keyword data; L 02b represents the standard deviation of the number of characters of the m keyword phrases in the second keyword data; R 01 represents a preset first reference coefficient, and the value range of the first reference coefficient is 1.37 - 1.48; R 02 represents a preset second reference coefficient, and the value range of the second reference coefficient is 1.24 - 1.53; f 02i represents the frequency of occurrence of the i-th keyword phrase in the second keyword data in the requirement document and the design document; f 02c represents the second frequency threshold; f 02b represents the standard deviation of the frequencies of occurrence of the m keyword phrases in the second keyword data in the requirement document and the design document;

[0126] Set the maximum text segment length by using the first setting coefficient and the second setting coefficient.

[0127] The technical effects of the above technical solution are as follows: By extracting the key information and keyword phrases in the requirement document and the design document, this technical solution can more accurately identify the core content of the requirement document and the design document. This helps to more precisely analyze and utilize this information in subsequent processing. By calculating the first setting coefficient and the second setting coefficient, this technical solution can comprehensively consider factors such as the number of characters, frequency of occurrence of keyword phrases, and the standard deviation of these data, so as to set a more reasonable maximum text segment length. This helps to ensure that the extracted text segments contain sufficient information and are not overly lengthy. This technical solution screens keyword phrases through a preset frequency threshold, reducing the amount of data to be processed, thereby improving the processing efficiency. At the same time, by introducing statistical quantities such as standard deviation to optimize the calculation of the setting coefficient, the calculation efficiency and accuracy are further improved. The setting coefficient calculation formula in this technical solution contains adjustable reference coefficients (R 01 and R 02 )), which enables the system to be flexibly adjusted according to specific application scenarios and requirements. This adaptability helps to enhance the flexibility and versatility of the system. Since this technical solution can process complex documents containing a large amount of key information and keyword phrases, it is applicable to various types of text data. This makes this technical solution have broad application prospects in the fields of text mining, information retrieval, natural language processing, etc.

[0128] In summary, the technical effects of this technical solution in terms of performance indicators are mainly reflected in improving the accuracy of information extraction, enhancing the rationality of the length of text segments, optimizing the processing efficiency, enhancing the adaptive ability of the system, and supporting complex document processing. These effects together constitute the unique advantages of this technical solution in setting the maximum text segment length.

[0129] Specifically, setting the maximum text segment length by using the first setting coefficient and the second setting coefficient includes:

[0130] Extracting the first setting coefficient and the second setting coefficient;

[0131] Comparing the first setting coefficient with a preset first coefficient threshold;

[0132] Comparing the second setting coefficient with a preset second coefficient threshold;

[0133] When the first setting coefficient exceeds the preset first coefficient threshold and the second setting coefficient exceeds the preset second coefficient threshold, the maximum text segment length is set by using the first setting coefficient and the second setting coefficient;

[0134] Among them, the maximum text segment length is obtained through the following formula:

[0135]

[0136] Among them, H max represents the maximum text segment length, and H max represents rounding up; J 02 represents the second setting coefficient; J 01 represents the first setting coefficient; J 02c represents the second coefficient threshold; J 01c represents the first coefficient threshold; R 01 represents a preset first reference coefficient, and the value range of the first reference coefficient is 1.37 - 1.48; R 02 represents a preset second reference coefficient, and the value range of the second reference coefficient is 1.24 - 1.53; H c represents a preset initial text segment length;

[0137] When the first setting coefficient does not exceed the preset first coefficient threshold or the second setting coefficient does not exceed the preset second coefficient threshold, and when the first setting coefficient does not exceed the preset first coefficient threshold and the second setting coefficient does not exceed the preset second coefficient threshold, the maximum text segment length is set by using the keyword that appears in the document with a frequency not lower than the preset first frequency threshold but lower than the preset second frequency threshold.

[0138] The technical effects of the above technical solution are as follows: By comparing the first setting coefficient with the preset first coefficient threshold, and the second setting coefficient with the preset second coefficient threshold, this technical solution can ensure that when setting the maximum text segment length, factors such as the character count, occurrence frequency of keyword phrases, and the standard deviation of these data are fully considered. This helps to improve the accuracy of the text segment length, making it closer to the actual content of the document. This technical solution provides multiple ways to set the maximum text segment length. When both the first setting coefficient and the second setting coefficient exceed the preset thresholds, a specific formula is used to calculate the maximum text segment length; when this condition is not met, the keyword phrases with occurrence frequencies within a specific range in the document are used to set it. This design enhances the flexibility of the system, enabling it to adapt to different types of documents and application scenarios. Through the preset coefficient thresholds and frequency thresholds, this technical solution can reduce the amount of data to be processed, thereby improving the processing efficiency. At the same time, using the setting coefficient for calculation also avoids complex iterative processes, further enhancing the calculation efficiency. When both the first setting coefficient and the second setting coefficient do not exceed the preset thresholds, this technical solution uses the keyword phrases with occurrence frequencies within a specific range in the document to set the maximum text segment length. This helps to ensure that the extracted text segments contain sufficient information and avoid missing important content. Since this technical solution can process complex documents containing a large amount of key information and keyword phrases and reasonably set the maximum text segment length according to the document content, it is applicable to various different types of text data. This makes this technical solution have broad application prospects in the fields of text mining, information retrieval, natural language processing, etc. The parameters such as the coefficient threshold, frequency threshold, and reference coefficient in this technical solution can be adjusted according to specific needs, which helps to improve the adaptability and scalability of the system. At the same time, this technical solution can also be used as the basis or component of other more complex algorithms to further expand its application scope.

[0139] In summary, the technical effects of this technical solution in terms of performance indicators are mainly reflected in improving the accuracy of the text segment length, enhancing the flexibility of the system, optimizing the processing efficiency, improving the integrity of information extraction, supporting the processing of complex documents, and being easy to adjust and expand. These effects together constitute the unique advantages of this technical solution in setting the maximum text segment length.

[0140] Specifically, when the first setting coefficient does not exceed the preset first coefficient threshold or the second setting coefficient does not exceed the preset second coefficient threshold, and when the first setting coefficient does not exceed the preset first coefficient threshold and the second setting coefficient does not exceed the preset second coefficient threshold, the keyword phrases with occurrence frequencies not less than the preset first frequency threshold but less than the preset second frequency threshold in the document are used to set the maximum text segment length, including:

[0141] When the first setting coefficient does not exceed the preset first coefficient threshold or the second setting coefficient does not exceed the preset second coefficient threshold, and when the first setting coefficient does not exceed the preset first coefficient threshold and the second setting coefficient does not exceed the preset second coefficient threshold, then keywords that appear in the requirement document and the design document with a frequency not lower than the preset first frequency threshold but lower than the preset second frequency threshold are retrieved as the third keyword data;

[0142] The third setting coefficient is obtained by using the number of characters of the keywords included in the third keyword data and their corresponding frequencies in the requirement document and the design document;

[0143] Among them, the third setting coefficient is obtained through the following formula:

[0144]

[0145] Among them, J 03 represents the third setting coefficient; k represents the total number of keywords included in the third keyword data; L 03i represents the number of characters of the i-th keyword in the third keyword data; L 03b represents the standard deviation of the number of characters of the k keywords in the third keyword data; R 03 represents the preset third reference coefficient, and the value range of the third reference coefficient is 1.19 - 1.42; f 03i represents the frequency of the i-th keyword in the third keyword data in the requirement document and the design document; f 01c represents the first frequency threshold; f 02c represents the second frequency threshold; f 03b represents the standard deviation of the frequencies of the k keywords in the third keyword data in the requirement document and the design document;

[0146] The maximum text segment length is set by using the third setting coefficient, where the maximum text segment length is obtained through the following formula:

[0147]

[0148] Among them, H max represents the maximum text segment length, and H max represents rounding up; H c represents the preset initial text segment length; J 03 represents the third setting coefficient; R 03 represents the preset third reference coefficient, and the value range of the third reference coefficient is 1.19 - 1.42; J 02 represents the second setting coefficient; J 01 represents the first setting coefficient.

[0149] The technical effects of the above technical solution are as follows: When both the first setting coefficient and the second setting coefficient do not exceed the preset threshold, this technical solution can automatically retrieve the keyword phrases that appear between the first frequency threshold and the second frequency threshold in the requirement document and the design document as the third keyword data, and use these data to calculate the third setting coefficient. This method enables the finally set maximum text segment length to better adapt to the actual content of the document, avoiding information omission or redundancy caused by overly strict or loose settings. By introducing the third setting coefficient and the corresponding calculation formula, this technical solution can provide a reasonable maximum text segment length setting under different circumstances. This design enhances the robustness of the system, enabling it to maintain stable performance in various complex scenarios. Using the number of characters and the appearance frequency of the keyword phrases in the third keyword data to calculate the third setting coefficient helps ensure that the extracted text segments contain sufficient information. At the same time, by reasonably setting the maximum text segment length, it is possible to avoid difficulties in processing due to overly long segments or omission of important information due to overly short segments. This technical solution directly calculates the maximum text segment length through the preset reference coefficient and calculation formula, avoiding complex iterative processes or manual adjustments. This helps improve processing efficiency, reduce calculation time and resource consumption. The parameters such as the reference coefficient and the initial text segment length in this technical solution can be adjusted according to specific requirements. This design makes the system more flexible and capable of adapting to different application scenarios and requirements. At the same time, this technical solution can also be used as the basis or component of other more complex algorithms to further expand its application scope. Since this technical solution can process complex documents containing a large amount of key information and keyword phrases and reasonably set the maximum text segment length according to the document content, it is applicable to various types of text data. This makes this technical solution have broad application prospects in the fields of text mining, information retrieval, natural language processing, etc.

[0150] In summary, the technical effects of this technical solution in terms of performance indicators are mainly reflected in improving the adaptability of the text segment length, enhancing the robustness of the system, optimizing the integrity of information extraction, improving processing efficiency, being easy to adjust and expand, and supporting the processing of complex documents. These effects together constitute the unique advantages of this technical solution in setting the maximum text segment length.

[0151] In this embodiment, by setting a context window and ensuring that the split point is not in the middle of the key information, it is ensured that the text splitter can intelligently identify and retain the context information of the text segment, avoiding the destruction of the continuity and integrity of the key information during splitting. The splitting strategy can also be adjusted according to the model feedback to optimize the context window size and split point selection;

[0152] In this embodiment, the step of parsing the content of the small text list in step two and constructing prompt words specifically includes:

[0153] Traverse the small text list, read each small text segment, and analyze the key information in each small text segment, such as function points, performance indicators, interface definitions, user stories, business rules, etc.;

[0154] Match the key information with a preset prompt word template, and based on the matching result, input the key information into the corresponding prompt word template to generate a complete prompt word;

[0155] Among them, the prompt word template includes: Test purpose: According to the function description in the text segment, explain the purpose of the test case, such as "Verify the user login function". Test steps: Guide the model to generate specific operation steps, and according to the function points, write guiding language for the test steps, such as "Enter a valid username and password, and click the login button". Expected result: According to the function expectation, describe the expected result after executing the test steps, such as "The user has successfully logged in and is redirected to the home page". Test conditions: According to the function requirements, point out the prerequisite conditions for executing the test, such as "The user has registered and the account is valid";

[0156] At the same time, obtain the test case feedback data generated by the large language model, and optimize the prompt words based on the test case feedback data.

[0157] In this embodiment, by constructing small text segments containing sufficient context information and combining with the prompt word template, it guides the large language model to generate more accurate and targeted test cases, improving the quality of the test cases. Traversing the small text list and analyzing the key information, combined with the use of prompt words, enhances the large language model's understanding ability of the text related to software configuration items, enabling it to generate test cases that meet the requirements more effectively, providing accurate input for the large language model, and thus significantly improving the overall performance and practicality of the automatic generation of software configuration item test cases.

[0158] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and should be covered by the protection scope of the present invention.

Claims

1. A method for automatically generating test cases for software configuration items based on a large language model, characterized in that: The following steps are involved: Step 1: Build the environment: Select the corresponding large language model according to the characteristics of the software configuration items and test requirements, install and configure the framework for building applications based on the large language model, and install and configure related components; Step 2: Construct prompt words: Based on the framework and related components, a software project document library is established, which includes requirement documents and design documents related to software configuration items, and the requirement documents and design documents obtained from the software project document library are divided into multiple small text fragments to generate a small text list; At the same time, it stores and memorizes the context information of the small text fragments, parses the content of the small text list, and constructs prompt words; In the step 2, the requirement document and design document obtained from the software project document library are divided into multiple small text fragments, specifically including: Load the requirement document and design document into the file format parser to parse the document content and determine the document structure of the requirement document and design document; Identify key information in requirement documents and design documents based on document structure and set a maximum text fragment length; The natural paragraphs in the requirement document and the design document are used as preliminary segmentation units, and the content length of each paragraph is obtained. If the content length of each paragraph exceeds the set maximum text segment length, the natural paragraph is segmented into sentences that meet the maximum text segment length; A context window is set at each segmentation point, including related sentences and words before and after, wherein the segmentation point is avoided to be located in the middle of the key information to maintain the integrity of the key information; Generate multiple small text segments based on the segmentation results, assign a unique number to each small text segment, arrange each small text segment according to the unique number, and generate a small text list; Step 3: Test case generation: passing the constructed prompt words to the large language model to generate test cases for the software configuration items, and feeding back the generated test cases to the testers in real time through the framework; Step 4: Test case evaluation and optimization: Save the generated test cases in a predetermined format. At the same time, establish a test case library to classify, annotate and optimize the test cases.

2. The method for automatically generating software configuration item test cases based on a large language model according to claim 1, characterized in that: The construction environment is specifically: Analyze software configuration items: Analyze software configuration items to determine key information on software functional characteristics, performance indicators, and interface design; Select the big prediction model: Determine the required natural language processing capabilities based on the software functional characteristics, performance indicators, and key information of interface design, and select the corresponding big language model based on the needs of generating test cases; Install and configure the framework: Create a Python virtual environment, install application development libraries that support large language models, and configure large language model parameters; Configure related components: Install and configure related components, including text segmenter, file format parser, requirement document analysis tool, and design document visualization tool. After the configuration is complete, test each related component.

3. The method for automatically generating software configuration item test cases based on a large language model as claimed in claim 2, characterized in that: The building environment also includes training the selected large language model, and the specific steps include: Data processing: collect text data of software configuration items, identify the text data, and filter out target text data that meets the test requirements; Feature extraction: extract text features from the filtered target text data, and generate a text feature dataset for training based on the extraction results; Model training: Train the large language model based on the text feature dataset to optimize the large language model's ability to understand and generate text related to software configuration items; Model output: Verify the trained large language model, determine whether the performance of the trained large language model meets the predetermined standards based on the verification results, and output the large language model that meets the predetermined standards.

4. The method for automatically generating software configuration item test cases based on a large language model as claimed in claim 3, characterized in that: The data processing also includes constructing a knowledge graph of software configuration items, specifically: Define the specific instance of the software configuration item as the key node in the software configuration item knowledge graph, match the specific instance with the existing software library, and input the detailed data of the software configuration item into the software library based on the matching result; Draw a knowledge graph of software configuration items based on the detailed data of the software library, and align entities between the knowledge graphs of software configuration items; Perform large language model training based on information in the software configuration item knowledge graph.

5. The method for automatically generating software configuration item test cases based on a large language model as claimed in claim 4, characterized in that: The step 2 of constructing prompt words also includes: classifying the demand documents and design documents collected in the software project document library according to document type and relevance, and preprocessing the classified demand documents and design documents.

6. The method for automatically generating software configuration item test cases based on a large language model as claimed in claim 5, characterized in that: Set the maximum text segment length, including: Extract key information from requirement documents and design documents; Extract key words contained in each key information from each key information; Extract the number of characters corresponding to each keyword and the frequency of each keyword appearing in the requirements document and design document; Retrieving key words in the requirement document and the design document whose frequencies are lower than a preset first frequency threshold as first keyword data; Obtaining a first setting coefficient using the number of characters of the key words included in the first keyword data and the corresponding frequencies of occurrence in the requirement document and the design document; The first setting coefficient is obtained by the following formula: Among them, J 01 represents the first setting coefficient; n represents the total number of key words contained in the first keyword data; L 01i represents the number of characters of the i-th keyword in the first keyword data; L 01b represents the standard deviation of the number of characters of the n key words in the first key word data; R 01 represents a preset first reference coefficient, and the value range of the first reference coefficient is 1.37-1.48; f 01i represents the frequency of the i-th keyword in the first keyword data appearing in the requirement document and the design document; f 01c represents the first frequency threshold; f 01b represents the frequency standard deviation of n key words in the first keyword data in the requirement document and the design document; Retrieving key words in the requirement document and the design document whose frequencies exceed a preset second frequency threshold as second keyword data; Obtaining a second setting coefficient using the number of characters of the key words included in the second keyword data and the corresponding frequencies of occurrence in the requirement document and the design document; The second setting coefficient is obtained by the following formula: Among them, J 02 represents the second setting coefficient; m represents the total number of key words contained in the second keyword data; L 02i represents the number of characters of the i-th keyword in the second keyword data; L 02b represents the standard deviation of the number of characters of the m key words in the second key word data; R 01 represents a preset first reference coefficient, and the value range of the first reference coefficient is 1.37-1.48; R 02 represents a preset second reference coefficient, and the value range of the second reference coefficient is 1.24-1.53; f 02i represents the frequency of the i-th keyword in the second keyword data appearing in the requirement document and the design document; f 02c represents the second frequency threshold; f 02b represents the frequency standard deviation of the m key words in the second key word data in the requirement document and the design document; The maximum text segment length is set using the first setting coefficient and the second setting coefficient.

7. The method for automatically generating software configuration item test cases based on a large language model as claimed in claim 6, characterized in that: Setting the maximum text segment length using the first setting coefficient and the second setting coefficient includes: extracting a first setting coefficient and a second setting coefficient; Comparing the first setting coefficient with a preset first coefficient threshold; comparing the second setting coefficient with a preset second coefficient threshold; When the first setting coefficient exceeds a preset first coefficient threshold, and the second setting coefficient exceeds a preset second coefficient threshold, the first setting coefficient and the second setting coefficient are used to set the maximum text segment length; The maximum text segment length is obtained by the following formula: Among them, H max represents the maximum text segment length, and H max Indicated as rounded up; J 02 Indicates the second setting coefficient; J 01 Indicates the first setting coefficient; J 02c represents the second coefficient threshold; J 01c represents the first coefficient threshold; R 01 represents a preset first reference coefficient, and the value range of the first reference coefficient is 1.37-1.48; R 02 represents a preset second reference coefficient, and the value range of the second reference coefficient is 1.24-1.53; H c Indicates the preset initial text segment length; When the first setting coefficient does not exceed the preset first coefficient threshold or the second setting coefficient does not exceed the preset second coefficient threshold, and when the first setting coefficient does not exceed the preset first coefficient threshold and the second setting coefficient does not exceed the preset second coefficient threshold, the maximum text fragment length is set using keywords whose frequency of appearance in the requirement document and the design document is not lower than the preset first frequency threshold, but lower than the preset second frequency threshold.

8. The method for automatically generating software configuration item test cases based on a large language model according to claim 7, characterized in that: When the first setting coefficient does not exceed the preset first coefficient threshold or the second setting coefficient does not exceed the preset second coefficient threshold, and the first setting coefficient does not exceed the preset first coefficient threshold and the second setting coefficient does not exceed the preset second coefficient threshold, the maximum text segment length is set using keywords whose frequencies in the requirement document and the design document are not lower than the preset first frequency threshold, but lower than the preset second frequency threshold, including: When the first setting coefficient does not exceed the preset first coefficient threshold or the second setting coefficient does not exceed the preset second coefficient threshold, and the first setting coefficient does not exceed the preset first coefficient threshold and the second setting coefficient does not exceed the preset second coefficient threshold, then retrieve the key words in the requirement document and the design document whose frequency is not lower than the preset first frequency threshold, but lower than the preset second frequency threshold as the third keyword data; Obtaining a third setting coefficient using the number of characters of the key words included in the third keyword data and the corresponding frequencies of occurrence in the requirement document and the design document; The third setting coefficient is obtained by the following formula: Among them, J 03 represents the third setting coefficient; k represents the total number of key words contained in the third keyword data; L 03i represents the number of characters of the i-th keyword in the third keyword data; L 03b represents the standard deviation of the number of characters of k key words in the third key word data; R 03 represents a preset third reference coefficient, and the value range of the third reference coefficient is 1.19-1.42; f 03i represents the frequency of the i-th keyword in the third keyword data appearing in the requirement document and the design document; f 01c represents the first frequency threshold; f 02c represents the second frequency threshold; f 03b represents the frequency standard deviation of k key words in the third keyword data in the requirement document and the design document; The third setting coefficient is used to set the maximum text segment length, wherein the maximum text segment length is obtained by the following formula: Among them, H max represents the maximum text segment length, and H max Indicates rounding up; H c Indicates the preset initial text segment length; J 03 Indicates the third setting coefficient; R 03 represents a preset third reference coefficient, and the value range of the third reference coefficient is 1.19-1.42; J 02 Indicates the second setting coefficient; J 01 Indicates the first setting coefficient.

9. The method for automatically generating software configuration item test cases based on a large language model according to claim 1, characterized in that: The second step of parsing the small text list content and constructing prompt words specifically includes: Traverse the small text list, read each small text segment, and analyze the key information in each small text segment; Matching the key information with a preset prompt word template, and inputting the key information into the corresponding prompt word template based on the matching result to generate a complete prompt word; At the same time, the test case feedback data generated by the large language model is obtained, and the prompt words are optimized based on the test case feedback data; The test cases generated in step 3 specifically include: Test case ID: Assign a unique identifier to each test case; Test purpose: The purpose and goal of the test case; Preconditions: The conditions that must be met before executing the test case; Test steps: The steps to execute the test; Input data: The data entered during the test; Expected results: The expected results after executing the test steps; Actual results: The actual results after executing the test steps; Test conclusion: Whether the test result is successful.

Images