Software fuzzy test case generation method based on semantic clustering

Abstract

The invention relates to a semantic clustering-based software fuzzy test case generation method. The method comprises the following steps of: forming a large model base based on a cue word sequence and software related knowledge data; the method comprises the following steps: converting each word of a software related document into a low-dimensional vector representation so as to obtain a semantic vector representation of the document, clustering the semantic vector representation, and cutting the document into semantic segments according to a clustering result; key words are extracted from the semantic fragments, entity recognition is carried out, and the relation between entities is recognized; determining a test key point and a test range according to the correlation of the software; and inputting the keyword, the relationship between the entities, the test key point and the test range into a large model base in the step, generating a test case, and optimizing the test case. According to the method, the document content is cut into a plurality of segments with independent semantics and classified according to the semantic similarity of the document content, key information in the document can be accurately extracted, effective integration and utilization of knowledge, accurate extraction of the key information of the document, intelligent collaboration and efficient retrieval are achieved, and the efficiency and quality of test case generation are improved.

Description

Technical Field

[0001] This invention belongs to the field of software testing technology, specifically relating to a method for generating software fuzzy test cases based on semantic clustering. Background Technology

[0002] In fields with high reliability requirements, such as aerospace, the testing of critical software is crucial. These software systems not only need high real-time performance and stability, but also must meet stringent safety and reliability standards to ensure normal operation even in extreme environments. However, the current process of generating software test cases often faces many challenges: on the one hand, traditional test case generation methods rely heavily on manual design or automated tools based on simple rules, making it difficult to comprehensively cover all potential execution paths and boundary conditions of the software, especially for testing requirements involving complex logic and dynamic behavior; on the other hand, as software scales up and functions become increasingly complex, the difficulty of manually designing high-quality test cases increases dramatically, becoming not only time-consuming and labor-intensive, but also prone to overlooking critical test scenarios due to human error or lack of experience.

[0003] Therefore, extremely high requirements are placed on the coverage and accuracy of test cases. Traditional methods are inadequate when dealing with software verification that has such high reliability requirements, and are unable to effectively discover potential errors and security vulnerabilities hidden deep within the code.

[0004] Traditional software fuzz test case generation faces a series of severe challenges. First, there is a lack of effective knowledge integration and utilization mechanisms. Software knowledge is complex, involving numerous business rules, technical specifications, and historical experience. Traditional methods struggle to effectively integrate this scattered knowledge and transform it into resources that can guide test case generation. This results in generated test cases that often lack specificity and comprehensiveness, failing to adequately cover various potential operating scenarios and boundary conditions of the software.

[0005] Secondly, there is insufficient document processing capability. Software-related documents are typically lengthy and complex in structure, containing a large amount of technical details and descriptive content. Traditional methods struggle to accurately extract key information from these documents, and are unable to properly segment and categorize them, thus failing to uncover valuable test clues and limiting the depth and breadth of test case generation.

[0006] Furthermore, intelligent collaboration and retrieval capabilities are limited. Generating fuzzy test cases requires comprehensive consideration of multiple factors, such as software functional requirements, performance metrics, and security requirements. Traditional methods lack effective intelligent collaboration mechanisms when dealing with these complex factors; each processing step is relatively independent, making information sharing and interaction difficult. Simultaneously, insufficient retrieval capabilities prevent the rapid and accurate acquisition of relevant knowledge and information during test case generation, resulting in low generation efficiency and inconsistent test case quality.

[0007] With the rapid development of artificial intelligence technology, automated test case generation technology has gradually become a research hotspot. However, most existing technologies focus only on the application of single technical means, such as model-based test generation, search-based test optimization, or simple machine learning-assisted testing. While these methods can achieve certain results in specific scenarios, they still suffer from insufficient coverage and poor adaptability when dealing with complex and dynamic software systems. Therefore, exploring a high-coverage test case generation method that can integrate multiple advanced technologies and has dynamic adaptability has become a key problem that urgently needs to be solved in the current software testing field. Summary of the Invention

[0008] The purpose of this invention is to provide a software fuzzy test case generation method based on semantic clustering. By establishing a large model base based on relevant knowledge, and extracting key information from the large model base using document segmentation technology based on semantic clustering, efficient, accurate and comprehensive software fuzzy test case generation is achieved, effectively solving many problems existing in traditional test case generation methods.

[0009] The above-mentioned objectives of the present invention are mainly achieved through the following technical solutions:

[0010] A method for generating software fuzzy test cases based on semantic clustering includes the following steps:

[0011] (1) Select a large language model, design a prompt word sequence and input relevant knowledge data of the software. Combine the prompt word sequence and knowledge data to adjust the large language model and form the large model base.

[0012] (2) Convert each word in the software-related document into a low-dimensional vector representation, average or weighted average the obtained low-dimensional vector representations to obtain the semantic vector representation of the document, set a similarity threshold, cluster the semantic vector representations, cluster the semantic vector representations with semantic similarity higher than the threshold into the same cluster, and cut the document into semantic fragments according to the clustering results.

[0013] (3) Use the TF-IDF algorithm to extract keywords from the semantic fragments obtained in step (2), calculate the TF-IDF value based on the frequency of the words in the document and the inverse document frequency in the entire document set, and select the words with higher TF-IDF values ​​as keywords;

[0014] (4) Perform entity recognition on the semantic fragments obtained in step (2), wherein the entities include software module names, function names and variable names; extract the relationships between the entities to identify the relationships between them;

[0015] (5) Determine the testing focus and scope based on the software's functional description document, performance indicator document, and security specification document;

[0016] (6) Input the keywords obtained in step (3), the relationships between entities obtained in step (4), and the test focus and test scope obtained in step (5) into the large model base obtained in step (1). The large model base generates test cases based on the input and uses a genetic algorithm to iteratively optimize the generated test cases.

[0017] (7) Output and store the test cases obtained in step (6).

[0018] In step (1), the software-related knowledge data includes the software's functional description, performance indicators, security specifications, and historical test data. The knowledge data is structured and stored as a knowledge graph or vector database. The knowledge graph organizes knowledge in the form of entity-relationship-entity, while the vector database converts knowledge text into high-dimensional vectors.

[0019] In step (2), the semantic similarity calculation method is the cosine similarity method.

[0020] In step (4), a conditional random field model is used for entity recognition.

[0021] In step (4), the relation extraction method includes formulating rules for matching and using a deep learning model for relation classification.

[0022] In step (5), the performance requirements of the software are determined according to the performance index document, the performance modeling tool is used to perform performance modeling of the software, different load conditions are simulated, the performance of the software under different loads is analyzed, and the focus and key indicators of performance testing are determined.

[0023] Based on the functional description document, identify the main functional modules of the software, the user operation flow, and the internal activities and state transition flow of the functional modules; analyze and determine the functional requirements of the software.

[0024] Based on security specification documents, identify potential security threats to the software, employ security threat modeling methods to conduct security threat analysis on the software's system architecture and business processes, and determine the scope and focus of security testing.

[0025] In step (6), after generating test cases, based on the current test cases, similar historical test cases, best practices and failure cases are searched in software-related knowledge data through RAG retrieval enhancement technology, and the retrieved relevant information is used as a supplement to optimize the test cases.

[0026] In step (6), the chromosomes of the genetic algorithm are test cases, the genes are attributes of each test case, and the quality of the test cases is evaluated by the fitness function.

[0027] In step (6), the optimized test cases are evaluated, and only those that pass the evaluation are output and stored. The evaluation methods include coverage analysis, manual review, and automated testing.

[0028] Coverage analysis methods involve calculating the coverage of software code for each test case and identifying the uncovered code portions.

[0029] Manual review includes checking whether test cases reflect test requirements and whether they are executable and repeatable;

[0030] Automated testing uses a simulated environment or actual software to run test cases, observe whether the software's running results meet expectations, and check whether the test cases can discover potential defects in the software.

[0031] The evaluation results are sent to the large model base. The large model base adjusts the test case generation strategy based on the evaluation results, and adjusts the keyword extraction method, entity recognition method, relation extraction method, and test focus and test scope determination method according to the adjusted test case generation strategy.

[0032] In step (7), the test case output information includes test case ID, test purpose, input data, expected output, test steps, preconditions and postconditions.

[0033] Compared with the prior art, the present invention has at least the following beneficial effects:

[0034] (1) This invention constructs a large model base based on relevant knowledge and extracts key information into the large model base by using document segmentation technology based on semantic clustering. Based on the semantic similarity of the document content, it segments the document into multiple segments with independent semantics and classifies them. This can accurately extract key information in the document, uncover potential test clues, and provide a foundation for the subsequent generation of comprehensive and targeted test cases. It realizes the effective integration and utilization of knowledge in the process of generating software test cases, the accurate extraction of key information in the document, and intelligent collaboration and efficient retrieval, thereby significantly improving the efficiency and quality of test case generation and solving the problems of insufficient knowledge utilization, limited document processing capabilities, and limited intelligent collaboration and retrieval capabilities of traditional methods.

[0035] (2) In the preferred embodiment of the present invention, a feedback loop mechanism is established, and the output results of each step are provided as feedback information to the preceding steps to prompt the system to optimize its processing method and task allocation strategy. Attached Figure Description

[0036] Figure 1 This is a flowchart of the software fuzzy test case generation method based on semantic clustering of the present invention;

[0037] Figure 2 This is a schematic diagram of the semantic clustering document segmentation technology process of the present invention. Detailed Implementation

[0038] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments:

[0039] like Figure 1 As shown, this invention discloses a software fuzzy test case generation technology based on semantic clustering. This method integrates a large model base based on the Prompt project and knowledge base, a document segmentation technology based on semantic clustering, and a multi-agent collaboration and RAG retrieval enhancement technology to achieve efficient, accurate, and comprehensive software fuzzy test case generation, effectively solving many problems existing in traditional test case generation methods.

[0040] To address the aforementioned problems, this invention proposes the following solution: First, a large-scale model foundation based on the Mind Chain Prompt project and a knowledge base is constructed. A structured knowledge base is formed by collecting and organizing a vast amount of knowledge in the software domain, including technical documents, testing experience, and failure cases. The Mind Chain Prompt project is then used to train and optimize the large-scale model, enabling it to better understand and utilize the information in the knowledge base, providing powerful knowledge support and intelligent reasoning capabilities for test case generation. This foundation effectively integrates scattered knowledge in the software domain, transforming it into intelligent resources that can guide test case generation, thus solving the problem of insufficient knowledge utilization in traditional methods.

[0041] Secondly, a document segmentation technique based on semantic clustering is employed. Semantic analysis is performed on software-related documents, segmenting them into multiple semantically independent fragments based on semantic similarity. These fragments are then classified using a semantic clustering algorithm, grouping fragments with similar semantics into one category, thus forming document sets on different topics. This segmentation and classification method can accurately extract key information from documents, uncover potential testing clues, and provide a foundation for generating comprehensive and targeted test cases, overcoming the limitations of traditional document processing methods.

[0042] Finally, multi-agent collaboration and RAG retrieval enhancement techniques are employed. A multi-agent framework is constructed, decomposing the test case generation process into multiple sub-tasks, each completed by a corresponding agent. These agents collaborate to share and interact information, jointly completing the test case generation task. Simultaneously, RAG retrieval enhancement technology is introduced, retrieving relevant knowledge and information from the knowledge base in real time during test case generation, providing decision support for the agents. This intelligent collaboration and retrieval enhancement mechanism fully considers various software factors, improving the efficiency and quality of test case generation and solving the problem of limited intelligent collaboration and retrieval capabilities in traditional methods.

[0043] In summary, this invention integrates semantic clustering technology and large-scale artificial intelligence models, effectively solving the problems existing in traditional software fuzz test case generation methods, and providing a more efficient, accurate and comprehensive solution for software testing.

[0044] A software fuzzy test case generation technique based on semantic clustering is characterized by automatically generating comprehensive and accurate software fuzzy test cases by utilizing a large model base based on the Prompt project and knowledge base, document segmentation technology based on semantic clustering, and multi-agent collaboration and RAG retrieval enhancement technology. The specific implementation steps are as follows:

[0045] Step 1: Construct a large model foundation based on the Mind Chain Prompt project and a knowledge base. Select a pre-trained large language model with strong language understanding and generation capabilities, such as the GPT series or Qwen series models. Fine-tune the model using the Mind Chain Prompt project, a technique that guides the model to generate output according to a specific logical reasoning process. Design a series of logically guiding prompt sequences, such as "First, analyze the functional requirements of the software, then consider possible abnormal situations, and then generate preliminary ideas for test cases based on these situations." Combine these prompts with knowledge data in the software domain (including technical documents, test reports, failure cases, etc.) to form the large model foundation.

[0046] This involves collecting and organizing a vast amount of knowledge in the software field, covering software functional descriptions, performance metrics, security specifications, historical test data, and more. This knowledge is then structured and stored in the form of knowledge graphs or vector databases. Knowledge graphs organize knowledge in an entity-relationship-entity format, such as "software module A - dependency - software module B"; vector databases convert knowledge text into high-dimensional vectors for easy retrieval and similarity calculation.

[0047] Integrating a large language model with a knowledge base allows the model to retrieve relevant information from the knowledge base in real time when generating output. This can be achieved by adding a knowledge retrieval module to the model's input layer. When the model generates test case ideas, it first retrieves knowledge related to the current task from the knowledge base and provides it as additional input to the model.

[0048] Step 2, as follows Figure 2 As shown, a document segmentation technique based on semantic clustering is used to process software-related documents. Word embedding models (such as the word embedding layer of Word2Vec, GloVe, or BERT) are used to convert each word in the document into a low-dimensional vector representation. For the entire document, the semantic vector representation can be obtained by averaging or weighted averaging all word vectors in the document. The K-Means clustering algorithm is selected to cluster the semantic vectors of the documents. The goal of the K-Means algorithm is to minimize the sum of squared distances within clusters. By iteratively optimizing this objective function, semantically similar documents are clustered into the same cluster. Based on the clustering results, documents within each cluster are further segmented into smaller semantic fragments according to semantic relevance. Sentence-level semantic similarity calculation methods, such as cosine similarity, can be used. When the similarity is lower than a preset threshold, sentences are divided into different semantic fragments.

[0049] Step 3: Multi-Agent Framework Construction and Task Allocation. Construct multiple agents of different types, including a document analysis agent, a test requirement extraction agent, a test case generation agent, and a result evaluation agent. Each agent has a specific function and knowledge domain. For example, the document analysis agent is responsible for processing the semantic fragments of the segmented document and extracting key information; the test requirement extraction agent determines the focus and scope of testing based on document information and software functional requirements; the test case generation agent uses the large model base and retrieved knowledge to generate specific test cases; and the result evaluation agent evaluates and optimizes the generated test cases.

[0050] Message queues (such as RabbitMQ or Kafka) are used as communication middleware between agents. Agents interact and collaborate on tasks by sending and receiving messages. For example, after completing document processing, the document analysis agent encapsulates the extracted key information into messages and sends them to the message queue. The test requirement extraction agent retrieves these messages from the message queue and extracts test requirements based on the information.

[0051] Based on the capabilities of each agent and the characteristics of the current task, a priority-based task allocation strategy is adopted. A priority attribute is set for each task, and the priority can be determined based on factors such as the urgency and importance of the task. The agent management module assigns tasks to appropriate agents for processing according to their priorities.

[0052] Step 4: Process semantic segments. Use the TF-IDF (Term Frequency-Inverse Document Frequency) algorithm to extract keywords from the semantic segments. The TF-IDF algorithm measures the importance of a word by calculating its frequency (TF) in a document and its inverse document frequency (IDF) across the entire document set. Select words with higher TF-IDF values ​​as keywords.

[0053] Entity recognition is performed using a Conditional Random Field (CRF) model. A CRF model is a probabilistic graphical model used for labeling sequential data. Taking a sequence of words from a semantic segment as input, the CRF model is trained to identify entities such as software module names, function names, and variable names. The training objective of the CRF model is to maximize the conditional probability (P(y|x)), where (x) is the input word sequence and (y) is the corresponding labeled sequence.

[0054] Relation extraction is performed using a combination of rule-based and deep learning approaches. For well-defined relation patterns, rules can be established for extraction; for example, rules can be formulated to match "software module A calls software module B". For complex relations, deep learning models, such as convolutional neural networks (CNNs) or recurrent neural networks (RNNs) and their variants (such as LSTM and GRUs), are used for relation classification. Word sequences and entity pairs are taken as input, and the model outputs the type of relationship between them.

[0055] Step 5: Determine the testing focus and scope. Based on the information extracted from the document analysis agent and combined with the software's functional description document, use UML modeling tools such as use case diagrams and activity diagrams to conduct a visual analysis of the software's functional requirements. By analyzing the use cases and actors in the use case diagrams, identify the main functional modules and user operation flows of the software; analyze the internal activity flow and state transitions of each functional module using activity diagrams.

[0056] Refer to the software's performance metrics documentation to determine its performance requirements, such as response time, throughput, and resource utilization. Use performance modeling tools (such as LoadRunner or JMeter) to perform performance modeling of the software, simulate different load conditions, and analyze the software's performance under different loads to determine the focus and key metrics for performance testing.

[0057] Based on the software's security specifications, identify potential security threats, such as data breaches, unauthorized access, and malicious attacks. Employ security threat modeling methods (such as the STRIDE model) to analyze the software's system architecture and business processes, determining the scope and focus of security testing, such as authentication, authorization management, and data encryption.

[0058] Step 6: Test Case Generation. The test requirement extraction agent, which defines the test focus and scope, along with key information extracted by the document analysis agent, serves as input to the large model base based on the Prompt engineering and knowledge base. The large model generates preliminary test case ideas and frameworks according to the pre-defined Prompt framework based on the input information. For example, the model might generate: "First, test the normal function of software module A, input valid data, and verify whether the output meets expectations; then test the abnormal situation of module A, input invalid data, and check the software's error handling mechanism."

[0059] During the test case generation process, the test case generation agent uses RAG retrieval enhancement technology to retrieve relevant knowledge and information from the knowledge base in real time. Based on the currently generated test case approach, the retrieval module searches the knowledge base for similar historical test cases, best practices, and failure cases. The retrieved information is then provided as supplementary input to the large model, helping it generate more comprehensive and accurate test cases.

[0060] Genetic algorithms are used to optimize the generated test cases. Test cases are viewed as chromosomes, and the attributes of each test case (such as input data, expected output, and test steps) are considered as genes. A fitness function is defined to evaluate the quality of the test cases; for example, the fitness function can comprehensively consider factors such as test case coverage, effectiveness, and executability. Through selection, crossover, and mutation operations in the genetic algorithm, the test cases are iteratively optimized to improve their quality.

[0061] Step 7: Result Evaluation. The agent evaluates the generated test cases. A code coverage tool (such as JaCoCo or Cobertura) is used to perform coverage analysis on the generated test cases. Coverage metrics include statement coverage, branch coverage, and path coverage. The agent calculates the coverage of the software code for each test case, identifies uncovered code segments, analyzes the reasons, and provides feedback to the test case generation agent for supplementation and optimization of the test cases.

[0062] The effectiveness of test cases is evaluated through a combination of manual review and automated testing. Manual review involves experienced testers examining test cases to ensure they accurately reflect the testing requirements and are executable and repeatable. Automated testing uses a simulated environment or actual software to run the test cases, observing whether the software's results meet expectations and checking whether the test cases can uncover potential defects in the software.

[0063] Assess the maintainability of test cases, including their readability, modifiability, and extensibility. Check whether the test case descriptions are clear and accurate, and whether they use a consistent naming convention and format. Analyze the difficulty of modifying test cases when software functionality changes, and whether they can be easily extended to adapt to new testing requirements.

[0064] Step 8: Multi-Agent Collaborative Optimization and Feedback. Based on the evaluation results of the result evaluation agents, the multiple agents collaborate to optimize the process. The document analysis agent further optimizes document processing and key information extraction methods based on the evaluation feedback; the test requirement extraction agent adjusts the strategy for determining test focus and scope; and the test case generation agent improves the test case generation algorithm and model parameters. Through multiple iterative collaborative optimizations, the quality and efficiency of test case generation are continuously improved.

[0065] Establish a feedback loop mechanism to provide the output of each step as feedback information to the preceding steps. For example, the evaluation result of the result evaluation agent is fed back to the test case generation agent, which adjusts its generation strategy based on the feedback and feeds the adjusted information back to the document analysis agent and the test requirement extraction agent, prompting them to optimize their processing methods and task allocation strategies.

[0066] Step 9: Test Case Output and Storage

[0067] The generated, evaluated, and optimized test cases will be output in a predefined format, such as an Excel spreadsheet or an XML file. The output format should include detailed information about the test cases, such as the test case ID, test purpose, input data, expected output, test steps, preconditions, and postconditions.

[0068] The output test cases are stored in a test case management system for easy subsequent test execution, maintenance, and management. The test case management system should have functions such as querying, retrieving, updating, and deleting, and should be able to classify, manage, and statistically analyze test cases based on different conditions (such as test type, software module, priority, etc.).

[0069] Example 1

[0070] Taking a certain power control software as an example, this software is a key software system in the aerospace field, responsible for monitoring and controlling the operating status of the power system and ensuring its stability and safety under various flight conditions. Due to the importance of this software and its high reliability requirements, the comprehensiveness and accuracy of its testing pose extremely high challenges. Traditional test case generation methods are insufficient to fully cover all potential execution paths and boundary conditions of the software. Therefore, this invention adopts a software fuzzy test case generation method based on semantic clustering. The specific process is detailed below. Figure 1 The implementation process is as follows:

[0071] Construct a large model foundation based on the Mind Chain Prompt project and a knowledge base. Select pre-trained large language models with powerful language understanding and generation capabilities, such as GPT-3.5 or Qwen-7B. Collect a large amount of knowledge in the power command software domain, including technical documents (such as the "Power Command Software Requirements Specification V2.0"), test reports, historical failure cases, etc., and perform structured processing, storing them in the form of a knowledge graph or vector database. Utilize the Mind Chain Prompt project to design a series of logically guided prompt sequences, such as "First, analyze the functional requirements of the power command software, then consider possible abnormal situations, and then generate preliminary ideas for test cases based on these situations." Combine the prompts with domain knowledge data to form the large model foundation.

[0072] Furthermore, a document segmentation technique based on semantic clustering was employed to process the uploaded documents, including "Power Command Software Requirements Specification V2.0," "Power System Operation Manual," and related historical test reports. The BERT word embedding model was used to convert each word in the document into a low-dimensional vector representation, and a weighted average was used to obtain the document's semantic vector. The K-Means clustering algorithm was selected to cluster the document's semantic vectors, and based on the clustering results, the document was segmented into multiple semantic fragments, such as "Power System Startup Process" and "Fault Diagnosis and Handling." The specific process is detailed in [link to documentation]. Figure 2 .

[0073] Furthermore, a multi-agent framework is constructed. This includes a document analysis agent, a test requirement extraction agent, a test case generation agent, and a result evaluation agent. The document analysis agent is responsible for processing the segmented semantic fragments and extracting key information; the test requirement extraction agent determines the testing focus and scope based on document information and software functional requirements; the test case generation agent generates test cases using a large model base; and the result evaluation agent evaluates the generated test cases. RabbitMQ is used as the communication middleware between the agents to realize the sending and receiving of information.

[0074] Further, semantic fragments are processed to extract key information. The TF-IDF algorithm is used to extract keywords from the semantic fragments, such as "power system," "startup," and "fault code." A Conditional Random Field (CRF) model is employed to identify entities in the semantic fragments, such as software module names ("power control module") and function names ("starting the power system"). Relationship extraction is performed by combining rules and deep learning models to identify relationships between entities, such as "the power control module calls the fault diagnosis module."

[0075] Furthermore, the testing focus and scope were determined. Use case diagrams and activity diagrams were used to visually analyze the functional requirements of the power control software, identifying key functional modules (such as "Power Startup" and "Status Monitoring") and user operation flows. Referring to performance indicator documents, performance testing priorities were determined, such as response time and resource utilization, and performance modeling was performed using LoadRunner. Based on security specification documents, security threats, such as unauthorized access and data breaches, were identified, and security threat analysis was conducted using the STRIDE model.

[0076] Further, test cases are generated. The test requirements extraction agent, which determines the test focus and scope, along with key information extracted by the document analysis agent, are input into the large model base to generate the initial ideas and framework for test cases. During the generation process, RAG retrieval enhancement technology is used to retrieve relevant information from the knowledge base in real time, such as historical test cases and best practices, and add them to the test cases. A genetic algorithm is used to optimize the generated test cases, defining a fitness function to evaluate test case quality. Selection, crossover, and mutation operations are used to improve the coverage, effectiveness, and executability of the test cases.

[0077] Further, the results are evaluated. The JaCoCo tool is used to perform coverage analysis on the generated test cases, statistically analyzing statement coverage, branch coverage, etc., and analyzing uncovered code sections. The effectiveness of the test cases is evaluated through a combination of manual review and automated testing to check whether the test cases accurately reflect the testing requirements and identify potential defects in the software. The readability, modifiability, and extensibility of the test cases are assessed to ensure that the test case descriptions are clear, accurate, and easy to maintain and extend in the future.

[0078] Furthermore, optimize and provide feedback on multi-agent collaboration.

[0079] Based on the feedback from the evaluation agents, multiple agents collaborate to optimize and adjust strategies for document processing, test requirement extraction, and test case generation. A feedback loop mechanism is established to feed the evaluation results back to previous steps, prompting each agent to optimize processing methods and task allocation strategies, thereby improving the quality and efficiency of test case generation.

[0080] Furthermore, test cases are output and stored. The generated, evaluated, and optimized test cases are output in Excel spreadsheet format, including detailed information such as test case ID, test objective, input data, and expected output. Test cases are stored in a test case management system for easy subsequent test execution, maintenance, and management, supporting categorized management and statistical analysis based on different criteria.

[0081] By implementing the above technical solutions, this invention achieves high flexibility, high accuracy, and high coverage in generating software fuzzy test cases, providing an innovative solution for software quality assurance.

[0082] The above description is only the best specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the protection scope of the present invention.

[0083] The contents not described in detail in this specification are common knowledge to those skilled in the art.

Claims

1. A method for generating software fuzzy test cases based on semantic clustering, characterized in that: Includes the following steps: (1) Select a large language model, design a prompt word sequence and input relevant knowledge data of the software. Combine the prompt word sequence and knowledge data to adjust the large language model and form the large model base. (2) Convert each word in the software-related document into a low-dimensional vector representation, average or weighted average the obtained low-dimensional vector representations to obtain the semantic vector representation of the document, set a similarity threshold, cluster the semantic vector representations, cluster the semantic vector representations with semantic similarity higher than the threshold into the same cluster, and cut the document into semantic fragments according to the clustering results. (3) Use the TF-IDF algorithm to extract keywords from the semantic fragments obtained in step (2), calculate the TF-IDF value based on the frequency of the words in the document and the inverse document frequency in the entire document set, and select the words with higher TF-IDF values ​​as keywords; (4) Perform entity recognition on the semantic fragments obtained in step (2), wherein the entities include software module names, function names and variable names; extract the relationships between the entities to identify the relationships between them; (5) Determine the testing focus and scope based on the software's functional description document, performance indicator document, and security specification document; (6) Input the keywords obtained in step (3), the relationships between entities obtained in step (4), and the test focus and test scope obtained in step (5) into the large model base obtained in step (1). The large model base generates test cases based on the input and uses a genetic algorithm to iteratively optimize the generated test cases. (7) Output and store the test cases obtained in step (6).

2. The method for generating software fuzzy test cases based on semantic clustering according to claim 1, characterized in that: In step (1), the software-related knowledge data includes the software's functional description, performance indicators, security specifications, and historical test data. The knowledge data is structured and stored as a knowledge graph or vector database. The knowledge graph organizes knowledge in the form of entity-relationship-entity, while the vector database converts knowledge text into high-dimensional vectors.

3. The method for generating software fuzzy test cases based on semantic clustering according to claim 1, characterized in that: In step (2), the semantic similarity calculation method is the cosine similarity method.

4. The method for generating software fuzzy test cases based on semantic clustering according to claim 1, characterized in that: In step (4), a conditional random field model is used for entity recognition.

5. The method for generating software fuzzy test cases based on semantic clustering according to claim 1, characterized in that: In step (4), the relation extraction method includes formulating rules for matching and using a deep learning model for relation classification.

6. The method for generating software fuzzy test cases based on semantic clustering according to claim 1, characterized in that: In step (5), the performance requirements of the software are determined according to the performance index document, the performance modeling tool is used to perform performance modeling of the software, different load conditions are simulated, the performance of the software under different loads is analyzed, and the focus and key indicators of performance testing are determined. Based on the functional description document, identify the main functional modules of the software, the user operation flow, and the internal activities and state transition flow of the functional modules; analyze and determine the functional requirements of the software. Based on security specification documents, identify potential security threats to the software, employ security threat modeling methods to conduct security threat analysis on the software's system architecture and business processes, and determine the scope and focus of security testing.

7. The method for generating software fuzzy test cases based on semantic clustering according to claim 1, characterized in that: In step (6), after generating test cases, based on the current test cases, similar historical test cases, best practices and failure cases are searched in software-related knowledge data through RAG retrieval enhancement technology, and the retrieved relevant information is used as a supplement to optimize the test cases.

8. The method for generating software fuzzy test cases based on semantic clustering according to claim 1, characterized in that: In step (6), the chromosomes of the genetic algorithm are test cases, the genes are attributes of each test case, and the quality of the test cases is evaluated by the fitness function.

9. The method for generating software fuzzy test cases based on semantic clustering according to claim 1, characterized in that: In step (6), the optimized test cases are evaluated, and only those that pass the evaluation are output and stored. The evaluation methods include coverage analysis, manual review, and automated testing. Coverage analysis methods involve calculating the coverage of software code for each test case and identifying the uncovered code portions. Manual review includes checking whether test cases reflect test requirements and whether they are executable and repeatable; Automated testing uses a simulated environment or actual software to run test cases, observe whether the software's running results meet expectations, and check whether the test cases can discover potential defects in the software.

10. A method for generating software fuzzy test cases based on semantic clustering according to claim 9, characterized in that: The evaluation results are sent to the large model base. The large model base adjusts the test case generation strategy based on the evaluation results, and adjusts the keyword extraction method, entity recognition method, relation extraction method, and test focus and test scope determination method according to the adjusted test case generation strategy.

11. The method for generating software fuzzy test cases based on semantic clustering according to claim 1, characterized in that: In step (7), the test case output information includes test case ID, test purpose, input data, expected output, test steps, preconditions and postconditions.

Images